Bryan J. Weiner Archives - Page 6 of 10 - Implementation Science at UW

✪ Structural readiness to implement community-wide mass drug administration programs for soil-transmitted helminth elimination: results from a three-country hybrid study

Authors:

Arianna Rubin Means, Elizabeth Orlan, Marie-Claire Gwayi-Chore, Angelin Titus, Saravanakumar Puthupalayam Kaliappan, Comlanvi Innocent Togbevi, Félicien Chabi, Katherine E. Halliday, Providence Nindi, Euripide Avokpaho, Khumbo Kalua, Moudachirou Ibikounlé, Sitara S. R. Ajjampur, Bryan J. Weiner, Judd L. Walson, & Kumudha Aruldas

^{University of Washington affiliated authors are displayed in bold.}

✪ Open Access

Published: July 2021

Read the full text in the open access journal Implementation Science Communications

Abstract:

Background

Current soil-transmitted helminth (STH) control programs target pre-school and school-age children with mass drug administration (MDA) of deworming medications, reducing morbidity without interrupting ongoing transmission. However, evidence suggests that STH elimination may be possible if MDA is delivered to all community members. Such a change to the STH standard-of-care would require substantial systems redesign. We measured baseline structural readiness to launch community-wide MDA for STH in Benin, India, and Malawi.

Methods

After field piloting and adaptation, the structural readiness survey included two constructs: Organizational Readiness for Implementing Change and Organizational Capacity for Change. Sub-constructs of organizational readiness include change commitment and change efficacy. Sub-constructs of organizational capacity include flexibility, organizational structure, and demonstrated capacity. Survey items were also separately organized into seven implementation domains. Surveys were administered to policymakers, mid-level managers, and implementers in each country using a five-point Likert scale. Item, sub-construct, construct, and domain-level medians and interquartile ranges were calculated for each stakeholder level within each country.

Results

Median organizational readiness for change scores were highest in Malawi (5.0 for all stakeholder groups). In India, scores were 5.0, 4.0, and 5.0 while in Benin, scores were 4.0, 3.0, and 4.0 for policymakers, mid-level managers, and implementers, respectively. Median change commitment was equal to or higher than median change efficacy across all countries and stakeholder groups.

Median organizational capacity for change was highest in India, with a median of 4.5 for policymakers and mid-level managers and 5.0 for implementers. In Malawi, the median capacity was 4.0 for policymakers and implementers, and 3.5 for mid-level managers. In Benin, the median capacity was 4.0 for policymakers and 3.0 for mid-level managers and implementers. Median sub-construct scores varied by stakeholder and country.

Across countries, items reflective of the implementation domain ‘policy environment’ were highest while items reflective of the ‘human resource’ domain were consistently lower.

Conclusion

Across all countries, stakeholders valued community-wide MDA for STH but had less confidence in their collective ability to effectively implement it. Perceived capacity varied by stakeholder group, highlighting the importance of accounting for multi-level stakeholder perspectives when determining organizational preparedness to launch new public health initiatives.

Trial registration

NCT03014167

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

Financial Incentives for Pediatric HIV Testing (FIT): Caregiver Insights on Incentive Mechanisms, Focus Populations, and Acceptability for Programmatic Scale Up

✪ Development of an Asthma Home-Visit Training Program for Community Health Workers and Their Supervisors in Washington State

Abstract:

The community health worker (CHW) asthma home-visiting model developed by Public Health-Seattle & King County (PHSKC) is an evidence-based approach proven to improve health outcomes and quality of life. In addition, it has been shown to be an effective and culturally appropriate approach to helping people with asthma understand the environmental and behavioral causes of uncontrolled asthma, while acquiring the skills they need to control their asthma.

This paper describes the development and implementation of training curricula for CHWs and supervisors in the asthma home visiting program. To facilitate dissemination, this program took advantage of the current healthcare landscape in Washington State resulting from Centers for Medicare & Medicaid Services (CMS) approval of the 1115 Medicaid Waiver project. Key aspects of the training program development included: (1) Engagement: forming a Community Advisory Board with multiple stakeholders to help prioritize training content; (2) Curriculum Development: building the training on evidence-based home-visit protocols previously developed at PHSKC; (3) Implementation of the training program; (4) Evaluation of the training; and (5) Adaptation of the training based on lessons learned.

We describe key factors in the training program's improvement including the use of a community-based participatory approach to engage stakeholders at multiple phases of the project and ensure regional adaption; combining in-person and online modules for delivery; and holding learning collaboratives for post-training and technical support. We also outline our training program evaluation plan and the planned evaluation of the home visit program which the trainees will deliver, both of which follow the RE-AIM framework. However, because the COVID-19 pandemic has curtailed training activities and prohibited the trainees from implementation of these CHW home visit practices, our evaluation is currently incomplete. Therefore, this case study provides insight into the adaptation of the training program, but not the delivery of the home visit program, the outcomes of which remain to be seen.

✪ Using Assisted Partner Services for HIV Testing and the Treatment of Males and Their Female Sexual Partners: Protocol for an Implementation Science Study

Abstract:

Background

Despite the effective scale-up of HIV testing and treatment programs, only 75% of people living with HIV (PLWH) globally know their status, and this rate is lower among men. This highlights the importance of implementing HIV testing and linkage interventions with a high uptake in this population. In a cluster randomized controlled trial conducted in Kenya between 2013 and 2015, we found that assisted partner services (APS) for HIV-exposed partners of newly diagnosed PLWH safely reached more HIV-exposed individuals with HIV testing compared with client referral alone. However, more data are needed to evaluate APS implementation in a real-world setting.

Objective

This study aims to evaluate the effectiveness, acceptability, fidelity, and cost of APS when integrated into existing HIV testing services (HTS) in Western Kenya.

Methods

Our study team from the University of Washington and PATH is integrating APS into 31 health facilities in Western Kenya. We are enrolling females newly diagnosed with HIV (index clients) who consent to receiving APS, their male sexual partners, and female sexual partners of male sexual partners who tested HIV positive. Female index clients and sexual partners testing HIV positive will be followed up at 6 weeks, 6 months, and 12 months postenrollment to assess linkage to care, antiretroviral therapy initiation, and HIV viral load suppression. We will evaluate the acceptability, fidelity, and cost of real-world implementation of APS via in-depth interviews conducted with national, county, and subcounty-level policy makers responsible for HTS. Facility health staff providing HTS and APS, in addition to staff working with the study project team, will also be interviewed. We will also conduct direct observations of facility infrastructure and clinical procedures and extract data from the facilities and county and national databases.

Results

As of March 2020, we have recruited 1724 female index clients, 3201 male partners, and 1585 female partners. We have completed study recruitment as well as 6-week (2936/2973, 98.75%), 6-month (1596/1641, 97.25%), and 12-month (725/797, 90.9%) follow-up visits. Preliminary analyses show that facilities scaling up APS identify approximately 12-18 new HIV-positive males for every 100 men contacted and tested. We are currently completing the remaining follow-up interviews and incorporating an HIV self-testing component into the study in response to the COVID-19 pandemic.

Conclusions

The results will help bridge the gap between clinical research findings and real-world practice and provide guidance regarding optimal strategies for APS integration into routine HIV service delivery.

International Registered Report Identifier (IRRID)

DERR1-10.2196/27262

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

✪ Measuring characteristics of individuals: An updated systematic review of instruments’ psychometric properties

Abstract:

Background

Identification of psychometrically strong implementation measures could (1) advance researchers’ understanding of how individual characteristics impact implementation processes and outcomes, and (2) promote the success of real-world implementation efforts. The current study advances the work that our team published in 2015 by providing an updated and enhanced systematic review that identifies and evaluates the psychometric properties of implementation measures that assess individual characteristics.

Methods

A full description of our systematic review methodology, which included three phases, is described in a previously published protocol paper. Phase I focused on data collection and involved search string generation, title and abstract screening, full-text review, construct assignment, and measure forward searches. During Phase II, we completed data extraction (i.e., coding psychometric information). Phase III involved data analysis, where two trained specialists independently rated each measurement tool using our psychometric rating criteria.

Results

Our team identified 124 measures of individual characteristics used in mental or behavioral health research, and 123 of those measures were deemed suitable for rating using Psychometric and Pragmatic Evidence Rating Scale. We identified measures of knowledge and beliefs about the intervention (n = 76), self-efficacy (n = 24), individual stage of change (n = 2), individual identification with organization (n = 7), and other personal attributes (n = 15). While psychometric information was unavailable and/or unreported for many measures, information about internal consistency and norms were the most commonly identified psychometric data across all individual characteristics’ constructs. Ratings for all psychometric properties predominantly ranged from “poor” to “good.”

Conclusion

The majority of research that develops, uses, or examines implementation measures that evaluate individual characteristics does not include the psychometric properties of those measures. The development and use of psychometric reporting standards could advance the use of valid and reliable tools within implementation research and practice, thereby enhancing the successful implementation and sustainment of evidence-based practice in community care.

Plain Language Summary

Measurement is the foundation for advancing practice in health care and other industries. In the field of implementation science, the state of measurement is only recently being targeted as an area for improvement, given that high-quality measures need to be identified and utilized in implementation work to avoid developing another research to practice gap. For the current study, we utilized the Consolidated Framework for Implementation Research to identify measures related to individual characteristics’ constructs, such as knowledge and beliefs about the intervention, self-efficacy, individual identification with the organization, individual stage of change, and other personal attributes. Our review showed that many measures exist for certain constructs (e.g., measures related to assessing providers’ attitudes and perceptions about evidence-based practice interventions), while others have very few (e.g., an individual’s stage of change). Also, we rated measures for their psychometric strength utilizing an anchored rating system and found that most measures assessing individual characteristics are in need of more research to establish their evidence of quality. It was also clear from our results that frequency of use/citations does not equate to high quality, psychometric strength. Ultimately, the state of the literature has demonstrated that assessing individual characteristics of implementation stakeholders is an area of strong interest in implementation work. It will be important for future research to focus on clearly delineating the psychometric properties of existing measures for saturated constructs, while for the others the emphasis should be on developing new, high-quality measures and make these available to stakeholders.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

✪ Applying human-centered design to maximize acceptability, feasibility, and usability of mobile technology supervision in Kenya: A mixed methods pilot study protocol

Abstract:

Background

Although research continues to support task-shifting as an effective model of delivering evidence-based practices (EBPs), little scholarship has focused how to scale up and sustain task-shifting in low- and middle-income countries, including how to sustainably supervise lay counselors. Ongoing supervision is critical to ensure EBPs are delivered with fidelity; however, the resources and expertise required to provide ongoing supervision may limit the potential to scale up and sustain task shifting. Opportunities may exist to leverage mobile technology to replace or supplement in-person supervision in low-resource contexts, but contextual variables, such as network connectivity and lay counselor preferences surrounding mobile technology, must be examined and considered when designing and implementing mobile technology supervision.

Methods

This study builds from an existing randomized trial in Kenya, wherein teachers and community health volunteers have been trained to provide trauma-focused cognitive behavioral therapy as lay counselors. The study will use an iterative and mixed methods approach, with qualitative interviews and a Human-Centered Design (HCD) workshop informing a non-randomized pilot trial. Semi-structured interviews will be conducted with lay counselors and supervisors to understand how mobile technology is currently being used for supervision and determine the barriers and facilitators to mobile technology supervision. Data from these interviews will inform an HCD workshop, where lay counselors and supervisors “re-design” supervision to most effectively leverage mobile technology. Workshop participants will also participate in focus groups to gather perceptions on the use of HCD techniques. The final outcome of the workshop will be a set of refined workflows, which will be tested in a mixed method, nonrandomized pilot with newly trained lay counselors and existing supervisors. The pilot trial will evaluate the acceptability, feasibility, and usability of mobile technology supervision through self-report questionnaires as well as perceptions of effectiveness through qualitative interviews with a subset of lay counselors and all supervisors.

Discussion

This study will provide a launching point for future research on supervision and methods to engage stakeholders to design and tailor interventions and implementation supports to fit low-resourced contexts.

Trial registration

The parent trial from which this study builds was registered on ClinicalTrials.gov on August 9, 2017 (NCT03243396).

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

Optimizing treatment cascades for mental healthcare in Mozambique: Preliminary effectiveness of the Systems Analysis and Improvement Approach for Mental Health (SAIA-MH)

✪ Implementation Science to Respond to the COVID-19 Pandemic

Abstract:

The COVID-19 pandemic continues to expand globally, requiring massive public health responses from national and local governments. These bodies have taken heterogeneous approaches to their responses, including when and how to introduce and enforce evidence-based interventions—such as social distancing, hand-washing, personal protective equipment (PPE), and testing. In this commentary, we reflect on opportunities for implementation science to contribute meaningfully to the COVID-19 pandemic response. We reflect backwards on missed opportunities in emergency preparedness planning, using the example of PPE stockpiling and supply management; this planning could have been strengthened through process mapping with consensus-building, microplanning with simulation, and stakeholder engagement.

We propose current opportunities for action, focusing on enhancing the adoption, fidelity, and sustainment of hand washing and social distancing; we can combine qualitative data, policy analysis, and dissemination science to inform agile and rapid adjustment to social marketing strategies to enhance their penetration. We look to future opportunities to enhance the integration of new evidence in decision-making, focusing on serologic and virologic testing systems; we can leverage simulation and other systems engineering modeling to identify ideal system structures. Finally, we discuss the ways in which the COVID-19 pandemic challenges implementation science to become more rapid, rigorous, and nimble in its approach, and integrate with public health practice. In summary, we articulate the ways in which implementation science can inform, and be informed by, the COVID-19 pandemic, looking backwards, proposing actions for the moment, and approaches for the future.

Contributions to the Literature

The COVID-19 pandemic response has illuminated gaps in emergency preparedness planning that could be addressed by implementation scientists working in coordination with public health practitioners.
Classical implementation science tools and measures can be operationalized within the context of the COVID-19 pandemic, and at each stage of the pandemic. Doing so empowers implementation scientists to apply their skillsets toward optimizing uptake of evidence-based interventions such as such as social distancing, hand washing, and personal protective equipment.
To be more responsive to public health emergencies, implementation science as a discipline must evolve to be more rapid, nimble, and policy-oriented in its approaches.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

✪ Measuring readiness for implementation: A systematic review of measures’ psychometric and pragmatic properties

Abstract:

Background

Systematic measure reviews can facilitate advances in implementation research and practice by locating reliable, valid, pragmatic measures; identifying promising measures needing refinement and testing; and highlighting measurement gaps. This review identifies and evaluates the psychometric and pragmatic properties of measures of readiness for implementation and its sub-constructs as delineated in the Consolidated Framework for Implementation Research: leadership engagement, available resources, and access to knowledge and information.

Methods

The systematic review methodology is described fully elsewhere. The review, which focused on measures used in mental or behavioral health, proceeded in three phases. Phase I, data collection, involved search string generation, title and abstract screening, full text review, construct assignment, and cited citation searches. Phase II, data extraction, involved coding relevant psychometric and pragmatic information. Phase III, data analysis, involved two trained specialists independently rating each measure using Psychometric and Pragmatic Evidence Rating Scales (PAPERS). Frequencies and central tendencies summarized information availability and PAPERS ratings.

Results

Searches identified 9 measures of readiness for implementation, 24 measures of leadership engagement, 17 measures of available resources, and 6 measures of access to knowledge and information. Information about internal consistency was available for most measures. Information about other psychometric properties was often not available. Ratings for internal consistency were “adequate” or “good.” Ratings for other psychometric properties were less than “adequate.” Information on pragmatic properties was most often available regarding cost, language readability, and brevity. Information was less often available regarding training burden and interpretation burden. Cost and language readability generally exhibited “good” or “excellent” ratings, interpretation burden generally exhibiting “minimal” ratings, and training burden and brevity exhibiting mixed ratings across measures.

Conclusion

Measures of readiness for implementation and its sub-constructs used in mental health and behavioral health care are unevenly distributed, exhibit unknown or low psychometric quality, and demonstrate mixed pragmatic properties. This review identified a few promising measures, but targeted efforts are needed to systematically develop and test measures that are useful for both research and practice.

Plain language abstract

Successful implementation of effective mental health or behavioral health treatments in service delivery settings depends in part on the readiness of the service providers and administrators to implement the treatment; the engagement of organizational leaders in the implementation effort; the resources available to support implementation, such as time, money, space, and training; and the accessibility of knowledge and information among service providers about the treatment and how it works. It is important that the methods for measuring these factors are dependable, accurate, and practical; otherwise, we cannot assess their presence or strength with confidence or know whether efforts to increase their presence or strength have worked. This systematic review of published studies sought to identify and evaluate the quality of questionnaires (referred to as measures) that assess readiness for implementation, leadership engagement, available resources, and access to knowledge and information. We identified 56 measures of these factors and rated their quality in terms of how dependable, accurate, and practical they are. Our findings indicate there is much work to be done to improve the quality of available measures; we offer several recommendations for doing so.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

✪ Measuring implementation outcomes: An updated systematic review of measures’ psychometric properties

Abstract:

Background

Systematic reviews of measures can facilitate advances in implementation research and practice by locating reliable and valid measures and highlighting measurement gaps. Our team completed a systematic review of implementation outcome measures published in 2015 that indicated a severe measurement gap in the field. Now, we offer an update with this enhanced systematic review to identify and evaluate the psychometric properties of measures of eight implementation outcomes used in behavioral health care.

Methods

The systematic review methodology is described in detail in a previously published protocol paper and summarized here. The review proceeded in three phases. Phase I, data collection, involved search string generation, title and abstract screening, full text review, construct assignment, and measure forward searches. Phase II, data extraction, involved coding psychometric information. Phase III, data analysis, involved two trained specialists independently rating each measure using PAPERS (Psychometric And Pragmatic Evidence Rating Scales).

Results

Searches identified 150 outcomes measures of which 48 were deemed unsuitable for rating and thus excluded, leaving 102 measures for review. We identified measures of acceptability (N = 32), adoption (N = 26), appropriateness (N = 6), cost (N = 31), feasibility (N = 18), fidelity (N = 18), penetration (N = 23), and sustainability (N = 14). Information about internal consistency and norms were available for most measures (59%). Information about other psychometric properties was often not available. Ratings for internal consistency and norms ranged from “adequate” to “excellent.” Ratings for other psychometric properties ranged mostly from “poor” to “good.”

Conclusion

While measures of implementation outcomes used in behavioral health care (including mental health, substance use, and other addictive behaviors) are unevenly distributed and exhibit mostly unknown psychometric quality, the data reported in this article show an overall improvement in availability of psychometric information. This review identified a few promising measures, but targeted efforts are needed to systematically develop and test measures that are useful for both research and practice.

Plain language abstract

When implementing an evidence-based treatment into practice, it is important to assess several outcomes to gauge how effectively it is being implemented. Outcomes such as acceptability, feasibility, and appropriateness may offer insight into why providers do not adopt a new treatment. Similarly, outcomes such as fidelity and penetration may provide important context for why a new treatment did not achieve desired effects. It is important that methods to measure these outcomes are accurate and consistent. Without accurate and consistent measurement, high-quality evaluations cannot be conducted. This systematic review of published studies sought to identify questionnaires (referred to as measures) that ask staff at various levels (e.g., providers, supervisors) questions related to implementation outcomes, and to evaluate the quality of these measures. We identified 150 measures and rated the quality of their evidence with the goal of recommending the best measures for future use. Our findings suggest that a great deal of work is needed to generate evidence for existing measures or build new measures to achieve confidence in our implementation evaluations.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

Authors:

✪ Open Access

Read the full text in the open access journal Implementation Science Communications

Abstract:

Background

Methods

Results

Conclusion

Trial registration

Authors:

Subscription Required for Full Text Access

Read the full text in the subscription access journal AIDS and Behavior

Authors:

✪ Open Access

Read the full text in the open access journal Frontiers in Public Health

Abstract:

Authors:

✪ Open Access

Read the full text in the open access journal JMIR Research Protocols

Abstract:

Background

Objective

Methods

Results

Conclusions

International Registered Report Identifier (IRRID)

Authors:

✪ Open Access

Read the full text in the open access journal Implementation Research and Practice

Abstract:

Background

Methods

Results

Conclusion

Plain Language Summary

Authors:

✪ Open Access

Read the full text in the open access journal Implementation Science Communications

Abstract:

Background

Methods

Discussion

Trial registration

Authors:

Subscription Required for Full Text Access

Read the full text in the subscription access journal Health Policy and Planning

Authors:

✪ Open Access

Read the full text in the open access journal Frontiers in Public Health

Abstract:

Contributions to the Literature

Authors:

✪ Open Access

Read the full text in the open access journal Implementation Research and Practice

Abstract:

Background

Methods

Results

Conclusion

Plain language abstract

Authors:

✪ Open Access

Read the full text in the open access journal Implementation Research and Practice

Abstract:

Background

Methods

Results

Conclusion

Plain language abstract