Hybrid Designs Archives - Page 2 of 2 - Implementation Science at UW

✪ A machine learning approach to improve implementation monitoring of family-based preventive interventions in primary care

Abstract:

Background

Evidence-based parenting programs effectively prevent the onset and escalation of child and adolescent behavioral health problems. When programs have been taken to scale, declines in the quality of implementation diminish intervention effects. Gold-standard methods of implementation monitoring are cost-prohibitive and impractical in resource-scarce delivery systems. Technological developments using computational linguistics and machine learning offer an opportunity to assess fidelity in a low burden, timely, and comprehensive manner.

Methods

In this study, we test two natural language processing (NLP) methods [i.e., Term Frequency-Inverse Document Frequency (TF-IDF) and Bidirectional Encoder Representations from Transformers (BERT)] to assess the delivery of the Family Check-Up 4 Health (FCU4Health) program in a type 2 hybrid effectiveness-implementation trial conducted in primary care settings that serve primarily Latino families. We trained and evaluated models using 116 English and 81 Spanish-language transcripts from the 113 families who initiated FCU4Health services. We evaluated the concurrent validity of the TF-IDF and BERT models using observer ratings of program sessions using the COACH measure of competent adherence. Following the Implementation Cascade model, we assessed predictive validity using multiple indicators of parent engagement, which have been demonstrated to predict improvements in parenting and child outcomes.

Results

Both TF-IDF and BERT ratings were significantly associated with observer ratings and engagement outcomes. Using mean squared error, results demonstrated improvement over baseline for observer ratings from a range of 0.83–1.02 to 0.62–0.76, resulting in an average improvement of 24%. Similarly, results demonstrated improvement over baseline for parent engagement indicators from a range of 0.81–27.3 to 0.62–19.50, resulting in an approximate average improvement of 18%.

Conclusions

These results demonstrate the potential for NLP methods to assess implementation in evidence-based parenting programs delivered at scale. Future directions are presented.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

✪ Implementation outcomes and strategies for depression interventions in low- and middle-income countries: A systematic review

Abstract:

Background

We systematically reviewed implementation research targeting depression interventions in low- and middle-income countries (LMICs) to assess gaps in methodological coverage.

Methods

PubMed, CINAHL, PsycINFO, and EMBASE were searched for evaluations of depression interventions in LMICs reporting at least one implementation outcome published through March 2019.

Results

A total of 8714 studies were screened, 759 were assessed for eligibility, and 79 studies met inclusion criteria. Common implementation outcomes reported were acceptability (n = 50; 63.3%), feasibility (n = 28; 35.4%), and fidelity (n = 18; 22.8%). Only four studies (5.1%) reported adoption or penetration, and three (3.8%) reported sustainability. The Sub-Saharan Africa region (n = 29; 36.7%) had the most studies. The majority of studies (n = 59; 74.7%) reported outcomes for a depression intervention implemented in pilot researcher-controlled settings. Studies commonly focused on Hybrid Type-1 effectiveness-implementation designs (n = 53; 67.1), followed by Hybrid Type-3 (n = 16; 20.3%). Only 21 studies (26.6%) tested an implementation strategy, with the most common being revising professional roles (n = 10; 47.6%). The most common intervention modality was individual psychotherapy (n = 30; 38.0%). Common study designs were mixed methods (n = 27; 34.2%), quasi-experimental uncontrolled pre-post (n = 17; 21.5%), and individual randomized trials (n = 16; 20.3).

Conclusions

Existing research has focused on early-stage implementation outcomes. Most studies have utilized Hybrid Type-1 designs, with the primary aim to test intervention effectiveness delivered in researcher-controlled settings. Future research should focus on testing and optimizing implementation strategies to promote scale-up of evidence-based depression interventions in routine care. These studies should use high-quality pragmatic designs and focus on later-stage implementation outcomes such as cost, penetration, and sustainability.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

✪ Study protocol for evaluating Six Building Blocks for opioid management implementation in primary care practices

Abstract:

Background

The Six Building Blocks for improving opioid management (6BBs) is a program for improving the management of patients in primary care practices who are on long-term opioid therapy for chronic pain. The 6BBs include building leadership and consensus; aligning policies, patient agreements, and workflows; tracking and monitoring patient care; conducting planned, patient-centered visits; tailoring care for complex patients; and measuring success. The Agency for Healthcare Research and Quality funded the development of a 6BBs implementation guide: a step-by-step approach for independently implementing the 6BBs in a practice. This mixed-method study seeks to assess practices’ use of the implementation guide to implement the 6BBs and the effectiveness of 6BBs implementation on opioid management processes of care among practices using the implementation guide.

Methods

Data collection is guided by the Consolidated Framework for Implementation Research, Proctor’s taxonomy of implementation outcomes, and the Centers for Disease Control and Prevention’s Guideline for Prescribing Opioids for Chronic Pain. A diverse group of health care organizations with primary care clinics across the USA will participate in the study over 15 months. Qualitative data collection will include semi-structured interviews with stakeholders at each organization at two time points, notes from routine check-in calls, and document review. These data will be used to understand practices’ motivation for participation, history with opioid management efforts, barriers and facilitators to implementation, and implementation progress. Quantitative data collection will consist of a provider and staff survey, an implementation milestones assessment, and quarterly opioid prescribing quality measures. These data will supplement our understanding of implementation progress and will allow us to assess changes over time in providers’ opioid prescribing practices, prescribing self-efficacy, challenges to providing guideline-driven care, and practices’ opioid prescribing quality measures. Qualitative data will be coded and analyzed for emergent themes. Quantitative data will be analyzed using descriptive statistics and clustered multivariate regression.

Discussion

This study contributes to the knowledge of the implementation and effectiveness of a team-based approach to opioid management in primary care practices. Information gleaned from this study can be used to inform efforts to curtail opioid prescribing and assist primary care practices considering implementing the 6BBs.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

✪ Research translation for military and veteran health: research, practice, policy

Authors:

Jeffrey P. Haibach, Katherine D. Hoerster, Lindsey Dorflinger, Lisa M. McAndrew, Daniel G. Cassidy, David E. Goodrich, Jill E. Bormann, Julie Lowery, Steven M. Asch, Susan D. Raffa, Tannaz Moin, Alan L. Peterson, Michael G. Goldstein, Tracy Neal-Walden, Gerald W. Talcott, Christopher L. Hunter, & Sara J. Knight

^{University of Washington affiliated authors are displayed in bold.}

✪ Open Access

Published: February 2020

Read the full text in the journal Translational Behavioral Medicine

✪ Advancing successful implementation of task-shifted mental health care in low-resource settings (BASIC): Protocol for a stepped wedge cluster randomized trial

Abstract:

Background

The mental health treatment gap—the difference between those with mental health need and those who receive treatment—is high in low- and middle-income countries. Task-shifting has been used to address the shortage of mental health professionals, with a growing body of research demonstrating the effectiveness of mental health interventions delivered through task-shifting. However, very little research has focused on how to embed, support, and sustain task-shifting in government-funded systems with potential for scale up. The goal of the Building and Sustaining Interventions for Children (BASIC) study is to examine implementation policies and practices that predict adoption, fidelity, and sustainment of a mental health intervention in the education sector via teacher delivery and the health sector via community health volunteer delivery.

Methods

BASIC is a Hybrid Type II Implementation-Effectiveness trial. The study design is a stepped wedge, cluster randomized trial involving 7 sequences of 40 schools and 40 communities surrounding the schools. Enrollment consists of 120 teachers, 120 community health volunteers, up to 80 site leaders, and up to 1280 youth and one of their primary guardians. The evidence-based mental health intervention is a locally adapted version of Trauma-focused Cognitive Behavioral Therapy, called Pamoja Tunaweza. Lay counselors are trained and supervised in Pamoja Tunaweza by local trainers who are experienced in delivering the intervention and who participated in a Train-the-Trainer model of skills transfer. After the first sequence completes implementation, in-depth interviews are conducted with initial implementing sites’ counselors and leaders. Findings are used to inform delivery of implementation facilitation for subsequent sequences’ sites. We use a mixed methods approach including qualitative comparative analysis to identify necessary and sufficient implementation policies and practices that predict 3 implementation outcomes of interest: adoption, fidelity, and sustainment. We also examine child mental health outcomes and cost of the intervention in both the education and health sectors.

Discussion

The BASIC study will provide knowledge about how implementation of task-shifted mental health care can be supported in government systems that already serve children and adolescents. Knowledge about implementation policies and practices from BASIC can advance the science of implementation in low-resource contexts.

Trial registration

ClinicalTrials.gov Identifier: NCT03243396. Registered 9th August 2017

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

✪ Expanding Hybrid Studies for Implementation Research: Intervention, Implementation Strategy, and Context

Abstract:

Successful implementation reflects the interplay between intervention, implementation strategy, and context. Hybrid effectiveness-implementation studies allow investigators to assess the effects of both intervention and implementation strategy, though the role of context as a third independent variable (IV) is incompletely specified. Our objective is to expand the hybrid typology to include mixtures of all three types of IVs: intervention, implementation strategy, and context.

We propose to use I to represent the IV of intervention, IS to represent implementation strategy, and C to represent context. Primary IVs are written first and in upper case. Secondary IVs are written after a forward slash and in lower case; co-primary IVs are written after a dash and in upper case. The expanded framework specifies nine two-variable hybrid types: I/is, I-IS, IS/i, IS/c, IS-C, C/is, C/i, I-C, and I/c. We describe four in detail: I/is, IS/c, IS-C, and C/is. We also specify seven three-variable hybrid types. We argue that many studies already meet our definitions of two- or three-variable hybrids.

Our proposal builds from the typology proposed by Curran et al. (1), but offers a more complete specification of hybrid study types. We need studies that measure the implementation-related effects of variations in contextual determinants, both to advance the science and to optimize intervention delivery in the real world. Prototypical implementation studies that evaluate the effectiveness of an implementation strategy, in isolation from its context, risk perpetuating the gap between evidence and practice, as they will not generate context-specific knowledge around implementation, scale-up, and de-implementation.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

✪ Using a continuum of hybrid effectiveness-implementation studies to put research-tested colorectal screening interventions into practice

Abstract:

Background

Few previous studies have applied the hybrid effectiveness-implementation design framework to illustrate the way in which an intervention was progressively implemented and evaluated across multiple studies in diverse settings.

Methods

We describe the design components and methodologies of three studies that sought to improve rates of colorectal cancer (CRC) screening using mailed outreach, and apply domains put forth by Curran et al.: research aims, research questions, comparison conditions, sample, evaluation methods, measures, and potential challenges.

The Hybrid 1 study (emphasis on effectiveness) was a patient-level randomized trial of a mailed fecal test and stepped phone-outreach intervention program delivered in an integrated healthcare system (21 clinics, 4673 patients). The primary outcome was effectiveness (CRC screening uptake). Implementation outcomes included cost-effectiveness and acceptability.

The Hybrid 2 study (shared emphasis on effectiveness and implementation) was a pragmatic cluster-randomized trial of mailed fecal immunochemical test (FIT) outreach implemented at safety net clinics (26 clinics, 41,000 patients). The intervention used electronic health record tools (adapted from Hybrid 1) and clinic personnel to deliver the intervention. Outcomes included effectiveness (FIT completion) and implementation (FIT kits delivered, clinic barriers and facilitators, cost-effectiveness).

Hybrid 3 study (emphasis on implementation) is a demonstration project being conducted by two Medicaid/Medicare insurance plans (2 states, 12,000 patients) comparing two strategies for implementing mailed FIT programs that addressed Hybrid 2 implementation barriers. Outcomes include implementation (activities delivered, barriers) and effectiveness (FIT completion).

Results

The effectiveness-implementation typology successfully identified a number of distinguishing features between the three studies. Two additional features, program design and program delivery, varied across our studies, and we propose adding them to the current typology. Program design and program delivery reflect the process by which and by whom a program is designed and delivered (e.g., research staff vs. clinic/health plan staff).

Conclusions

We describe three studies that demonstrate the hybrid effectiveness to implementation continuum and make recommendations for expanding the hybrid typology to include new descriptive features. Additional comparisons of Hybrid 1, 2, and 3 studies may help confirm whether our hybrid typology refinements are generalizable markers of the pipeline from research to practice.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

✪ Using implementation facilitation to implement primary care mental health integration via clinical video telehealth in rural clinics: protocol for a hybrid type 2 cluster randomized stepped-wedge design

Abstract:

Background

Integrating mental health providers into primary care clinics improves access to and outcomes of mental health care. In the Veterans Health Administration (VA) Primary Care Mental Health Integration (PCMHI) program, mental health providers are co-located in primary care clinics, but the implementation of this model is challenging outside large VA medical centers, especially for rural clinics without full mental health staffing. Long wait times for mental health care, little collaboration between mental health and primary care providers, and sub-optimal outcomes for rural veterans could result. Telehealth could be used to provide PCMHI to rural clinics; however, the clinical effectiveness of the tele-PCMHI model has not been tested. Based on evidence that implementation facilitation is an effective implementation strategy to increase uptake of PCMHI when delivered on-site at larger VA clinics, it is hypothesized that this strategy may also be effective with regard to ensuring adequate uptake of the tele-PCMHI model at rural VA clinics.

Methods

This study is a hybrid type 2 pragmatic effectiveness-implementation trial of tele-PCMHI in six sites over 24 months. Tele-PCMHI, which will be delivered by clinical staff available in routine care settings, will be compared to usual care. Fidelity to the care model will be monitored but not controlled. We will use the Reach Effectiveness Adoption Implementation Maintenance (RE-AIM) framework to evaluate the patient-level clinical effectiveness of tele-PCMHI in rural VA clinics and also to evaluate the fidelity to and outcomes of the implementation strategy, implementation facilitation. The proposed study will employ a stepped-wedge design in which study sites sequentially begin implementation in three steps at 6-month intervals. Each step will include (1) a 6-month period of implementation planning, followed by (2) a 6-month period of active implementation, and (3) a final period of stepped-down implementation facilitation.

Discussion

This study will evaluate the effectiveness of PCMHI in a novel setting and via a novel method (clinical video telehealth). We will test the feasibility of using implementation facilitation as an implementation strategy to deploy tele-PCMHI in rural VA clinics.

Trial registration

ClinicalTrials.gov registration number NCT02713217. Registered on 18 March 2016.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

✪ Testing the effectiveness and implementation of a brief version of the Common Elements Treatment Approach (CETA) in Ukraine: A study protocol for a randomized controlled trial

Abstract:

Background

Mental illness is a major public health concern. Despite progress understanding which treatments work, a significant treatment gap remains. An ongoing concern is treatment length. Modular, flexible, transdiagnostic approaches have been offered as one solution to scalability challenges. The Common Elements Treatment Approach (CETA) is one such approach and offers the ability to treat a wide range of common mental health problems. CETA is supported by two randomized trials from low- and middle-income countries showing strong effectiveness and implementation outcomes.

Methods/design

This trial evaluates the effectiveness and implementation of two versions of CETA using a non-inferiority design to test two primary hypotheses: (1) a brief five-session version of CETA (Brief CETA) will provide similar effectiveness for reducing the severity of common mental health problems such as depression, post-traumatic stress, impaired functioning, anxiety, and substance use problems compared with the standard 8–12-session version of CETA (Standard CETA); and (2) both Brief and Standard CETA will have superior impact on the outcomes compared to a wait-list control condition. For both hypotheses, the main effect will be assessed using longitudinal data and mixed-effects regression models over a 6-month period post baseline. A secondary aim includes exploration of implementation factors. Additional planned analyses will include exploration of: moderators of treatment impact by disorder severity and comorbidity; the impact of individual therapeutic components; and trends in symptom change between end of treatment and 6-month assessment for all participants.

Discussion

This trial is the first rigorous study comparing a standard-length (8–12 sessions) modular, flexible, transdiagnostic, cognitive-behavioral approach to a shortened version of the approach (five sessions). Brief CETA entails “front-loading” with elements that research suggests are strong mechanisms of change. The study design will allow us to draw conclusions about the effects of both Brief and Standard CETA as well as which elements are integral to their mechanisms of action, informing future implementation and fidelity efforts. The results from this trial will inform future dissemination, implementation and scale-up of CETA in Ukraine and contribute to our understanding of the effects of modular, flexible, transdiagnostic approaches in similar contexts.

Trial registration

ClinicalTrials.gov, ID: NCT03058302 (U.S. National Library of Medicine). Registered on 20 February 2017.

Authors:

✪ Open Access

Read the full text in the open access journal Implementation Research and Practice

Abstract:

Background

Methods

Results

Conclusions

Authors:

✪ Open Access

Read the full text in the open access journal Global Mental Health

Abstract:

Background

Methods

Results

Conclusions

Authors:

✪ Open Access

Read the full text in the open access journal Implementation Science Communications

Abstract:

Background

Methods

Discussion

Authors:

✪ Open Access

Read the full text in the journal Translational Behavioral Medicine

Authors:

✪ Open Access

Read the full text in the open access journal BMC Psychiatry

Abstract:

Background

Methods

Discussion

Trial registration

Authors:

✪ Open Access

Read the full text in the open access journal Frontiers in Public Health

Abstract:

Authors:

Read the full text in the open access journal Implementation Science

Abstract:

Background

Methods

Results

Conclusions

Authors:

✪ Open Access

Read the full text in the open access journal Implementation Science

Abstract:

Background

Methods

Discussion

Trial registration

Authors:

✪ Open Access

Read the full text in the open access journal Trials

Abstract:

Background

Methods/design

Discussion

Trial registration