Skip to content

✪ A framework for designing hybrid effectiveness-implementation trials for digital health interventions

Authors:

Theresa E. Matson, Eric D.A. Hermes, Aaron R. Lyon, Andrew Quanbeck, Stephen M. Schueller, Sarah M. Wilson, and Joseph E. Glass

University of Washington affiliated authors are displayed in bold.

✪ Open Access

Published: February 2025

Read the full text open access in the journal Annals of Epidemiology

Abstract:

This article proposes methods for designing randomized controlled trials studying the implementation and effectiveness of digital interventions, meaning websites or applications ("apps") that patients use in healthcare. Deploying digital interventions for behavioral health differs from implementing traditional interventions such as medications or human-delivered therapy. Prior trial design guidance has ignored the existence of international governmental evidence standards, has paid insufficient attention to implementation reporting guidelines, and has not described methods for empirically testing the approach for organizing the delivery of digital interventions.

This framework for designing hybrid effectiveness-implementation trials of digital behavioral health interventions helps researchers articulate research questions that matter to decision-makers and meaningfully contribute to implementation. The framework outlines three phases: 1) frame effectiveness and implementation questions in terms of the digital intervention components, types of clinical support for the digital intervention, and specific strategies for implementing the digital intervention; 2) define and delineate actors, activities, action targets, dose, temporality, and outcomes to maximize inference and reproducibility; and 3) specify trial design features used for hybrid classification.

We illustrate the utility of this framework with two effectiveness-implementation studies of digital interventions for substance use. This framework can help researchers decide on appropriate methodology and help decision-makers apply findings.

**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**

The difference-making role of staff support in implementing nurse care management for opioid use disorder treatment: A configurational analysis

Authors:

Theresa E Matson, Amy K Lee, Edward J Miech, Paige D Wartko, Rebecca C Phillps, Mary Shea, Andrea Altschuler, Aimee N C Campbell, Colleen T Labelle, Julia H Arnsten, Jordan M Braciszewski, Joseph E Glass, Viviana E Horigian, Mark T Murphy, Mohammad Zare-Mehrjerdi, and Katharine A Bradley

University of Washington affiliated authors are displayed in bold.

Subscription Required for Full Text Access

Published: February 2025

Read the full text in the subscription access Journal of Substance Use and Addiction Treatment

✪ Formative evaluation of the implementation of digital therapeutics for opioids and other substance use disorders in primary care (DIGITS trial)

Authors:

Lorella Palazzo, Caitlin N Dorsey, Jess Mogk, Tara Beatty, Deborah King, Kelsey Stefanik-Guizlo, Dustin Key, Tessa E Matson, Mary Shea, Ryan M Caldeiro, Angela Garza McWethy, Edwin S Wong, Abisola E Idu, and Joseph E Glass

University of Washington affiliated authors are displayed in bold.

✪ Open Access

Published: December 2024

Read the full text in the open access journal Implementation Research and Practice

Abstract:

Background

Substance use disorders (SUDs) result in individual and societal burden. However, most individuals with SUD receive no treatment. Implementing SUD interventions in primary care could address this population's treatment needs. In the USA, reSET® and reSET-O® were the first prescription digital therapeutics (PDTs) for SUDs and opioid use disorder (OUD), respectively. The Digital Treatments for Substance Use Disorder (DIGITS) study tested the effectiveness of practice facilitation and health coaching strategies to support reSET and reSET-O implementation into primary care. A formative evaluation was conducted to monitor implementation, inform adaptations, and learn what promotes PDT sustainment.

Method

The Dynamic Sustainability Framework and the Framework for Reporting Adaptations and Modifications to Evidence-based Implementation Strategies guided the evaluation. Using rapid qualitative methods, we collected and analyzed observational fieldnotes, key informant interviews, and document sources (e.g., meeting minutes) for synthesis and dissemination to clinical partners and the study team via formative reports. We analyzed the reports to generate evaluation results.

Results

Twenty-four primary care clinics participated. Evaluation data included 98 observational fieldnotes, 16 interviews, and 253 document sources. We produced nine formative reports. The study encountered barriers and facilitators in each DSF domain (ecological system, practice setting, and intervention). In the ecological system, the PDT vendor enabled the study, but the COVID-19 pandemic, laws, regulations, and contracting delayed implementation. In the practice setting, staff shortages and low clinic capacity were implementation challenges, while electronic health record capabilities were both barriers and facilitators. At the intervention level, non-routine workflows, clinician burden, and low patient engagement were barriers despite clinicians’ efforts.

Conclusions

Digital therapeutics are promising SUD and OUD treatments, but integration into primary care requires conducive laws and regulations, organizational capacity, and patient and clinician engagement. Formative evaluation identified important lessons for future PDT implementation.

Plain Language Summary

Evaluation of the integration of digital treatments for opioid and other substance use disorders in primary care

Most people with substance use disorders (SUDs) receive no treatment. In the USA, two smartphone app-based digital treatments for SUD and opioid use disorders (OUDs) became available by prescription. The Digital Treatments for Substance Use Disorder (DIGITS) study, a randomized implementation trial, tested how best to integrate these digital treatments for SUD and OUD into primary care. Throughout the study, we conducted a formative evaluation to observe progress, recommend implementation improvements, and understand how digital treatments could be offered to patients after the study ended. For this evaluation, we collected qualitative data through observing study meetings and interviewing primary care leaders, clinicians, and clinical and study staff. The data were regularly summarized and reported to our clinical partners and the study team. We used two implementation frameworks to interpret the data: the Dynamic Sustainability Framework and the Framework for Reporting Adaptations and Modifications to Evidence-based Implementation Strategies. The evaluation revealed factors that helped and hindered implementation. Obstacles from outside the health care system included the COVID-19 pandemic, laws, regulations, and delays in setting up contractual agreements, whereas a partnership with the digital treatment vendor proved helpful. Clinics had difficulties with clinician burden, staff shortages, lack of time and available appointments, and performing new and unfamiliar tasks. The electronic health record system both supported and impended implementation. Last, few patients accepted the offered digital treatment, or used it persistently. Formative evaluation results suggest that digital treatments for SUD and OUD in primary care are promising, however their successful use requires supportive laws and regulations, health system resources, and increased patient and clinician engagement.

**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**

✪ Study protocol: A cluster-randomized trial implementing Sustained Patient-centered Alcohol-related Care (SPARC trial)

Authors:

Joseph E. Glass, Jennifer F. Bobb, Amy K. Lee, Julie E. Richards, Gwen T. Lapham, Evette Ludman, Carol Achtmeyer, Ryan M. Caldeiro, Rebecca Parrish, Emily C. Williams, Paula Lozano and Katharine A. Bradley

University of Washington affiliated authors are displayed in bold.

✪ Open Access

Published: August 2018

Read the full text in the open access journal Implementation Science

Abstract:

Background

Experts recommend that alcohol-related care be integrated into primary care (PC) to improve prevention and treatment of unhealthy alcohol use. However, few healthcare systems offer such integrated care. To address this gap, implementation researchers and clinical leaders at Kaiser Permanente Washington (KPWA) partnered to design a high-quality program of evidence-based care for unhealthy alcohol use: the Sustained Patient-centered Alcohol-related Care (SPARC) program. SPARC implements systems of clinical care designed to increase both prevention and treatment of unhealthy alcohol use. This clinical care for unhealthy alcohol use was implemented using three strategies: electronic health record (EHR) decision support, performance monitoring and feedback, and front-line support from external practice coaches with expertise in alcohol-related care (“SPARC implementation intervention” hereafter).

The purpose of this report is to describe the protocol of the SPARC trial, a pragmatic, cluster-randomized, stepped-wedge implementation trial to evaluate whether the SPARC implementation intervention increased alcohol screening and brief alcohol counseling (so-called brief interventions), and diagnosis and treatment of alcohol use disorders (AUDs) in 22 KPWA PC clinics.

Methods/Design

The SPARC trial sample includes all adult patients who had a visit to any of the 22 primary care sites in the trial during the study period (January 1, 2015–July 31, 2018). The 22 sites were randomized to implement the SPARC program on different dates (in seven waves, approximately every 4 months). Primary outcomes are the proportion of patients with PC visits who (1) screen positive for unhealthy alcohol use and have documented brief interventions and (2) have a newly recognized AUD and subsequently initiate and engage in alcohol-related care. Main analyses compare the rates of these primary outcomes in the pre- and post-implementation periods, following recommended approaches for analyzing stepped-wedge trials. Qualitative analyses assess barriers and facilitators to implementation and required adaptations of implementation strategies.

Discussion

The SPARC trial is the first study to our knowledge to use an experimental design to test whether practice coaches with expertise in alcohol-related care, along with EHR clinical decision support and performance monitoring and feedback to sites, increase both preventive care—alcohol screening and brief intervention—as well as diagnosis and treatment of AUDs.

Trial registration

The trial is registered at ClinicalTrials.Gov: NCT02675777. Registered February 5, 2016, https://clinicaltrials.gov/ct2/show/NCT02675777.

**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**