Skip to content

✪ Formative evaluation of the implementation of digital therapeutics for opioids and other substance use disorders in primary care (DIGITS trial)

Authors:

Lorella Palazzo, Caitlin N Dorsey, Jess Mogk, Tara Beatty, Deborah King, Kelsey Stefanik-Guizlo, Dustin Key, Tessa E Matson, Mary Shea, Ryan M Caldeiro, Angela Garza McWethy, Edwin S Wong, Abisola E Idu, and Joseph E Glass

University of Washington affiliated authors are displayed in bold.

✪ Open Access

Published: December 2024

Read the full text in the open access journal Implementation Research and Practice

Abstract:

Background

Substance use disorders (SUDs) result in individual and societal burden. However, most individuals with SUD receive no treatment. Implementing SUD interventions in primary care could address this population's treatment needs. In the USA, reSET® and reSET-O® were the first prescription digital therapeutics (PDTs) for SUDs and opioid use disorder (OUD), respectively. The Digital Treatments for Substance Use Disorder (DIGITS) study tested the effectiveness of practice facilitation and health coaching strategies to support reSET and reSET-O implementation into primary care. A formative evaluation was conducted to monitor implementation, inform adaptations, and learn what promotes PDT sustainment.

Method

The Dynamic Sustainability Framework and the Framework for Reporting Adaptations and Modifications to Evidence-based Implementation Strategies guided the evaluation. Using rapid qualitative methods, we collected and analyzed observational fieldnotes, key informant interviews, and document sources (e.g., meeting minutes) for synthesis and dissemination to clinical partners and the study team via formative reports. We analyzed the reports to generate evaluation results.

Results

Twenty-four primary care clinics participated. Evaluation data included 98 observational fieldnotes, 16 interviews, and 253 document sources. We produced nine formative reports. The study encountered barriers and facilitators in each DSF domain (ecological system, practice setting, and intervention). In the ecological system, the PDT vendor enabled the study, but the COVID-19 pandemic, laws, regulations, and contracting delayed implementation. In the practice setting, staff shortages and low clinic capacity were implementation challenges, while electronic health record capabilities were both barriers and facilitators. At the intervention level, non-routine workflows, clinician burden, and low patient engagement were barriers despite clinicians’ efforts.

Conclusions

Digital therapeutics are promising SUD and OUD treatments, but integration into primary care requires conducive laws and regulations, organizational capacity, and patient and clinician engagement. Formative evaluation identified important lessons for future PDT implementation.

Plain Language Summary

Evaluation of the integration of digital treatments for opioid and other substance use disorders in primary care

Most people with substance use disorders (SUDs) receive no treatment. In the USA, two smartphone app-based digital treatments for SUD and opioid use disorders (OUDs) became available by prescription. The Digital Treatments for Substance Use Disorder (DIGITS) study, a randomized implementation trial, tested how best to integrate these digital treatments for SUD and OUD into primary care. Throughout the study, we conducted a formative evaluation to observe progress, recommend implementation improvements, and understand how digital treatments could be offered to patients after the study ended. For this evaluation, we collected qualitative data through observing study meetings and interviewing primary care leaders, clinicians, and clinical and study staff. The data were regularly summarized and reported to our clinical partners and the study team. We used two implementation frameworks to interpret the data: the Dynamic Sustainability Framework and the Framework for Reporting Adaptations and Modifications to Evidence-based Implementation Strategies. The evaluation revealed factors that helped and hindered implementation. Obstacles from outside the health care system included the COVID-19 pandemic, laws, regulations, and delays in setting up contractual agreements, whereas a partnership with the digital treatment vendor proved helpful. Clinics had difficulties with clinician burden, staff shortages, lack of time and available appointments, and performing new and unfamiliar tasks. The electronic health record system both supported and impended implementation. Last, few patients accepted the offered digital treatment, or used it persistently. Formative evaluation results suggest that digital treatments for SUD and OUD in primary care are promising, however their successful use requires supportive laws and regulations, health system resources, and increased patient and clinician engagement.

**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**

Qualitative Study of Health Care Team Perception of the Benefits and Limitations of Remote Symptom Monitoring

Authors:

Emma K Hendrix, Nicole L Henderson, Tanvi V Padalkar, Tara Kaufmann, Stacey A Ingram, D'Ambra N Dent, Chao-Hui Sylvia Huang, J Nicholas Odom, Bryan J Weiner, Doris Howell, Angela M Stover, Ethan M Basch, Chelsea McGowan, Jennifer Young Pierce, and Gabrielle B Rocque

University of Washington affiliated authors are displayed in bold.

Subscription Required for Full Text Access

Published: December 2024

Read the full text in the subscription access journal JCO Oncology Practice

✪ Editorial: Learning for action in policy implementation

Authors:

Yanfang Su, Heather L Bullock, Michael Trisolini, Karen M Emmons

University of Washington affiliated authors are displayed in bold.

✪ Open Access

Published: November 2024

Read the full text in the open access journal Frontiers in Health Services

Excerpt:

Introduction

Policy implementation science (IS) is an emerging field that intersects implementation science and public policy studies to support the translation of evidence into policy. As a multidisciplinary field, Policy IS uses methodologies and frameworks from economics, political science, sociology, public administration, knowledge translation, and other fields. Policy IS would benefit from broader consensus on definitions, theories, frameworks, methodologies, and outcomes, so that a wide range of studies could build on a common conceptual foundation and thus build scientific knowledge more rapidly. The objective of this Research Topic is to reach consensus and drive learning for action in policy implementation by strengthening...

**This excerpt is posted with permission under the Creative Commons Attribution 4.0 International License**

Provider Identified Access Barriers to Delivering Acupuncture to Persons with Traumatic Brain Injury and Chronic Pain: A Cross-Sectional Self-Report Survey

Authors:

Jolie N. Haun, Risa Nakase-Richardson, Jeanne M. Hoffman, Mitch Sevigny, Mark D. Sodders, Flora M. Hammond, Bridget A. Cotner, Amanda Tweed, Robin Hanks, and Aaron M. Martin

University of Washington affiliated authors are displayed in bold.

Subscription Required for Full Text Access

Published: November 2024

Read the full text in the subscription access journal Journal of Integrative and Complementary Medicine

✪ Applying the Cognitive Walkthrough for Implementation Strategies methodology to inform the redesign of a selection-quality implementation toolkit for use in schools

Authors:

Kelsey S. Dickson, Olivia G. Michael, Amy Drahota, Aksheya Sridhar, Jessica E. Tschida, & Jill Locke

University of Washington affiliated authors are displayed in bold.

✪ Open Access

Published: November 2024

Read the full text in the open access journal Implementation Science Communications

Abstract:

Background

Implementation strategies are key to enhancing the translation of new innovations but there is a need to systematically design and tailor strategies to match the targeted implementation context and address determinants. There are increasing methods to inform the development and tailoring of implementation strategies to maximize their usability, feasibility, and appropriateness in new settings such as the Cognitive Walkthrough for Implementation Strategies (CWIS) approach. The aim of the current project is to apply the CWIS approach to inform the redesign of a multifaceted selection-quality implementation toolkit entitled Adoption of Curricular supports Toolkit: Systematic Measurement of Appropriateness and Readiness for Translation in Schools (ACT SMARTS) for use in middle and high schools.

Methods

We systematically applied CWIS as the second part of a community-partnered iterative redesign of ACT SMARTS for schools to evaluate the usability and inform further toolkit redesign areas. We conducted three CWIS user testing sessions with key end users of school district administrators (n = 3), school principals (n = 6), and educators (n = 6).

Results

Our CWIS application revealed that end users found ACT SMARTS acceptable and relevant but anticipate usability issues engaging in the ACT SMARTS process. Results informed the identification of eleven usability issues and corresponding redesign solutions to enhance the usability of ACT SMARTS for use in middle and high schools.

Conclusions

Results indicated the utility of CWIS in assessing implementation strategy usability in service of informing strategy modification as part of our broader redesign to improve alignment with end user, end recipient, and setting needs. Recommendations regarding the use of this participatory approach are discussed.

Contributions to the literature

  • The current case study adds to the nascent literature applying the Cognitive Walkthrough for Implementation Strategies (CWIS) method as part of a larger implementation project, including one of the first independent applications outside of the CWIS method developers.
  • The current case study illustrates the impact and utility of the CWIS method for designing and tailoring implementation strategies.
  • This case study provides further guidance and recommendations for the application of CWIS to future implementation projects.

**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**

✪ Operational and organizational variation in determinants of policy implementation success: the case of policies that earmark taxes for behavioral health services

Authors:

Jonathan Purtle, Nicole A. Stadnick, Amanda I. Mauri, Sarah C. Walker, Eric J. Bruns, & Gregory A. Aarons

University of Washington affiliated authors are displayed in bold.

✪ Open Access

Published: October 2024

Read the full text in the open access journal Implementation Science

Abstract:

Background

Research on determinants of health policy implementation is limited, and conceptualizations of evidence and implementation success are evolving in the field. This study aimed to identify determinants of perceived policy implementation success and assess whether these determinants vary according to: (1) how policy implementation success is operationally defined [i.e., broadly vs. narrowly related to evidence-based practice (EBP) reach] and (2) the role of a person’s organization in policy implementation. The study focuses on policies that earmark taxes for behavioral health services.

Methods

Web-based surveys of professionals involved with earmarked tax policy implementation were conducted between 2022 and 2023 (N = 272). The primary dependent variable was a 9-item score that broadly assessed perceptions of the tax policy positively impacting multiple dimensions of outcomes. The secondary dependent variable was a single item that narrowly assessed perceptions of the tax policy increasing EBP reach. Independent variables were scores mapped to determinants in the Exploration, Preparation, Implementation, and Sustainment (EPIS) framework. Multiple linear regression estimated associations between measures of determinants and policy implementation success.

Results

Perceptions of tax attributes (innovation determinant), tax EBP implementation climate (inner-context determinant), and inter-agency collaboration in tax policy implementation (outer-context and bridging factor determinant) were significantly associated with perceptions of policy implementation success. However, the magnitude of associations varied according to how success was operationalized and by respondent organization type. For example, the magnitude of the association between tax attributes and implementation success was 42% smaller among respondents at direct service organizations than non-direct service organizations when implementation success was operationalized broadly in terms of generating positive impacts (β = 0.37 vs. β = 0.64), and 61% smaller when success was operationalized narrowly in terms of EBP reach (β = 0.23 vs. β = 0.59). Conversely, when success was operationalized narrowly as EBP reach, the magnitude of the association between EBP implementation climate and implementation success was large and significant among respondents at direct service organizations while it was not significant among respondents from non-direct service organizations (β = 0.48 vs. β=-0.06).

Conclusion

Determinants of perceived policy implementation success may vary according to how policy implementation success is defined and the role of a person’s organization in policy implementation. This has implications for implementation science and selecting policy implementation strategies.

Contributions to the literature

  • The current study contributes to limited knowledge focused on determinants of policy implementation and scholarship about broadening definitions of evidence and implementation success in the field of implementation science.
  • Attributes of earmarked tax policies, tax policy EBP implementation climate, and frequency of inter-agency collaboration in tax policy implementation are significant determinants of tax policy implementation success and could be targeted by policy implementation strategies.
  • Determinants of policy implementation success vary according to how policy implementation success is operationally defined (broadly vs. narrowly related to EBP reach) and the role of a person’s organization in policy implementation the process.
  • Linking policy implementation determinants to the EPIS framework advances understanding of how implementation science frameworks can advance the study of policy implementation.

**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**

✪ VA’s EHR transition and health professions trainee programs: Findings and impacts of a multistakeholder learning community

Authors:

Julian Brunner, Ellen A. Ahlness, Ekaterina Anderson, Brianne K. Molloy-Paolillo, Alexandre Braga, Sarah L. Cutrona, Christian D. Helfrich, Deborah Levy, Erin Matteau, Edward Walton, George Sayre, Seppo T. Rinne

University of Washington affiliated authors are displayed in bold.

✪ Open Access

Published: October 2024

Read the full text in the open access journal Learning Health System

Abstract:

Introduction

The Department of Veterans Affairs (VA) is undergoing an unprecedented electronic health record (EHR) transition, switching from its homegrown EHR to a commercial system. The transition affects nearly every clinical employee but is particularly disruptive to health professions trainees (HPTs)—an often-overlooked population in EHR transitions. To better understand and address trainee challenges with the EHR transition, we formed a multistakeholder learning community. In this study, we describe the findings of this learning community and the practices and policies developed in response.

Methods

In the qualitative study designed and executed by our learning community, we conducted 51 interviews with HPTs, program leaders, and preceptors before and multiple times after an EHR transition site's go-live (February 16, 2022 to April 7, 2023). We merged interview transcripts with 125 survey free-text responses from a survey conducted with preceptors 2 months post-go-live and conducted thematic analysis to identify key themes. To complement qualitative findings, we also include a quantitative survey finding, and, where applicable, we note policy and practice responses spurred by our learning community.

Results

Interviews yielded six key themes: (1) High satisfaction with HPT programs, despite negative impacts of the EHR transition; (2) early delays, then substantial improvements, in HPTs' EHR access; (3) persistent challenges with HPTs' EHR training and support, mitigated by local and national efforts; (4) the challenge of learning to use a rapidly evolving EHR during clinical training; (5) reduced visit volume as a continuing barrier to education; and (6) an impression that HPTs' relative lack of exposure to the prior EHR facilitated their proficiency with the new EHR.

Conclusions

Findings highlighted challenges for HPT programs related to the EHR transition, which spurred important changes including the creation of a national VA council to represent the needs of HPTs in the EHR transition, and improvements to HPTs' EHR training and access.

**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**