FRAME-IS Archives - Implementation Science at UW

✪ Evaluating behavioral intervention technologies: Integrating human-centered design and implementation science outcomes

Abstract:

Background

Behavioral intervention technologies (BITs) offer scalable, cost-effective healthcare solutions but often show reduced impacts in community settings and are rarely sustained. Human-centered design (HCD) enhances usability by tailoring BITs to stakeholder needs, while implementation science (IS) identifies contextual barriers and strategies to promote uptake and sustainability. Both HCD and IS aim to improve BIT usability and implementation through iterative, user-focused processes but are rarely integrated.

Objective

We introduced the user-centered and sustainable implementation science (USIS) model, a novel and systematic framework that combines HCD and IS principles to enhance BIT effectiveness and sustainability. We aimed to (1) synthesize the HCD and IS outcomes and integrate them into a USIS framework; (2) apply USIS to a case study: the sleep shared-management intervention for children with juvenile idiopathic arthritis and their parents (SLEEPSMART).

Methods

We conducted a narrative literature review on HCD and IS outcomes for BITs from the PubMed, CINAHL, and Web of Science databases. Articles were selected based on their focus on usability, implementation outcomes, and evidence-based healthcare practices. This synthesis informed the development of the USIS model.

Results

The USIS model incorporates five domains: (1) user-centeredness (empathy, engagement, and equity), (2) efficiency (cost, timeliness, and rapidity), (3) feasibility (learnability, memorability, error reduction, and low cognitive load), (4) satisfaction (acceptability and appropriateness), and (5) fidelity (adoption, penetration, and sustainability). We applied the USIS model to evaluate the SLEEPSMART project to demonstrate the application of the USIS model in guiding BIT design and refinement. Insights from the review informed the design principles applied during the development and testing stages of SLEEPSMART.

Conclusion

The USIS model serves as a practical and theoretical guide to improve BIT design and evaluation. By emphasizing implementation considerations early and centering user needs, USIS provides a pathway to bridge HCD and IS approaches to enhance the real-world impact and sustainability of digital health innovations.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

✪ Finding the balance between rigour and relevance: implementing adaptations to the implementation of a pragmatic randomised controlled trial of a two-way texting intervention for voluntary medical male circumcision in South Africa

Abstract:

Objectives

To document adaptations that were made to the implementation of the two-way texting (2wT) randomised controlled trial (RCT) for voluntary medical male circumcision (VMMC) in South Africa and to provide a nuanced discussion on the differences between adaptations and fidelity in this context.

Design

We conducted a qualitative study using the Framework for Reporting Adaptations and Modifications in Evidence-based Implementation Strategies (FRAME-IS) to examine 2wT adaptations. We reported adaptations to the 2wT intervention using two steps. First, we categorised adaptations in a shared study-specific Google Docs that documented participant engagement with the 2wT system, tracked daily RCT implementation notes, reported software bugs and noted reminder emails about adaptations for the research team. Second, we conducted a qualitative assessment of the influence of adaptations on project outcomes via 10 periodic reflection meetings with VMMC implementers. Reflection documentation included notes from field observations, meeting minutes and informal partner check-ins to complete adaptation documentation. Using the FRAME-IS as a codebook, adaptations were categorised.

Settings

The RCT was conducted in rural and urban VMMC clinics in the North West and Gauteng districts of South Africa.

Participants

Implementation scientists and VMMC implementers who implemented the 2wT pragmatic randomised controlled trial (pRCT) were participants for the adaptation study.

Primary and secondary outcome measures

The primary outcome measure was the adaptations that were made during the implementation of the 2wT pRCT. The secondary outcome measures were fidelity and rigour of implementing adaptations to the 2wT pRCT.

Results

Between June 2021 and February 2022, 13 adaptations were identified in three phases during the implementation of the 2wT pRCT. The first phase of adaptations aimed to augment study recruitment, including conducting weekend VMMC recruitment camps, using mobile outreach services in the rural site, adding two urban sites to increase recruitment, using weekly WhatsApp calls for updates with all implementing teams, using virtual meetings to implement the 2wT strategy remotely during COVID-19 restrictions and allocating one clinician to work outside of normal working hours. The second phase of adaptations further enhanced enrolments, including adding two local language translations in the usability survey for 2wT men and contributing a portion towards the salary of the implementing staff by the research partner. The third phase included the exclusion of two rural clinics as recruitment sites due to inconsistent mobile phone networks, adding another layer of data quality checks to ensure data quality, training non-clinical counsellors to help with enrolling clients, retraining of staff in the rural site with high staff turnover and using both primary and alternative phone numbers for enrolment to reduce loss to follow-up.

Conclusions

This study made adaptations to the 2wT pRCT without compromising the fidelity of the study. The 2wT pRCT balances rigour (fidelity) and relevance (adaptation). Adaptations should not be confined by rigour but should also not go unchallenged or unverified. We conclude that fidelity can be maintained with adaptations that are implemented to close the gap between research in the laboratory and practice.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

✪ Harnessing Human-Centered Design for Evidence-Based Psychosocial Interventions and Implementation Strategies in Community Settings: Protocol for Redesign to Improve Usability, Engagement, and Appropriateness

Authors:

Aaron R Lyon, Sean A Munson, Michael D Pullmann, Brittany Mosser, Tricia Aung , John Fortney, Alex Dopp, Katie P Osterhage, Helen G Haile, Kathryn E Bruzios, Brittany E Blanchard, Ryan Allred, Macey R Fuller, Patrick J Raue, Ian Bennett, Jill Locke, Karen Bearss, Denise Walker, Elizabeth Connors, Eric J Bruns, Jenna Van Draanen, Doyanne Darnell, Patricia A Areán

^{University of Washington affiliated authors are displayed in bold.}

✪ Open Access

Published: January 2025

Read the full text in the open access journal JMIR Research Protocols

Abstract:

Background

Although substantial progress has been made in establishing evidence-based psychosocial clinical interventions and implementation strategies for mental health, translating research into practice—particularly in more accessible, community settings—has been slow.

Objective

This protocol outlines the renewal of the National Institute of Mental Health–funded University of Washington Advanced Laboratories for Accelerating the Reach and Impact of Treatments for Youth and Adults with Mental Illness Center, which draws from human-centered design (HCD) and implementation science to improve clinical interventions and implementation strategies. The Center’s second round of funding (2023-2028) focuses on using the Discover, Design and Build, and Test (DDBT) framework to address 3 priority clinical intervention and implementation strategy mechanisms (ie, usability, engagement, and appropriateness), which we identified as challenges to implementation and scalability during the first iteration of the center. Local redesign teams work collaboratively and share decision-making to carry out DDBT.

Methods

All 4 core studies received institutional review board approval by June 2024, and each pilot project will pursue institutional review board approval when awarded. We will provide research infrastructure to 1 large effectiveness study and 3 exploratory pilot studies as part of the center grant. At least 4 additional small pilot studies will be solicited and funded by the center. All studies will explore the use of DDBT for clinical interventions and implementation strategies to identify modification targets to improve usability, engagement, and appropriateness in accessible nonspecialty settings (Discover phase); develop redesign solutions with local teams to address modification targets (Design and Build phase); and determine if redesign improves usability, engagement, and appropriateness (Test phase), as well as implementation outcomes. Center staff will collaborate with local redesign teams to develop and test clinical interventions and implementation strategies for community settings. We will collaborate with teams to use methods and centerwide measures that facilitate cross-project analysis of the effects of DDBT-driven redesign on outcomes of interest.

Results

As of January 2025, three of the 4 core studies are underway. We will generate additional evidence on the robustness of DDBT and whether combining HCD and implementation science is an asset for improving clinical interventions and implementation strategies.

Conclusions

During the first round of the center, we established that DDBT is a useful approach to systematically identify and address chronic challenges of implementing clinical interventions and implementation strategies. In this subsequent grant, we expect to increase evidence of DDBT’s impact on clinical interventions and implementation strategies by expanding a list of common challenges that could benefit from modification, a list of exemplary solutions to address these challenges, and guidance on using the DDBT framework. These resources will contribute to broader discourse on how to enhance implementation of clinical interventions and implementation strategies that integrate HCD and implementation science.

International Registered Report Identifier (IRRID)

PRR1-10.2196/65446

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

✪ Implementation research logic model in the design and execution of eHealth innovations for maternal and newborn healthcare in Ethiopia

Abstract:

Background

The use of eHealth innovations is becoming increasingly important in improving health outcomes, especially for maternal and newborn health. However, planning and executing these innovations can be challenging due to their complex nature. To provide guidance and clarity on implementation approaches, researchers need to use implementation research (IR) tools. We conducted IR to recognize the challenges in implementing eHealth innovations in the context of maternal and newborn healthcare using the implementation research logic model (IRLM). Therefore, this paper aims to describe the practical application of IRLM to design, execute and evaluate eHealth innovations that improve maternal and newborn care in public facilities in Ethiopia.

Methods

We employed rapid review, formative assessment and process evaluation of an eHealth innovation in selected healthcare facilities serving maternal and newborn care. The eHealth innovation we developed and deployed was named ‘ADHERE’ (Antenatal Care, ChilDbirtH CarE and Postnatal CaRE), representing the continuum of maternal care. The rapid review was conducted as an initial step using the Consolidated Framework for Implementation Research (CFIR). We employed a mix of data collection methods: interview/discussion, eHealth system or document review and direct observation. Furthermore, we executed various stakeholder engagement activities: two co-creation workshops and on-site iterative discussions. We applied the Framework for Reporting Adaptations and Modifications to Evidence-based Implementation Strategies (FRAME-IS) to capture ongoing implementation learnings.

Results

We developed IRLM of the eHealth innovation implementation for three contexts: urban, peri-urban and remote public healthcare facilities. The model depicted the mechanism of interaction between implementation determinants and implementation strategies to produce the intended implementation outcomes. The IRLM helped to identify more than 35 implementation barriers or facilitators for eHealth interventions and to develop over 17 mitigation strategies for the study contexts. The initial IRLM was refined through ongoing implementation learnings and the mitigation strategies that were executed.

Conclusions

The IRLM is a comprehensive and effective guiding tool for the development, implementation and evaluation of innovations in various low- and middle-income contexts. Researchers and implementing partners should adapt and use it.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

✪ Formative evaluation of the implementation of digital therapeutics for opioids and other substance use disorders in primary care (DIGITS trial)

Abstract:

Background

Substance use disorders (SUDs) result in individual and societal burden. However, most individuals with SUD receive no treatment. Implementing SUD interventions in primary care could address this population's treatment needs. In the USA, reSET® and reSET-O® were the first prescription digital therapeutics (PDTs) for SUDs and opioid use disorder (OUD), respectively. The Digital Treatments for Substance Use Disorder (DIGITS) study tested the effectiveness of practice facilitation and health coaching strategies to support reSET and reSET-O implementation into primary care. A formative evaluation was conducted to monitor implementation, inform adaptations, and learn what promotes PDT sustainment.

Method

The Dynamic Sustainability Framework and the Framework for Reporting Adaptations and Modifications to Evidence-based Implementation Strategies guided the evaluation. Using rapid qualitative methods, we collected and analyzed observational fieldnotes, key informant interviews, and document sources (e.g., meeting minutes) for synthesis and dissemination to clinical partners and the study team via formative reports. We analyzed the reports to generate evaluation results.

Results

Twenty-four primary care clinics participated. Evaluation data included 98 observational fieldnotes, 16 interviews, and 253 document sources. We produced nine formative reports. The study encountered barriers and facilitators in each DSF domain (ecological system, practice setting, and intervention). In the ecological system, the PDT vendor enabled the study, but the COVID-19 pandemic, laws, regulations, and contracting delayed implementation. In the practice setting, staff shortages and low clinic capacity were implementation challenges, while electronic health record capabilities were both barriers and facilitators. At the intervention level, non-routine workflows, clinician burden, and low patient engagement were barriers despite clinicians’ efforts.

Conclusions

Digital therapeutics are promising SUD and OUD treatments, but integration into primary care requires conducive laws and regulations, organizational capacity, and patient and clinician engagement. Formative evaluation identified important lessons for future PDT implementation.

Plain Language Summary

Evaluation of the integration of digital treatments for opioid and other substance use disorders in primary care

Most people with substance use disorders (SUDs) receive no treatment. In the USA, two smartphone app-based digital treatments for SUD and opioid use disorders (OUDs) became available by prescription. The Digital Treatments for Substance Use Disorder (DIGITS) study, a randomized implementation trial, tested how best to integrate these digital treatments for SUD and OUD into primary care. Throughout the study, we conducted a formative evaluation to observe progress, recommend implementation improvements, and understand how digital treatments could be offered to patients after the study ended. For this evaluation, we collected qualitative data through observing study meetings and interviewing primary care leaders, clinicians, and clinical and study staff. The data were regularly summarized and reported to our clinical partners and the study team. We used two implementation frameworks to interpret the data: the Dynamic Sustainability Framework and the Framework for Reporting Adaptations and Modifications to Evidence-based Implementation Strategies. The evaluation revealed factors that helped and hindered implementation. Obstacles from outside the health care system included the COVID-19 pandemic, laws, regulations, and delays in setting up contractual agreements, whereas a partnership with the digital treatment vendor proved helpful. Clinics had difficulties with clinician burden, staff shortages, lack of time and available appointments, and performing new and unfamiliar tasks. The electronic health record system both supported and impended implementation. Last, few patients accepted the offered digital treatment, or used it persistently. Formative evaluation results suggest that digital treatments for SUD and OUD in primary care are promising, however their successful use requires supportive laws and regulations, health system resources, and increased patient and clinician engagement.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

✪ Using FRAME to characterize provider-identified adaptations to a stepped care intervention for adolescents and youth living with HIV in Kenya: a mixed methods approach

Abstract:

Introduction

The Data-informed Stepped Care (DiSC) study is a cluster-randomized trial implemented in 24 HIV care clinics in Kenya, aimed at improving retention in care for adolescents and youth living with HIV (AYLHIV). DiSC is a multi-component intervention that assigns AYLHIV to different intensity (steps) of services according to risk. We used the Framework for Reporting Adaptations and Modifications-Expanded (FRAME) to characterize provider-identified adaptations to the implementation of DiSC to optimize uptake and delivery, and determine the influence on implementation outcomes.

Methods

Between May and December 2022, we conducted continuous quality improvement (CQI) meetings with providers to optimize DiSC implementation at 12 intervention sites. The meetings were guided by plan-do-study-act processes to identify challenges during early phase implementation and propose targeted adaptations. Meetings were audio-recorded and analysed using FRAME to categorize the level, context and content of planned adaptations and determine if adaptations were fidelity consistent. Providers completed surveys to quantify perceptions of DiSC acceptability, appropriateness and feasibility. Mixed effects linear regression models were used to evaluate these implementation outcomes over time.

Results

Providers participated in eight CQI meetings per facility over a 6-month period. A total of 65 adaptations were included in the analysis. The majority focused on optimizing the integration of DiSC within the clinic (83%, n = 54), and consisted of improving documentation, addressing scheduling challenges and improving clinic workflow. Primary reasons for adaptation were to align delivery with AYLHIV needs and preferences and to increase reach among AYLHIV: with reminder calls to AYLHIV, collaborating with schools to ensure AYLHIV attended clinic appointments and addressing transportation challenges. All adaptations to optimize DiSC implementation were fidelity-consistent. Provider perceptions of implementation were consistently high throughout the process, and on average, slightly improved each month for intervention acceptability (β = 0.011, 95% CI: 0.002, 0.020, p = 0.016), appropriateness (β = 0.012, 95% CI: 0.007, 0.027, p<0.001) and feasibility (β = 0.013, 95% CI: 0.004, 0.022, p = 0.005).

Conclusions

Provider-identified adaptations targeted improved integration into routine clinic practices and aimed to reduce barriers to service access unique to AYLHIV. Characterizing types of adaptations and adaptation rationale may enrich our understanding of the implementation context and improve abilities to tailor implementation strategies when scaling to new settings.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

✪ Methodology for adapting a co-created early childhood development intervention and implementation strategies for use by frontline workers in India and Guatemala: a systematic application of the FRAME-IS framework

Authors:

Amruta Bandal, Sara Hernández, Revan Mustafa, Karyn Choy, Namrata Edwards, Magdalena Guarchaj, Marinés Mejía Alvarez, Anushree Sane, Scott Tschida , Chetna Maliye, Ann Miller, Abhishek Raut, Roopa Srinivasan, Morgan Turner, Bradley H. Wagenaar , Ilgi Ertem, Maria del Pilar Grazioso, Subodh S. Gupta, Vibha Krishnamurthy, and Peter Rohloff.

^{University of Washington affiliated authors are displayed in bold.}

✪ Open Access

Published: May 2026

Read the full text in the open access journal Global Health Action

Abstract:

There is little evidence on optimizing the effectiveness and implementation of evidence-based early childhood development (ECD) interventions when task-shifted to frontline workers. In this Methods Forum paper, we describe our adaptation of the International Guide for Monitoring Child Development (GMCD) for task-shifting to frontline workers in Guatemala and India. In 2021–2022, implementers, trainers, frontline workers, caregivers, and international GMCD experts collaborated to adapt the GMCD for a task shifted implementation by frontline workers. We used an eight-step co-creating process: assembling a multidisciplinary team, training on the existing package, working groups to begin modifications, revision of draft modifications, tailoring of visual materials and language, train-the-trainers activities, pilot frontline worker trainings, final review and feedback. Preliminary effectiveness of adaptations was evaluated through narrative notes and group-based qualitative feedback following pilot trainings with 16 frontline workers in India and 6 in Guatemala. Final adaptations included: refining training techniques to match skill levels and learning styles of frontline workers; tailoring all visual materials to local languages and contexts; design of job aids for providing developmental support messages; modification of referral and triage processes for children in need of enhanced support and specialty referral; and creation of post-training support procedures. Feedback from pilot trainings included: (1) group consensus that training improved ECD skills and knowledge across multiple domains; and (2) feedback on ongoing needed adjustments to pacing, use of video-based vs. role-playing materials, and time allocated to small group work. We use the Framework for Reporting Adaptations and Modifications to Evidence-based Implementation Strategies (FRAME-IS) framework to document our adaptations. The co-creating approach we use, as well as systematic documentation of adaptation decisions will be of use to other community-based early childhood interventions and implementation strategies.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

✪ Characterizing provider-led adaptations to mobile phone delivery of the Adolescent Transition Package (ATP) in Kenya using the Framework for Reporting Adaptations and Modifications to Evidence-based Implementation Strategies (FRAME-IS): a mixed methods approach

Abstract:

Background

The COVID-19 pandemic resulted in disruptions to routine HIV services for youth living with HIV (YLH), provoking rapid adaptation to mitigate interruptions in care. The Adolescent Transition to Adult Care for HIV-infected adolescents (ATTACH) study (NCT03574129) was a hybrid I cluster randomized trial testing the effectiveness of a healthcare worker (HCW)-delivered disclosure and transition intervention — the Adolescent Transition Package (ATP). During the pandemic, HCWs leveraged phone delivery of the ATP and were supported to make adaptations. We characterized real-time, provider-driven adaptations made to support phone delivery of the ATP.

Methods

We conducted continuous quality improvement (CQI) meetings with HCWs involved in phone delivery of the ATP at 10 intervention sites. CQI meetings used plan-do-study-act (PDSA) cycles and were audio-recorded. Adaptations were coded by two-independent coders using the Framework for Reporting Adaptations and Modifications to Evidence-based Implementation Strategies (FRAME-IS). Adaptation testing outcomes (adopt, retest, or abandon) and provider experience implementing the adaptations were also recorded. We summarized adaptation characteristics, provider experience, and outcomes.

Results

We identified 72 adaptations, 32 were unique. Overall, adaptations included modification to context (53%, n = 38), content (49%, n = 35), and evaluation processes (13%, n = 9). Context adaptations primarily featured changes to personnel, format, and setting, while content and evaluation adaptations were frequently achieved by simple additions, repetition, and tailoring/refining of the phone delivery strategy. Nine adaptations involved abandoning, then returning to phone delivery. HCWs sought to increase reach, improve fidelity, and intervention fit within their context. Most adaptations (96%, n = 69) were perceived to increase the feasibility of phone delivery when compared to before the changes were introduced, and HCWs felt 83% (n = 60) of adaptations made phone delivery easier. Most adaptations were either incorporated into routine workflows (47%) or tested again (47%).

Conclusion

Adaptation of phone delivery was a feasible and effective way of addressing challenges with continuity of care for YLH during the COVID-19 pandemic. Adaptations were primarily context adaptions. While FRAME-IS was apt for characterizing adaptations, more use cases are needed to explore the range of its utility.

Trial registration

Trial registered on ClinicalTrial.gov as NCT03574129.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

Authors:

✪ Open Access

Read the full text in the open access journal Digital Health

Abstract:

Background

Objective

Methods

Results

Conclusion

Authors:

✪ Open Access

Read the full text in the open access journal BMJ Open

Abstract:

Objectives

Design

Settings

Participants

Primary and secondary outcome measures

Results

Conclusions

Authors:

✪ Open Access

Read the full text in the open access journal JMIR Research Protocols

Abstract:

Background

Objective

Methods

Results

Conclusions

International Registered Report Identifier (IRRID)

Authors:

✪ Open Access

Read the full text open access in the journal Health Research Policy and Systems

Abstract:

Background

Methods

Results

Conclusions

Authors:

✪ Open Access

Read the full text in the open access journal Implementation Research and Practice

Abstract:

Background

Method

Results

Conclusions

Plain Language Summary

Evaluation of the integration of digital treatments for opioid and other substance use disorders in primary care

Authors:

✪ Open Access

Read the full text in the open access journal Journal of the International AIDS Society

Abstract:

Introduction

Methods

Results

Conclusions

Authors:

✪ Open Access

Read the full text in the open access journal Global Health Action

Abstract:

Authors:

✪ Open Access

Read the full text in the open access journal Implementation Science Communication

Abstract:

Background

Methods

Results

Conclusion

Trial registration