Skip to content

✪ Development of a method for Making Optimal Decisions for Intervention Flexibility during Implementation (MODIFI): a modified Delphi study

Authors:

Stephanie K. Brewer, Catherine M. Corbin, Ana A. Baumann, Shannon Wiltsey Stirman, Janine M. Jones, Michael D. Pullmann, Aaron R. Lyon & MODIFI Expert Panel

University of Washington affiliated authors are displayed in bold.

✪ Open Access

Published: June 2024

Read the full text in the open access journal Implementation Science Communications

Abstract:

Background

Intervention adaptation is often necessary to improve the fit between evidence-based practices/programs and implementation contexts. Existing frameworks describe intervention adaptation processes but do not provide detailed steps for prospectively designing adaptations, are designed for researchers, and require substantial time and resources to complete. A pragmatic approach to guide implementers through developing and assessing adaptations in local contexts is needed. The goal of this project was to develop Making Optimal Decisions for Intervention Flexibility during Implementation (MODIFI), a method for intervention adaptation that leverages human centered design methods and is tailored to the needs of intervention implementers working in applied settings with limited time and resources.

Method

MODIFI was iteratively developed via a mixed-methods modified Delphi process. Feedback was collected from 43 implementation research and practice experts. Two rounds of data collection gathered quantitative ratings of acceptability and inclusion (Round 1) and feasibility (Round 2), as well as qualitative feedback regarding MODIFI revisions analyzed using conventional content analysis.

Results

In Round 1, most participants rated all proposed components as essential but identified important avenues for revision which were incorporated into MODIFI prior to Round 2. Round 2 emphasized feasibility, where ratings were generally high and fewer substantive revisions were recommended. Round 2 changes largely surrounded operationalization of terms/processes and sequencing of content. Results include a detailed presentation of the final version of the three-step MODIFI method (Step 1: Learn about the users, local context, and intervention; Step 2: Adapt the intervention; Step 3: Evaluate the adaptation) along with a case example of its application.

Discussion

MODIFI is a pragmatic method that was developed to extend the contributions of other research-based adaptation theories, models, and frameworks while integrating methods that are tailored to the needs of intervention implementers. Guiding teams to tailor evidence-based interventions to their local context may extend for whom, where, and under what conditions an intervention can be effective.

Contributions to the literature
  • Adaptation of evidence-based practices, programs, and treatments is often needed to ensure successful implementation. However, existing theories/models/frameworks do not outline concrete adaptation steps that are feasible for real-world settings.
  • We created a resource-efficient method (MODIFI) for developing and assessing intervention adaptations in local settings. This method was iteratively developed through a mixed-methods modified Delphi process, which resulted in consensus among experts in implementation science and practice.
  • MODIFI leverages human centered design techniques and is tailored to the needs of intervention implementers in applied settings, filling a crucial gap in the research to practice pipeline.

**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**

✪ Methodology for adapting a co-created early childhood development intervention and implementation strategies for use by frontline workers in India and Guatemala: a systematic application of the FRAME-IS framework

Authors:

Amruta Bandal, Sara Hernández, Revan Mustafa, Karyn Choy, Namrata Edwards, Magdalena Guarchaj, Marinés Mejía Alvarez, Anushree Sane, Scott Tschida , Chetna Maliye, Ann Miller, Abhishek Raut, Roopa Srinivasan, Morgan Turner, Bradley H. Wagenaar , Ilgi Ertem, Maria del Pilar Grazioso, Subodh S. Gupta, Vibha Krishnamurthy, and Peter Rohloff.

University of Washington affiliated authors are displayed in bold.

✪ Open Access

Published: May 2026

Read the full text in the open access journal Global Health Action

Abstract:

There is little evidence on optimizing the effectiveness and implementation of evidence-based early childhood development (ECD) interventions when task-shifted to frontline workers. In this Methods Forum paper, we describe our adaptation of the International Guide for Monitoring Child Development (GMCD) for task-shifting to frontline workers in Guatemala and India. In 2021–2022, implementers, trainers, frontline workers, caregivers, and international GMCD experts collaborated to adapt the GMCD for a task shifted implementation by frontline workers. We used an eight-step co-creating process: assembling a multidisciplinary team, training on the existing package, working groups to begin modifications, revision of draft modifications, tailoring of visual materials and language, train-the-trainers activities, pilot frontline worker trainings, final review and feedback. Preliminary effectiveness of adaptations was evaluated through narrative notes and group-based qualitative feedback following pilot trainings with 16 frontline workers in India and 6 in Guatemala. Final adaptations included: refining training techniques to match skill levels and learning styles of frontline workers; tailoring all visual materials to local languages and contexts; design of job aids for providing developmental support messages; modification of referral and triage processes for children in need of enhanced support and specialty referral; and creation of post-training support procedures. Feedback from pilot trainings included: (1) group consensus that training improved ECD skills and knowledge across multiple domains; and (2) feedback on ongoing needed adjustments to pacing, use of video-based vs. role-playing materials, and time allocated to small group work. We use the Framework for Reporting Adaptations and Modifications to Evidence-based Implementation Strategies (FRAME-IS) framework to document our adaptations. The co-creating approach we use, as well as systematic documentation of adaptation decisions will be of use to other community-based early childhood interventions and implementation strategies.

**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**

✪ Centering School Leaders’ Expertise: Usability Evaluation of a Leadership-Focused Implementation Strategy to Support Tier 1 Programs in Schools

Authors:

Vaughan K. Collins, Catherine M. Corbin, Jill J. Locke, Clayton R. Cook, Mark G. Ehrhart, Kurt D. Hatch & Aaron R. Lyon

University of Washington affiliated authors are displayed in bold.

✪ Open Access

Published: March 2024

Read the full text open access in the journal School Mental Health

Abstract:

Although there are a growing number of implementation strategies to increase the adoption, fidelity, and sustainment of evidence-based practices (EBP), they are often complex and bulky, which can interfere with their widespread application. To address these challenges, the Helping Educational Leaders Mobilize Evidence (HELM) strategy was created as an adaptation of the evidence-based Leadership and Organizational Change for Implementation (LOCI) implementation strategy to enhance elementary school principals’ use of strategic implementation leadership to support the adoption and delivery of Tier 1 (i.e., universal social, emotional, and behavioral) EBP. In service of its iterative development, a human-centered design methodology was employed to increase the successful uptake and use of HELM. The Cognitive Walkthrough for Implementation Strategies (CWIS), a novel mixed-methods approach to evaluate implementation strategy usability, was applied to identify and test HELM strategy tasks of critical importance. A sample of 15 elementary school principals participated in group cognitive walkthrough testing as either school principal recipients or HELM coaches. Both user types rated the strategy as acceptable (principal M = 77.8, SD = 15.5; coach M = 87.5, SD = 7.9). Five usability issues were identified using a highly structured common usability issue framework and provided direction for the generation of redesign solutions to be incorporated in a subsequent version of the HELM strategy. The evaluation of the strategy’s usability improved its alignment with user needs, expectations, and contextual constraints, rendering a more usable strategy and broadly applicable information surrounding the development of psychosocial implementation strategies in real-world settings.

**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**

✪ Adaptation of a Tailored Lung Cancer Screening Decision Aid for People With HIV

Authors:

Meagan C. Brown, Madison Snidarich, Jehan Z. Budak, Nicholas Murphy, Nicholas Giustini, Perrin E. Romine, Bryan J. Weiner, Tanner Caverly, Kristina Crothers, and Matthew Triplette.

University of Washington affiliated authors are displayed in bold.

✪ Open Access

Published: February 2024

Read the full text in the open access journal CHEST Pulmonary

Abstract:

Background

People with HIV are both at elevated risk of lung cancer and at high risk of multimorbidity, which makes shared decision-making (SDM) for lung cancer screening (LCS) in people with HIV complex. Currently no known tools have been adapted for SDM in people with HIV.

Research Question

Can an SDM decision aid be adapted to include HIV-specific measures with input from both people with HIV and their providers?

Study Design and Methods

This study used qualitative methods including focus groups of people with HIV and interviews with HIV care providers to adapt and iterate an SDM tool for people with HIV. Eligible participants were those with HIV enrolled in an HIV primary care clinic who met age and smoking eligibility criteria for LCS and HIV care providers at the clinic. Both the focus groups and interviews included semistructured discussions of SDM and decision aid elements for people with HIV. We used a framework-guided thematic analysis, mapping themes onto the Health Equity Implementation framework.

Results

Forty-three people with HIV participated in eight focus groups; 10 providers were interviewed. Key themes from patients included broad interest in adapting LCS SDM specifically for people with HIV, a preference for clear LCS recommendations, and the need for positive framing emphasizing survival. Providers were enthusiastic about personalized LCS risk assessments and point-of-care tools. Both patients and providers gave mixed views on the usefulness of HIV-specific risk measures in patient-facing tools. Themes were used to adapt a personalized and flexible SDM tool for LCS in people with HIV.

Interpretation

People with HIV and providers were enthusiastic about specific tools for SDM that are personalized and tailored for people with HIV, that make recommendations, and that inform LCS decision-making. Divergent views on presenting patient-facing quantitative risk assessments suggests that these elements could be optional but available for review. This tool may have usefulness in complex decision-making for LCS in this population and currently is being evaluated in a pilot prospective trial.

**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**

✪ Acceptability and Utility of a Digital Group Intervention to Prevent Perinatal Depression in Youths via Interactive Maternal Group for Information and Emotional Support (IMAGINE): Pilot Cohort Study

Authors:

Keshet Ronen, Anupa Gewali, Kristin Dachelet, Erica White, Marimirca Jean-Baptiste, Yolanda N Evans, Jennifer A Unger, S. Darius Tandon, Amritha Bhat

University of Washington affiliated authors are displayed in bold.

✪ Open Access

Published: February 2024

Read the full text in the open access journal JMIR Formative Research

Abstract:

Background

Perinatal depression (depression during pregnancy or the first year postpartum) affects 10%-25% of perinatal individuals, with a higher risk among youths aged <25 years. The Mothers and Babies Course (MB) is an evidence-based intervention for the prevention of perinatal depression, grounded in cognitive behavioral therapy, attachment theory, and psychoeducation.

Objective

We developed a digital adaptation of MB (Interactive Maternal Group for Information and Emotional Support [IMAGINE]) and evaluated it in a pre-post mixed methods pilot among young perinatal people in the United States

Methods

IMAGINE was a structured digital group of up to 7 participants, with scheduled MB content and open discussion for 12 weeks, facilitated by a social worker. Scheduled content included asynchronous SMS text messages, graphics, prerecorded videos, mood polls, and optional weekly synchronous video calls. Eligible participants were pregnant or ≤80 days postpartum, aged 16 to 24 years, had access to a smartphone, spoke English, and had a Patient Health Questionnaire score <10. Participants were recruited throughout the United States from August 2020 to January 2021 through paid social media ads, in-person outreach at clinics, and respondent-driven sampling. Participants completed quantitative questionnaires at enrollment and 3 months, and qualitative interviews at 3 months. We determined uptake, acceptability (by Acceptability of Intervention Measure score), and utility (by use of cognitive behavioral therapy skills). We compared depression symptoms (by Patient Health Questionnaire score), social support (by abbreviated Social Support Behavior score), and perceived stress (by Perceived Stress Score) between enrollment and follow-up by paired 2-tailed t test.

Results

Among 68 individuals who contacted this study, 22 were screened, 13 were eligible, and 10 enrolled, for an uptake of 76.9%. Furthermore, 4 (40%) participants were pregnant at enrollment. Participants had a median age of 17.9 (IQR 17.4-21.7) years, 6 (67%) identified as Black, 5 (56%) Latinx, and 6 (67%) using Medicaid health insurance. Further, 9 (90%) participants completed follow-up. Among these, the mean acceptability score was 4.3 out of 5 (SD 0.6) and all participants said they would recommend IMAGINE to a friend. Participants reported using a median of 7 of 11 skills (IQR 5-7 skills) at least half the days. We found no significant changes in depression symptoms, perceived stress, or social support. Qualitatively, participants reported one-to-one support from the facilitator, connection with other parents, and regular mood reflection were especially helpful aspects of the intervention. Additionally, participants reported that the intervention normalized their mental health challenges, improved their ability to manage their mood, and increased their openness to mental health care.

Conclusions

This pilot study provides promising evidence of the acceptability and utility of IMAGINE among perinatal youths. Our study’s small sample size did not detect changes in clinical outcomes; our findings suggest IMAGINE warrants larger-scale evaluation.

**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**

✪ A case study of pragmatic measure development of the Treatment Integrity for Elementary Settings self-report measure for teachers

Authors:

Emma R Dear, Bryce D McLeod, Nicole M Peterson, Kevin S Sutherland, Michael D Broda, Alex R Dopp & Aaron R Lyon

University of Washington affiliated authors are displayed in bold.

✪ Open Access

Published: January 2024

Read the full text in the open access journal Implementation Research and Practice

Abstract:

Introduction

Due to usability, feasibility, and acceptability concerns, observational treatment fidelity measures are often challenging to deploy in schools. Teacher self-report fidelity measures with specific design features might address some of these barriers. This case study outlines a community-engaged, iterative process to adapt the observational Treatment Integrity for Elementary Settings (TIES-O) to a teacher self-report version designed to assess the use of practices to support children's social-emotional competencies in elementary classrooms.

Method

Cognitive walkthrough interviews were conducted with teachers to improve the usability of the teacher self-report measure, called the Treatment Integrity for Elementary Schools–Teacher Report (TIES-T). Qualitative content analysis was used to extract themes from the interviews and inform changes to the measure.

Results

Increasing clarity and interactive elements in the measure training were the dominant themes, but suggestions for the measure format and jargon were also suggested.

Conclusion

The suggested changes resulted in a brief measure, training, and feedback system designed to support the teacher's use of practices to support children's social-emotional competencies in elementary classrooms. Future research with the TIES-T will examine the score reliability and validity of the measure.

**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**

✪ Implementation measurement in global mental health: Results from a modified Delphi panel and investigator survey

Authors:

Christopher G. Kemp, Kristen Danforth, Luke Aldridge, Laura K. Murray, and Emily E. Haroz

University of Washington affiliated authors are displayed in bold.

✪ Open Access

Published: October 2023

Read the full text open access in the journal Cambridge Prisms: Global Mental Health

Abstract:

Limited guidance exists to support investigators in the choice, adaptation, validation and use of implementation measures for global mental health implementation research. Our objectives were to develop consensus on best practices for implementation measurement and identify strengths and opportunities in current practice. We convened seven expert panelists. Participants rated approaches to measure adaptation and validation according to appropriateness and feasibility. Follow-up interviews were conducted and a group discussion was held. We then surveyed investigators who have used quantitative implementation measures in global mental health implementation research. Participants described their use of implementation measures, including approaches to adaptation and validation, alongside challenges and opportunities. Panelists agreed that investigators could rely on evidence of a measure’s validity, reliability and dimensionality from similar contexts. Panelists did not reach consensus on whether to establish the pragmatic qualities of measures in novel settings. Survey respondents (n = 28) most commonly reported using the Consolidated Framework for Implementation Research Inner Setting Measures (n = 9) and the Program Assessment Sustainability Tool (n = 5). All reported adapting measures to their settings; only two reported validating their measures. These results will support guidance for implementation measurement in support of mental health services in diverse global settings.

**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**

✪ Enhancing the Introduction and Scale Up of Self-Administered Injectable Contraception (DMPA-SC) in Health Systems (the EASIER Project): Protocol for Embedded Implementation Research

Authors:

Colin Baynes, Petrus S Steyn, Kenneth Sherr, and James Kiarie

University of Washington affiliated authors are displayed in bold.

✪ Open Access

Published:August 2023

Read the full text in the open access journal JMIR Research Protocols

Abstract:

Background

The introduction of self-administered injectable contraception presents an opportunity to address the unmet need for family planning. As ministries of health scale up self-administered injectable contraception, there is a scarcity of knowledge on the implementation practices and contextual conditions that help and hinder these efforts. The World Health Organization has launched the “enhancing self-administered family planning through embedded research project” (EASIER) to address this challenge.

Objective

EASIER’s objectives are to: (1) assess the coverage of self-injectable contraception, and the readiness of health systems to integrate it into the contraceptive method mix; (2) document strategies used to introduce and scale up self-injectable contraception and understand practices that have led to success and challenges; (3) identify the contextual factors that affect the adoption and implementation of self-injectable contraception throughout health systems; (4) understand whether implementation addresses users’ preferences and needs; (5) strengthen collaboration between decision makers, researchers, and implementers; support and build capacity to use evidence.

Methods

EASIER developed a global protocol that implementation research (IR) teams in Burkina Faso, Ghana, and Kenya adapted into country-level embedded IR projects. In all countries (1) at the national level, IR teams evaluate the policy environment for scaling up by conducting a desk review and in-depth interviews; (2) at the local level, IR teams implement quantitative questionnaires on structural and organizational readiness to integrate self-injection into the method mix; (3) in “case study” localities, IR teams conduct in-depth interviews and focus group discussions with implementers, method users, and community members; and (4) IR teams use participatory action research to elicit stakeholder participation and translate findings into programmatic decisions.

Results

EASIER has been launched in all 3 countries. Preliminary findings are available from Burkina Faso and Kenya. In Burkina Faso, IR teams identified the need to strengthen health worker training approaches to ensure that family planning providers at primary health care facilities are adequately oriented to depo-medroxyprogesterone acetate subcutaneous (DMPA-SC) and self-injection and capacitated to initiate women to the method. In addition, they report the need for service communication strategies that reach potential users of the method with knowledge about self-injection and how to initiate the practice. In Kenya, the findings illuminate the need for practice guidelines that county health teams can use to coordinate the rollout of self-administered DMPA-SC. In addition, Kenya’s findings underscore the importance of addressing logistical bottlenecks to help avoid stock-outs.

Conclusions

EASIER presents a strategy to embed IR in contraceptive method introduction and scale-up, address local knowledge needs, devise ways to maximize the impact of new technologies in health systems, and build capacity for using evidence in programmatic decisions. Adaptation and implementation of country-level IR studies will advance the use of IR to strengthen family planning programs.

**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**

✪ Characterizing provider-led adaptations to mobile phone delivery of the Adolescent Transition Package (ATP) in Kenya using the Framework for Reporting Adaptations and Modifications to Evidence-based Implementation Strategies (FRAME-IS): a mixed methods approach

Authors:

Dorothy Imbuka Mangale, Alvin Onyango, Cyrus Mugo, Caren Mburu, Nok Chhun, Dalton Wamalwa, Irene Njuguna, Arianna Rubin Means, Grace John-Stewart, Bryan J. Weiner & Kristin Beima-Sofie

University of Washington affiliated authors are displayed in bold.

✪ Open Access

Published: August 2023

Read the full text in the open access journal Implementation Science Communication

Abstract:

Background

The COVID-19 pandemic resulted in disruptions to routine HIV services for youth living with HIV (YLH), provoking rapid adaptation to mitigate interruptions in care. The Adolescent Transition to Adult Care for HIV-infected adolescents (ATTACH) study (NCT03574129) was a hybrid I cluster randomized trial testing the effectiveness of a healthcare worker (HCW)-delivered disclosure and transition intervention — the Adolescent Transition Package (ATP). During the pandemic, HCWs leveraged phone delivery of the ATP and were supported to make adaptations. We characterized real-time, provider-driven adaptations made to support phone delivery of the ATP.

Methods

We conducted continuous quality improvement (CQI) meetings with HCWs involved in phone delivery of the ATP at 10 intervention sites. CQI meetings used plan-do-study-act (PDSA) cycles and were audio-recorded. Adaptations were coded by two-independent coders using the Framework for Reporting Adaptations and Modifications to Evidence-based Implementation Strategies (FRAME-IS). Adaptation testing outcomes (adopt, retest, or abandon) and provider experience implementing the adaptations were also recorded. We summarized adaptation characteristics, provider experience, and outcomes.

Results

We identified 72 adaptations, 32 were unique. Overall, adaptations included modification to context (53%, n = 38), content (49%, n = 35), and evaluation processes (13%, n = 9). Context adaptations primarily featured changes to personnel, format, and setting, while content and evaluation adaptations were frequently achieved by simple additions, repetition, and tailoring/refining of the phone delivery strategy. Nine adaptations involved abandoning, then returning to phone delivery. HCWs sought to increase reach, improve fidelity, and intervention fit within their context. Most adaptations (96%, n = 69) were perceived to increase the feasibility of phone delivery when compared to before the changes were introduced, and HCWs felt 83% (n = 60) of adaptations made phone delivery easier. Most adaptations were either incorporated into routine workflows (47%) or tested again (47%).

Conclusion

Adaptation of phone delivery was a feasible and effective way of addressing challenges with continuity of care for YLH during the COVID-19 pandemic. Adaptations were primarily context adaptions. While FRAME-IS was apt for characterizing adaptations, more use cases are needed to explore the range of its utility.

Trial registration

Trial registered on ClinicalTrial.gov as NCT03574129.

**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**