Skip to content

Integrating Implementation Outcomes into Effectiveness Studies: A Practical Guide for Clinical Interventionists

Authors:

Danielle M. Wesolowicz, William C. Becker, Margarita Alegria , Lia Chin-Purcell, Joseph E. Glass, Hannah K. Knudsen, Patience M. Dow, Karen H. Seal & Emma B. McGinty

University of Washington affiliated authors are displayed in bold.

Subscription Required for Full Text Access

Published: April 2025

Read the full text in the subscription access journal Journal of General Internal Medicine

✪ Lessons learnt from the 2021 Pacific Northwest heat dome: a qualitative study of western Washington’s healthcare community response

Authors:

Matias Korfmacher, Cat Hartwell, Kelly Hill, Nathaniel Matthews-Trigg, Jeremy Hess, Amruta Nori-Sarma, & Gregory Wellenius

University of Washington affiliated authors are displayed in bold.

✪ Open Access

Published: April 2025

Read the full text in the open access journal BMJ Open

Abstract:

Objective

The 2021 Pacific Northwest heat dome was Washington state’s deadliest recorded weather event and presented unprecedented response challenges to the state’s health sector. Understanding the impacts of this extreme heat event (EHE) on the sector as well as the barriers to and facilitators of implementing effective heat response is critical to preparing for future events, which are happening more frequently in the region due to climate change.

Design

Guided by an implementation science framework, we convened listening sessions and focus groups of the health sector in western Washington to reflect on regional response efforts.

Setting

Health sector organisations in 15 counties in western Washington State, USA: Clallam, Grays Harbor, Island, Jefferson, King, Kitsap, Lewis, Mason, Pacific, Pierce, San Juan, Skagit, Snohomish, Thurston and Whatcom.

Participants

A convenience sample of 109 listening group participants was recruited through the professional networks of the Northwest Healthcare Response Network, a regional healthcare coalition. 27 of the health sector professionals were recruited using purposive sampling to participate in seven focus groups organised by organisation type.

Results

The co-presence of the COVID-19 pandemic, limited staff capacity, resource acquisition challenges and inadequate regional collaboration emerged as key barriers, while advanced planning, indoor cooling capabilities, adapting strategies to local needs, robust internal relationships and strong external partnerships were reported to facilitate effective response. Establishing centralised coordination ahead of heat events, making improvements to the cooling capabilities of the built environment, developing plans and policies for EHEs that have co-benefits for other events, adopting evidence-informed response strategies, institutionalising the knowledge and relationships developed through prior events and improving evaluative processes (such as developing real-time monitoring capacity) will enable more effective response to future EHEs.

Conclusions

Western Washington’s health sector implemented EHE response activities that enabled essential service continuity, despite limited resources, unfamiliarity with EHEs and other systemic challenges. The recency of the heat dome presents an opportunity to incorporate lessons learnt into practice, policies, plans and built environment; these are necessary improvements ahead of future large-scale events the region may experience in the coming decades.

**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**

✪ Practical considerations for engaging staff in resource-constrained healthcare settings in implementation research: A qualitative focus group and consensus building study

Authors:

Kelly A. Aschbrenner, Callie Walsh-Bailey, Meagan C. Brown, Tanveer Khan, Travis P. Baggett, Salene M.W. Jones, Douglas E. Levy, Lydia E. Pace, and Jonathan P. Winickoff

University of Washington affiliated authors are displayed in bold.

✪ Open Access

Published: March 2025

Read the full text open access in the Journal of Clinical and Translational Science

Abstract:

Background

The primary purpose of this study was to assess perceived burdens and benefits of participating in implementation research among staff employed in resource-constrained healthcare settings. Another objective was to use findings to generate considerations for engaging staff in research across different phases of implementation research.

Methods

This qualitative focus group and consensus building study involved researchers affiliated with the National Cancer Institute Implementation Science Centers in Cancer Control program and nine Community Health Centers (CHCs) in Massachusetts. Six focus groups (n = 3 with CHC staff; n = 3 with researchers) assessed barriers and facilitators to staff participation in implementation research. During consensus discussions, we used findings to develop considerations for engaging staff as participants and partners throughout phases of implementation research.

Results

Sixteen researchers and 14 staff participated in separate focus groups; nine researchers and seven staff participated in separate consensus discussions. Themes emerged across participant groups in three domains: (1) influences on research participation; (2) research burdens and benefits; and (3) ways to facilitate staff participation in research. Practical considerations included: (a) aligning research with organizational and staff values and priorities; (b) applying user-centered design to research methods; (c) building organizational and individual research capacity; and (d) offering equitable incentives for staff participation.

Conclusions

Engaging staff as participants and partners across different phases of implementation research requires knowledge about what contributes to research burden and benefits and addressing context-specific burdens and benefits.

**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**

✪ Development of a method for qualitative data integration to advance implementation science within research consortia

Authors:

Lisa DiMartino, Allison J. Carroll, Jennifer L. Ridgeway, Anna Revette, Joan M. Griffin, Bryan J. Weiner, Sandra A. Mitchell, Wynne E. Norton, Christine Cronin, Andrea L. Cheville, Ann Marie Flores, Justin D. Smith on behalf of the IMPACT Consortium

University of Washington affiliated authors are displayed in bold.

✪ Open Access

Published: February 2025

Read the full text in the open access journal Implementation Science Communications

Abstract:

Background

Methods of integrating qualitative data across diverse studies and within multi-site research consortia are less developed than those for integrating quantitative data. The development ofsuchmethods is essential to support the data exchange needed for cross-study qualitative inquiry and given the increasing emphasis on data sharing and open science. We describe methods for qualitative data integration within the National Cancer Institute’s Improving the Management of symPtoms During And following Cancer Treatment (IMPACT) Consortium funded by the Cancer MoonshotSM. Data collection and analysis were guided by the Consolidated Framework for Implementation Research (CFIR). Our case study highlights potential solutions for unique challenges faced when integrating qualitative data across multiple settings in a research consortium.

Methods

The IMPACT consortium is comprised of three research centers (RCs) each conducting pragmatic trials examining the effectiveness of routine symptom management on patient-centered outcomes. After reaching consensus on use of CFIR as the common implementation determinant framework, RCs developed a semi-structured interview guide and tailored it to features of their healthcare setting and symptom management interventions. RCs conducted interviews/focus groups with healthcare system partners to examine contextual factors impacting implementation. RCs exchanged 1–2 transcripts (n = 5 total) for purposes of pilot testing the methodology.

Results

Given the heterogeneity of study settings and contexts, it was challenging to simultaneously assign codes at both domain and construct levels and the process was resource intensive. Recommendations include employing a common framework for data collection and analyses from the outset, coding at domain level first and then incorporating construct codes, and centralizing processes via a coordinating center (or similar entity) and combining coded transcripts using qualitative software. We also generated an iteratively refined codebook that employed the CFIR schema and incorporated CFIR 2.0 to provide detailed guidance for coders conducting cross-study qualitative inquiry.

Conclusions

Limited guidance exists on how to support qualitative data integration, data exchange, and sharing across multiple studies. This paper describes a systematic method for employing an implementation determinant framework-guided approach to foster data integration. This methodology can be adopted by other research consortia to support qualitative data integration, cross-site qualitative inquiry, and generate improved understanding of evidence-based intervention implementation.

Contributions to the literature
  • Few standard practices exist for integrating qualitative data across different studies to conduct cross-study qualitative inquiry. Developing data integration procedures is critical to generating insights about practice improvement across settings and achieving goals of open science.
  • This report describes a method for integrating qualitative data using the Consolidated Framework for Implementation Research illustrated within the context of NCI’s Improving the Management of symPtoms During And following Cancer Treatment (IMPACT) Consortium.
  • The method describedcan be applied within research consortia and across multi-site studies to support qualitative data integration, thereby producing valuable insights to improve implementation of evidence-based interventions.

**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**

✪ Implementation Outcomes From a Pilot Study of Mindful Awareness in Body-Oriented Therapy (MABT) as a Chronic Pain Treatment Modality in an Integrative Health Clinic

Authors:

Kathryn A. Hansen, Erin Blakeney, and Cynthia J. Price

University of Washington affiliated authors are displayed in bold.

✪ Open Access

Published: February 2025

Read the full text in the open access journal Integrative Medicine & Health

Abstract:

Background

As regulation of opioid prescribing evolves, primary care and pain clinics are shifting to provide non-pharmacological and interdisciplinary chronic pain care. An under-utilized but growing area of health care for chronic pain is complementary and integrative health (CIH). However, there is limited availability of CIH approaches within the health care system. Mindful Awareness in Body-Oriented Therapy (MABT) is an evidence-based mind-body therapy, with a manualized protocol, that focuses on developing interoceptive sensibility for improved self-awareness and nervous system regulation. Prior MABT research shows MABT improves self-report and physiological indicators of interoception as well as mental and physical symptoms of distress.

Methods

This pilot single-group study used a hybrid implementation-effectiveness design and mixed methods to study implementation strategies and outcomes for bringing MABT into an integrative chronic pain clinic. Administrative data, staff surveys, and focus groups were used to understand the implementation process and outcomes (see Additional files 2, 3, and 4). Descriptive statistics were used to analyze survey and administrative data. A content analysis approach was used to analyze qualitative data from focus groups.

Results

7 staff surveys were administered over the 24-month study period and showed high acceptability and appropriateness that increased over time. Adoption, feasibility, and sustainability were also high. Clinicians made 70 referrals to MABT, 56 patients scheduled a session, 41 patients completed at least one session, and 71% of these completed the protocol. Focus groups identified MABT as a therapy that filled a gap in services, particularly for patients with a lack of body awareness and high emotion dysregulation.

Conclusion

Implementation of MABT was highly successful in an integrative health clinic focused on chronic pain treatment.

ClinicalTrials.gov registration

NCT05289024 Registered March 11, 2022

**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**

Barriers and facilitators to implementing enteral resuscitation for major burn injuries: Reflections from Nepalese care providers

Authors:

Raslina Shrestha, Kajal Mehta, Aldina Mesic, Dinasha Dahanayake, Manish Yadav, Shankar Rai, Kiran Nakarmi, Pariwesh Bista, Tam Pham, and Barclay T. Stewart

University of Washington affiliated authors are displayed in bold.

Subscription Required for Full Text Access

Published: February 2025

Read the full text in the subscription access journal Burns

✪ A Team Coaching Intervention for Dementia Care: Acceptability and Feasibility Study in Care Communities

Authors:

Jessica Welsch, Boeun Kim, Lorna Prophater, Basia Belza, & Sam Fazio

University of Washington affiliated authors are displayed in bold.

✪ Open Access

Published: December 2024

Read the full text open access in the Journal of the American Medical Directors Association

Abstract:

Objectives

In 2018, the Alzheimer's Association published the Dementia Care Practice Recommendations (DCPR) to provide guidance on quality care for professionals. This study aimed to examine the feasibility and acceptability of a consultative coaching program to increase adoption of the DCPR.

Design

A single-group pre-post design was employed. A 6-month team coaching intervention was implemented to make organization-wide and measurable changes in the areas of (1) policies and procedures, (2) education and training, or (3) care practices to increase person-centered care practices.

Setting and Participants

A total of 36 care communities in Washington and Montana states were enrolled in this program.

Methods

Focus groups were conducted with participants to explore the acceptability and feasibility of the program. A 35-item employees satisfaction survey was administered both before and after the program.

Results

The findings offer preliminary evidence that implementing a coaching program is desired by care community staff and has a positive impact on staff job satisfaction and the use of person-centered care. The coaching model was received by staff as a supportive and affirming program.

Conclusions and Implications

The team coaching intervention, designed to improve person-centered care and employee satisfaction, is acceptable. However, its efficacy needs to be tested with a larger sample using a randomized controlled design.

**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**

Barriers and facilitators to implementing enteral resuscitation for major burn injuries: Reflections from Nepalese care providers

Authors:

Raslina Shrestha, Kajal Mehta, Aldina Mesic, Dinasha Dahanayake, Manish Yadav, Shankar Rai, Kiran Nakarmi, Pariwesh Bista, Tam Pham, & Barclay T Stewart

University of Washington affiliated authors are displayed in bold.

Subscription Required for Full Text Access

Published: October 2024

Read the full text in the subscription access journal Burns

✪ A research agenda to advance the study of implementation mechanisms

Authors:

Cara C. Lewis, Hannah E. Frank, Gracelyn Cruden, Bo Kim, Aubyn C. Stahmer, Aaron R. Lyon, Bianca Albers, Gregory A. Aarons, Rinad S. Beidas, Brian S. Mittman, Bryan J. Weiner, Nate J. Williams, Byron J. Powell & MNoE Group

University of Washington affiliated authors are displayed in bold.

✪ Open Access

Published: September 2024

Read the full text in the open access journal Implementation Science Communications

Abstract:

Background

Implementation science scholars have made significant progress identifying factors that enable or obstruct the implementation of evidence-based interventions, and testing strategies that may modify those factors. However, little research sheds light on how or why strategies work, in what contexts, and for whom. Studying implementation mechanisms—the processes responsible for change—is crucial for advancing the field of implementation science and enhancing its value in facilitating equitable policy and practice change. The Agency for Healthcare Research and Quality funded a conference series to achieve two aims: (1) develop a research agenda on implementation mechanisms, and (2) actively disseminate the research agenda to research, policy, and practice audiences. This article presents the resulting research agenda, including priorities and actions to encourage its execution.

Method

Building on prior concept mapping work, in a semi-structured, 3-day, in-person working meeting, 23 US-based researchers used a modified nominal group process to generate priorities and actions for addressing challenges to studying implementation mechanisms. During each of the three 120-min sessions, small groups responded to the prompt: “What actions need to be taken to move this research forward?” The groups brainstormed actions, which were then shared with the full group and discussed with the support of facilitators trained in structured group processes. Facilitators grouped critical and novel ideas into themes. Attendees voted on six themes they prioritized to discuss in a fourth, 120-min session, during which small groups operationalized prioritized actions. Subsequently, all ideas were collated, combined, and revised for clarity by a subset of the authorship team.

Results

From this multistep process, 150 actions emerged across 10 priority areas, which together constitute the research agenda. Actions included discrete activities, projects, or products, and ways to shift how research is conducted to strengthen the study of implementation mechanisms.

Conclusions

This research agenda elevates actions to guide the selection, design, and evaluation of implementation mechanisms. By delineating recommended actions to address the challenges of studying implementation mechanisms, this research agenda facilitates expanding the field of implementation science, beyond studying what works to how and why strategies work, in what contexts, for whom, and with which interventions.

Contributions to the literature
  • This research agenda operationalizes a set of activities to strengthen the implementation science field’s focus on why and how strategies work.
  • The research agenda addresses the following activities: accumulating knowledge, innovating methods and overcoming design challenges, improving measurement, providing guidance for specifying causal mechanisms, increasing focus on theorizing, engaging the policy and practice community, engaging funders, building capacity, enhancing equity, and effectively disseminating methods.
  • Studying implementation mechanisms can promote pragmatic strategy development, equitable processes and outcomes, and policy relevance by clarifying pathways for overcoming contextually specific barriers and achieving outcomes of interest.

**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**

✪ Changing hearts and minds: theorizing how, when, and under what conditions three social influence implementation strategies work

Authors:

Bryan J Weiner, Rosemary D Meza, Predrag Klasnja, Rebecca Lengnick-Hall, Gretchen J Buchanan, Aaron R Lyon, Kayne D Mettert, Marcella H Boynton, Byron J Powell, and Cara C Lewis

University of Washington affiliated authors are displayed in bold.

✪ Open Access

Published: September 2024

Read the full text in the open access journal Frontiers in Health Services

Abstract:

Background

Opinion leadership, educational outreach visiting, and innovation championing are commonly used strategies to address barriers to implementing innovations and evidence-based practices in healthcare settings. Despite voluminous research, ambiguities persist in how these strategies work and under what conditions they work well, work poorly, or work at all. The current paper develops middle-range theories to address this gap.

Methods

Conceptual articles, systematic reviews, and empirical studies informed the development of causal pathway diagrams (CPDs). CPDs are visualization tools for depicting and theorizing about the causal process through which strategies operate, including the mechanisms they activate, the barriers they address, and the proximal and distal outcomes they produce. CPDs also clarify the contextual conditions (i.e., preconditions and moderators) that influence whether, and to what extent, the strategy's causal process unfolds successfully. Expert panels of implementation scientists and health professionals rated the plausibility of these preliminary CPDs and offered comments and suggestions on them.

Findings

Theoretically, opinion leadership addresses potential adopters' uncertainty about likely consequences of innovation use (determinant) by promoting positive attitude formation about the innovation (mechanism), which results in an adoption decision (proximal outcome), which leads to innovation use (intermediate outcome). As this causal process repeats, penetration, or spread of innovation use, occurs (distal outcome). Educational outreach visiting addresses knowledge barriers, attitudinal barriers, and behavioral barriers (determinants) by promoting critical thinking and reflection about evidence and practice (mechanism), which results in behavioral intention (proximal outcome), behavior change (intermediate outcome), and fidelity, or guideline adherence (distal outcome). Innovation championing addresses organizational inertia, indifference, and resistance (determinants) by promoting buy-in to the vision, fostering a positive implementation climate, and increasing collective efficacy (mechanisms), which leads to participation in implementation activities (proximal outcome), initial use of the innovation with increasing skill (intermediate outcome) and, ultimately, greater penetration and fidelity (distal outcomes). Experts found the preliminary CPDs plausible or highly plausible and suggested additional mechanisms, moderators, and preconditions, which were used to amend the initial CPD.

Discussion

The middle-range theories depicted in the CPDs furnish testable propositions for implementation research and offer guidance for selecting, designing, and evaluating these social influence implementation strategies in both research studies and practice settings.

**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**