Open Access Archives - Page 8 of 35 - Implementation Science at UW

✪ Usability, Acceptability, and Barriers to Implementation of a Collaborative Agenda-Setting Intervention (CASI) to Promote Person-Centered Ovarian Cancer Care: Development Study

Abstract:

Background

People with advanced ovarian cancer and their caregivers report unmet supportive care needs. We developed a Collaborative Agenda-Setting Intervention (CASI) to elicit patients’ and caregivers’ needs through the patient portal before a clinic visit and to communicate these needs to clinicians using the electronic health record.

Objective

We aimed to assess the usability and acceptability of the CASI and identify barriers to and facilitators of its implementation.

Methods

We recruited English- and Spanish-speaking patients, caregivers, and clinicians from the gynecologic oncology program at a comprehensive cancer center. Participants used the CASI prototype and then completed individual cognitive interviews and surveys. We assessed usability with the System Usability Scale (scores range 0‐100, scores ≥70 indicate acceptable usability) and acceptability with the Acceptability of Intervention Measure and Intervention Appropriateness Measure (scores for both measures range from 1 to 5, higher scores indicate greater acceptability). Interviews were audio recorded, transcribed, and analyzed using directed content analysis. Domains and constructs from the Consolidated Framework for Implementation Research comprised the initial codebook. We analyzed survey data using descriptive statistics and compared usability and acceptability scores across patients, caregivers, and clinicians using analyses of variance.

Results

We enrolled 15 participants (5 patients, 5 caregivers, and 5 clinicians). The mean System Usability Scale score was 72 (SD 16). The mean Acceptability of Intervention Measure and Intervention Appropriateness Measure scores were 3.9 (SD 1.0) and 4.1 (SD 0.8), respectively. Participants viewed the CASI content and format positively overall. Several participants appreciated the CASI’s integration into the clinical workflow and its potential to increase attention to psychosocial concerns. Suggestions to refine the CASI included removing redundant items, simplifying item language, and adding options to request a conversation or opt out of supportive care referrals. Key barriers to implementing the CASI include its complexity and limited resources available to address patients’ and caregivers’ needs.

Conclusions

The CASI is usable and acceptable to patients with advanced ovarian cancer, caregivers, and clinicians. We identified several barriers to and facilitators of implementing the CASI. In future research, we will apply these insights to a pilot randomized controlled trial to assess the feasibility of comparing the CASI to usual care in a parallel group-randomized efficacy trial.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

✪ A framework for designing hybrid effectiveness-implementation trials for digital health interventions

Abstract:

This article proposes methods for designing randomized controlled trials studying the implementation and effectiveness of digital interventions, meaning websites or applications ("apps") that patients use in healthcare. Deploying digital interventions for behavioral health differs from implementing traditional interventions such as medications or human-delivered therapy. Prior trial design guidance has ignored the existence of international governmental evidence standards, has paid insufficient attention to implementation reporting guidelines, and has not described methods for empirically testing the approach for organizing the delivery of digital interventions.

This framework for designing hybrid effectiveness-implementation trials of digital behavioral health interventions helps researchers articulate research questions that matter to decision-makers and meaningfully contribute to implementation. The framework outlines three phases: 1) frame effectiveness and implementation questions in terms of the digital intervention components, types of clinical support for the digital intervention, and specific strategies for implementing the digital intervention; 2) define and delineate actors, activities, action targets, dose, temporality, and outcomes to maximize inference and reproducibility; and 3) specify trial design features used for hybrid classification.

We illustrate the utility of this framework with two effectiveness-implementation studies of digital interventions for substance use. This framework can help researchers decide on appropriate methodology and help decision-makers apply findings.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

✪ Development of a method for qualitative data integration to advance implementation science within research consortia

Abstract:

Background

Methods of integrating qualitative data across diverse studies and within multi-site research consortia are less developed than those for integrating quantitative data. The development ofsuchmethods is essential to support the data exchange needed for cross-study qualitative inquiry and given the increasing emphasis on data sharing and open science. We describe methods for qualitative data integration within the National Cancer Institute’s Improving the Management of symPtoms During And following Cancer Treatment (IMPACT) Consortium funded by the Cancer MoonshotSM. Data collection and analysis were guided by the Consolidated Framework for Implementation Research (CFIR). Our case study highlights potential solutions for unique challenges faced when integrating qualitative data across multiple settings in a research consortium.

Methods

The IMPACT consortium is comprised of three research centers (RCs) each conducting pragmatic trials examining the effectiveness of routine symptom management on patient-centered outcomes. After reaching consensus on use of CFIR as the common implementation determinant framework, RCs developed a semi-structured interview guide and tailored it to features of their healthcare setting and symptom management interventions. RCs conducted interviews/focus groups with healthcare system partners to examine contextual factors impacting implementation. RCs exchanged 1–2 transcripts (n = 5 total) for purposes of pilot testing the methodology.

Results

Given the heterogeneity of study settings and contexts, it was challenging to simultaneously assign codes at both domain and construct levels and the process was resource intensive. Recommendations include employing a common framework for data collection and analyses from the outset, coding at domain level first and then incorporating construct codes, and centralizing processes via a coordinating center (or similar entity) and combining coded transcripts using qualitative software. We also generated an iteratively refined codebook that employed the CFIR schema and incorporated CFIR 2.0 to provide detailed guidance for coders conducting cross-study qualitative inquiry.

Conclusions

Limited guidance exists on how to support qualitative data integration, data exchange, and sharing across multiple studies. This paper describes a systematic method for employing an implementation determinant framework-guided approach to foster data integration. This methodology can be adopted by other research consortia to support qualitative data integration, cross-site qualitative inquiry, and generate improved understanding of evidence-based intervention implementation.

Contributions to the literature

Few standard practices exist for integrating qualitative data across different studies to conduct cross-study qualitative inquiry. Developing data integration procedures is critical to generating insights about practice improvement across settings and achieving goals of open science.
This report describes a method for integrating qualitative data using the Consolidated Framework for Implementation Research illustrated within the context of NCI’s Improving the Management of symPtoms During And following Cancer Treatment (IMPACT) Consortium.
The method describedcan be applied within research consortia and across multi-site studies to support qualitative data integration, thereby producing valuable insights to improve implementation of evidence-based interventions.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

✪ Usability of a cross-system housing stability program for juvenile courts: A multimethod study of probation counselor perspectives

Abstract:

Background

Youth in contact with the juvenile legal system experience disproportionate rates of homelessness. While system contact is a critical intervening point, juvenile courts do not typically offer housing services. One solution is to refer youth to evidence-based, community-based services to meet housing-related needs, but a myriad of individual and organizational factors often impede court staff from making such referrals. Housing Stability for Youth in Courts (HSYNC) is a novel, cross-system service linkage model for court-involved youth facing housing instability. HSYNC was developed using codesign as a strategy to improve usability and incorporated evidence on juvenile court linkage strategies effective at increasing service referrals and improving outcomes.

Method

The current study utilized a multimethod approach to evaluate the usability of HSYNC from the perspective of juvenile probation counselors (JPCs) as one group of end-users. As part of the pilot implementation study, survey (N = 15) and semi-structured interview (N = 20) data were obtained from JPCs from four juvenile courts in one northwestern state.

Results

Survey results indicated good usability for HSYNC (mean implementation strategy usability scale score of 80). Themes from the interviews complemented survey findings, pointing to HSYNC as a value-added program that integrated well with existing probation practices. The housing navigator's approach was highlighted as a strong facilitator of program usability. Themes also provided greater insight into implementation barriers (i.e., individual and contextual factors) and areas for improvement (e.g., eligibility screening process).

Conclusions

Drivers of program usability and implications for program improvement to address identified implementation challenges to ensure the HSYNC model is effective and sustainable long-term are discussed.

Plain Language Summary

A study examining the key factors impacting successful implementation of a novel housing stability program to address homelessness risk for court-involved youth and their families.
Court-involved youth experience disproportionate rates of homelessness, yet housing services are not traditionally offered by juvenile courts. Court staff, such as probation staff, can refer youth to community-based services to meet their housing needs. Unfortunately, many factors complicate whether these referrals are made, including staff feeling overburdened by other work obligations and uncertainty about whether referring youth to community services fits within their job responsibilities. In this study, we surveyed and interviewed probation staff who were responsible for implementing a new referral-based housing stability program—Housing Stability for Youth in Courts (HSYNC)—to understand whether the program addressed previously identified barriers to referral-making. A unique feature of HSYNC is that there is a dedicated housing navigator, who works as a “linkage specialist” to support probation staff in making referrals while working directly with youth and families to access housing-related services in the community. Our findings suggest that, from the perspective of probation staff, HSYNC is considered to be a user-friendly program that is well-aligned with existing court practices. The housing navigator and their unique approach to working with probation staff, youth, and families was perceived as the strongest facilitator of implementation success; meanwhile, screening for program eligibility was the biggest barrier, pointing to the need for improvement before HSYNC is widely implemented. The persistent intersection of youth housing instability and court involvement calls for better coordination between juvenile courts and community agencies providing housing-related services. From the probation perspective, HSYNC offers a viable solution to meeting youths’ housing needs without creating additional workload burden for court staff.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

✪ Outcome preferences in fidelity-adaptation scenarios across evidence-based parenting programs: A discrete choice experiment

Abstract:

Background

Implementing evidence-based parenting programs often involves navigating fidelity-adaptation decisions. While research has explored various aspects of this dilemma, little is known about how practitioners’ outcome preferences influence their decisions in real-world scenarios.

Methods

This study employed a discrete choice experiment (DCE) to investigate the relative importance of five outcomes (Relationship Quality, Satisfaction, Workload Strain, Value Conflict, and Reach) in fidelity-adaptation decisions among 209 practitioners delivering evidence-based parenting programs in Sweden. The DCE presented 25 choice sets across five contextual scenarios, analyzed using Bayesian hierarchical logistic regression.

Results

All five outcomes significantly influenced practitioners’ choices, with Relationship Quality emerging as the most impactful (log-odds: 4.56, 95% CI [4.16, 4.91]). Satisfaction and minimizing Value Conflict showed similar importance (log odds: 2.45 and -2.40, respectively), while Workload Strain and Reach had slightly less impact (log odds: -2.10 and 1.96, respectively).

Conclusions

This study offers a novel perspective on the role of outcome preference in navigating fidelity-adaptation decisions. The strong preference for improving parent-child relationships aligns with core parenting program goals, while consideration of other outcomes reflects practitioners’ holistic approach to implementation. These findings can inform the design of interventions and implementation strategies that balance effectiveness with real-world constraints, potentially enhancing parenting programs’ adoption, sustainability, and impact.

Contributions to the literature

This study pioneers the application of discrete choice experiments in exploring fidelity-adaptation decisions.
Quantifying practitioners’ outcome preferences provides empirical evidence of complex trade-offs in implementing evidence-based parenting programs.
The study’s results provide actionable insights for program developers, policymakers, and implementation scientists on designing interventions that resonate with practitioners’ priorities while maintaining program effectiveness.
This research bridges theoretical frameworks and real-world decision-making, advancing our understanding of how various factors influence intervention delivery in practice.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

✪ Implementation Outcomes From a Pilot Study of Mindful Awareness in Body-Oriented Therapy (MABT) as a Chronic Pain Treatment Modality in an Integrative Health Clinic

Abstract:

Background

As regulation of opioid prescribing evolves, primary care and pain clinics are shifting to provide non-pharmacological and interdisciplinary chronic pain care. An under-utilized but growing area of health care for chronic pain is complementary and integrative health (CIH). However, there is limited availability of CIH approaches within the health care system. Mindful Awareness in Body-Oriented Therapy (MABT) is an evidence-based mind-body therapy, with a manualized protocol, that focuses on developing interoceptive sensibility for improved self-awareness and nervous system regulation. Prior MABT research shows MABT improves self-report and physiological indicators of interoception as well as mental and physical symptoms of distress.

Methods

This pilot single-group study used a hybrid implementation-effectiveness design and mixed methods to study implementation strategies and outcomes for bringing MABT into an integrative chronic pain clinic. Administrative data, staff surveys, and focus groups were used to understand the implementation process and outcomes (see Additional files 2, 3, and 4). Descriptive statistics were used to analyze survey and administrative data. A content analysis approach was used to analyze qualitative data from focus groups.

Results

7 staff surveys were administered over the 24-month study period and showed high acceptability and appropriateness that increased over time. Adoption, feasibility, and sustainability were also high. Clinicians made 70 referrals to MABT, 56 patients scheduled a session, 41 patients completed at least one session, and 71% of these completed the protocol. Focus groups identified MABT as a therapy that filled a gap in services, particularly for patients with a lack of body awareness and high emotion dysregulation.

Conclusion

Implementation of MABT was highly successful in an integrative health clinic focused on chronic pain treatment.

ClinicalTrials.gov registration

NCT05289024 Registered March 11, 2022

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

✪ How integrated knowledge translation worked to reduce federal policy barriers to the implementation of medication abortion in Canada: a realist evaluation

Abstract:

Background

Initial Canadian federal regulations for the abortion pill, mifepristone, had the potential to impede safe and equitable access to this medication. To catalyze evidence-based regulatory change, we engaged health policy, health system, and health services decision makers, and health professional organizations in integrated knowledge translation (iKT), a research approach that engages the users of research as equal partners.

Methods

We conducted a realist evaluation of what iKT strategies worked, for whom, and in what context to impact federal mifepristone regulations. We constructed initial program theories (if–then statements about how iKT worked). We tested the initial program theories using interviews with researchers and knowledge partners and triangulated with analysis of research programme documents. We configured the evidence in relation to the initial program theories, and refined program theories into causal explanatory configurations.

Results

We analyzed 38 interviews with researchers, health professional leaders, advocacy group leaders, and administrative government policy makers, as well as 49 program documents. Our results indicated that researcher partnerships with stakeholders had a meaningful impact on the removal of restrictions. We found key components of the causal explanatory configurations included: researcher motivation to move evidence into action, trusted reputations as credible sources of evidence, strategic partnerships, understanding of health policy processes, and researcher roles as a trusted convenor between key groups and decision makers.

Conclusions

Our study identifies several practical and transferable approaches to impactful iKT. The findings may be of relevance to researchers focused on public health topics subject to stigma.

Contributions to the literature

The effect of integrated knowledge translation (iKT) on public health policy is ambiguous, especially for issues that may be stigmatized, like abortion.
To bridge this knowledge gap, we evaluated the iKT used to support Canadian federal drug regulatory decisions to remove restrictions for the medication abortion pill, mifepristone.
Our realist evaluation identified several practical and transferable approaches to impactful iKT in complex policy systems that comprise an ‘iKT mindset’.
Key approaches for researchers include acting on motivations to move evidence into policy, building trusted reputations as credible sources of evidence, understanding and targeting policy levers, acting as a source of strategic evidence, and acting as a trusted convenor between key groups and decision makers.
Our use of realist methodology contributes to theoretical and empirical understandings of iKT with policy makers and has relevance for other stigmatized topics including opioid safe supply, vaccination, and climate change.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

✪ Human-Centered Design of an mHealth Tool for Optimizing HIV Index Testing in Wartime Ukraine: Formative Research Case Study

Abstract:

Background

Assisted partner services (APSs; sometimes called index testing) are now being brought to scale as a high-yield HIV testing strategy in many nations. However, the success of APSs is often hampered by low levels of partner elicitation. The Computer-Assisted Self-Interview (CASI)–Plus study sought to develop and test a mobile health (mHealth) tool to increase the elicitation of sexual and needle-sharing partners among persons with newly diagnosed HIV. CASI-Plus provides client-facing information on APS methods and uses a standardized, self-guided questionnaire with nonjudgmental language for clients to list partners who would benefit from HIV testing. The tool also enables health care workers (HCWs) to see summarized data to facilitate partner tracking.

Objective

The formative research phase of the CASI-Plus study aimed to gather client and HCW input on the design of the CASI-Plus tool to ensure its acceptability, feasibility, and usability.

Methods

This study gathered input to prioritize features and tested the usability of CASI-Plus with HCWs and clients receiving HIV services in public health clinics in wartime Ukraine. The CASI-Plus study’s formative phase, carried out from May 2023 to July 2024, adapted human-centered design (HCD) methods grounded in principles of empathy, iteration, and creative ideation. The study involved 3 steps: formative HCD, including in-depth individual interviews with clients, such as men who have sex with men and people who inject drugs, and internet-based design workshops with HCWs from rural and urban HIV clinics in Chernihiv and Dnipro; software platform assessment and heuristic evaluation, including assessment of open-source mHealth platforms against CASI-Plus requirements, prototype development, and testing of the REDCap (Research Electronic Data Capture) prototype based on usability heuristics; and usability walk-throughs, including simulated cases with HCWs and clients.

Results:

The formative phase of the CASI-Plus study included in-depth individual interviews with 10 clients and 3 workshops with 22 HCWs. This study demonstrated how simplified HCD methods, adapted to the wartime context, gathered rich input on prioritized features and tool design. The CASI-Plus design reflected features that are both culturally sensitive and in alignment with the constraints of Ukraine’s wartime setting. Prioritized features included information about the benefits of HIV index testing; a nonjudgmental, self-guided questionnaire to report partners; client stories; and bright images to accompany the text. Two-way SMS text messaging between clients and HCWs was deemed impractical based on risks of privacy breaches, national patient privacy regulations, and HCW workload./p>

Conclusions

It was feasible to conduct HCD research in Ukraine in a wartime setting. The CASI-Plus mHealth tool was acceptable to both HCWs and clients. The next step for this research is a randomized clinical trial of the effect of the REDCap-based CASI-Plus tool on the number of partners named and the rate of partners completing HIV testing.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

✪ Development of learning health system competency items related to health and healthcare equity and justice for rehabilitation researchers

Authors:

Pamela M. Dunlap, Kathleen M. Poploski, Catherine A. Anderson, Thiru M. Annaswamy, Melissa A. Clark, Peter C. Coyle, Natalie F. Douglas, Ann Marie Flores, Janet K. Freburger, Brian J. Hafner, Kenneth J. Harwood, Jeanne M. Hoffman, Adam R. Kinney, Linda Resnik, Kristin Ressel, Margarite J. Whitten, and Christine M. McDonough

^{University of Washington affiliated authors are displayed in bold.}

✪ Open Access

Published: January 2025

Read the full text in the open access journal Learning Health Systems

Abstract:

Introduction

In 2021, the Learning Health Systems Rehabilitation Research Network (LeaRRn) developed and administered a needs assessment survey, based on the Agency on Healthcare Research and Quality's (AHRQ's) original seven domains of learning health systems (LHS) researcher core competencies, to identify knowledge and interest in LHS research competencies among rehabilitation researchers. In 2022, the AHRQ added a new health and healthcare equity and justice (HE) domain to the existing seven domains for LHS researcher core competencies.

Methods

LeaRRn utilized methods similar to those employed in the development of their original needs assessment survey to generate and refine competency items for the HE domain. In this report, we describe the methods used to develop these HE competency items.

Results & Conclusions

Other training programs and LHS researchers may use the competency items developed for this needs assessment survey to identify training opportunities in the HE domain.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

✪ Not getting better but not getting worse: A cluster randomized controlled pilot trial of a leadership implementation strategy

Abstract:

Background

Implementation of evidence-based practices (EBPs) in schools is fraught with challenges. Even when EBPs are initiated, deterioration of implementation efforts often hinders their long-term success. School leadership behaviors can influence teachers’ EBP implementation. Our study tested an implementation strategy called Helping Educational Leaders Mobilize Evidence (HELM), adapted from the Leadership and Organizational Change for Implementation strategy, to enhance EBP implementation through improvements in school leadership teams’ implementation leadership and climate to buffer against the deterioration of implementation efforts. This study explores the impact of HELM on theorized mechanisms of change (i.e., implementation leadership, climate), educator-level factors (i.e., implementation citizenship), and implementation outcomes (i.e., fidelity, initiative stability).

Method

One school district and 10 schools in Washington participated. Five of the schools were randomized to receive the HELM strategy and the remaining five schools received an alternative leadership training as an implementation attention control. Teachers at every school (n = 341) received training for an EBP called Positive Greetings at the Door that has been previously demonstrated to reduce student behavior problems. Principals and Assistant Principals (n = 18) received the HELM strategy or alternative leadership training. Three district Administrators also participated in HELM as part of the Organizational Strategy Development meetings.

Results

HELM significantly slowed the average decline of implementation leadership (perseverant leadership and communication), three dimensions of implementation climate (recognition, rewards, and existing supports) and total implementation climate, and one dimension of implementation citizenship (keeping informed). No significant effects were found with regard to implementation outcomes (i.e., fidelity, initiative stability).

Conclusions

HELM shows promise in buffering the deterioration of EBP implementation efforts in schools. HELM positively influenced implementation leadership and climate, which are the hypothesized mechanisms for promoting successful long-term implementation efforts. An appropriately powered trial is needed to determine the efficacy of HELM in the future.

Trial Registration Information

Name of the registry: clinicaltrials.gov
Trial registration number: NCT06340074
Date of registration: March 29, 2024. Retrospectively registered
URL of trial registry record: https://clinicaltrials.gov/study/NCT06340074?intr=helm&rank=

Plain Language Summary

Using practices that have been proven to work in research to support youth social, emotional, and behavioral well-being in schools is hard. Efforts to use these practices decrease across the school year. School leadership behaviors can influence educators’ use of these practices. Our study tested an implementation strategy or support designed to help educators use a research-backed practice called Helping Educational Leaders Mobilize Evidence (HELM). HELM was adapted from the Leadership and Organizational Change for Implementation (LOCI) strategy, to enhance school leadership teams’ specific leadership behaviors to protect against the decline in implementation effort. This study explores the impact of HELM on outcomes related to practice use. One school district and 10 schools in western Washington State participated. Five of the schools received the HELM strategy and the remaining five schools received an alternative leadership training. Teachers at every school (n = 341) received training in a practice called Positive Greetings at the Door (PGD) that has been shown to reduce student behavior problems. Principals and Assistant Principals (n = 18) received the HELM strategy or alternative leadership training. Three district administrators also participated in HELM. HELM unexpectedly slowed the average decline of implementation leadership (perseverant leadership and communication), three dimensions of implementation climate (recognition, rewards, and existing supports), total implementation climate, and one dimension of implementation citizenship (keeping informed). No significant effects were found with regard to use of the practice. The HELM strategy shows promise in slowing the deterioration of practice in schools and positively influenced implementation leadership behaviors.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

Authors:

✪ Open Access

Read the full text in the open access journal JMIR Cancer

Abstract:

Background

Objective

Methods

Results

Conclusions

Authors:

✪ Open Access

Read the full text open access in the journal Annals of Epidemiology

Abstract:

Authors:

✪ Open Access

Read the full text in the open access journal Implementation Science Communications

Abstract:

Background

Methods

Results

Conclusions

Contributions to the literature

Authors:

✪ Open Access

Read the full text in the open access journal Implementation Research and Practice

Abstract:

Background

Method

Results

Conclusions

Plain Language Summary

Authors:

✪ Open Access

Read the full text in the open access journal Implementation Science

Abstract:

Background

Methods

Results

Conclusions

Contributions to the literature

Authors:

✪ Open Access

Read the full text in the open access journal Integrative Medicine & Health

Abstract:

Background

Methods

Results

Conclusion

ClinicalTrials.gov registration

Authors:

✪ Open Access

Read the full text in the open access journal Implementation Science Communications

Abstract:

Background

Methods

Results

Conclusions

Contributions to the literature

Authors:

✪ Open Access

Read the full text in the open access journal JMIR Formative Research

Abstract:

Background

Objective

Methods

Results:

Conclusions

Authors:

✪ Open Access

Read the full text in the open access journal Learning Health Systems

Abstract:

Introduction

Methods

Results & Conclusions

Authors:

✪ Open Access

Read the full text in the open access journal Implementation Research and Practice

Abstract:

Background

Method