Research Design Archives - Implementation Science at UW

✪ Exploring fidelity and its influence on effectiveness in an audit and feedback strategy implemented in a low-resource setting: an application of regression modeling

Abstract:

Background

Assessing implementation fidelity—the degree to which a program is implemented as intended—is essential to understand whether poor outcomes are due to implementation problems or the design of an intervention. Few studies in health research have documented the association between implementation fidelity and effectiveness. The Integrated District Evidence-to-Action (IDEAs) is a multicomponent audit and feedback strategy designed to improve the implementation of maternal and child clinical guidelines in Mozambique. In a previous study, we found mixed results of IDEAs effectiveness. The objective of the present study is to understand how implementation fidelity may have influenced the effectiveness of the strategy.

Methods

IDEAs was implemented in 154 health facilities across 12 districts in Manica and Sofala provinces in Mozambique between 2016 and 2020. We used the conceptual framework for implementation fidelity to guide descriptive analysis of IDEAs adherence. Regression modeling was used to study patterns of the direction of association between measures of fidelity and effectiveness for ten service delivery outcomes and five service readiness outcomes.

Results

We describe adherence on 15 measures of fidelity, of which 12 had high fidelity. Poor fidelity was found in conducting facility service readiness assessments and completing micro-interventions from action plans. Service delivery measures tended to be positively associated with participation and degree of micro-intervention completion and negatively associated with a higher number of action plans elaborated by participating teams. For the service readiness outcomes, delivery of essential care was positively associated with participation and micro-intervention completion, and staff availability was negatively associated with supervision.

Conclusion

Participation in audit and feedback meetings, the number of action plans elaborated, and the degree of completion of micro-interventions seem to be related to the effectiveness results. IDEAs should be adapted to reduce the number of action plans elaborated and promote better micro-intervention completion. Additionally, combining audit and feedback strategies with other strategies might enhance effectiveness in service outcomes. This study examines how to analyze the link between fidelity and effectiveness of a strategy to inform better design and recommend context-specific improvements.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

✪ “Though there are some challenges, we try our best to follow the guidelines”: a qualitative study exploring determinants of providing guideline-adherent care to hospitalized children in Kenya

Abstract:

Background

Clinical treatment guidelines are designed to improve quality of care by offering clear recommendations for health workers; however, across many settings, implementing guidelines is a challenge. For over 20 years, comprehensive World Health Organization guidelines for care of hospitalized children have provided guidance for treating the primary causes of childhood death. These recommendations were adapted in Kenya and codified in the Basic Paediatric Protocol; however, adherence to these evidence-based interventions remains suboptimal.

Methods

This qualitative study identified the barriers and facilitators of providing guideline-adherent care to children admitted to two hospitals in Kenya. The semi-structured question guide was informed by the Theoretical Domains Framework (TDF). Three focus group discussions and 16 in-depth interviews were conducted with 35 health workers. Participants were nurses, doctors, clinical officers, or nutritionists (including in-charges and clinical supervisors). Interviews were audio recorded and transcribed. Transcripts were coded using a TDF-based codebook, validated, and analyzed for themes.

Results

TDF domains with the greatest influence on the thematic findings were environmental context and resources, social influences, beliefs about consequences, and beliefs about capabilities. Health workers were knowledgeable about guidelines and felt strongly that adherence to them was beneficial; however, they also faced barriers implementing them. Challenges related to the environmental context were pervasive, including shortages of staff and supplies. Participants praised the simplicity of guidelines but also noted that simplified diagnostic criteria don’t align with complex patient presentations. While guidelines empowered some nurses to make clinical decisions, respondents reported that strict professional roles sometimes delayed diagnosis and treatment. Further, health workers reported some deviations were intentional (ex. when guidelines were not aligned to their beliefs) and other times they were unintentional (ex. when complex patient presentations make guideline adherence difficult).

Conclusions

Knowledge of guidelines and motivation to use them were not major determinants in this analysis. Rather, human and material resource shortages presented the greatest barrier to guideline adherence in this setting. Improving guidelines by involving health workers in participatory development of guidelines would improve both clarity and feasibility. Health workers are well-versed in guideline recommendations, so educational strategies should focus on knowledge gaps or changes in the latest version of guidelines.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

✪ Community-Engaged Research in HIV Implementation Science: Characterizing Academic and Community Partnerships in the “Ending the HIV Epidemic” Supplement Awards

✪ Development of a method for qualitative data integration to advance implementation science within research consortia

Abstract:

Background

Methods of integrating qualitative data across diverse studies and within multi-site research consortia are less developed than those for integrating quantitative data. The development ofsuchmethods is essential to support the data exchange needed for cross-study qualitative inquiry and given the increasing emphasis on data sharing and open science. We describe methods for qualitative data integration within the National Cancer Institute’s Improving the Management of symPtoms During And following Cancer Treatment (IMPACT) Consortium funded by the Cancer MoonshotSM. Data collection and analysis were guided by the Consolidated Framework for Implementation Research (CFIR). Our case study highlights potential solutions for unique challenges faced when integrating qualitative data across multiple settings in a research consortium.

Methods

The IMPACT consortium is comprised of three research centers (RCs) each conducting pragmatic trials examining the effectiveness of routine symptom management on patient-centered outcomes. After reaching consensus on use of CFIR as the common implementation determinant framework, RCs developed a semi-structured interview guide and tailored it to features of their healthcare setting and symptom management interventions. RCs conducted interviews/focus groups with healthcare system partners to examine contextual factors impacting implementation. RCs exchanged 1–2 transcripts (n = 5 total) for purposes of pilot testing the methodology.

Results

Given the heterogeneity of study settings and contexts, it was challenging to simultaneously assign codes at both domain and construct levels and the process was resource intensive. Recommendations include employing a common framework for data collection and analyses from the outset, coding at domain level first and then incorporating construct codes, and centralizing processes via a coordinating center (or similar entity) and combining coded transcripts using qualitative software. We also generated an iteratively refined codebook that employed the CFIR schema and incorporated CFIR 2.0 to provide detailed guidance for coders conducting cross-study qualitative inquiry.

Conclusions

Limited guidance exists on how to support qualitative data integration, data exchange, and sharing across multiple studies. This paper describes a systematic method for employing an implementation determinant framework-guided approach to foster data integration. This methodology can be adopted by other research consortia to support qualitative data integration, cross-site qualitative inquiry, and generate improved understanding of evidence-based intervention implementation.

Contributions to the literature

Few standard practices exist for integrating qualitative data across different studies to conduct cross-study qualitative inquiry. Developing data integration procedures is critical to generating insights about practice improvement across settings and achieving goals of open science.
This report describes a method for integrating qualitative data using the Consolidated Framework for Implementation Research illustrated within the context of NCI’s Improving the Management of symPtoms During And following Cancer Treatment (IMPACT) Consortium.
The method describedcan be applied within research consortia and across multi-site studies to support qualitative data integration, thereby producing valuable insights to improve implementation of evidence-based interventions.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

Perspectives on Implementing a Communication Facilitator Intervention From a Critical Care Setting

Design of a pragmatic trial integrating human papillomavirus (HPV) self-sampling into primary care to reduce cervical cancer screening disparities in Somali American individuals: The Isbaar project

Qualitative Study of Health Care Team Perception of the Benefits and Limitations of Remote Symptom Monitoring

A Guide to Implementation Science for Phase 3 Clinical Trialists: Designing Trials for Evidence Uptake

Formative qualitative research to guide implementation of the Collaborative Care Model in a low-barrier HIV clinic

✪ Use of concept mapping to inform a participatory engagement approach for implementation of evidence-based HPV vaccination strategies in safety-net clinics

Abstract:

Background

Multiple evidence-based strategies (EBS) for promoting HPV vaccination exist. However, adolescent HPV vaccination rates remain below target levels in communities at high risk for HPV-associated cancers and served by safety-net clinics. Participatory engaged approaches are needed to leverage the expertise of community and clinical partners in selecting EBS relevant to their local context. We engaged concept mapping as a method to inform the adoption and adaptation of EBS that seeks to empower implementation partners to prioritize, select, and ultimately implement context-relevant EBS for HPV vaccination.

Methods

Using 38 EBS statements generated from qualitative interviews and national HPV vaccine advocacy sources, we conducted a modified concept mapping activity with partners internal to safety-net clinics and external community members in two study sites of a larger implementation study (Greater Los Angeles and New Jersey), to sort EBS into clusters and rate each EBS by importance and feasibility for increasing HPV vaccination within safety-net clinics. Concept mapping findings (EBS statement ratings, ladder graphs and go-zones) were shared with leaders from a large federally qualified health center (FQHC) system (focusing on three clinic sites), to select and implement EBS over 12 months.

Results

Concept mapping participants (n=23) sorted and rated statements, resulting in an eight-cluster solution: 1) Community education and outreach; 2) Advocacy and policy; 3) Data access/quality improvement monitoring; 4) Provider tracking/audit and feedback; 5) Provider recommendation/communication; 6) Expanding vaccine access; 7) Reducing missed opportunities; and 8) Nurse/staff workflow and training. The FQHC partner then selected to intervene on eight of 17 EBS statements in the “go-zone” for action, with three from “reducing missed opportunities,” two from “nurse/staff workflow and training,” and one each from “provider tracking/audit and feedback,” “provider recommendation/communication,” and “expanding vaccine access,” which the research team addressed through the implementation of three multi-level intervention strategies (e.g., physician communication training, staff training and workflow assessment, audit and feedback of clinic processes).

Conclusions

Concept mapping provided a powerful participatory approach to identify multilevel EBS for HPV vaccination relevant to the local safety-net clinic context, particularly when several strategies exist, and prioritization is necessary. This study demonstrates how a clinic system benefited directly from the ratings and prioritization of EBS by multilevel clinic and community partners within the broader safety-net clinic context to identify and adapt prioritized solutions needed to advance HPV vaccine equity.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

Authors:

✪ Open Access

Read the full text in the open access journal Implementation Science Communications

Abstract:

Background

Methods

Results

Conclusion

Authors:

✪ Open Access

Read the full text in the open access journal Implementation Science Communication

Abstract:

Background

Methods

Results

Conclusions

Authors:

✪ Open Access

Read the full text open access in the journal JAIDS: Journal of Acquired Immune Deficiency Syndrome

Authors:

✪ Open Access

Read the full text in the open access journal Implementation Science Communications

Abstract:

Background

Methods

Results

Conclusions

Contributions to the literature

Authors:

Subscription Required for Full Text Access

Read the full text in the subscription access Journal of Pain and Symptom Management

Authors:

Subscription Required for Full Text Access

Read the full text in the subscription access journal Contemporary Clinical Trials

Authors:

Subscription Required for Full Text Access

Read the full text in the subscription access journal JCO Oncology Practice

Authors:

Subscription Required for Full Text Access

Read the full text in the subscription access Journal of the American College of Cardiology

Authors:

Subscription Required for Full Text Access

Read the full text in the subscription access journal AIDS Care

Authors:

✪ Open Access

Read the full text in the open access journal Implementation Science Communications

Abstract:

Background

Methods

Results

Conclusions