Stephen Gloyd Archives - Implementation Science at UW

✪ Exploring fidelity and its influence on effectiveness in an audit and feedback strategy implemented in a low-resource setting: an application of regression modeling

Abstract:

Background

Assessing implementation fidelity—the degree to which a program is implemented as intended—is essential to understand whether poor outcomes are due to implementation problems or the design of an intervention. Few studies in health research have documented the association between implementation fidelity and effectiveness. The Integrated District Evidence-to-Action (IDEAs) is a multicomponent audit and feedback strategy designed to improve the implementation of maternal and child clinical guidelines in Mozambique. In a previous study, we found mixed results of IDEAs effectiveness. The objective of the present study is to understand how implementation fidelity may have influenced the effectiveness of the strategy.

Methods

IDEAs was implemented in 154 health facilities across 12 districts in Manica and Sofala provinces in Mozambique between 2016 and 2020. We used the conceptual framework for implementation fidelity to guide descriptive analysis of IDEAs adherence. Regression modeling was used to study patterns of the direction of association between measures of fidelity and effectiveness for ten service delivery outcomes and five service readiness outcomes.

Results

We describe adherence on 15 measures of fidelity, of which 12 had high fidelity. Poor fidelity was found in conducting facility service readiness assessments and completing micro-interventions from action plans. Service delivery measures tended to be positively associated with participation and degree of micro-intervention completion and negatively associated with a higher number of action plans elaborated by participating teams. For the service readiness outcomes, delivery of essential care was positively associated with participation and micro-intervention completion, and staff availability was negatively associated with supervision.

Conclusion

Participation in audit and feedback meetings, the number of action plans elaborated, and the degree of completion of micro-interventions seem to be related to the effectiveness results. IDEAs should be adapted to reduce the number of action plans elaborated and promote better micro-intervention completion. Additionally, combining audit and feedback strategies with other strategies might enhance effectiveness in service outcomes. This study examines how to analyze the link between fidelity and effectiveness of a strategy to inform better design and recommend context-specific improvements.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

✪ Can audit and feedback improve health service readiness and delivery outcomes in a low-resource setting? Effectiveness results of the IDEAs strategy from central Mozambique

Abstract:

Limited evidence exists on audit and feedback (A&F) in low-resource contexts. We tested the Integrated District Evidence-to-Action (IDEAs), a multicomponent A&F strategy in Mozambique’s maternal and child health (MCH) services. IDEAs include A&F meetings, readiness assessments, and facility support. We report the effectiveness results. IDEAs were implemented in 2016–2020 across two provinces, 12 districts, and 154 primary health facilities in Mozambique. We assessed 1) ten service delivery outcomes across antenatal, maternity, postpartum, childcare, and reproductive health services and 2) five service readiness outcomes (medicines, infrastructure, equipment, care, and staffing availability). We used propensity score matching to minimize bias and a controlled interrupted time series with a negative binomial mixed effects model for service delivery analysis, presenting incidence rate ratios (IRR) with 95% confidence intervals (95% CI). For service readiness outcomes, we created composite scores for each domain and a difference-in-difference analysis using an ordinal mixed effects model, reporting odds ratios (OR) and 95% CI. Significant associations were found with first at-risk child appointments (IRR = 1.06 [1.04, 1.07]), first polymerase chain reaction tests for HIV-exposed children (IRR = 1.02 [1.01, 1.03]), new contraceptive users (IRR = 0.95 [0.94, 0.96]), women starting long-lasting contraceptives (IRR = 0.94 [0.93, 0.95]), availability of infrastructure (OR = 5.84 [1.32, 25.88]) and essential care (OR = 0.13 [0.03, 0.54]). No significant associations were found between IDEAs and six of 10 service delivery outcomes (women with a fourth dose of preventive malaria treatment; women protected with tetanus vaccine; women with four+ antenatal visits; deliveries with active management of the third stage of labor; first postpartum consultations and fully vaccinated children) and on medicine, equipment, and staffing availability. We observed mixed effectiveness in implementing IDEAs, with null and sub-optimal effects suggesting the need to refine and adapt strategy components to more effectively address clinical and readiness outcomes.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

✪ Implementation outcomes of the integrated district evidence to action (IDEAs) program to reduce neonatal mortality in central Mozambique: an application of the RE-AIM evaluation framework

Authors:

Aneth Dinis, Quinhas Fernandes, Bradley H Wagenaar, Sarah Gimbel, Bryan J Weiner, Grace John-Stewart, Ermyas Birru, Stephen Gloyd, Ruth Etzioni, Dorlim Uetela, Isaías Ramiro, Artur Gremu, Orvalho Augusto, Stélio Tembe, Jaime L Mário, Jalilo E Chinai, Alfredo F Covele, Cassimo M Sáide, Nélia Manaca & Kenneth Sherr

^{University of Washington affiliated authors are displayed in bold.}

✪ Open Access

Published: February 2024

Read the full text in the open access journal BMC Health Services Research

Abstract:

Background

Scarce evidence exists on audit and feedback implementation processes in low-resource health systems. The Integrated District Evidence to Action (IDEAs) is a multi-component audit and feedback strategy designed to improve the implementation of maternal and child guidelines in Mozambique. We report IDEAs implementation outcomes.

Methods

IDEAs was implemented in 154 health facilities across 12 districts in Manica and Sofala provinces between 2016 and 2020 and evaluated using a quasi-experimental design guided by the Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) framework. Reach is the proportion of pregnant women attending IDEAs facilities. Adoption is the proportion of facilities initiating audit and feedback meetings. Implementation is the fidelity to the strategy components, including readiness assessments, meetings (frequency, participation, action plan development), and targeted financial support and supervision. Maintenance is the sustainment at 12, 24, and 54 months.

Results

Across both provinces, 56% of facilities were exposed to IDEAs (target 57%). Sixty-nine and 73% of pregnant women attended those facilities’ first and fourth antenatal consultations (target 70%). All facilities adopted the intervention. 99% of the expected meetings occurred with an average interval of 5.9 out of 6 months. Participation of maternal and child managers was high, with 3076 attending meetings, of which 64% were from the facility, 29% from the district, and 7% from the province level. 97% of expected action plans were created, and 41 specific problems were identified. “Weak diagnosis or management of obstetric complications” was identified as the main problem, and “actions to reinforce norms and protocols” was the dominant subcategory of micro-interventions selected. Fidelity to semiannual readiness assessments was low (52% of expected facilities), and in completing micro-interventions (17% were completed). Ninety-six and 95% of facilities sustained the intervention at 12 and 24 months, respectively, and 71% had completed nine cycles at 54 months.

Conclusion

Maternal and child managers can lead audit and feedback processes in primary health care in Mozambique with high reach, adoption, and maintenance. The IDEAs strategy should be adapted to promote higher fidelity around implementing action plans and conducting readiness assessments. Adding effectiveness to these findings will help to inform strategy scale-up.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

✪ Stepped-Wedge Cluster Randomized Controlled Trial to Promote Option B+ Retention in Central Mozambique

Health facility factors and quality of services to prevent mother-to-child HIV transmission in Côte d’Ivoire, Kenya, and Mozambique

✪ Evaluation of a Systems Analysis and Improvement Approach to Optimize Prevention of Mother-To-Child Transmission of HIV Using the Consolidated Framework for Implementation Research

✪ Experiences in implementation and publication of operations research interventions: Gaps and a way forward

Abstract

Introduction

According to UNAIDS, the world currently has an adequate collection of proven HIV prevention, treatment and diagnostic tools, which, if scaled up, can lay the foundation for ending the AIDS epidemic. HIV operations research (OR) tests and promotes the use of interventions that can increase the demand for and supply of these tools. However, current publications of OR mainly focus on outcomes, leaving gaps in reporting of intervention characteristics, which are essential to address for the utilization of OR findings. This has prompted WHO and other international public health agencies to issue reporting requirements for OR studies. The objective of this commentary is to review experiences in HIV OR intervention design, implementation, process data collection and publication in order to identify gaps, contribute to the body of knowledge and propose a way forward to improve the focus on “implementation” in implementation research.

Discussion

Interventions in OR, like ordinary service delivery programmes, are subject to the programme cycle, which continually uses insights from implementation and the local context to modify service delivery modalities. Given that some of these modifications in the intervention may influence study outcomes, the documentation of process data becomes vital in OR. However, a key challenge is that study resources tend to be skewed towards documentation and the reporting of study outcomes to the detriment of process data, even though process data is vital for understanding factors influencing the outcomes.

Conclusions

Interventions in OR should be viewed using the lens of programme evaluation, which includes formative assessment (to determine concept and design), followed by process evaluation (to monitor inputs and outputs) and effectiveness evaluation (to assess outcomes and effectiveness). Study resources should be equitably used between process evaluation and outcome measurement to facilitate inclusion of data about fidelity and dose in publications in order to enable explanation of the relationship between dosing and study outcomes for purposes of scaling up and further refinement through research.

Authors:

✪ Open Access

Read the full text in the open access journal Implementation Science Communications

Abstract:

Background

Methods

Results

Conclusion

Authors:

✪ Open Access

Read the full text in the open access journal PLOS Global Public Health

Abstract:

Authors:

✪ Open Access

Read the full text in the open access journal BMC Health Services Research

Abstract:

Background

Methods

Results

Conclusion

Authors:

✪ Open Access

Read the full text in the subscription access JAIDS: Journal of Acquired Immune Deficiency Syndromes

Authors:

Subscription Required for Full Text Access

Read the full text in the subscription access International Journal of STD & AIDS

Authors:

✪ Open Access

Read the full text open access in the subscription journal JAIDS

Authors:

✪ Open Access

Read the full text open access in the Journal of the International AIDS Society

Abstract

Introduction

Discussion

Conclusions