difference-in-differences Archives - Implementation Science at UW

✪ Impact of a stakeholder selected implementation strategy package – fast tracking, provider re-training, and co-location – on PrEP implementation for pregnant women in antenatal care clinics in western Kenya

Abstract:

Background

Pre-exposure prophylaxis (PrEP) is recommended for HIV prevention in pregnant and postpartum women at substantial ongoing risk for HIV. In resource-limited settings, there exist gaps in the integration of PrEP into antenatal care.

Methods

We conducted a difference-in-differences analytic approach (3 months pre- and 3 months post) between January 2022 and July 2022 in 8 facilities (4 intervention and 4 comparison) in western Kenya. During the 6-month period, we tested a combination of 2 stakeholder selected implementation strategies – retraining health providers and fast tracking PrEP clients– to improve PrEP delivery. All study facilities dispensed PrEP in the Maternal and Child health clinics (MCH). We evaluated absolute changes in: PrEP penetration, PrEP fidelity, client PrEP knowledge, client satisfaction, and client waiting and service times as outcomes specified a priori while PrEP offer and HIV testing were outcomes specified post hoc. We measured acceptability and appropriateness by providers of the implementation strategies using AIM and IAM respectively.

Results

We observed statistically significant improvements in PrEP penetration and PrEP offer (p < 0.05) and non-significant improvements in fidelity. PrEP penetration increased 6 percent points (p = 0.002), PrEP offer increased nearly 6 percentage points (p = 0.002), and PrEP fidelity increased 4 percentage points (p = 0.202) in intervention vs comparison facilities. Client PrEP knowledge increased 0.45 out of 7 total points (p < 0.001) and PrEP screening increased 13 percentage points (p = 0.001). We observed no significant changes in service time (0.13-min increase; p = 0.249), waiting time (0.03-min decrease; p = 0.796), or client satisfaction (0.04/24 total point decrease; p = 0.849) in intervention vs comparison facilities. HIV testing did not significantly change (7 percentage point decrease, p = 0.305). The implementation strategy bundle was deemed appropriate and acceptable by the providers (appropriateness: 18.5/20; acceptability: 18.5/20). Overall, the implementation strategy bundle was associated with larger increases in implementation outcomes among women receiving a visit other than their first ANC visit, as well as among sites without stockouts of HIV test kits.

Conclusions

A stakeholder-selected implementation strategy bundle that included retraining healthcare workers, fast tracking PrEP clients to reduce waiting time, and PrEP dispensing in MCH improved several implementation outcomes without significantly affecting waiting time or reducing service time.

Contributions to the literature

Evidence is sparse on effective implementation strategies that focus on structural- and provider-level barriers to enhance delivery of integrated care in resource-limited settings.
Stakeholder-selected strategies are hypothesized to have a better fit to local context and potentially be more effective than externally selected strategies.
We tested a package of two stakeholder-selected implementation strategies – retraining health providers, fast tracking PrEP clients in a difference-in-differences study in 8 clinics in Kenya.
The package was associated with significant improvements in PrEP penetration, PrEP offer, PrEP knowledge, and PrEP screening. It was not associated with significant changes in PrEP fidelity, HIV testing, service time, waiting time, or client satisfaction.
The improvement associated with the package was most pronounced among clinics that did not experience substantial stockouts of supplies, suggesting that implementation strategies are not sufficient to overcome gaps in basic resourcing in the presence of critical events.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

✪ Can audit and feedback improve health service readiness and delivery outcomes in a low-resource setting? Effectiveness results of the IDEAs strategy from central Mozambique

Abstract:

Limited evidence exists on audit and feedback (A&F) in low-resource contexts. We tested the Integrated District Evidence-to-Action (IDEAs), a multicomponent A&F strategy in Mozambique’s maternal and child health (MCH) services. IDEAs include A&F meetings, readiness assessments, and facility support. We report the effectiveness results. IDEAs were implemented in 2016–2020 across two provinces, 12 districts, and 154 primary health facilities in Mozambique. We assessed 1) ten service delivery outcomes across antenatal, maternity, postpartum, childcare, and reproductive health services and 2) five service readiness outcomes (medicines, infrastructure, equipment, care, and staffing availability). We used propensity score matching to minimize bias and a controlled interrupted time series with a negative binomial mixed effects model for service delivery analysis, presenting incidence rate ratios (IRR) with 95% confidence intervals (95% CI). For service readiness outcomes, we created composite scores for each domain and a difference-in-difference analysis using an ordinal mixed effects model, reporting odds ratios (OR) and 95% CI. Significant associations were found with first at-risk child appointments (IRR = 1.06 [1.04, 1.07]), first polymerase chain reaction tests for HIV-exposed children (IRR = 1.02 [1.01, 1.03]), new contraceptive users (IRR = 0.95 [0.94, 0.96]), women starting long-lasting contraceptives (IRR = 0.94 [0.93, 0.95]), availability of infrastructure (OR = 5.84 [1.32, 25.88]) and essential care (OR = 0.13 [0.03, 0.54]). No significant associations were found between IDEAs and six of 10 service delivery outcomes (women with a fourth dose of preventive malaria treatment; women protected with tetanus vaccine; women with four+ antenatal visits; deliveries with active management of the third stage of labor; first postpartum consultations and fully vaccinated children) and on medicine, equipment, and staffing availability. We observed mixed effectiveness in implementing IDEAs, with null and sub-optimal effects suggesting the need to refine and adapt strategy components to more effectively address clinical and readiness outcomes.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

Design of a pragmatic trial integrating human papillomavirus (HPV) self-sampling into primary care to reduce cervical cancer screening disparities in Somali American individuals: The Isbaar project

✪ Integrating early child development into an existing health and nutrition program: evidence from a cluster-randomized controlled trial

Abstract:

Introduction

In low-resource settings, introducing child health programs into community services may compete for caregiver time. We analyzed the impact of a new early childhood development (ECD) program in rural Madagascar on family attendance at other health services and of adding at-home play materials on program attendance.

Methods

We randomized 75 communities where community health workers (CHWs) implement an existing child health and nutrition program (Projet d’Amélioration des Résultats Nutritionnels or PARN), the status quo. We offered two 6-month cycles of 12 ECD sessions to eligible caregiver-child dyads (6–30 months) in 25 sites [T]; we added take-home play materials in Cycle 2 to 25 sites [T+]. We used differences-in-differences with administrative data to analyze the effect of offering ECD sessions on monthly PARN attendance (T+/T vs. C) among age-eligible children and the impact of toy boxes/libraries on monthly ECD session attendance (T + vs. T). We used random intercept models to analyze characteristics associated with program registration.

Results

We analyzed data for 9,408 dyads; 30% and 32% registered for the program in Cycle 1 and 2 (respectively). On average, CHWs delivered 11.4 sessions (SD: 1.5). Children from wealthier households who already attended PARN sessions were more likely to register, and we found no effect of T or T + on PARN attendance. Adding play materials did not affect monthly ECD session attendance. Children from more populated sites were less likely to participate in both ECD and PARN sessions.

Conclusions

Integrating new services for ECD into the health system was feasible and did not reduce dyad participation in existing services. Investment in health services in more populated areas is needed to provide coverage to all eligible children. Novel strategies should be explored to engage the most vulnerable children in new and existing health services.

Trial registration

AEA Social Science Registry (AEARCTR-0004704) on November 15, 2019 (prospective registration) and ClinicalTrials.gov (NCT05129696) on November 22, 2021 (retrospective registration).

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

✪ An implementation strategy package (video education, HIV self-testing, and co-location) improves PrEP implementation for pregnant women in antenatal care clinics in western Kenya

Abstract:

Background

Pre-exposure prophylaxis (PrEP) is recommended by the World Health Organization and the Kenyan Ministry of Health for HIV prevention in pregnancy and postpartum for women at risk for HIV. Integration of PrEP into antenatal care is promising, but delivery gaps exist in the face of healthcare provider shortages in resource-limited settings.

Methods

Between May and November 2021, we conducted a difference-in-differences study (3 months pre-intervention data collection and 3 months post-intervention data collection) analyzing four intervention facilities, where the strategies were implemented, and four comparison facilities, where no strategies were implemented. We tested a combination of three implementation strategies—video-based PrEP information in the waiting bay, HIV self-testing, and dispensing of PrEP in the antenatal care rooms—to improve PrEP delivery. We compared absolute changes in the proportion of antenatal attendees screened for PrEP (PrEP penetration), the proportion receiving all PrEP-specific steps in a visit (HIV testing, risk screening, and PrEP counseling) (PrEP fidelity), and client PrEP knowledge, client satisfaction, and waiting time and service time (a priori outcomes); post hoc, we compared the proportion offered PrEP (PrEP offer) and completing HIV testing. We measured provider perceptions of the acceptability and appropriateness of the implementation strategies.

Results

We observed significant improvements in PrEP penetration, PrEP offer, satisfaction, and knowledge (p < 0.05) and improvements in fidelity that trended towards significance (p = 0.057). PrEP penetration increased 5 percentage points (p = 0.008), PrEP fidelity increased 8 percentage points (p = 0.057), and PrEP offer increased 4 percentage points (p = 0.003) in intervention vs. comparison facilities. Client PrEP knowledge increased by 1.7 out of 6 total points (p < 0.001) and client satisfaction increased by 0.7 out of 24 total points (p = 0.003) in intervention vs. comparison facilities. We observed no changes in service time (0.09-min decrease; p = 0.435) and a small increase in waiting time (0.33-min increase; p = 0.005). HIV testing among those eligible did not change (1.5 percentage point decrease, p = 0.800). Providers felt the implementation strategies were acceptable and appropriate (median acceptability: 20/20; median appropriateness: 19.5/20). However, absolute levels of each step of the PrEP cascade remained suboptimal

Conclusions

An implementation strategy package with video information, HIV self-testing, and co-location of medication dispensing enhanced PrEP delivery across several implementation outcomes and client satisfaction, while not substantially increasing wait time or decreasing provider-client contact time.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

✪ Selecting implementation strategies to improve implementation of integrated PrEP for pregnant and postpartum populations in Kenya: a sequential explanatory mixed methods analysis

Abstract:

Background

There is a higher risk for HIV acquisition during pregnancy and postpartum. Pre-exposure prophylaxis (PrEP) is recommended during this period for those at high risk of infection; integrated delivery in maternal and child health (MCH) clinics is feasible and acceptable but requires implementation optimization.

Methods

The PrEP in Pregnancy, Accelerating Reach and Efficiency study (PrEPARE; NCT04712994) engaged stakeholders to prioritize determinants of PrEP delivery (using Likert scores) and prioritize PrEP delivery implementation strategies. Using a sequential explanatory mixed methods design, we conducted quantitative surveys with healthcare workers at 55 facilities in Western Kenya and a stakeholder workshop (including nurses, pharmacists, counselors, and county and national policymakers), yielding visual plots of stakeholders’ perceived feasibility and effectiveness of the strategies. A stepwise elimination process was used to identify seven strategies for empirical testing. Facilitator debriefing reports from the workshop were used to qualitatively assess the decision-making process.

Results

Among 146 healthcare workers, the strongest reported barriers to PrEP delivery were insufficient providers and inadequate training, insufficient space, and high volume of patients. Sixteen strategies were assessed, 14 of which were included in the final analysis. Among rankings from 182 healthcare workers and 44 PrEP policymakers and implementers, seven strategies were eliminated based on low post-workshop ranking scores (bottom 50th percentile) or being perceived as low feasibility or low effectiveness for at least 50% of the workshop groups. The top seven strategies included delivering PrEP within MCH clinics instead of pharmacies, fast-tracking PrEP clients to reduce waiting time, delivering PrEP-related health talks in waiting bays, task shifting PrEP counseling, task shifting PrEP risk assessments, training different providers to deliver PrEP, and retraining providers on PrEP delivery. All top seven ranked strategies were grouped into bundles for subsequent testing. Facilitator debriefing reports generally aligned with rankings but noted how stakeholders’ decision-making changed when considering the impact of strategies on facility staff and non-PrEP clients.

Conclusions

The most impactful barriers to integrated PrEP delivery in MCH clinics were insufficient staffing and space. Implementation strategies prioritized through multiple methods of stakeholder input focused on co-location of services and increasing clinic efficiency. Future testing of these stakeholder-prioritized strategy bundles will be conducted to assess the effectiveness and implementation outcomes.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

Authors:

✪ Open Access

Read the full text in the open access journal Implementation Science Communications

Abstract:

Background

Methods

Results

Conclusions

Contributions to the literature

Authors:

✪ Open Access

Read the full text in the open access journal PLOS Global Public Health

Abstract:

Authors:

Subscription Required for Full Text Access

Read the full text in the subscription access journal Contemporary Clinical Trials

Authors:

✪ Open Access

Read the full text in the open access journal BMC Public Health

Abstract:

Introduction

Methods

Results

Conclusions

Trial registration

Authors:

✪ Open Access

Read the full text in the open access journal Frontiers in Reproductive Health

Abstract:

Background

Methods

Results

Conclusions

Authors:

✪ Open Access

Read the full text in the open access journal Implementation Science Communications

Abstract:

Background

Methods

Results

Conclusions