quasi-experimental design Archives - Implementation Science at UW

✪ Looking under the hood of a hybrid two-way texting intervention to improve early retention on antiretroviral therapy in Malawi: an implementation fidelity evaluation

Abstract:

Background

While key to interpreting findings and assessing generalizability, implementation fidelity is underreported in mobile health (mHealth) literature. We evaluated implementation fidelity of an opt-in, hybrid, two-way texting (2wT) intervention previously demonstrated to improve 12-month retention on antiretroviral therapy (ART) among people living with HIV (PLHIV) in a quasi-experimental study in Lilongwe, Malawi.

Methods

Short message service (SMS) data and ART refill visit records were used to evaluate adherence to 2wT content, frequency and duration through the lens of the Conceptual Framework for Implementation Fidelity. Message delivery and 2wT participant interactions were considered across four core 2wT components: 1) weekly motivational SMS messages; 2) proactive SMS appointment reminders; 3) SMS reminders after missed appointments; and 4) interactive messaging with 2wT staff about transfers and appointment rescheduling. Using mixed-effects logistic regression models adjusted for participant demographics, we examined the effect of core 2wT component fidelity on a) on-time appointment attendance and b) timely return to care after a missed appointment, presenting adjusted odds ratios (aORs) and 95% confidence intervals (CIs).

Results

The 468 2wT participants had a median of 52 study weeks (interquartile range [IQR] 34 – 52) with 6 ART appointments (IQR 4—7) of which 2 (IQR 1 – 3) were missed. On average, participants received a motivation message for 75% (IQR 56%—83%) of enrolled weeks, a reminder before 83% (IQR 67%—100%) of appointments, and after 67% (IQR 0%—100%) of missed appointments. Participants reported 9 transfers and rescheduled 46 appointments through 2wT prompts; 196 appointments were changed via unprompted interaction. Participants with 10% higher expected motivation message delivery were more likely to attend clinic appointments on time (aOR: 1.08; 95%CI: 1.01 – 1.16, p = 0.03). Receiving and responding to an appointment reminder in any way were also associated with increased on-time appointment attendance (aOR: 1.35; 95%CI: 1.03 – 1.79, p = 0.03 and aOR: 1.47, 95%CI: 1.16 – 1.87, p = 0.001, respectively). No associations were found for 2wT messages and timely return to care following a missed appointment.

Conclusion

Greater 2wT implementation fidelity was associated with improved care outcomes. Although implementation fidelity monitoring of mHealth interventions is complex, it should be integrated into study design.

Contributions to the literature

Despite the recognized importance of implementation fidelity in digital health research, few studies evaluate and transparently report the extent to which mHealth interventions were implemented as intended.
This study presents a framework-based evaluation of the implementation fidelity of a two-way texting intervention for HIV retention in a routine care setting in Malawi. Findings demonstrate how fidelity assessment not only facilitates replication of findings and generalizability assessment, but uncovers implementation challenges and generates hypotheses about intervention mechanisms of action.
The benefits and complexity of implementation fidelity evaluation underscore the need to consider fidelity early and integrate monitoring mechanisms into study design.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

✪ Systems analysis and improvement to optimize opioid use disorder care quality and continuity for patients exiting jail (SAIA-MOUD)

Abstract:

Background

Between 2012–2022 opioid-related overdose deaths in the United States, including Washington State, have risen dramatically. Opioid use disorder (OUD) is a complex, chronic, and criminalized illness with biological, environmental, and social causes. One-fifth of people with OUD have recent criminal-legal system involvement; > 50% pass through WA jails annually.

Medications for Opioid Use Disorder (MOUD) can effectively treat OUD. WA has prioritized improving access to MOUD, including for those in jails. As patients in jail settings are systematically marginalized due to incarceration, it is critical to foster connections to MOUD services upon release, an acknowledged period of high overdose risk. Currently, there is insufficient focus on developing strategies to foster linkages between jail-based MOUD and referral services.

The Systems Analysis and Improvement Approach (SAIA), an evidence-based implementation strategy, may optimize complex care cascades like MOUD provision and improve linkages between jail- and community-based providers. SAIA bundles systems engineering tools into an iterative process to guide care teams to visualize cascade drop-offs and prioritize steps for improvement; identify modifiable organization-level bottlenecks; and propose, implement, and evaluate modifications to overall cascade performance. The SAIA-MOUD study aims to strengthen the quality and continuity of MOUD care across jail and referral clinics in King County, WA, and ultimately reduce recidivism and mortality.

Methods

We will conduct a quasi-experimental evaluation of SAIA effectiveness on improving MOUD care cascade quality and continuity for patients receiving care in jail and exiting to referral clinics; examine determinants of SAIA-MOUD adoption, implementation, and sustainment; and determine SAIA-MOUD’s cost and cost-effectiveness. Clinic teams with study team support will deliver the SAIA-MOUD intervention at the jail-based MOUD program and three referral clinics over a two-year intensive phase, followed by a one-year sustainment phase where SAIA implementation will be led by King County Jail MOUD staff without study support to enable pragmatic evaluation of sustained implementation.

Discussion

SAIA packages user-friendly systems engineering tools to guide decision-making by front-line care providers to identify low-cost, contextually appropriate health care improvement strategies. By integrating SAIA into MOUD care provision in jail and linked services, this pragmatic trial is designed to test a model for national scale-up.

Trial registration

ClinicalTrials.gov NCT06593353 (registered 09/06/2024).

Contributions to the literature

Our study examines whether the introduction of an evidence-based, user-friendly, low-cost package of systems engineering tools can optimize investments in providing medications for opioid use disorder for individuals during the high-risk period immediately after exiting jail.
This is the first application of the Systems Analysis and Improvement Approach (SAIA) across health system levels to optimize both care delivery and continuity of care.
Our study is designed to provide decision-makers with evidence they need to decide whether to invest in systems engineering approaches during the expansion of MOUD in jail settings, including information on acceptability, feasibility, effectiveness and cost.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

Peer Review reports

✪ Integrating causal pathway diagrams into practice facilitation to address colorectal cancer screening disparities in primary care

Abstract:

Background

Colorectal cancer (CRC) is the second leading cause of cancer death and the second most common cancer diagnosis among the Hispanic population in the United States. However, CRC screening prevalence remains lower among Hispanic adults than among non-Hispanic white adults. To reduce CRC screening disparities, efforts to implement CRC screening evidence-based interventions in primary care organizations (PCOs) must consider their potential effect on existing screening disparities. More research is needed to understand how to leverage existing implementation science methodologies to improve health disparities. The Coaching to Improve Colorectal Cancer Screening Equity (CoachIQ) pilot study explores whether integrating two implementation science tools, Causal Pathway Diagrams and practice facilitation, is a feasible and effective way to address CRC screening disparities among Hispanic patients.

Methods

We used a quasi-experimental, mixed methods design to evaluate feasibility and assess initial signals of effectiveness of the CoachIQ approach. Three PCOs received coaching from CoachIQ practice facilitators over a 12-month period. Three non-equivalent comparison group PCOs received coaching during the same period as participants in a state quality improvement program. We conducted descriptive analyses of screening rates and coaching activities.

Results

The CoachIQ practice facilitators discussed equity, facilitated prioritization of QI activities, and reviewed CRC screening disparities during a higher proportion of coaching encounters than the comparison group practice facilitator. While the mean overall CRC screening rate in the comparison PCOs increased from 34 to 41%, the mean CRC screening rate for Hispanic patients did not increase from 30%. In contrast, the mean overall CRC screening rate at the CoachIQ PCOs increased from 41 to 44%, and the mean CRC screening rate for Hispanic patients increased from 35 to 39%.

Conclusions

The CoachIQ program merges two implementation science methodologies, practice facilitation and causal pathway diagrams, to help PCOs focus quality improvement efforts on improving CRC screening while also reducing screening disparities. Results from this pilot study demonstrate key differences between CoachIQ facilitation and standard facilitation, and point to the potential of the CoachIQ approach to decrease disparities in CRC screening.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

✪ The effect of proactive, interactive, two-way texting on 12-month retention in antiretroviral therapy: Findings from a quasi-experimental study in Lilongwe, Malawi

Abstract:

Background

Retaining clients on antiretroviral therapy (ART) is challenging, especially during the first year on ART. Mobile health (mHealth) interventions show promise to close retention gaps. We aimed to assess reach (who received the intervention?) and effectiveness (did it work?) of a hybrid two-way texting (2wT) intervention to improve ART retention at a large public clinic in Lilongwe, Malawi.

Methods

Between August 2021—June 2023, in a quasi-experimental study, outcomes were compared between two cohorts of new ART clients: 1) those opting into 2wT who received automated, weekly motivation short messaging service (SMS) messages and response-requested appointment reminders; and 2) a matched historical cohort receiving standard of care (SoC). Reach was defined as “the proportion clients ≤6 months of ART initiation eligible for 2wT”. 2wT effectiveness was assessed in time-to-event analysis. Retention was presented in a Kaplan-Meier plot and compared between 2wT and SoC using a log-rank test. The effect of 2wT on ART dropout (lost to follow-up or stopped ART) was estimated using Fine-Gray competing risk regression models, adjusting for sex, age and WHO HIV stage at ART initiation.

Results

Of 1,146 clients screened, 501 were eligible for 2wT, a reach of 44%. Lack of phone (393/645; 61%) and illiteracy (149/645; 23%) were the most common ineligibility reasons. Among 468 participants exposed to 2wT, 12-month probability of ART retention was 91% (95% CI: 88% - 94%) compared to 76% (95% CI: 72% - 80%) among 468 SoC participants (p<0.001). Compared to SoC, 2wT participants had a 65% lower hazard of ART dropout at any timepoint (sub-distribution hazard ratio 0.35, 95% CI: 0.24–0.51; p<0.001).

Conclusions

2wT did not reach all clients. For those who opted-in, 2wT significantly increased 12-month ART retention. Expansion of 2wT as a complement to other retention interventions should be considered in other low-resource, routine ART settings.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

✪ Strengthening integrated depression services within routine primary health care using the RE-AIM framework in South Africa

Authors:

Inge Petersen, Christopher G. Kemp, Deepa Rao, Bradley H. Wagenaar, Max Bachmann, Kenneth Sherr, Tasneem Kathree, Zamasomi Luvuno, André Van Rensburg, Sithabisile Gugulethu Gigaba, Londiwe Mthethwa, Merridy Grant, One Selohilwe, Nikiwe Hongo, Gillian Faris, Christy-Joy Ras, Lara Fairall, Sanah Bucibo, & Arvin Bhana

^{University of Washington affiliated authors are displayed in bold.}

✪ Open Access

Published: November 2023

Read the full text in the open access journal PLOS Global Public Health

Abstract:

Integration of mental health into routine primary health care (PHC) services in low-and middle-income countries is globally accepted to improve health outcomes of other conditions and narrow the mental health treatment gap. Yet implementation remains a challenge. The aim of this study was to identify implementation strategies that improve implementation outcomes of an evidence-based depression care collaborative implementation model integrated with routine PHC clinic services in South Africa. An iterative, quasi-experimental, observational implementation research design, incorporating the Reach, Effectiveness, Adoption, Implementation and Maintenance (RE-AIM) framework, was applied to evaluate implementation outcomes of a strengthened package of implementation strategies (stage two) compared with an initial evaluation of the model (stage one). The first stage package was implemented and evaluated in 10 PHC clinics and the second stage strengthened package in 19 PHC clinics (inclusive of the initial 10 clinics) in one resource-scarce district in the province of KwaZulu-Natal, South Africa. Diagnosed service users were more likely to be referred for counselling treatment in the second stage compared with stage one (OR 23.15, SE = 18.03, z = 4.04, 95%CI [5.03–106.49], p < .001). Training in and use of a validated, mandated mental health screening tool, including on-site educational outreach and technical support visits, was an important promoter of nurse-level diagnosis rates (OR 3.75, 95% CI [1.19, 11.80], p = 0.02). Nurses who perceived the integrated care model as acceptable were also more likely to successfully diagnose patients (OR 2.57, 95% CI [1.03–6.40], p = 0.043). Consistent availability of a clinic counsellor was associated with a greater probability of referral (OR 5.9, 95%CI [1.29–27.75], p = 0.022). Treatment uptake among referred service users remained a concern across both stages, with inconsistent co-located counselling services associated with poor uptake. The importance of implementation research for strengthening implementation strategies along the cascade of care for integrating depression care within routine PHC services is highlighted.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

An EMR-Based Alert with Brief Provider-Led ART Adherence Counseling: Promising Results of the InfoPlus Adherence Pilot Study Among Haitian Adults with HIV Initiating ART

✪ An integrated community health worker intervention in rural Nepal: A type 2 hybrid effectiveness-implementation study protocol

Abstract:

Background

Evidence-based medicines, technologies, and protocols exist to prevent many of the annual 300,000 maternal, 2.7 million neonatal, and 9 million child deaths, but they are not being effectively implemented and utilized in rural areas. Nepal, one of South Asia’s poorest countries with over 80% of its population living in rural areas, exemplifies this challenge.
Community health workers are an important cadre in low-income countries where human resources for health and health care infrastructure are limited. As local women, they are uniquely positioned to understand and successfully navigate barriers to health care access. Recent case studies of large community health worker programs have highlighted the
importance of training, both initial and ongoing, and accountability through structured management, salaries, and ongoing monitoring and evaluation. A gap in the evidence regarding whether such community health worker systems can change health outcomes, as well as be sustainably adopted at scale, remains. In this study, we plan to evaluate a community
health worker system delivering an evidence-based integrated reproductive, maternal, newborn, and child health intervention as it is scaled up in rural Nepal.

Methods

We will conduct a type 2 hybrid effectiveness-implementation study to test both the effect of an integrated reproductive, maternal, newborn, and child health intervention and the implementation process via a professional community health worker system. The intervention integrates five evidence-based approaches: (1) home-based antenatal care and post-natal
care counseling and care coordination; (2) continuous surveillance of all reproductive age women, pregnancies, and children under age 2 years via a mobile application; (3) Community-Based Integrated Management of Newborn and Childhood Illness; (4) group antenatal and postnatal care; and 5) the Balanced Counseling Strategy to post-partum
contraception. We will evaluate effectiveness using a pre-post quasi-experimental design with stepped implementation and implementation using the RE-AIM framework.

Discussion

This is the first hybrid effectiveness-implementation study of an integrated reproductive, maternal, newborn, and child health intervention in rural Nepal that we are aware of. As Nepal takes steps towards achieving the Sustainable Development Goals, the data from this three-year study will be useful in the detailed planning of a professionalized
community health worker cadre delivering evidence-based reproductive, maternal, newborn, and child health interventions to the country’s rural population.

Trial registration

ClinicalTrials.gov Identifier: NCT03371186, registered 04 December 2017, retrospectively registered.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

The role of implementation science training in global health: from the perspective of graduates of the field’s first dedicated doctoral program

Abstract:

Bridging the ‘know-do gap’ is an enormous challenge for global health practitioners. They must be able to understand local health dynamics within the operational and social contexts that engender them, test and adjust approaches to implementation in collaboration with communities and stakeholders, interpret data to inform policy decisions, and design adaptive and resilient health systems at scale. These skills and methods have been formalized within the nascent field of Implementation Science (IS). As graduates of the world's first PhD program dedicated explicitly to IS, we have a unique perspective on the value of IS and the training, knowledge, and skills essential to bridging the ‘know-do gap’. In this article, we describe the philosophy and curricula at the core of our program, outline the methods vital to IS in a global health context, and detail the role that we believe IS will increasingly play in global health practice. At this junction of enormous challenges and opportunities, we believe that IS offers the necessary tools for global health professionals to address complex problems in context and raises the bar of success for the global health programs of the future.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

Authors:

✪ Open Access

Read the full text in the open access journal Implementation Science

Abstract:

Background

Methods

Results

Conclusion

Contributions to the literature

Authors:

✪ Open Access

Read the full text in the open access journal Implementation Science

Abstract:

Background

Methods

Discussion

Trial registration

Contributions to the literature

Authors:

✪ Open Access

Read the full text in the open access journal BMC Health Services Research

Abstract:

Background

Methods

Results

Conclusions

Authors:

✪ Open Access

Read the full text in the open access journal PLOS One

Abstract:

Background

Methods

Results

Conclusions

Authors:

✪ Open Access

Read the full text in the open access journal PLOS Global Public Health

Abstract:

Authors:

Subscription Required for Full Text Access

Read the full text in the subscription access journal AIDS and Behavior

Authors:

✪ Open Access

Read the full text in the open access journal Implementation Science

Abstract:

Background

Methods

Discussion

Trial registration

Authors:

✪ Open Access

Read the full text in the open access journal Implementation Science

Abstract: