Operations Research Archives - Implementation Science at UW

✪ Protocol for the process evaluation of a randomised clinical trial of incremental-start versus conventional haemodialysis: the TwoPlus study

Authors:

Mariana Murea, Kristie L Foley, Samir C Gautam, Jennifer E Flythe , Jochen G Raimann, Emaad Abdel-Rahman, Alaa S Awad, Vandana Dua Niyyar, Cassandra Kovach, Glenda V Roberts, Nicole M Jefferson, Paul T Conway, Laura M Rosales, Jobira Woldemichael, Hiba I Sheikh, Gaurav Raman, Anne M Huml, Daphne H Knicely, Irtiza Hasan, Bhaktidevi Makadia , Janice Lea, John T Daugirdas, Nihan Gencerliler, Jasmin Divers, Peter Kotanko, TwoPlus Research Consortium & Ucheoma C Nwaozuru

^{University of Washington affiliated authors are displayed in bold.}

✪ Open Access

Published: November 2025

Read the full text in the open access journal BMJ Open

Abstract:

Introduction

Process evaluation provides insight into how interventions are delivered across varying contexts and why interventions work in some contexts and not in others. This manuscript outlines the protocol for a process evaluation embedded in a hybrid type 1 effectiveness-implementation randomised clinical trial of incremental-start haemodialysis (HD) versus conventional HD delivered to patients starting chronic dialysis (the TwoPlus Study). The trial will simultaneously assess the effectiveness of incremental-start HD in real-world settings and the implementation strategies needed to successfully integrate this intervention into routine practice. This manuscript describes the rationale and methods used to capture how incremental-start HD is implemented across settings and the factors influencing its implementation success or failure within this trial.

Methods and analysis

We will use the Consolidated Framework for Implementation Research (CFIR) and the Reach, Effectiveness, Adoption, Implementation and Maintenance (RE-AIM) frameworks to inform process evaluation. Mixed methods include surveys conducted with treating providers (physicians) and dialysis personnel (nurses and dialysis administrators); semi-structured interviews with patient participants, caregivers of patient participants, treating providers (physicians and advanced practice practitioners), dialysis personnel (nurses, dieticians and social workers); and focus group meetings with study investigators and stakeholder partners. Data will be collected on the following implementation determinants: (a) organisational readiness to change, intervention acceptability and appropriateness; (b) inner setting characteristics underlying barriers and facilitators to the adoption of HD intervention at the enrollment centres; (c) external factors that mediate implementation; (d) adoption; (e) reach; (f) fidelity, to assess adherence to serial timed urine collection and HD treatment schedule; and (g) sustainability, to assess barriers and facilitators to maintaining intervention. Qualitative and quantitative data will be analysed iteratively and triangulated following a convergent parallel and pragmatic approach. Mixed methods analysis will use qualitative data to lend insight to quantitative findings. Process evaluation is important to understand factors influencing trial outcomes and identify potential contextual barriers and facilitators for the potential implementation of incremental-start HD into usual workflows in varied outpatient dialysis clinics and clinical practices. The process evaluation will help interpret and contextualise the trial clinical outcomes’ findings.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

✪ Looking under the hood of a hybrid two-way texting intervention to improve early retention on antiretroviral therapy in Malawi: an implementation fidelity evaluation

Abstract:

Background

While key to interpreting findings and assessing generalizability, implementation fidelity is underreported in mobile health (mHealth) literature. We evaluated implementation fidelity of an opt-in, hybrid, two-way texting (2wT) intervention previously demonstrated to improve 12-month retention on antiretroviral therapy (ART) among people living with HIV (PLHIV) in a quasi-experimental study in Lilongwe, Malawi.

Methods

Short message service (SMS) data and ART refill visit records were used to evaluate adherence to 2wT content, frequency and duration through the lens of the Conceptual Framework for Implementation Fidelity. Message delivery and 2wT participant interactions were considered across four core 2wT components: 1) weekly motivational SMS messages; 2) proactive SMS appointment reminders; 3) SMS reminders after missed appointments; and 4) interactive messaging with 2wT staff about transfers and appointment rescheduling. Using mixed-effects logistic regression models adjusted for participant demographics, we examined the effect of core 2wT component fidelity on a) on-time appointment attendance and b) timely return to care after a missed appointment, presenting adjusted odds ratios (aORs) and 95% confidence intervals (CIs).

Results

The 468 2wT participants had a median of 52 study weeks (interquartile range [IQR] 34 – 52) with 6 ART appointments (IQR 4—7) of which 2 (IQR 1 – 3) were missed. On average, participants received a motivation message for 75% (IQR 56%—83%) of enrolled weeks, a reminder before 83% (IQR 67%—100%) of appointments, and after 67% (IQR 0%—100%) of missed appointments. Participants reported 9 transfers and rescheduled 46 appointments through 2wT prompts; 196 appointments were changed via unprompted interaction. Participants with 10% higher expected motivation message delivery were more likely to attend clinic appointments on time (aOR: 1.08; 95%CI: 1.01 – 1.16, p = 0.03). Receiving and responding to an appointment reminder in any way were also associated with increased on-time appointment attendance (aOR: 1.35; 95%CI: 1.03 – 1.79, p = 0.03 and aOR: 1.47, 95%CI: 1.16 – 1.87, p = 0.001, respectively). No associations were found for 2wT messages and timely return to care following a missed appointment.

Conclusion

Greater 2wT implementation fidelity was associated with improved care outcomes. Although implementation fidelity monitoring of mHealth interventions is complex, it should be integrated into study design.

Contributions to the literature

Despite the recognized importance of implementation fidelity in digital health research, few studies evaluate and transparently report the extent to which mHealth interventions were implemented as intended.
This study presents a framework-based evaluation of the implementation fidelity of a two-way texting intervention for HIV retention in a routine care setting in Malawi. Findings demonstrate how fidelity assessment not only facilitates replication of findings and generalizability assessment, but uncovers implementation challenges and generates hypotheses about intervention mechanisms of action.
The benefits and complexity of implementation fidelity evaluation underscore the need to consider fidelity early and integrate monitoring mechanisms into study design.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

✪ Comparing Patients’ Experiences in Three Differentiated Service Delivery Models for HIV Treatment in South Africa

Abstract:

Differentiated service delivery for HIV treatment seeks to enhance medication adherence while respecting the preferences of people living with HIV. Nevertheless, patients’ experiences of using these differentiated service delivery models or approaches have not been qualitatively compared. Underpinned by the tenets of descriptive phenomenology, we explored and compared the experiences of patients in three differentiated service delivery models using the National Health Services’ Patient Experience Framework. Data were collected from 68 purposively selected people living with HIV receiving care in facility adherence clubs, community adherence clubs, and quick pharmacy pick-up. Using the constant comparative thematic analysis approach, we compared themes identified across the different participant groups. Compared to facility adherence clubs and community adherence clubs, patients in the quick pharmacy pick-up model experienced less information sharing; communication and education; and emotional/psychological support. Patients’ positive experience with a differentiated service delivery model is based on how well the model fits into their HIV disease self-management goals.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

✪ Development and Implementation of a Mobile Phone–Based Prevention of Mother-To-Child Transmission of HIV Cascade Analysis Tool: Usability and Feasibility Testing in Kenya and Mozambique

Abstract:

Background

Prevention of mother-to-child HIV transmission (PMTCT) care cascade failures drive pediatric HIV infections in sub-Saharan Africa. As nurses’ clinical and management role in PMTCT expand, decision-support tools for nurses are needed to facilitate identification of cascade inefficiencies and solutions. The mobile phone–based PMTCT cascade analysis tool (mPCAT) provides health facility staff a quick summary of the number of patients and percentage drop-off at each step of the PMCTC care cascade, as well as how many women-infant pairs would be retained if a step was optimized.

Objective

The objective of this study was to understand and improve the mPCAT’s core usability factors and assess the health workers’ experience with using the mPCAT.

Methods

Overall, 2 rounds of usability testing were conducted with health workers from 4 clinics and leading experts in maternal and child health in Kenya and Mozambique using videotaped think aloud assessment techniques. Semistructured group interviews gauged the understanding of mPCAT’s core usability factors, based on the Nielsen Usability Framework, followed by development of cognitive demand tables describing the needed mPCAT updates. Post adaptation, feasibility was assessed in 3 high volume clinics over 12 weeks. Participants completed a 5-point Likert questionnaire designed to measure ease of use, convenience of integration into work, and future intention to use the mPCAT. Focus group discussions with nurse participants at each facility and in-depth interviews with nurse managers were also conducted to assess the acceptability, use, and recommendations for adaptations of the mPCAT.

Results

Usability testing with software engineers enabled real-time feedback to build a tool following empathic design principles. The revised mPCAT had improved navigation and simplified data entry interface, with only 1 data entry field per page. Improvements to the results page included a data visualization feature and the ability to share results through WhatsApp. Coding was simplified to enable future revisions by nontechnical staff—critical for context-specific adaptations for scale-up. Health care workers and facility managers found the tool easy to use (mean=4.3), used the tool very often (mean=4.1), and definitely intended to continue to use the tool (mean=4.8). Ease of use was the most common theme identified, with emphasis on how the tool readily informed system improvement decision making.

Conclusions

The mPCAT was well accepted by frontline health workers and facility managers. The collaborative process between software developer and user led to the development of a more user-friendly, context-specific tool that could be easily integrated into routine clinical practice and workflow. The mPCAT gave frontline health workers and facility managers an immediate, direct, and tangible way to use their clinical documentation and routinely reported data for decision making for their own clinical practice and facility-level improvements.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

Integration of PrEP Services Into Routine Antenatal and Postnatal Care Experiences From an Implementation Program in Western Kenya

✪ Data-driven quality improvement in low-and middle-income country health systems: lessons from seven years of implementation experience across Mozambique, Rwanda, and Zambia

Abstract:

Background

Well-functioning health systems need to utilize data at all levels, from the provider, to local and national-level decision makers, in order to make evidence-based and needed adjustments to improve the quality of care provided. Over the last 7 years, the Doris Duke Charitable Foundation’s African Health Initiative funded health systems strengthening projects at the facility, district, and/or provincial level to improve population health. Increasing data-driven decision making was a common strategy in Mozambique, Rwanda and Zambia. This paper describes the similar and divergent approaches to increase data-driven quality of care improvements (QI) and implementation challenge and opportunities encountered in these three countries.

Methods

Eight semi-structured in-depth interviews (IDIs) were administered to program staff working in each country. IDIs for this paper included principal investigators of each project, key program implementers (medically-trained support staff, data managers and statisticians, and country directors), as well as Ministry of Health counterparts. IDI data were collected through field notes; interviews were not audio recorded. Data were analyzed using thematic analysis but no systematic coding was conducted. IDIs were supplemented through donor report abstractions, a structured questionnaire, one-on-one phone calls, and email exchanges with country program leaders to clarify and expand on key themes emerging from IDIs.

Results

Project successes ranged from over 450 collaborative action-plans developed, implemented, and evaluated in Mozambique, to an increase from <10% to >80% of basic clinical protocols followed in intervention facilities in rural Zambia, and a shift from a lack of awareness of health data among health system staff to collaborative ownership of data and using data to drive change in Rwanda.

Conclusion

Based on common successes across the country experiences, we recommend future data-driven QI interventions begin with data quality assessments to promote that rapid health system improvement is possible, ensure confidence in available data, serve as the first step in data-driven targeted improvements, and improve staff data analysis and visualization skills. Explicit Ministry of Health collaborative engagement can ensure performance review is collaborative and internally-driven rather than viewed as an external “audit.”

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

Measuring health systems strength and its impact: experiences from the African Health Initiative

Abstract:

Background

Health systems are essential platforms for accessible, quality health services, and population health improvements. Global health initiatives have dramatically increased health resources; however, funding to strengthen health systems has not increased commensurately, partially due to concerns about health system complexity and evidence gaps demonstrating health outcome improvements. In 2009, the African Health Initiative of the Doris Duke Charitable Foundation began supporting Population Health Implementation and Training Partnership projects in five sub-Saharan African countries (Ghana, Mozambique, Rwanda, Tanzania, and Zambia) to catalyze significant advances in strengthening health systems. This manuscript reflects on the experience of establishing an evaluation framework to measure health systems strength, and associate measures with health outcomes, as part of this Initiative.

Methods

Using the World Health Organization’s health systems building block framework, the Partnerships present novel approaches to measure health systems building blocks and summarize data across and within building blocks to facilitate analytic procedures. Three Partnerships developed summary measures spanning the building blocks using principal component analysis (Ghana and Tanzania) or the balanced scorecard (Zambia). Other Partnerships developed summary measures to simplify multiple indicators within individual building blocks, including health information systems (Mozambique), and service delivery (Rwanda). At the end of the project intervention period, one to two key informants from each Partnership’s leadership team were asked to list – in rank order – the importance of the six building blocks in relation to their intervention.

Results

Though there were differences across Partnerships, service delivery and information systems were reported to be the most common focus of interventions, followed by health workforce and leadership and governance. Medical products, vaccines and technologies, and health financing, were the building blocks reported to be of lower focus.

Conclusion

The African Health Initiative experience furthers the science of evaluation for health systems strengthening, highlighting areas for further methodological development – including the development of valid, feasible measures sensitive to interventions in multiple contexts (particularly in leadership and governance) and describing interactions across building blocks; in developing summary statistics to facilitate testing intervention effects on health systems and associations with health status; and designing appropriate analytic models for complex, multi-level open health systems.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

Tackling the hard problems: Implementation experience and lessons learned in newborn health from the African Health Initiative

Abstract:

Background

The Doris Duke Charitable Foundation’s African Health Initiative supported the implementation of Population Health Implementation and Training (PHIT) Partnership health system strengthening interventions in designated areas of five countries: Ghana, Mozambique, Rwanda, Tanzania, and Zambia. All PHIT programs included health system strengthening interventions with child health outcomes from the outset, but all increasingly recognized the need to increase focus to improve health and outcomes in the first month of life. This paper uses a case study approach to describe interventions implemented in newborn health, compare approaches, and identify lessons learned across the programs’ collective implementation experience.

Methods

Case studies were built using quantitative and qualitative methods, applying the World Health Organization Health Systems Strengthening Framework, and maternal, newborn and child health continuum of care framework. We identified the following five primary themes in health systems strengthening intervention strategies used to target improvement in newborn health, which were incorporated by all PHIT projects with varying results: health service delivery at the community level (Tanzania), combining community and health facility level interventions (Zambia), participatory information feedback and clinical training (Ghana), performance review and enhancement (Mozambique), and integrated clinical and system-level improvement (Rwanda), and used individual case studies to illustrate each of these themes.

Results

Tanzania and Zambia included significant community-based components, including mobilization and sensitization for increased uptake of essential services, while Ghana, Mozambique, and Rwanda focused more efforts on improving the quality of services delivered once a patient enters a health facility. All countries included aspects that improved communication across levels of the health system, whether through district-wide data sharing and peer learning networks in Mozambique and Rwanda, or improved referral processes and systems in Tanzania, Zambia, and Ghana.

Conclusion

Key lessons learned include the importance of focusing intervention components on addressing drivers of neonatal mortality across the maternal and newborn care continuum at all levels of the health system, matching efforts to improve service utilization with provision of high quality facility-based services, and the critical role of leadership to catalyze improvements in newborn health.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

Authors:

✪ Open Access

Read the full text in the open access journal BMJ Open

Abstract:

Introduction

Methods and analysis

Authors:

✪ Open Access

Read the full text in the open access journal Implementation Science

Abstract:

Background

Methods

Results

Conclusion

Contributions to the literature

Authors:

✪ Open Access

Read the full text in the open access journal Qualitative Health Research

Abstract:

Authors:

Read the full text in the open access journal JMIR mHealth and uHealth

Abstract:

Background

Objective

Methods

Results

Conclusions

Authors:

Read the full text in the subscription access journal JAIDS – Journal of Acquired Immune Deficiency Syndromes

Authors:

✪ Open Access

Read the full text in the open access journal BMC Health Services Research

Abstract:

Background

Methods

Results

Conclusion

Authors:

✪ Open Access

Read the full text in the open access journal BMC Health Services Research

Abstract:

Background

Methods

Results

Conclusion

Authors:

✪ Open Access

Read the full text in the open access journal BMC Health Services Research

Abstract:

Background

Methods

Results

Conclusion

Authors:

✪ Open Access

Read the full text in the subscription access JAIDS: Journal of Acquired Immune Deficiency Syndromes

Authors:

Subscription Required for Full Text Access

Read the full text in the subscription access International Journal of STD & AIDS