Caryl Feldacker Archives - Implementation Science at UW

✪ When evidence is not enough: A qualitative exploration of healthcare workers’ perspectives on expansion of two-way texting (2wT) for post- circumcision follow-up in South Africa

Abstract:

As per South African national guidelines, in-person follow-up visits after voluntary medical male circumcision (VMMC) are required but may be unnecessary. Two-way texting (2wT), an mHealth platform, engages clients in post-operative care and triages those with complications to in-person review. 2wT was found to be safe, effective, and efficient. In South Africa, to understand provider perspectives on 2wT and potential for expansion, 20 key informant interviews were conducted with management, clinicians, data officials and support staff involved in 2wT scale-up. Interviews were analyzed using rapid qualitative methods and informed by two implementation science frameworks: the Reach, Effectiveness, Adoption, Implementation and Maintenance (RE-AIM) framework and the Pragmatic, Robust, Implementation and Sustainability Model (PRISM). Participants shared mixed and multi-faceted feedback, including that 2wT improves monitoring and evaluation of clients and clinical outcomes while also reducing follow-up visits. Challenges included duplicative routine and 2wT reporting systems and perceptions that 2wT increased workload. To improve the likelihood of successful 2wT scale-up in routine VMMC settings, participants suggested: further 2wT sensitization to ensure clinician and support staff buy-in; a dedicated clinician or nurse to manage telehealth clients; improved dashboards to better visualize 2wT client data; mobilizing 2wT champions at facilities to garner support for 2wT as routine care; and updating VMMC guidelines to support VMMC telehealth. As attendance at follow-up visits may not be as high as reported, implementing 2wT may require more effort but also brings added benefits of client verification and documented follow-up. The transition from research to routine practice is challenging, but use of RE-AIM and PRISM indicate that it is not impossible. As VMMC funding is decreasing, more effort to share the evidence base for 2wT as a safe, cost-effective, high-quality approach for VMMC follow-up is needed to encourage widespread uptake and adoption.

Author summary

South Africa has a high prevalence of HIV, and voluntary medical male circumcision is a prevention method recommended by the World Health Organization. Two-way texting (2wT) is a digital health intervention that allows males to receive educational messages after circumcision about healing and send messages directly to a provider via SMS or WhatsApp. This system detects as many complications as routine follow-up care, though is more efficient. As we scale up to new locations in South Africa, we wanted to ensure the approach is well-adapted for health workers. Through 20 interviews with health workers, we found that it saves clients in-person visits and helps with data collection. However, more effort is needed to introduce the intervention to clinicians, including providing better training and appointing champions to help motivate clinicians. Funding for HIV and VMMC programs is decreasing, so it is important to find tools such as digital health to reduce the burden on facilities while maintaining quality client care.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

✪ “If he returns, receive him because he has realized that he needs assistance”: A qualitative study exploring preferences for retention on antiretroviral therapy support in Malawi among Lighthouse Clinic clients

Abstract:

Introduction

Retention of people living with HIV (PLHIV) on antiretroviral therapy (ART) is critical. Retention is not static: clients cycle in and out of care over the course of treatment. Retention support is also evolving, with interventions like mobile health (mHealth) gaining traction. We aimed to explore ART clients’ perspectives and recommend strategies for improved retention support including a two-way-texting (2wT) mHealth intervention in two clinics in Malawi.

Materials and methods

We conducted focus group discussions (FGDs) with PLHIV on ART receiving 2wT or standard of care (SoC) retention support to understand client perspectives. FGDs were audio-recorded in Chichewa, translated and transcribed into English. We used the RE-AIM implementation science framework to report findings on the reach, effectiveness, adoption, implementation, and maintenance potential of retention interventions considering individual, clinic/organization, and community factors. Through rapid qualitative analysis, we identified key themes and subthemes.

Results

Ten FGDs were conducted with 89 ART clients (aged ≥18 years; 55% female): four FGDs among 2wT participants and six among SoC participants. Few differences were observed between 2wT and SoC clients. All clients appreciated accessing at least one form of support and recommended continuous assistance, regardless of age, sex, or duration on ART. Clients wished to be encouraged by other PLHIV to allay common fears. Groups discussed the need for intensified support to retain young clients or those out of care longer. Individual responsibility was identified as necessary, but insufficient, to improve retention: respondents desired a more positive clinic environment to encourage persistence in care. Fear of stigma, unintended status disclosure, and negative interactions with clinic staff prevent some clients from returning to care.

Conclusion

PLHIV want differentiated, continuous retention support from other PLHIV coupled with a positive clinic environment to promote retention and reengagement in care. As differentiated retention support is expensive, consideration of cost, feasibility, and sustainability is needed.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

✪ Finding the balance between rigour and relevance: implementing adaptations to the implementation of a pragmatic randomised controlled trial of a two-way texting intervention for voluntary medical male circumcision in South Africa

Abstract:

Objectives

To document adaptations that were made to the implementation of the two-way texting (2wT) randomised controlled trial (RCT) for voluntary medical male circumcision (VMMC) in South Africa and to provide a nuanced discussion on the differences between adaptations and fidelity in this context.

Design

We conducted a qualitative study using the Framework for Reporting Adaptations and Modifications in Evidence-based Implementation Strategies (FRAME-IS) to examine 2wT adaptations. We reported adaptations to the 2wT intervention using two steps. First, we categorised adaptations in a shared study-specific Google Docs that documented participant engagement with the 2wT system, tracked daily RCT implementation notes, reported software bugs and noted reminder emails about adaptations for the research team. Second, we conducted a qualitative assessment of the influence of adaptations on project outcomes via 10 periodic reflection meetings with VMMC implementers. Reflection documentation included notes from field observations, meeting minutes and informal partner check-ins to complete adaptation documentation. Using the FRAME-IS as a codebook, adaptations were categorised.

Settings

The RCT was conducted in rural and urban VMMC clinics in the North West and Gauteng districts of South Africa.

Participants

Implementation scientists and VMMC implementers who implemented the 2wT pragmatic randomised controlled trial (pRCT) were participants for the adaptation study.

Primary and secondary outcome measures

The primary outcome measure was the adaptations that were made during the implementation of the 2wT pRCT. The secondary outcome measures were fidelity and rigour of implementing adaptations to the 2wT pRCT.

Results

Between June 2021 and February 2022, 13 adaptations were identified in three phases during the implementation of the 2wT pRCT. The first phase of adaptations aimed to augment study recruitment, including conducting weekend VMMC recruitment camps, using mobile outreach services in the rural site, adding two urban sites to increase recruitment, using weekly WhatsApp calls for updates with all implementing teams, using virtual meetings to implement the 2wT strategy remotely during COVID-19 restrictions and allocating one clinician to work outside of normal working hours. The second phase of adaptations further enhanced enrolments, including adding two local language translations in the usability survey for 2wT men and contributing a portion towards the salary of the implementing staff by the research partner. The third phase included the exclusion of two rural clinics as recruitment sites due to inconsistent mobile phone networks, adding another layer of data quality checks to ensure data quality, training non-clinical counsellors to help with enrolling clients, retraining of staff in the rural site with high staff turnover and using both primary and alternative phone numbers for enrolment to reduce loss to follow-up.

Conclusions

This study made adaptations to the 2wT pRCT without compromising the fidelity of the study. The 2wT pRCT balances rigour (fidelity) and relevance (adaptation). Adaptations should not be confined by rigour but should also not go unchallenged or unverified. We conclude that fidelity can be maintained with adaptations that are implemented to close the gap between research in the laboratory and practice.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

✪ Looking under the hood of a hybrid two-way texting intervention to improve early retention on antiretroviral therapy in Malawi: an implementation fidelity evaluation

Abstract:

Background

While key to interpreting findings and assessing generalizability, implementation fidelity is underreported in mobile health (mHealth) literature. We evaluated implementation fidelity of an opt-in, hybrid, two-way texting (2wT) intervention previously demonstrated to improve 12-month retention on antiretroviral therapy (ART) among people living with HIV (PLHIV) in a quasi-experimental study in Lilongwe, Malawi.

Methods

Short message service (SMS) data and ART refill visit records were used to evaluate adherence to 2wT content, frequency and duration through the lens of the Conceptual Framework for Implementation Fidelity. Message delivery and 2wT participant interactions were considered across four core 2wT components: 1) weekly motivational SMS messages; 2) proactive SMS appointment reminders; 3) SMS reminders after missed appointments; and 4) interactive messaging with 2wT staff about transfers and appointment rescheduling. Using mixed-effects logistic regression models adjusted for participant demographics, we examined the effect of core 2wT component fidelity on a) on-time appointment attendance and b) timely return to care after a missed appointment, presenting adjusted odds ratios (aORs) and 95% confidence intervals (CIs).

Results

The 468 2wT participants had a median of 52 study weeks (interquartile range [IQR] 34 – 52) with 6 ART appointments (IQR 4—7) of which 2 (IQR 1 – 3) were missed. On average, participants received a motivation message for 75% (IQR 56%—83%) of enrolled weeks, a reminder before 83% (IQR 67%—100%) of appointments, and after 67% (IQR 0%—100%) of missed appointments. Participants reported 9 transfers and rescheduled 46 appointments through 2wT prompts; 196 appointments were changed via unprompted interaction. Participants with 10% higher expected motivation message delivery were more likely to attend clinic appointments on time (aOR: 1.08; 95%CI: 1.01 – 1.16, p = 0.03). Receiving and responding to an appointment reminder in any way were also associated with increased on-time appointment attendance (aOR: 1.35; 95%CI: 1.03 – 1.79, p = 0.03 and aOR: 1.47, 95%CI: 1.16 – 1.87, p = 0.001, respectively). No associations were found for 2wT messages and timely return to care following a missed appointment.

Conclusion

Greater 2wT implementation fidelity was associated with improved care outcomes. Although implementation fidelity monitoring of mHealth interventions is complex, it should be integrated into study design.

Contributions to the literature

Despite the recognized importance of implementation fidelity in digital health research, few studies evaluate and transparently report the extent to which mHealth interventions were implemented as intended.
This study presents a framework-based evaluation of the implementation fidelity of a two-way texting intervention for HIV retention in a routine care setting in Malawi. Findings demonstrate how fidelity assessment not only facilitates replication of findings and generalizability assessment, but uncovers implementation challenges and generates hypotheses about intervention mechanisms of action.
The benefits and complexity of implementation fidelity evaluation underscore the need to consider fidelity early and integrate monitoring mechanisms into study design.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

✪ WhatsApp Versus SMS for 2-Way, Text-Based Follow-Up After Voluntary Medical Male Circumcision in South Africa: Exploration of Messaging Platform Choice

Abstract:

Background

Telehealth is growing, especially in areas where access to health facilities is difficult. We previously used 2-way texting (2wT) via SMS to improve the quality of postoperative care after voluntary medical male circumcision in South Africa. In this study, we offered males aged 15 years and older WhatsApp or SMS as their message delivery and interaction platform to explore user preferences and behaviors.

Objective

The objectives of this process evaluation embedded within a larger 2wT expansion trial were to (1) explore 2wT client preferences, including client satisfaction, with WhatsApp or SMS; (2) examine response rates (participation) by SMS and WhatsApp; and (3) gather feedback from the 2wT implementation team on the WhatsApp approach.

Methods

Males aged 15 years and older undergoing voluntary medical male circumcision in program sites could choose their follow-up approach, selecting 2wT via SMS or WhatsApp or routine care (in-person postoperative visits). The 2wT system provided 1-way educational messages and an open 2-way communication channel between providers and clients. We analyzed quantitative data from the 2wT database on message delivery platforms (WhatsApp vs SMS), response rates, and user behaviors using chi-square tests, z tests, and t tests. The team conducted short phone calls with WhatsApp and SMS clients about their perceptions of this 2wT platform using a short, structured interview guide. We consider informal reflections from the technical team members on the use of WhatsApp. We applied an implementation science lens using the RE-AIM (reach, effectiveness, adoption, implementation, and maintenance) framework to focus results on practice and policy improvement.

Results

Over a 2-month period—from August to October, 2023—337 males enrolled in 2wT and were offered WhatsApp or SMS and were included in the analysis. For 2wT reach, 177 (53%) participants chose WhatsApp as their platform (P=.38). Mean client age was 30 years, and 253 (75%) participants chose English for automated messages. From quality assurance calls, almost all respondents (87/89, 98%) were happy with the way they were followed up. For effectiveness, on average for the days on which responses were requested, 58 (33%) WhatsApp clients and 44 (28%) SMS clients responded (P=.50). All 2wT team members believed WhatsApp limited the automated message content, language choices, and inclusivity as compared with the SMS-based 2wT approach.

Conclusions

When presented with a choice of 2wT communication platform, clients appear evenly split between SMS and WhatsApp. However, WhatsApp requires a smartphone and data plan, potentially reducing reach at scale. Clients using both platforms responded to 2wT interactive prompts, demonstrating similar effectiveness in engaging clients in follow-up. For telehealth interventions, digital health designers should maintain an SMS-based platform and carefully consider adding WhatsApp as an option for clients, using an implementation science approach to present evidence that guides the best implementation approach for their setting.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

✪ The effect of proactive, interactive, two-way texting on 12-month retention in antiretroviral therapy: Findings from a quasi-experimental study in Lilongwe, Malawi

Abstract:

Background

Retaining clients on antiretroviral therapy (ART) is challenging, especially during the first year on ART. Mobile health (mHealth) interventions show promise to close retention gaps. We aimed to assess reach (who received the intervention?) and effectiveness (did it work?) of a hybrid two-way texting (2wT) intervention to improve ART retention at a large public clinic in Lilongwe, Malawi.

Methods

Between August 2021—June 2023, in a quasi-experimental study, outcomes were compared between two cohorts of new ART clients: 1) those opting into 2wT who received automated, weekly motivation short messaging service (SMS) messages and response-requested appointment reminders; and 2) a matched historical cohort receiving standard of care (SoC). Reach was defined as “the proportion clients ≤6 months of ART initiation eligible for 2wT”. 2wT effectiveness was assessed in time-to-event analysis. Retention was presented in a Kaplan-Meier plot and compared between 2wT and SoC using a log-rank test. The effect of 2wT on ART dropout (lost to follow-up or stopped ART) was estimated using Fine-Gray competing risk regression models, adjusting for sex, age and WHO HIV stage at ART initiation.

Results

Of 1,146 clients screened, 501 were eligible for 2wT, a reach of 44%. Lack of phone (393/645; 61%) and illiteracy (149/645; 23%) were the most common ineligibility reasons. Among 468 participants exposed to 2wT, 12-month probability of ART retention was 91% (95% CI: 88% - 94%) compared to 76% (95% CI: 72% - 80%) among 468 SoC participants (p<0.001). Compared to SoC, 2wT participants had a 65% lower hazard of ART dropout at any timepoint (sub-distribution hazard ratio 0.35, 95% CI: 0.24–0.51; p<0.001).

Conclusions

2wT did not reach all clients. For those who opted-in, 2wT significantly increased 12-month ART retention. Expansion of 2wT as a complement to other retention interventions should be considered in other low-resource, routine ART settings.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

✪ ‘I understood the texting process well’ – Participant perspectives on usability and acceptability of SMS-based telehealth follow-up after voluntary medical male circumcision in South Africa

Abstract:

Background

Voluntary medical male circumcision (MC) is a biomedical HIV prevention method that requires post-operative follow-up for healing confirmation. Recent research found that a two-way texting (2wT) app providing SMS-based telehealth for MC patients was safe and reduced provider workload. We evaluated 2wT usability among MC clients in South Africa assigned the 2wT intervention within a larger randomized controlled trial (RCT) of 2wT safety and workload.

Methods

This quantitative usability study is within an RCT where 547 men used 2wT to interact with an MC provider via SMS. The sub-study involved the first 100 men assigned to 2wT who completed a usability survey 14 days after surgery. Acceptability was assessed through 2wT response rates of the 547 men. Regression models analyzed associations between age, wage, location, potential adverse events (AEs), and 2wT responses.

Results

Men assigned to 2wT found it safe, comfortable, and convenient, reporting time and cost savings. High response rates (88%) to daily messages indicated acceptability. Age, wage, and location didn't affect text responses or potential AEs.

Conclusion

2wT for post-MC follow-up was highly usable and acceptable, suggesting its viability as an alternative to in-person visits. It enhanced confidence in wound self-management. This SMS-based telehealth can enhance MC care quality and be adapted to similar contexts for independent healing support, particularly for men.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

Authors:

✪ Open Access

Read the full text in the open access journal PLOS Digital Health

Abstract:

Author summary

Authors:

✪ Open Access

Read the full text in the open access journal PLOS One

Abstract:

Introduction

Materials and methods

Results

Conclusion

Authors:

✪ Open Access

Read the full text in the open access journal BMJ Open

Abstract:

Objectives

Design

Settings

Participants

Primary and secondary outcome measures

Results

Conclusions

Authors:

✪ Open Access

Read the full text in the open access journal Implementation Science

Abstract:

Background

Methods

Results

Conclusion

Contributions to the literature

Authors:

✪ Open Access

Read the full text in the open access journal JMIR Formative Research

Abstract:

Background

Objective

Methods

Results

Conclusions

Authors:

✪ Open Access

Read the full text in the open access journal PLOS One

Abstract:

Background

Methods

Results

Conclusions

Authors:

✪ Open Access

Read the full text in the open access journal Digital Health

Abstract:

Background

Methods

Results

Conclusion