Skip to content
Implementation Science at UW

Keywords: Assessment

✪ Usability of a Web-Based App for Increasing Adolescent Vaccination in Primary Care Settings: Think-Aloud and Survey Assessment

Authors:

Stephanie A S Staras, Justin Tauscher, Michelle Vinson, Lindsay A Thompson, Mary A Gerend & Elizabeth A Shenkman

University of Washington affiliated authors are displayed in bold.

✪ Open Access

Published: September 2024

Read the full text in the open access journal JMIR Formative Research

Abstract:

Background

In the United States, only 58% of teens receive the recommended 2 doses of the human papillomavirus vaccine by 15 years of age. Overcoming vaccine hesitancy often requires effective communication between clinicians and parents to address specific concerns. To support this, we developed ProtectMe4, a multilevel, theory-informed web-based intervention designed to address parents’ vaccine-related questions and assist clinicians in discussing vaccine concerns for 4 adolescent vaccines.

Objective

This study aims to evaluate the usability of ProtectMe4 in routine care settings across 3 pediatric primary care clinics. Specifically, the study aims to (1) observe the proposed workflow in practice, (2) identify usability issues experienced by parents and clinicians, and (3) assess the perceptions of both parents and clinicians regarding the app’s usability.

Methods

On designated days in 2020 and 2021, the study team recruited parents of 11- to 12-year-old patients attending appointments with participating clinicians. We conducted think-aloud assessments during routine care visits and administered a usability survey after participants used the app. For parents, we simultaneously video-recorded the app screens and audio-recorded their commentary. For clinicians, observational notes were taken regarding their actions and comments. Timings recorded within the app provided data on the length of use. We reviewed the recordings and notes to compile a list of identified issues and calculated the frequencies of survey responses.

Results

Out of 12 parents invited to use the app, 9 (75%) participated. Two parents who were invited outside of the planned workflow, after seeing the clinician, refused to participate. For the parents whose child’s vaccination record was identified by the app, the median time spent using the app was 9 (range 6-28) minutes. Think-aloud assessment results for parents were categorized into 2 themes: (1) troubleshooting vaccine record identification and (2) clarifying the app content and purpose. Among the 8 parents who completed the survey, at least 75% (6/8) agreed with each acceptability measure related to user satisfaction, perceived usefulness, and acceptance. These parents’ children were patients of 4 of the 7 participating clinicians. Consistent with the planned workflow, clinicians viewed the app before seeing the patient in 4 of 9 (44%) instances. The median time spent on the app per patient was 95 (range 5-240) seconds. Think-aloud assessment results for clinicians were grouped into 2 themes: (1) trust of app vaccine results and (2) clarifying the app content. On the survey, clinicians were unanimously positive about the app, with an average System Usability Scale score of 87.5 (SE 2.5).

Conclusion

This mixed methods evaluation demonstrated that ProtectMe4 was usable and acceptable to both parents and clinicians in real-world pediatric primary care. Improved coordination among clinic staff is needed to ensure the app is consistently offered to patients and reviewed by clinicians before seeing the patient.

**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**

✪ The effect of proactive, interactive, two-way texting on 12-month retention in antiretroviral therapy: Findings from a quasi-experimental study in Lilongwe, Malawi

Authors:

Caryl Feldacker, Robin E. Klabbers, Jacqueline Huwa, Christine Kiruthu-Kamamia, Agness Thawani, Petros Tembo, Joseph Chintedza, Geldert Chiwaya, Aubrey Kudzala, Pachawo Bisani, Dumisani Ndhlovu, Johnnie Seyani & Hannock Tweya

University of Washington affiliated authors are displayed in bold.

✪ Open Access

Published: August 2024

Read the full text in the open access journal PLOS One

Abstract:

Background

Retaining clients on antiretroviral therapy (ART) is challenging, especially during the first year on ART. Mobile health (mHealth) interventions show promise to close retention gaps. We aimed to assess reach (who received the intervention?) and effectiveness (did it work?) of a hybrid two-way texting (2wT) intervention to improve ART retention at a large public clinic in Lilongwe, Malawi.

Methods

Between August 2021—June 2023, in a quasi-experimental study, outcomes were compared between two cohorts of new ART clients: 1) those opting into 2wT who received automated, weekly motivation short messaging service (SMS) messages and response-requested appointment reminders; and 2) a matched historical cohort receiving standard of care (SoC). Reach was defined as “the proportion clients ≤6 months of ART initiation eligible for 2wT”. 2wT effectiveness was assessed in time-to-event analysis. Retention was presented in a Kaplan-Meier plot and compared between 2wT and SoC using a log-rank test. The effect of 2wT on ART dropout (lost to follow-up or stopped ART) was estimated using Fine-Gray competing risk regression models, adjusting for sex, age and WHO HIV stage at ART initiation.

Results

Of 1,146 clients screened, 501 were eligible for 2wT, a reach of 44%. Lack of phone (393/645; 61%) and illiteracy (149/645; 23%) were the most common ineligibility reasons. Among 468 participants exposed to 2wT, 12-month probability of ART retention was 91% (95% CI: 88% - 94%) compared to 76% (95% CI: 72% - 80%) among 468 SoC participants (p<0.001). Compared to SoC, 2wT participants had a 65% lower hazard of ART dropout at any timepoint (sub-distribution hazard ratio 0.35, 95% CI: 0.24–0.51; p<0.001).

Conclusions

2wT did not reach all clients. For those who opted-in, 2wT significantly increased 12-month ART retention. Expansion of 2wT as a complement to other retention interventions should be considered in other low-resource, routine ART settings.

**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**

✪ Protocol for the Australian Type 1 Diabetes National Screening Pilot: Assessing the feasibility and acceptability of three general population screening models in children

Authors:

Kirstine J. Bell, Shannon Brodie, Jennifer J. Couper, Peter Colman, Elizabeth Davis, Gary Deed, William Hagopian, Aveni Haynes, Christel Hendrieckx, Amanda Henry, Adrienne Gordon, Kirsten Howard, Tony Huynh, Bernadette Kerr, Kara Mikler, Natasha Nassar, Sarah Norris, Richard Oram, Dorota Pawlak, Antonia Shand, Richard O. Sinnott, Bethany Wadling, John M. Wentworth, Maria E. Craig & the Type 1 Diabetes National Screening Pilot Study Group

University of Washington affiliated authors are displayed in bold.

✪ Open Access

Published: August 2024

Read the full text in the open access journal Diabetes Medicine

Abstract:

Aim

One third of Australian children diagnosed with type 1 diabetes present with life-threatening diabetic ketoacidosis (DKA) at diagnosis. Screening for early-stage, presymptomatic type 1 diabetes, with ongoing follow-up, can substantially reduce this risk (<5% risk). Several screening models are being trialled internationally, without consensus on the optimal approach. This pilot study aims to assess three models for a routine, population-wide screening programme in Australia.

Methods

An implementation science-guided pilot study to evaluate the feasibility, acceptability and costs of three screening models in children will be conducted between July 2022 and June 2024. These models are as follows: (1) Genetic risk-stratified screening using newborn heel prick dried bloodspots, followed by autoantibody testing from 11 months of age; (2) genetic risk-stratified screening of infant (6–12 months) saliva followed by autoantibody testing from 10 months of age; and (3) autoantibody screening using capillary dried bloodspots collected from children aged 2, 6 or 10 years. Cohorts for each model will be recruited from targeted geographic areas across Australia involving ≥2 states per cohort, with a recruitment target of up to 3000 children per cohort (total up to 9000 children). The primary outcome is screening uptake for each cohort. Secondary outcomes include programme feasibility, costs, parental anxiety, risk perception, satisfaction, well-being and quality of life, and health professional attitudes and satisfaction.

Conclusions

This pilot is the first direct comparison of three screening implementation models for general population screening. Findings will provide evidence to inform a potential national screening programme for Australian children.

**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**

Measuring perceived utility of genomic sequencing: Development and validation of the GENEtic Utility (GENE-U) scale for adult screening

Authors:

Hadley Stevens Smith, Caryn Kseniya Rubanovich, Jill Oliver Robinson, Ariel N. Levchenko, Sarah A. Classen, Janet Malek, Adam H. Buchanan, Barbara Biesecker, Kyle B. Brothers, Benjamin S. Wilfond, Christine Rini, Cinnamon S. Bloss, Amy L. McGuire & Sara J. Knight

University of Washington affiliated authors are displayed in bold.

Subscription Required for Full Text Access

Published: August 2024

Read the full text in the subscription access journal Genetics in Medicine

✪ Implementation Science for HIV Prevention and Treatment in Indigenous Communities: a Systematic Review and Commentary

Authors:

Christopher G. Kemp (Ngāi Tahu), Abagail J. Edwards, Lauren White (Choctaw Nation of Oklahoma), Gauri Kore, Pamela Jumper Thurman (Cherokee), Tommi Gaines (Navajo, Hopi), Paula Toko King (Te Aupōuri, Te Rarawa, Ngāpuhi, Ngāti Whātua, Waikato Tainui, Ngāti Maniapoto), Marama Cole (Ngāti Kahungunu, Ngāti Porou) & E. Roberto Orellana (Maya)

University of Washington affiliated authors are displayed in bold.

✪ Open Access

Published: August 2024

Read the full text in the open access journal Current HIV/AIDS Reports

Abstract:

Purpose of Review

We systematically reviewed implementation research conducted in Indigenous communities in the Americas and the Pacific that focused on improving delivery of HIV preventive or treatment services. We highlight strengths and opportunities in the literature and outline principles for Indigenous-led, HIV-related implementation science.

Recent Findings

We identified 31 studies, revealing a consistent emphasis on cultural tailoring of services to Indigenous communities. Common barriers to implementation included stigma, geographic limitations, confidentiality concerns, language barriers, and mistrust. Community involvement in intervention development and delivery emerged as a key facilitator, and nearly half of the studies used community-based participatory research methods. While behavioral HIV prevention, especially among Indigenous youth, was a major focus, there was limited research on biomedical HIV prevention and treatment. No randomized implementation trials were identified.

Summary

The findings underscore the importance of community engagement, the need for interventions developed within Indigenous communities rather than merely adapted, and the value of addressing the social determinants of implementation success. Aligned to these principles, an indigenized implementation science could enhance the acceptability and reach of critical HIV preventive and treatment services in Indigenous communities while also honoring their knowledge, wisdom, and strength.

**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**

Employing user-centered design to develop a remote technology kit for studying young children’s social communication skills

Authors:

Marisa Petruccelli, Mya Howard, Andres Morelos, Allison Wainer, Sarabeth Broder-Fingert, Brooke Ingersoll, Wendy L Stone & Alice S Carter

University of Washington affiliated authors are displayed in bold.

Subscription Required for Full Text Access

Published: August 2024

Read the full text in the subscription access journal Journal of Pediatric Psychology

✪ Implementation of a bilingual and bicultural navigation program for Hispanic children with traumatic brain injuries

Authors:

Celeste N. Garcia, Magaly Ramirez, Tasnia Alam, Linda K. Ko & Nathalia Jimenez

University of Washington affiliated authors are displayed in bold.

✪ Open Access

Published: August 2024

Read the full text in the open access journal Brain Injury

Abstract:

Background

Traumatic brain injury (TBI) disproportionately affects Hispanic children, with higher rates of severe injuries and worse outcomes compared to non-Hispanic White children. Patient navigation has shown effectiveness in improving health outcomes among Hispanic populations. However, culturally tailored navigation programs for pediatric TBI are lacking, and knowledge gaps remain regarding contextual factors influencing the implementation of these programs. This study examined factors that facilitate or hinder the implementation of a bilingual and bicultural navigation program for caregivers of Hispanic children with TBIs.

Methods

In-depth interviews were conducted with 12 personnel involved in implementing the 1st Brain Injury Education and Outpatient Navigation (1st BIEN) program across four states. Directed content analysis was used based on the Consolidated Framework for Implementation Research (CFIR) to guide the development of the interview guide and data analysis.

Results

Key facilitators included the program’s adaptability to patient needs, bilingual/bicultural patient navigator’s skills, and alignment with organizational values. Barriers included restrictive healthcare regulatory policies hindering interinstitutional work, incomplete medical records, and gaps in behavioral support training.

Discussion

These findings offer insights for optimizing the implementation of patient navigation programs in pediatric TBI care, potentially improving access to care and reducing disparities for Hispanic children with brain injuries.

**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**

Measuring perceived utility of genomic sequencing: Development and validation of the GENEtic Utility (GENE-U) scale for pediatric diagnostic testing

Authors:

Hadley Stevens Smith, Caryn Kseniya Rubanovich, Jill Oliver Robinson, Ariel N. Levchenko, Sarah A. Classen, Janet Malek, Barbara Biesecker, Kyle B. Brothers, Benjamin S. Wilfond, Christine Rini, Sara J. Knight, Amy L. McGuire & Cinnamon S. Bloss

University of Washington affiliated authors are displayed in bold.

Subscription Required for Full Text Access

Published: August 2024

Read the full text in the subscription access journal Genetics in Medicine

✪ Providing HIV-assisted partner services to partners of partners in western Kenya: an implementation science study

Authors:

Monisha Sharma, Barbara Wanjiku Mambo, Hanley Kingston, George Otieno, Sarah Masyuko, Harison Lagat, David A. Katz, Beatrice Wamuti, Paul Macharia, Rose Bosire , Mary Mugambi, Edward Kariithi, and Carey Farquhar

University of Washington affiliated authors are displayed in bold.

✪ Open Access

Published: July 2024

Read the full text in the open access Journal of the International AIDS Society

Abstract:

Introduction

Assisted partner services (APS), or exposure notification and HIV testing for sexual partners of persons diagnosed HIV positive (index clients), is recommended by the World Health Organization. Most APS literature focuses on outcomes among index clients and their partners. There is little data on the benefits of providing APS to partners of partners diagnosed with HIV.

Methods

We utilized data from a large-scale APS implementation project across 31 facilities in western Kenya from 2018 to 2022. Females testing HIV positive at facilities were offered APS; those who consented provided contact information for all male sexual partners in the last 3 years. Male partners were notified of their potential HIV exposure and offered HIV testing services (HTS). Males newly testing positive were also offered APS and asked to provide contact information for their female partners in the last 3 years. Female partners of male partners (FPPs) were provided exposure notification and HTS. All participants with HIV were followed up at 12 months post-enrolment to assess linkage-to antiretroviral treatment (ART) and viral suppression. We compared HIV positivity, demographics and linkage outcomes among female index clients and FPPs.

Results

Overall, 5708 FPPs were elicited from male partners, of whom 4951 received HTS through APS (87% coverage); 291 FPPs newly tested HIV positive (6% yield), an additional 1743 (35.2%) reported a prior HIV diagnosis, of whom 99% were on ART at baseline. At 12 months follow-up, most FPPs were taking ART (92%) with very few adverse events: <1% reported intimate partner violence or reported relationship dissolution. FPPs were more likely than female index clients to report HIV risk behaviours including no condom use at last sex (45% vs. 30%) and multiple partners (38% vs. 19%).

Conclusions

Providing HIV testing via APS to FPP is a safe and effective strategy to identify newly diagnosed females and achieve high linkage and retention to ART and can be an efficient means of identifying HIV cases in the era of declining HIV incidence. The high proportion of FPPs reporting HIV risk behaviours suggests APS may help interrupt community HIV transmission via increased knowledge of HIV status and linkage to treatment.

**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**

✪ Using FRAME to characterize provider-identified adaptations to a stepped care intervention for adolescents and youth living with HIV in Kenya: a mixed methods approach

Authors:

Nok Chhun, Dorothy Oketch, Kawango Agot, Dorothy I. Mangale, Jacinta Badia, James Kibugi, Wenwen Jiang, Mary Kirk, Barbra A. Richardson, Pamela K. Kohler, Grace John-Stewart & Kristin Beima-Sofie

University of Washington affiliated authors are displayed in bold.

✪ Open Access

Published: July 2024

Read the full text in the open access journal Journal of the International AIDS Society

Abstract:

Introduction

The Data-informed Stepped Care (DiSC) study is a cluster-randomized trial implemented in 24 HIV care clinics in Kenya, aimed at improving retention in care for adolescents and youth living with HIV (AYLHIV). DiSC is a multi-component intervention that assigns AYLHIV to different intensity (steps) of services according to risk. We used the Framework for Reporting Adaptations and Modifications-Expanded (FRAME) to characterize provider-identified adaptations to the implementation of DiSC to optimize uptake and delivery, and determine the influence on implementation outcomes.

Methods

Between May and December 2022, we conducted continuous quality improvement (CQI) meetings with providers to optimize DiSC implementation at 12 intervention sites. The meetings were guided by plan-do-study-act processes to identify challenges during early phase implementation and propose targeted adaptations. Meetings were audio-recorded and analysed using FRAME to categorize the level, context and content of planned adaptations and determine if adaptations were fidelity consistent. Providers completed surveys to quantify perceptions of DiSC acceptability, appropriateness and feasibility. Mixed effects linear regression models were used to evaluate these implementation outcomes over time.

Results

Providers participated in eight CQI meetings per facility over a 6-month period. A total of 65 adaptations were included in the analysis. The majority focused on optimizing the integration of DiSC within the clinic (83%, n = 54), and consisted of improving documentation, addressing scheduling challenges and improving clinic workflow. Primary reasons for adaptation were to align delivery with AYLHIV needs and preferences and to increase reach among AYLHIV: with reminder calls to AYLHIV, collaborating with schools to ensure AYLHIV attended clinic appointments and addressing transportation challenges. All adaptations to optimize DiSC implementation were fidelity-consistent. Provider perceptions of implementation were consistently high throughout the process, and on average, slightly improved each month for intervention acceptability (β = 0.011, 95% CI: 0.002, 0.020, p = 0.016), appropriateness (β = 0.012, 95% CI: 0.007, 0.027, p<0.001) and feasibility (β = 0.013, 95% CI: 0.004, 0.022, p = 0.005).

Conclusions

Provider-identified adaptations targeted improved integration into routine clinic practices and aimed to reduce barriers to service access unique to AYLHIV. Characterizing types of adaptations and adaptation rationale may enrich our understanding of the implementation context and improve abilities to tailor implementation strategies when scaling to new settings.

**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**