Assessment Archives - Page 6 of 7 - Implementation Science at UW

✪ Acceptability and Utility of a Digital Group Intervention to Prevent Perinatal Depression in Youths via Interactive Maternal Group for Information and Emotional Support (IMAGINE): Pilot Cohort Study

Abstract:

Background

Perinatal depression (depression during pregnancy or the first year postpartum) affects 10%-25% of perinatal individuals, with a higher risk among youths aged <25 years. The Mothers and Babies Course (MB) is an evidence-based intervention for the prevention of perinatal depression, grounded in cognitive behavioral therapy, attachment theory, and psychoeducation.

Objective

We developed a digital adaptation of MB (Interactive Maternal Group for Information and Emotional Support [IMAGINE]) and evaluated it in a pre-post mixed methods pilot among young perinatal people in the United States

Methods

IMAGINE was a structured digital group of up to 7 participants, with scheduled MB content and open discussion for 12 weeks, facilitated by a social worker. Scheduled content included asynchronous SMS text messages, graphics, prerecorded videos, mood polls, and optional weekly synchronous video calls. Eligible participants were pregnant or ≤80 days postpartum, aged 16 to 24 years, had access to a smartphone, spoke English, and had a Patient Health Questionnaire score <10. Participants were recruited throughout the United States from August 2020 to January 2021 through paid social media ads, in-person outreach at clinics, and respondent-driven sampling. Participants completed quantitative questionnaires at enrollment and 3 months, and qualitative interviews at 3 months. We determined uptake, acceptability (by Acceptability of Intervention Measure score), and utility (by use of cognitive behavioral therapy skills). We compared depression symptoms (by Patient Health Questionnaire score), social support (by abbreviated Social Support Behavior score), and perceived stress (by Perceived Stress Score) between enrollment and follow-up by paired 2-tailed t test.

Results

Among 68 individuals who contacted this study, 22 were screened, 13 were eligible, and 10 enrolled, for an uptake of 76.9%. Furthermore, 4 (40%) participants were pregnant at enrollment. Participants had a median age of 17.9 (IQR 17.4-21.7) years, 6 (67%) identified as Black, 5 (56%) Latinx, and 6 (67%) using Medicaid health insurance. Further, 9 (90%) participants completed follow-up. Among these, the mean acceptability score was 4.3 out of 5 (SD 0.6) and all participants said they would recommend IMAGINE to a friend. Participants reported using a median of 7 of 11 skills (IQR 5-7 skills) at least half the days. We found no significant changes in depression symptoms, perceived stress, or social support. Qualitatively, participants reported one-to-one support from the facilitator, connection with other parents, and regular mood reflection were especially helpful aspects of the intervention. Additionally, participants reported that the intervention normalized their mental health challenges, improved their ability to manage their mood, and increased their openness to mental health care.

Conclusions

This pilot study provides promising evidence of the acceptability and utility of IMAGINE among perinatal youths. Our study’s small sample size did not detect changes in clinical outcomes; our findings suggest IMAGINE warrants larger-scale evaluation.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

✪ Study protocol: Novel Methods for Implementing Measurement-Based Care with youth in Low-Resource Environments (NIMBLE)

Abstract:

Introduction

For youth receiving care in community mental health centers, comorbidities are the rule rather than the exception. Using measurement-based care (MBC), or the routine evaluation of symptoms to inform care decisions, as the foundation of treatment for youth with comorbid problems significantly improves the impact of psychotherapy by focusing care and building engagement and alliance. MBC increases the rate of symptom improvement, detects clients who would otherwise deteriorate, and alerts clinicians to non-responders. Despite its demonstrated utility, MBC is rarely implemented with fidelity; less than 15% of providers report using MBC per recommendations. Previous efforts to support MBC implementation have yielded suboptimal outcomes, in part, due to organizations’ challenges with identifying and prioritizing barriers and selecting and developing strategies to overcome them. New methods are needed for identifying and prioritizing barriers, and matching strategies to barriers to optimize MBC implementation and treatment quality to improve youth mental health outcomes in community settings.

Methods

Pragmatic implementation methods will be piloted in four diverse community mental health centers. Methods include (a) rapid evidence synthesis; (b) rapid ethnography; (c) design kits (e.g., kits with disposable cameras, journals, maps); (d) barrier prioritization, and (e) causal pathway diagramming. These activities will generate actionable barriers; subsequently, we will use facilitated group processes to prioritize barriers and develop causal pathway diagrams to match strategies to barriers to create implementation plans that optimize MBC fidelity (Aim 1). We will track strategy deployment for 6 months, then compare MBC fidelity for another 6 months post-implementation with data from 2 years of historical controls (Aim 2). Finally, we will co-design a toolkit for design kit methods with youth and the practice and scientific communities (Aim 3).

Discussion

Optimizing MBC implementation in community mental health centers could transform youth mental health care by ensuring the most pressing symptoms are targeted early in treatment. The discussion section highlights expected challenges and limits to using the five methods, including recruitment and engagement given the high pressure on community mental health settings.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

✪ The implementation and effectiveness of multi-tasked, paid community health workers on maternal and child health: A cluster-randomized pragmatic trial and qualitative process evaluation in Tanzania

Abstract:

Community health worker programs have proliferated worldwide based on evidence that they help prevent mortality, particularly among children. However, there is limited evidence from randomized studies on the processes and effectiveness of implementing community health worker programs through public health systems. This paper describes the results of a cluster-randomized pragmatic implementation trial (registration number ISRCTN96819844) and qualitative process evaluation of a community health worker program in Tanzania that was implemented from 2011–2015. Program effects on maternal, newborn and child health service utilization, childhood morbidity and sick childcare seeking were evaluated using difference-in-difference regression analysis with outcomes measured through pre- and post-intervention household surveys in intervention and comparison trial arms. A qualitative process evaluation was conducted between 2012 and 2014 and comprised of in-depth interviews and focus group discussions with community health workers, community members, facility-based health workers and staff of district health management teams. The community health worker program reduced incidence of illness and improved access to timely and appropriate curative care for children under five; however, there was no effect on facility-based maternal and newborn health service utilization. The positive outcomes occurred because of high levels of acceptability of community health workers within communities, as well as the durability of community health workers’ motivation and confidence. Implementation factors that generated these effects were the engagement of communities in program startup; the training, remuneration and supervision of the community health workers from the local health system and community. The lack of program effects on maternal and newborn health service utilization at facilities were attributed to lapses in the availability of needed care at facilities. Strategies that strengthen and align communities’ and health systems core capacities, and their ability to learn, adapt and integrate evidence-based interventions, are needed to maximize the health impact of community health workers.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

✪ A machine learning approach to improve implementation monitoring of family-based preventive interventions in primary care

Abstract:

Background

Evidence-based parenting programs effectively prevent the onset and escalation of child and adolescent behavioral health problems. When programs have been taken to scale, declines in the quality of implementation diminish intervention effects. Gold-standard methods of implementation monitoring are cost-prohibitive and impractical in resource-scarce delivery systems. Technological developments using computational linguistics and machine learning offer an opportunity to assess fidelity in a low burden, timely, and comprehensive manner.

Methods

In this study, we test two natural language processing (NLP) methods [i.e., Term Frequency-Inverse Document Frequency (TF-IDF) and Bidirectional Encoder Representations from Transformers (BERT)] to assess the delivery of the Family Check-Up 4 Health (FCU4Health) program in a type 2 hybrid effectiveness-implementation trial conducted in primary care settings that serve primarily Latino families. We trained and evaluated models using 116 English and 81 Spanish-language transcripts from the 113 families who initiated FCU4Health services. We evaluated the concurrent validity of the TF-IDF and BERT models using observer ratings of program sessions using the COACH measure of competent adherence. Following the Implementation Cascade model, we assessed predictive validity using multiple indicators of parent engagement, which have been demonstrated to predict improvements in parenting and child outcomes.

Results

Both TF-IDF and BERT ratings were significantly associated with observer ratings and engagement outcomes. Using mean squared error, results demonstrated improvement over baseline for observer ratings from a range of 0.83–1.02 to 0.62–0.76, resulting in an average improvement of 24%. Similarly, results demonstrated improvement over baseline for parent engagement indicators from a range of 0.81–27.3 to 0.62–19.50, resulting in an approximate average improvement of 18%.

Conclusions

These results demonstrate the potential for NLP methods to assess implementation in evidence-based parenting programs delivered at scale. Future directions are presented.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

✪ Organizational contexts, implementation processes, and capacity outcomes of multicultural, multilingual Home-Based Programs in public initiatives: A Mixed-Methods study

Abstract:

Aims

The study aim was to examine the impact of a home-based programme intervention on organizational contexts, implementation processes and organizational capacity outcomes from multicultural, multilingual participants working at community-based organizations.

Design

This was a sequential exploratory, mixed-methods longitudinal study using community-based participatory research principles.

Sample

Twenty participants from nine multicultural, multilingual community-based organizations were in this public health initiative's intervention to develop community-designed, home-based programmes.

Methods

Capacity building providers delivered the intervention selected by the funders. Workshop outcomes were descriptively measured in April/May 2019. In April/May and November 2019, participants completed surveys about organizational contexts, implementation processes and organizational capacity outcomes, which were analysed with t-tests using the organization as the unit of analysis. Qualitative data were analysed using content analysis.

Results

Seven programmes were new and two were modified. As workshop outcomes, 59% of participants reported increased overall implementation knowledge and 74% reported capacity building providers as the most helpful resource. After 6 to 7 months, no statistically significant changes were noted in organizational contexts, implementation processes or organizational capacity outcomes. Participants benefited from capacity building because they had programmes developed, formed partnerships with capacity building providers, gained implementation knowledge, and engaged in networking.

Conclusion

Participants reported excellent individual and organizational strengths. Many Initiative factors contributed to no statistical changes. Namely, there was no opportunity for baseline data; limited community-based organization engagement in the intervention model selection, timeline and processes; the Initiative's timeline did not fit participants' timeline; insufficient time to develop culturally and linguistically appropriate programmes; late literature review abstracts; lack of adequate, planful and paid capacity building time; and a contract requirement to have the programme due when it was not implementable. These Initiative design factors, as reported by participants, limited the Initiative's home-based programme development.

Impact

This study highlights the strengths of participants, community-based organizations and capacity building providers. Model selection, timeline and budget were identified as key factors for equitable implementation in multicultural, multilingual organizations.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

✪ Assessing drivers of implementing “Scaling-up the Systems Analysis and Improvement Approach” for Prevention of Mother-to-Child HIV Transmission in Mozambique (SAIA-SCALE) over implementation waves

Authors:

Celso Inguane, Sarah Gimbel, Caroline Soi, Esperança Tavede, Filipe Murgorgo, Xavier Isidoro, Yaesh Sidat, Regina Nassiaca, Joana Coutinho, Maria Cruz, Mery Agostinho, Fernando Amaral, Aneth Dinis, Kristjana Ásbjörnsdóttir, Jonny Crocker, Nélia Manaca, Isaias Ramiro, James Pfeiffer, Maria de Fátima Cuembelo, & Kenneth Sherr

^{University of Washington affiliated authors are displayed in bold.}

✪ Open Access

Published: July 2023

Read the full text in the open access journal Implementation Science Communication

Abstract:

Background

The Systems Analysis and Improvement Approach (SAIA) is an evidence-based package of systems engineering tools originally designed to improve patient flow through the prevention of Mother-to-Child transmission of HIV (PMTCT) cascade. SAIA is a potentially scalable model for maximizing the benefits of universal antiretroviral therapy (ART) for mothers and their babies. SAIA-SCALE was a stepped wedge trial implemented in Manica Province, Mozambique, to evaluate SAIA’s effectiveness when led by district health managers, rather than by study nurses. We present the results of a qualitative assessment of implementation determinants of the SAIA-SCALE strategy during two intensive and one maintenance phases.

Methods

We used an extended case study design that embedded the Consolidated Framework for Implementation Research (CFIR) to guide data collection, analysis, and interpretation. From March 2019 to April 2020, we conducted in-depth individual interviews (IDIs) and focus group discussions (FGDs) with district managers, health facility maternal and child health (MCH) managers, and frontline nurses at 21 health facilities and seven districts of Manica Province (Chimoio, Báruè, Gondola, Macate, Manica, Sussundenga, and Vanduzi).

Results

We included 85 participants: 50 through IDIs and 35 from three FGDs. Most study participants were women (98%), frontline nurses (49.4%), and MCH health facility managers (32.5%). An identified facilitator of successful intervention implementation (regardless of intervention phase) was related to SAIA’s compatibility with organizational structures, processes, and priorities of Mozambique’s health system at the district and health facility levels. Identified barriers to successful implementation included (a) inadequate health facility and road infrastructure preventing mothers from accessing MCH/PMTCT services at study health facilities and preventing nurses from dedicating time to improving service provision, and (b) challenges in managing intervention funds.

Conclusions

The SAIA-SCALE qualitative evaluation suggests that the scalability of SAIA for PMTCT is enhanced by its fit within organizational structures, processes, and priorities at the primary level of healthcare delivery and health system management in Mozambique. Barriers to implementation that impact the scalability of SAIA include district-level financial management capabilities and lack of infrastructure at the health facility level. SAIA cannot be successfully scaled up to adequately address PMTCT needs without leveraging central-level resources and priorities.

Trial registration

ClinicalTrials.gov, NCT03425136. Registered on 02/06/2018.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

✪ Implementation and scale-up of a single-visit, screen-and-treat approach with thermal ablation for sustainable cervical cancer prevention services: a protocol for a stepped-wedge cluster randomized trial in Kenya

Abstract:

Background

An important cervical cancer prevention strategy in low- and middle-income countries (LMICs) has been single-visit screen-and-treat (SV-SAT) approach, using visual inspection with acetic acid (VIA) and ablative treatment with cryotherapy to manage precancerous lesions. While SV-SAT with VIA and cryotherapy have established efficacy, its population level coverage and impact on reducing cervical cancer burden remains low. In Kenya, the estimated cervical cancer screening uptake among women aged 30–49 is 16% and up to 70% of screen-positive women do not receive treatment. Thermal ablation for treatment of precancerous lesions of the cervix is recommended by the World Health Organization and has the potential to overcome logistical challenges associated with cryotherapy and facilitate implementation of SV-SAT approach and increase treatment rates of screen-positive women. In this 5-year prospective, stepped-wedge randomized trial, we plan to implement and evaluate the SV-SAT approach using VIA and thermal ablation in ten reproductive health clinics in central Kenya.

Methods

The study aims to develop and evaluate implementation strategies to inform the national scale-up of SV-SAT approach with VIA and thermal ablation through three aims: (1) develop locally tailored implementation strategies using multi-level participatory method with key stakeholders (patient, provider, system-level), (2) implement SV-SAT approach with VIA and thermal ablation and evaluate clinical and implementation outcomes, and (3) assess the budget impact of SV-SAT approach with VIA and thermal ablation compared to single-visit, screen-and-treat method using cryotherapy.

Discussion

Our findings will inform national scale-up of the SV-SAT approach with VIA and thermal ablation. We anticipate that this intervention, along with tailored implementation strategies will enhance the adoption and sustainability of cervical cancer screening and treatment compared to the standard of care using cryotherapy.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

✪ ADCC’s Improving Goal Concordant Care Initiative: Implementing Primary Palliative Care Principles

Abstract:

Background

High-quality, timely goals of care communication (GOCC) may improve patient and caregiver outcomes and promote care that is consistent with patient preferences.

Problem

Cancer patients, and their loved ones, appreciate GOCC; however, oncologists often lack formal communication training, institutional support and structures necessary to promote the delivery, documentation, and longitudinal follow-up of GOCC.

Proposed solution

The Alliance of Dedicated Cancer Centers (ADCC), representing 10 U.S. academic cancer hospitals, undertook the Improving Goal Concordant Care Initiative (IGCC). This national, 3-year implementation initiative was designed in Fall 2019 by a workgroup of quality, oncology, and palliative care leaders, as well as patient and family advisory committee members (PFAC). IGCC addresses systemic gaps by requiring four core components for participation: 1) Implementation of a formal communication skills training (CST) program, 2) Structured GOCC documentation in the electronic medical record that is visible to all clinicians, 3) Expectations regarding the timing and patient populations for GOCC, and 4) Implementation of a measurement framework.

Method

Dyads of palliative and oncology leaders committed to attend regularly scheduled, ADCC-led, virtual meetings during the design and implementation phase, incorporating PFAC feedback at every stage. Using the RE-AIM framework, we describe process and outcome evaluation measures defined by implementation and measures workgroups and collected routinely, including: CST completion; trainee evaluation response rate, trainee-reported quality of CST, trainee changes in self-efficacy and distress; percent of high-priority patients participating in GOCC, and patient-reported response to the “Heard and Understood” scale (HU). IGCC's impact will be assessed using claims-based utilization metrics near the end of life (EOLM) and followed longitudinally. Qualitative evaluations near the completion of IGCC will provide insight into perceived barriers, enabling factors, and sustainability.

Outcomes

Implementation of all IGCC components has begun at all sites. ADCC-wide, 35% of MD/DOs have completed CST (range by site: 8%–100%). CST is highly rated; in Quarter 3, 2022, 93%–100%, 90%–100% and 87%–100% of respondents reported above average to excellent CST quality, likelihood to use the skills and likelihood to recommend CST to others, respectively. Clinician self-efficacy and distress ratings are expected in late 2023. All sites have identified patient populations and continue to refine automated triggers and timelines; uptake of GOCC documentation has been slow. Eight of 10 sites have submitted patient-reported HU data. EOLM data are expected for all sites in early 2024.

Lesson learned

Flexibility in implementation with shared definitions, measures, and learnings about approaches optimizes the ability of all centers to collaborate and make progress in improving GOCC. Flexibility adds to the complexity of understanding intervention effectiveness, the critical intervention components and the fidelity necessary to achieve specific outcomes.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

✪ Implementation fidelity to HIV assisted partner services (aPS) during scale-up in western Kenya: a convergent mixed methods study

Abstract:

Background

HIV assisted partner services (aPS) is an intervention to improve HIV status awareness among sex and drug-injecting partners of people newly diagnosed with HIV (index clients). Implementation fidelity—the degree to which an intervention is conducted as intended – is critical to effectiveness, but there are limited data about aPS fidelity when delivered by HIV testing service (HTS) providers. We explored factors affecting implementation fidelity to aPS in two high-HIV prevalence counties in western Kenya.

Methods

We used convergent mixed methods adapting the conceptual framework for implementation fidelity within the aPS scale-up project. This was an implementation study examining scale-up of APS within HTS programs in Kisumu and Homa Bay counties that recruited male sex partners (MSPs) of female index clients. We defined implementation fidelity as the extent to which HTS providers followed the protocol for phone and in-person participant tracing at six expected tracing attempts. Quantitative data were collected from tracing reports in 31 facilities between November 2018 and December 2020, and in-depth interviews (IDIs) were conducted with HTS providers. Descriptive statistics were used to describe tracing attempts. IDIs were analyzed using thematic content analysis.

Results

Overall, 3017 MSPs were mentioned of whom 98% (2969/3017) were traced, with most tracing attempts being successful (2831/2969, 95%). Fourteen HTS providers participated in the IDIs—mostly females (10/14, 71%) with a median age of 35 years (range 25–52), who all had post-secondary education (14/14, 100%). The proportion of tracing attempts occurring by phone ranged from 47 to 66%, with the highest proportion occurring on the first attempt and lowest on the sixth attempt. Contextual factors either enhanced or impeded implementation fidelity to aPS. Positive provider attitudes towards aPS and conducive work environment factors promoted implementation fidelity, while negative MSP responses and challenging tracing conditions impeded it.

Conclusion

Interactions at the individual (provider), interpersonal (client—provider), and health systems (facility) levels affected implementation fidelity to aPS. As policymakers prioritize strategies to reduce new HIV infections, our findings highlight the importance of conducting fidelity assessments to better anticipate and mitigate the impact of contextual factors during the scale-up of interventions.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

✪ Content validity of an item bank to assess intervention characteristic determinants of implementing evidence-based practices

Abstract:

Background

Barriers and facilitators, collectively called determinants, of evidence-based practice implementation are key to identifying the best strategies for promoting implementation. Assessing determinants before implementation can help tailor strategies to those that would be most effective. Current measures of determinants are not comparable across implementation settings and implementation scientists and practitioners often have to create their own measures. This study was the first step in creating determinants item banks that are usable across settings and focused on intervention characteristics. We aimed to establish the content validity of the item bank.

Method

This study used a concurrent mixed methods approach. Items for assessing intervention characteristic determinants were first identified through systematic reviews. Implementation scientists then completed a survey where they provided both quantitative and qualitative feedback on the items. Finally, three experts with both clinical and implementation experience provided feedback on redundancy and representativeness.

Results

The systematic reviews identified over 1,959 items so subsequent steps were limited to focus on intervention characteristic determinants (271 items) such as adaptability of the practice. Based on feedback from thirty implementation scientists, the items were reduced to 92 but an additional 53 were added, most due to qualitative feedback. Items were also rewritten based on qualitative results. Three experts reviewed the remaining 145 items. Based on their feedback, the number of items was reduced to 109.

Conclusions

Creating a determinants item bank was feasible and the final items had content validity. The next steps include testing reliability and validity in a larger sample of clinicians implementing evidence-based practices.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

Authors:

✪ Open Access

Read the full text in the open access journal JMIR Formative Research

Abstract:

Background

Objective

Methods

Results

Conclusions

Authors:

✪ Open Access

Read the full text in the open access journal Implementation Science Communications

Abstract:

Introduction

Methods

Discussion

Authors:

✪ Open Access

Read the full text in the open access journal PLOS Global Public Health

Abstract:

Authors:

✪ Open Access

Read the full text in the open access journal Implementation Research and Practice

Abstract:

Background

Methods

Results

Conclusions

Authors:

✪ Open Access

Read the full text in the open access Journal of Advanced Nursing

Abstract:

Aims

Design

Sample

Methods

Results

Conclusion

Impact

Authors:

✪ Open Access

Read the full text in the open access journal Implementation Science Communication

Abstract:

Background

Methods

Results

Conclusions

Trial registration

Authors:

✪ Open Access

Read the full text in the open access journal Implementation Science

Abstract:

Background

Methods

Discussion

Authors:

✪ Open Access

Read the full text in the open access journal Journal of Pain and Symptom Management

Abstract:

Background

Problem

Proposed solution

Method

Outcomes

Lesson learned

Authors:

✪ Open Access

Read the full text in the open access journal BMC Health Services Research

Abstract:

Background

Methods

Results

Conclusion

Authors:

✪ Open Access

Read the full text in the open access journal Implementation Research and Practice

Abstract:

Background

Method

Results