Fidelity Archives - Page 4 of 6 - Implementation Science at UW

✪ Study protocol: Novel Methods for Implementing Measurement-Based Care with youth in Low-Resource Environments (NIMBLE)

Abstract:

Introduction

For youth receiving care in community mental health centers, comorbidities are the rule rather than the exception. Using measurement-based care (MBC), or the routine evaluation of symptoms to inform care decisions, as the foundation of treatment for youth with comorbid problems significantly improves the impact of psychotherapy by focusing care and building engagement and alliance. MBC increases the rate of symptom improvement, detects clients who would otherwise deteriorate, and alerts clinicians to non-responders. Despite its demonstrated utility, MBC is rarely implemented with fidelity; less than 15% of providers report using MBC per recommendations. Previous efforts to support MBC implementation have yielded suboptimal outcomes, in part, due to organizations’ challenges with identifying and prioritizing barriers and selecting and developing strategies to overcome them. New methods are needed for identifying and prioritizing barriers, and matching strategies to barriers to optimize MBC implementation and treatment quality to improve youth mental health outcomes in community settings.

Methods

Pragmatic implementation methods will be piloted in four diverse community mental health centers. Methods include (a) rapid evidence synthesis; (b) rapid ethnography; (c) design kits (e.g., kits with disposable cameras, journals, maps); (d) barrier prioritization, and (e) causal pathway diagramming. These activities will generate actionable barriers; subsequently, we will use facilitated group processes to prioritize barriers and develop causal pathway diagrams to match strategies to barriers to create implementation plans that optimize MBC fidelity (Aim 1). We will track strategy deployment for 6 months, then compare MBC fidelity for another 6 months post-implementation with data from 2 years of historical controls (Aim 2). Finally, we will co-design a toolkit for design kit methods with youth and the practice and scientific communities (Aim 3).

Discussion

Optimizing MBC implementation in community mental health centers could transform youth mental health care by ensuring the most pressing symptoms are targeted early in treatment. The discussion section highlights expected challenges and limits to using the five methods, including recruitment and engagement given the high pressure on community mental health settings.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

Implementation fidelity of the Promoting First Relationships intervention program in a Native community

✪ Association of organizational culture and climate with variation in the clinical outcomes of collaborative care for maternal depression in community health centers

Abstract:

Background

Organizational factors may help explain variation in the effectiveness of evidence-based clinical innovations through implementation and sustainment. This study tested the relationship between organizational culture and climate and variation in clinical outcomes of the Collaborative Care Model (CoCM) for treatment of maternal depression implemented in community health centers.

Method

Organizational cultures and climates of 10 community health centers providing CoCM for depression among low-income women pregnant or parenting were assessed using the organizational social context (OSC) measure. Three-level hierarchical linear models tested whether variation in culture and climate predicted variation in improvement in depression symptoms from baseline to 6.5-month post-baseline for N = 468 women with care ±1 year of OSC assessment. Depression symptomology was measured using the Patient Health Questionnaire (PHQ-9).

Results

After controlling for patient characteristics, case mix, center size, and implementation support, patients served by centers with more proficient cultures improved significantly more from baseline to 6.5-month post-baseline than patients in centers with less proficient cultures (mean improvement = 5.08 vs. 0.14, respectively, p = .020), resulting in a large adjusted effect size of dadj = 0.78. A similar effect was observed for patients served by centers with more functional climates (mean improvement = 5.25 vs. 1.12, p < .044, dadj = 0.65). Growth models indicated that patients from all centers recovered on average after 4 months of care. However, those with more proficient cultures remained stabilized whereas patients served by centers with less proficient cultures deteriorated by 6.5-month post-baseline. A similar pattern was observed for functional climate.

Conclusions

Variation in clinical outcomes for women from historically underserved populations receiving Collaborative Care for maternal depression was associated with the organizational cultures and climates of community health centers. Implementation strategies targeting culture and climate may improve the implementation and effectiveness of integrated behavioral health care for depression.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

✪ A machine learning approach to improve implementation monitoring of family-based preventive interventions in primary care

Abstract:

Background

Evidence-based parenting programs effectively prevent the onset and escalation of child and adolescent behavioral health problems. When programs have been taken to scale, declines in the quality of implementation diminish intervention effects. Gold-standard methods of implementation monitoring are cost-prohibitive and impractical in resource-scarce delivery systems. Technological developments using computational linguistics and machine learning offer an opportunity to assess fidelity in a low burden, timely, and comprehensive manner.

Methods

In this study, we test two natural language processing (NLP) methods [i.e., Term Frequency-Inverse Document Frequency (TF-IDF) and Bidirectional Encoder Representations from Transformers (BERT)] to assess the delivery of the Family Check-Up 4 Health (FCU4Health) program in a type 2 hybrid effectiveness-implementation trial conducted in primary care settings that serve primarily Latino families. We trained and evaluated models using 116 English and 81 Spanish-language transcripts from the 113 families who initiated FCU4Health services. We evaluated the concurrent validity of the TF-IDF and BERT models using observer ratings of program sessions using the COACH measure of competent adherence. Following the Implementation Cascade model, we assessed predictive validity using multiple indicators of parent engagement, which have been demonstrated to predict improvements in parenting and child outcomes.

Results

Both TF-IDF and BERT ratings were significantly associated with observer ratings and engagement outcomes. Using mean squared error, results demonstrated improvement over baseline for observer ratings from a range of 0.83–1.02 to 0.62–0.76, resulting in an average improvement of 24%. Similarly, results demonstrated improvement over baseline for parent engagement indicators from a range of 0.81–27.3 to 0.62–19.50, resulting in an approximate average improvement of 18%.

Conclusions

These results demonstrate the potential for NLP methods to assess implementation in evidence-based parenting programs delivered at scale. Future directions are presented.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

✪ Implementation fidelity to HIV assisted partner services (aPS) during scale-up in western Kenya: a convergent mixed methods study

Abstract:

Background

HIV assisted partner services (aPS) is an intervention to improve HIV status awareness among sex and drug-injecting partners of people newly diagnosed with HIV (index clients). Implementation fidelity—the degree to which an intervention is conducted as intended – is critical to effectiveness, but there are limited data about aPS fidelity when delivered by HIV testing service (HTS) providers. We explored factors affecting implementation fidelity to aPS in two high-HIV prevalence counties in western Kenya.

Methods

We used convergent mixed methods adapting the conceptual framework for implementation fidelity within the aPS scale-up project. This was an implementation study examining scale-up of APS within HTS programs in Kisumu and Homa Bay counties that recruited male sex partners (MSPs) of female index clients. We defined implementation fidelity as the extent to which HTS providers followed the protocol for phone and in-person participant tracing at six expected tracing attempts. Quantitative data were collected from tracing reports in 31 facilities between November 2018 and December 2020, and in-depth interviews (IDIs) were conducted with HTS providers. Descriptive statistics were used to describe tracing attempts. IDIs were analyzed using thematic content analysis.

Results

Overall, 3017 MSPs were mentioned of whom 98% (2969/3017) were traced, with most tracing attempts being successful (2831/2969, 95%). Fourteen HTS providers participated in the IDIs—mostly females (10/14, 71%) with a median age of 35 years (range 25–52), who all had post-secondary education (14/14, 100%). The proportion of tracing attempts occurring by phone ranged from 47 to 66%, with the highest proportion occurring on the first attempt and lowest on the sixth attempt. Contextual factors either enhanced or impeded implementation fidelity to aPS. Positive provider attitudes towards aPS and conducive work environment factors promoted implementation fidelity, while negative MSP responses and challenging tracing conditions impeded it.

Conclusion

Interactions at the individual (provider), interpersonal (client—provider), and health systems (facility) levels affected implementation fidelity to aPS. As policymakers prioritize strategies to reduce new HIV infections, our findings highlight the importance of conducting fidelity assessments to better anticipate and mitigate the impact of contextual factors during the scale-up of interventions.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

✪ Adapting Evidence-Based Falls Prevention Programs for Remote Delivery — Implementation Insights through the RE-AIM Evaluation Framework to Promote Health Equity

Abstract:

COVID-19 disproportionally impacted the health and well-being of older adults—many of whom live with chronic conditions—due to their higher risk of dying and being hospitalized. It also created several secondary pandemics, including increased falls risk, sedentary behavior, social isolation, and physical inactivity due to limitations in mobility from lock-down policies. With falls as the leading cause of preventable death and hospitalizations, it became vital for in-person evidence-based falls prevention programs (EBFPPs) to pivot to remote delivery. In Spring 2020, many EBFPP administrators began re-designing programs for remote delivery to accommodate physical distancing guidelines necessitated by the pandemic. Transition to remote delivery was essential for older adults and persons with disabilities to access EBFPPs for staying healthy, falls and injury free, out of hospitals, and also keeping them socially engaged. We collaborated with the Administration on Community Living (ACL), the National Council on Aging (NCOA), and the National Falls Prevention Resource Center (NFPRC), for an in-depth implementation evaluation of remotely delivered EBFPPs. We examined the process of adapting and implementing four EBFPPs for remote delivery, best practices for implementing the programs remotely within the RE-AIM evaluation framework. This enhances NFPRC’s ongoing work supporting dissemination, implementation, and sustainability of EBFPPs. We purposively sampled organizations for maximum variation in organization and provider type, geographic location, and reach of underserved older populations (Black, Indigenous, or other People of Color (BIPOC), rural, disabilities).

This qualitative evaluation includes provider-level data from semi-structured interviews (N = 22) with program administrators, staff, and leaders. The interview guide included what, why, and how adaptations were made to EBFPP interventions and implementation strategies using Wiltsey-Stirman (2019) adaptations framework (FRAME), reach, and implementation outcomes (acceptability, feasibility, fidelity, and costs; Proctor et al., 2011), focusing on equity to learn for whom these programs were working and opportunities to address inequities. Findings demonstrate remote EBFPPs made planned and fidelity-consistent adaptations to remote delivery in partnership with researchers and community organizations, focusing on participant safety both in program content and delivery. Supports using and accessing technology were needed for delivery sites and leaders to facilitate engagement, and improved over time. While remote EBFPP delivery has increased access to EBFPPs for some populations from the perspective of program administrator, leaders, and staff (e.g., caregivers, rural-dwellers, persons with physical disabilities), the digital divide remains a barrier in access to and comfort using technology. Remote-delivered EBFPPs were acceptable and feasible to delivery organizations and leaders, were able to be delivered with fidelity using adaptations from program developers, but were more resource intensive and costly to implement compared to in-person. This work has important implications beyond the pandemic. Remote delivery has expanded access to groups traditionally underserved by in-person programming, particularly disability communities. This work will help answer important questions about reach, accessibility, feasibility, and cost of program delivery for older adults and people with disabilities at risk for falls, those living with chronic conditions, and communities most vulnerable to disparities in access to health care, health promotion programming, and health outcomes. It will also provide critical information to funders about elements required to adapt EBFPPs proven effective in in-person settings for remote delivery with fidelity to achieve comparable outcomes.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

Lessons from the implementation of a trauma center-based program to support primary care providers in managing opioids and pain after trauma hospitalization

Abstract:

Background

Decreasing exposure to prescription opioids is critical to lowering risk of opioid misuse, overdose and opioid use disorder. This study reports a secondary analysis of a randomized controlled trial implementing an opioid taper support program directed to primary care providers (PCPs) of patients discharged from a level I trauma center to their homes distant from the center, and shares lessons for trauma centers in supporting these patients.

Methods

This longitudinal descriptive mixed-methods study uses quantitative/qualitative data from trial intervention arm patients to examine implementation challenges and outcomes: adoption, acceptability, appropriateness, feasibility, fidelity. In the intervention, a physician assistant (PA) contacted patients after discharge to review their discharge instructions and pain management plan, confirm their PCP’s identity and encourage PCP follow-up. The PA reached out to the PCP to review the discharge instructions and offer ongoing opioid taper and pain management support.

Results

The PA reached 32 of 37 patients randomized to the program. Of these 32, 81% discussed topics not targeted by the intervention (eg, social/financial). The PA identified and reached a PCP’s office for only 51% of patients. Of these, all PCP offices (100% adoption) received one to four consults (mean 1.9) per patient (fidelity). Few consults were with PCPs (22%); most were with medical assistants (56%) or nurses (22%). The PA reported that it was not routinely clear to patients or PCPs who was responsible for post-trauma care and opioid taper, and what the taper instructions were.

Conclusions

This level I trauma center successfully implemented a telephonic opioid taper support program during COVID-19 but adapted the program to allow nurses and medical assistants to receive it. This study demonstrates a critical need to improve care transition from hospitalization to home for patients discharged after trauma.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

✪ “We are the sun for our community:” Partnering with community health workers/promotores to adapt, deliver and evaluate a home-based collaborative care model to improve equity in access to quality depression care for older U.S. Latino adults who are underserved

Abstract:

Background:

While depression is a leading cause of poor health, less than half of older adults receive adequate care. Inequities in both access and outcomes are even more pronounced for socially disadvantaged older adults. The collaborative care model (CCM) has potential to reduce this burden through community-based organizations (CBOs) who serve these populations. However, CCM has been understudied in diverse cultural and resource-constrained contexts. We evaluated the implementation and effectiveness of PEARLS, a home-based CCM adapted with and for community health workers/promotores (CHWs/Ps).

Methods:

We used an instrumental case study design. Our case definition is a community-academic partnership to build CHW/P capacity for evidence-based depression care for older U.S. Latino adults in the Inland Empire region of California (2017–2020). We aimed to understand adaptations to fit local context; acceptability, feasibility, and fidelity; clinical effectiveness; and contextual determinants of implementation success or failure. Data sources included quantitative and qualitative administrative and evaluation data from participants and providers. We used descriptive statistics and paired t-tests to characterize care delivery and evaluate effectiveness post-intervention, and deductive thematic analysis to answer other aims.

Findings:

This case study included 152 PEARLS participants and nine data sources (N = 67 documents). The CBO including their CHWs/Ps partnered with the external implementation team made adaptations to PEARLS content, context, and implementation strategies to support CHWs/Ps and older adults. PEARLS was acceptable, feasible and delivered with fidelity. Participants showed significant reductions in depression severity at 5 months (98% clinical response rate [mean (SD), 13.7 (3.9) drop in pre/post PHQ-9; p < 0.001] and received support for 2.6 social needs on average. PEARLS delivery was facilitated by its relative advantage, adaptability, and trialability; the team's collective efficacy, buy-in, alignment with organization mission, and ongoing reflection and evaluation during implementation. Delivery was challenged by weak partnerships with clinics for participant referral, engagement, reimbursement, and sustainability post-grant funding.

Discussion:

This case study used existing data to learn how home-based CCM was adapted by and for CHWs/Ps to reduce health inequities in late-life depression and depression care among older Latino immigrants. The CBOs and CHWs/Ps strong trust and rapport, addressing social and health needs alongside depression care, and regular internal and external coaching and consultation, appeared to drive successful implementation and effectiveness.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

✪ Project MIMIC (Maximizing Implementation of Motivational Incentives in Clinics): A cluster-randomized type 3 hybrid effectiveness-implementation trial

Abstract:

Background

Opioid-related overdoses and harms have been declared a public health emergency in the United States, highlighting an urgent need to implement evidence-based treatments. Contingency management (CM) is one of the most effective behavioral interventions when delivered in combination with medication for opioid use disorder, but its implementation in opioid treatment programs is woefully limited. Project MIMIC (Maximizing Implementation of Motivational Incentives in Clinics) was funded by the National Institute on Drug Abuse to identify effective strategies for helping opioid treatment programs improve CM implementation as an adjunct to medication. Specific aims will test the impact of two different strategies on implementation outcomes (primary aim) and patient outcomes (secondary aims), as well as test putative mediators of implementation effectiveness (exploratory aim).

Methods

A 3-cohort, cluster-randomized, type 3 hybrid design is used with the opioid treatment programs as the unit of randomization. Thirty programs are randomized to one of two conditions. The control condition is the Addiction Technology Transfer Center (ATTC) Network implementation strategy, which consists of three core approaches: didactic training, performance feedback, and on-going consultation. The experimental condition is an enhanced ATTC strategy, with the same core ATTC elements plus two additional theory-driven elements. The two additional elements are Pay-for-Performance, which aims to increase implementing staff’s extrinsic motivations, and Implementation & Sustainment Facilitation, which targets staff’s intrinsic motivations. Data will be collected using a novel, CM Tracker tool to document CM session delivery, session audio recordings, provider surveys, and patient surveys. Implementation outcomes include CM Exposure (number of CM sessions delivered per patient), CM Skill (ratings of CM fidelity), and CM Sustainment (number of patients receiving CM after removal of support). Patient outcomes include self-reported opioid abstinence and opioid-related problems (both assessed at 3- and 6-months post-baseline).

Discussion

There is urgent public health need to improve the implementation of CM as an adjunct to medication for opioid use disorder. Consistent with its hybrid type 3 design, Project MIMIC is advancing implementation science by comparing impacts of these two multifaceted strategies on both implementation and patient outcomes, and by examining the extent to which the impacts of those strategies can be explained by putative mediators.

Trial registration

This clinical trial has been registered with clinicaltrials.gov (NCT03931174).

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

✪ Implementing Dialectical Behavior Therapy Skills Training for Emotional Problem Solving for Adolescents (DBT STEPS-A) in a Low-Income School

Abstract:

Adolescents living in low-income areas often have high need for mental health supports due to experiences of poverty and trauma, coupled with limited access and availability of such supports. This study investigated the implementation of a socio-emotional learning curriculum titled, “Dialectical Behavior Therapy Skills Training for Emotional Problem Solving for Adolescents (DBT STEPS-A),” which was integrated into health classes in a low-income high school.

While preliminary evidence suggests that DBT STEPS-A can be effective in reducing mental health symptoms in high school students, this study is the first to explore the program’s acceptability, appropriateness, and feasibility when implemented in a low-income school. The implementation presented here also diverged from recommended training protocols due to time and cost limitations. Quantitative and qualitative data were collected from 29 school stakeholders prior to implementation and from 23 school stakeholders post-implementation.

Our results indicate that DBT STEPS-A is acceptable and feasible for teachers involved in offering the program and that more work is needed to address appropriateness of the content for racially and socio-economically diverse students, ease of implementing lessons, and support for teachers using DBT STEPS-A skills outside of class. We conclude with a discussion of key implementation challenges and solutions generated.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

Authors:

✪ Open Access

Read the full text in the open access journal Implementation Science Communications

Abstract:

Introduction

Methods

Discussion

Authors:

Subscription Required for Full Text Access

Read the full text in the subscription access journal Translational Behavioral Medicine

Authors:

✪ Open Access

Read the full text in the open access journal Implementation Research and Practice

Abstract:

Background

Method

Results

Conclusions

Authors:

✪ Open Access

Read the full text in the open access journal Implementation Research and Practice

Abstract:

Background

Methods

Results

Conclusions

Authors:

✪ Open Access

Read the full text in the open access journal BMC Health Services Research

Abstract:

Background

Methods

Results

Conclusion

Authors:

✪ Open Access

Read the full text in the open access journal Prevention Science

Abstract:

Authors:

✪ Open Access

Read the full text in the open access journal Trauma Surgery and Acute Care Open

Abstract:

Background

Methods

Results

Conclusions

Authors:

✪ Open Access

Read the full text in the open access journal Frontiers in Public Health

Abstract:

Background:

Methods:

Findings:

Discussion:

Authors:

✪ Open Access

Read the full text in the open access journal Addiction Science & Clinical Practice

Abstract:

Background

Methods

Discussion

Trial registration

Authors:

✪ Open Access

Read the full text in the open access journal School Mental Health

Abstract: