Shannon Dorsey Archives - Implementation Science at UW

✪ Testing Three Alternate Methods to Direct Observation in Measuring Use of Discrete Youth Cognitive Behavioral Techniques: A Secondary Analysis

Abstract:

Background

Cognitive behavioral therapy (CBT), an umbrella term for therapeutic techniques guided by cognitive behavioral theory, is an evidence-based approach for many psychiatric conditions in youth. A stronger dose of CBT delivery is thought to improve youth clinical outcomes. While a critical indicator of care quality, measuring the use of CBT techniques feasibly and affordably is challenging. Certain CBT techniques (e.g., more concrete and observable) may be easier to measure than others using low-cost methods, such as clinician self-report; however, this has not been studied.

Method

To assess the concordance of three methods of measuring CBT technique use with direct observation (DO), clinicians from 27 community agencies (n = 126; Mage = 37.7 years, SD = 12.8; 76% female) were randomized 1:1:1 to a self-report, chart-stimulated recall (CSR; semistructured interviews with the chart available), or behavioral rehearsal (BR; simulated role-plays) condition. In previous work using a global score aggregating 12 CBT techniques, only BR produced scores that did not differ from DO. This secondary analysis examined the concordance of these alternate methods with DO for each discrete CBT technique, testing for differential concordance across cognitive techniques (e.g., cognitive education) compared to behavioral techniques (e.g., behavioral activation).

Results

Results of three-level mixed effects regression models indicated that BR scores did not differ significantly from DO for any techniques, and for nine techniques, neither did CSR (all ps > .05). Contrastingly, self-report scores differed from DO for all but one technique, with greater concordance for behavioral than cognitive techniques (z = −3.29, p < .001).

Conclusions

Unlike previous findings using an aggregate score, we found that both BR and CSR did not differ significantly from DO for most techniques tested. These findings have implications within implementation research and usual care settings; they support multiple viable measurement methods that are less resource-intensive than DO.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

✪ A randomized controlled trial testing supervision strategies in community mental health

Abstract:

Background

Clinicians need supports beyond training to deliver evidence-based treatments with fidelity. Workplace-based clinical supervision often is a commonly provided support in community mental health, yet too few studies have empirically examined supervision and its impact on clinician fidelity and treatment delivery.

Method

Building on a Washington State-funded evidence-based treatment initiative (CBT+), we conducted a randomized controlled trial (RCT), testing two supervision conditions delivered by workplace-based supervisors (supervisors employed by community mental health organizations). The RCT followed a supervision-as-usual (SAU) phase for comparison. The treatment of focus was trauma-focused cognitive behavioral therapy (TF-CBT). Clinicians (N = 238) from 25 organizations participated in the study across the SAU baseline and RCT phases. In the RCT phase, clinicians were randomized to either symptom and fidelity monitoring (SFM) or SFM and behavioral rehearsal (SFM + BR). For BR, clinicians engaged in a short role play of an upcoming treatment element. Supervisors delivered both conditions, with regular study monitoring for drift. Clinicians audiorecorded therapy sessions with enrolled clients, and masked coders coded a subset of recordings for adherence to TF-CBT. One hundred and thirty-three clinicians had recorded TF-CBT session data for 258 youth. We examined six adherence outcomes, including potential moderators.

Results

Results of generalized estimating equations indicated that there were no real differences on adherence outcomes for experimental conditions (SFM, SFM + BR) compared to SAU. Adherence scores in the baseline SAU phase and the RCT conditions were high. Only one interaction was significant.

Conclusions

Contrary to our hypotheses, we did not see improvements in adherence with the RCT conditions. However, nonsignificant findings seem best explained by clinicians’ acceptable/high adherence in SAU. This study was conducted within the context of a long-standing, state-funded EBT initiative, in which clinicians and their supervisors receive training and support, and in which participating community mental health organizations have adopted and supported TF-CBT.

Trial Registration

ClinicalTrials.gov ID: NCT01800266

Plain Language Summary

Clinicians often receive training in evidence-based treatments (EBTs), but training alone is not enough for clinicians to deliver EBTs well. They need other supports. One of the necessary supports is clinical supervision. In public community mental health organizations, which are often resource-constrained, routine clinical supervision is a commonly available support. However, its potential for supporting high-quality EBT delivery is not well understood. This study looked at supervision and clinicians’ delivery of trauma-focused cognitive behavioral therapy (TF-CBT), an EBT for youth who have mental health problems subsequent to trauma. We found that clinicians who received workplace-based supervision on TF-CBT from TF-CBT-trained supervisors, within the context of a state-funded EBT initiative, delivered TF-CBT well. This initiative supported delivery by providing training and supervisor-specific supports. In a second phase of the study, we rigorously tested supervisors’ use of two specific technique packages. These packages included having supervisors review: (a) graphed client mental health symptoms across treatment and clinicians’ report of TF-CBT elements delivered each session or (b) these plus having clinicians do a brief demonstration of how they would deliver an upcoming TF-CBT element to their client, so supervisors could “see” clinicians’ skill and provide feedback. Contrary to expectations, we did not see important differences in clinicians’ TF-CBT delivery. This may be because the clinicians enrolled in our study were already delivering TF-CBT well and, therefore, there was less opportunity for changes. Future studies should explore if these techniques are more helpful for specific situations or clinicians.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

✪ “A problem shared is a problem solved:” integrating human-centered design and implementation science to optimize lay counselor supervision in Western Kenya

Abstract:

Implementation science and human-centered design (HCD) offer useful frameworks and methods for considering and designing for individuals' needs and preferences when implementing new interventions or technologies in global health. When used in tandem, the two approaches may blend creative and partnered research methods with a focus on the factors necessary to design, implement, and sustain interventions. However, research is needed that describes the process of blending these two approaches and explores the experiences of community partners. This study builds from a stepped-wedge cluster-randomized trial in Western Kenya, wherein teachers and community health volunteers have been trained to provide trauma-focused cognitive behavioral therapy (TF-CBT). Mobile phones emerged as a tool to supervise lay counselors from afar; however, their use was characterized by unique challenges. Informed by human-centered design and implementation science, we first engaged lay counselors (n = 24) and supervisors (n = 3) in individual semi-structured interviews then hosted an in-person participatory workshop to “co-design” solutions to optimize the use of mobile phone supervision. Lay counselors participated in focus group discussions regarding their experiences in the workshop. Focus group transcripts were analyzed using thematic analysis. We describe our approach as well as focus group discussion results. Counselors felt the workshop was a valuable experience to learn new strategies from their colleagues, and they enjoyed the “collaborative spirit” that emerged as they worked together. Counselors felt that varying small and large group discussions fostered participation by creating opportunities for more people to engage and share their thoughts. Counselors suggested the approach be improved by providing more tangible materials (e.g., hand-outs) and more closely following a schedule of activities. It is important to also center stakeholders’ experiences as partners in the research process. Though counselors largely expressed positive sentiments, they also shared valuable suggestions for how to improve participatory research practices in the future.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

✪ Uncovering determinants of perceived feasibility of TF-CBT through coincidence analysis

Abstract:

Introduction

A mental health provider's perception of how well an intervention can be carried out in their context (i.e., feasibility) is an important implementation outcome. This article aims to identify determinants of feasibility of trauma-focused cognitive behavioral therapy (TF-CBT) through a case-based causal approach.

Methods

Data come from an implementation-effectiveness study in which lay counselors (teachers and community health volunteers) implemented a culturally adapted manualized mental health intervention, TF-CBT, delivered to teens who were previously orphaned and were experiencing posttraumatic stress symptoms and prolonged grief in Western Kenya. The intervention team identified combinations of determinants that led to feasibility among teacher- and community health volunteer-counselors through coincidence analysis.

Results

Among teacher-counselors, organizational-level factors (implementation climate, implementation leadership) determined moderate and high levels of feasibility. Among community health volunteer-counselors, a strong relationship between a clinical supervisor and the supervisee was the most influential determinant of feasibility.

Conclusion

Methodology and findings from this article can guide the assessment of determinants of feasibility and the development of implementation strategies for manualized mental health interventions in contexts like Western Kenya.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

✪ Study protocol: Novel Methods for Implementing Measurement-Based Care with youth in Low-Resource Environments (NIMBLE)

Abstract:

Introduction

For youth receiving care in community mental health centers, comorbidities are the rule rather than the exception. Using measurement-based care (MBC), or the routine evaluation of symptoms to inform care decisions, as the foundation of treatment for youth with comorbid problems significantly improves the impact of psychotherapy by focusing care and building engagement and alliance. MBC increases the rate of symptom improvement, detects clients who would otherwise deteriorate, and alerts clinicians to non-responders. Despite its demonstrated utility, MBC is rarely implemented with fidelity; less than 15% of providers report using MBC per recommendations. Previous efforts to support MBC implementation have yielded suboptimal outcomes, in part, due to organizations’ challenges with identifying and prioritizing barriers and selecting and developing strategies to overcome them. New methods are needed for identifying and prioritizing barriers, and matching strategies to barriers to optimize MBC implementation and treatment quality to improve youth mental health outcomes in community settings.

Methods

Pragmatic implementation methods will be piloted in four diverse community mental health centers. Methods include (a) rapid evidence synthesis; (b) rapid ethnography; (c) design kits (e.g., kits with disposable cameras, journals, maps); (d) barrier prioritization, and (e) causal pathway diagramming. These activities will generate actionable barriers; subsequently, we will use facilitated group processes to prioritize barriers and develop causal pathway diagrams to match strategies to barriers to create implementation plans that optimize MBC fidelity (Aim 1). We will track strategy deployment for 6 months, then compare MBC fidelity for another 6 months post-implementation with data from 2 years of historical controls (Aim 2). Finally, we will co-design a toolkit for design kit methods with youth and the practice and scientific communities (Aim 3).

Discussion

Optimizing MBC implementation in community mental health centers could transform youth mental health care by ensuring the most pressing symptoms are targeted early in treatment. The discussion section highlights expected challenges and limits to using the five methods, including recruitment and engagement given the high pressure on community mental health settings.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

✪ Higher rates of mental health screening of adolescents recorded after provider training using simulated patients in a Kenyan HIV clinic: results of a pilot study

Abstract:

Background

Kenyan adolescent girls and young women (AGYW) experience a dual burden of HIV and common mental disorders (CMD). HIV clinics are a key entry point for AGYW in need of integrated CMD and HIV care; however, rates of screening and referral for CMDs are low. Our objective was to test an evidence-based provider training strategy, simulated patient encounters (SPEs), on CMD service delivery for AGYW in a Kenyan HIV clinic.

Methods

This pilot study was conducted in a public HIV clinic in Thika, Kenya from January to November 2021. The simulated patient encounter (SPE) implementation strategy included case script development from prior qualitative work, patient actor training, and a three-day SPE training including four standardized mock clinical encounters followed by quantitative surveys assessing provider competencies for each encounter. We abstracted medical record data related to HIV and CMDs such as HIV status, reason for visit, CMD screening test performed, and counselling or referral information. We conducted an interrupted time series analysis using abstracted HIV and CMD screening rates from AGYW ages 16–25 years visiting the clinic 7 months before and 3 months after SPE training. We used generalized linear models to assess changes in CMD screening rates after training.

Results

A total of 10 providers participated in the training. Competency ratings improved across four mock encounters (mean score from 8.1 to 13.7) between first and fourth encounters. We abstracted all medical records (n = 1,154) including from 888 (76%) AGYW seeking HIV treatment, 243 (21%) seeking prevention services, and 34 (3%) seeking other services. CMD screening rates increased immediately following training from 8 to 21% [relative risk (RR) = 2.57, 95% confidence interval (CI) = 1.34–4.90, p < 0.01]. The 3 months following the SPE training resulted in an 11% relative increase in CMD screening proportion compared to the 7 months pre-SPE (RR: 1.11, 95% CI: 1.04–1.17, p < 0.01). Finally, 1% of all pre-SPE screens resulted in referral versus 5% of post-SPE screens (p = 0.07).

Conclusions

The SPE model is a promising implementation strategy for improving HIV provider competencies and CMD service delivery for adolescents in HIV clinics. Future research is needed to explore effects on adolescent clinical outcomes in larger trials.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

✪ Implementation science in plain language: The use of nonjargon terms to facilitate collaboration

Abstract:

Background

Interdisciplinary collaboration and stakeholder engagement are key ingredients in implementation science research. However, effective and efficient collaboration can be limited by the complexity of implementation science terms. In this article, we argue that the development and use of plain language implementation science terms is an essential step to facilitate collaboration and engagement.

Methods

We present an example of plain language development to portray the process and the potential benefits plain language can have on implementation science research. Implementation scientists and intervention experts codeveloped plain language implementation terms as a part of an implementation-effectiveness trial in western Kenya and in preparation for a stakeholder collaborative design meeting.

Results

The developed plain language terms facilitated wider stakeholder understanding and integration of implementation science findings that could inform the design of a stakeholder-led implementation coaching program.

Conclusions

We encourage the use of the plain language terms presented in this article, further translation, and additional development of other plain language terms for implementation science constructs.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

✪ Co-developed implementation guidelines to maximize acceptability, feasibility, and usability of mobile phone supervision in Kenya

Abstract:

Opportunities exist to leverage mobile phones to replace or supplement in-person supervision of lay counselors. However, contextual variables, such as network connectivity and provider preferences, must be considered. Using an iterative and mixed methods approach, we co-developed implementation guidelines to support the implementation of mobile phone supervision with lay counselors and supervisors delivering a culturally adapted trauma-focused cognitive behavioral therapy in Western Kenya. Guidelines were shared and discussed with lay counselors in educational outreach visits led by supervisors. We evaluated the impact of guidelines and outreach on the acceptability, feasibility, and usability of mobile phone supervision. Guidelines were associated with significant improvements in acceptability and usability of mobile phone supervision. There was no evidence of a significant difference in feasibility. Qualitative interviews with lay counselors and supervisors contextualized how guidelines impacted acceptability and feasibility – by setting expectations for mobile phone supervision, emphasizing importance, increasing comfort, and sharing strategies to improve mobile phone supervision. Introducing and discussing co-developed implementation guidelines significantly improved the acceptability and usability of mobile phone supervision. This approach may provide a flexible and scalable model to address challenges with implementing evidence-based practices and implementation strategies in lower-resourced areas.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

Acute and intensive care nurses’ perspectives on suicide prevention with medically hospitalized patients: Exploring barriers, facilitators, interests, and training opportunities

✪ The Cognitive Walkthrough for Implementation Strategies (CWIS): a pragmatic method for assessing implementation strategy usability

Abstract:

Background

Implementation strategies have flourished in an effort to increase integration of research evidence into clinical practice. Most strategies are complex, socially mediated processes. Many are complicated, expensive, and ultimately impractical to deliver in real-world settings. The field lacks methods to assess the extent to which strategies are usable and aligned with the needs and constraints of the individuals and contexts who will deliver or receive them. Drawn from the field of human-centered design, cognitive walkthroughs are an efficient assessment method with potential to identify aspects of strategies that may inhibit their usability and, ultimately, effectiveness. This article presents a novel walkthrough methodology for evaluating strategy usability as well as an example application to a post-training consultation strategy to support school mental health clinicians to adopt measurement-based care.

Method

The Cognitive Walkthrough for Implementation Strategies (CWIS) is a pragmatic, mixed-methods approach for evaluating complex, socially mediated implementation strategies. CWIS includes six steps: (1) determine preconditions; (2) hierarchical task analysis; (3) task prioritization; (4) convert tasks to scenarios; (5) pragmatic group testing; and (6) usability issue identification, classification, and prioritization. A facilitator conducted two group testing sessions with clinician users (N = 10), guiding participants through 6 scenarios and 11 associated subtasks. Clinicians reported their anticipated likelihood of completing each subtask and provided qualitative justifications during group discussion. Following the walkthrough sessions, users completed an adapted quantitative assessment of strategy usability.

Results

Average anticipated success ratings indicated substantial variability across participants and subtasks. Usability ratings (scale 0–100) of the consultation protocol averaged 71.3 (SD = 10.6). Twenty-one usability problems were identified via qualitative content analysis with consensus coding, and classified by severity and problem type. High-severity problems included potential misalignment between consultation and clinical service timelines as well as digressions during consultation processes.

Conclusions

CWIS quantitative usability ratings indicated that the consultation protocol was at the low end of the “acceptable” range (based on norms from the unadapted scale). Collectively, the 21 resulting usability issues explained the quantitative usability data and provided specific direction for usability enhancements. The current study provides preliminary evidence for the utility of CWIS to assess strategy usability and generate a blueprint for redesign.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

Authors:

✪ Open Access

Read the full text in the open access journal Implementation Research and Practice

Abstract:

Background

Method

Results

Conclusions

Authors:

✪ Open Access

Read the full text in the open access journal Implementation Research and Practice

Abstract:

Background

Method

Results

Conclusions

Trial Registration

Plain Language Summary

Authors:

✪ Open Access

Read the full text in the open access journal Social Science and Medicine - Mental Health

Abstract:

Authors:

✪ Open Access

Read the full text in the open access journal Implementation Research and Practice

Abstract:

Introduction

Methods

Results

Conclusion

Authors:

✪ Open Access

Read the full text in the open access journal Implementation Science Communications

Abstract:

Introduction

Methods

Discussion

Authors:

✪ Open Access

Read the full text in the open access journal Frontiers in Public Health

Abstract:

Background

Methods

Results

Conclusions

Authors:

✪ Open Access

Read the full text in the open access journal Implementation Research and Practice

Abstract:

Background

Methods

Results

Conclusions

Authors:

✪ Open Access

Read the full text in the open access journal Cambridge Prisms: Global Mental Health

Abstract:

Authors:

Subscription Required for Full Text Access

Read the full text in the subscription access journal Journal of Advanced Nursing

Authors:

✪ Open Access

Read the full text in the open access journal Implementation Science Communications

Abstract:

Background

Method

Results

Conclusions