Michael D. Pullmann Archives - Page 2 of 2 - Implementation Science at UW

✪ Use of Human-Centered Design to Improve Implementation of Evidence-Based Psychotherapies in Low-Resource Communities: Protocol for Studies Applying a Framework to Assess Usability 

Abstract:

Background

This paper presents the protocol for the National Institute of Mental Health (NIMH)–funded University of Washington’s ALACRITY (Advanced Laboratories for Accelerating the Reach and Impact of Treatments for Youth and Adults with Mental Illness) Center (UWAC), which uses human-centered design (HCD) methods to improve the implementation of evidence-based psychosocial interventions (EBPIs). We propose that usability—the degree to which interventions and implementation strategies can be used with ease, efficiency, effectiveness, and satisfaction—is a fundamental, yet poorly understood determinant of implementation.

Objective

We present a novel Discover, Design/Build, and Test (DDBT) framework to study usability as an implementation determinant. DDBT will be applied across Center projects to develop scalable and efficient implementation strategies (eg, training tools), modify existing EBPIs to enhance usability, and create usable and nonburdensome decision support tools for quality delivery of EBPIs.

Methods

Stakeholder participants will be implementation practitioners/intermediaries, mental health clinicians, and patients with mental illness in nonspecialty mental health settings in underresourced communities. Three preplanned projects and 12 pilot studies will employ the DDBT model to (1) identify usability challenges in implementing EBPIs in underresourced settings; (2) iteratively design solutions to overcome these challenges; and (3) compare the solution to the original version of the EPBI or implementation strategy on usability, quality of care, and patient-reported outcomes. The final products from the center will be a streamlined modification and redesign model that will improve the usability of EBPIs and implementation strategies (eg, tools to support EBPI education and decision making); a matrix of modification targets (ie, usability issues) that are both common and unique to EBPIs, strategies, settings, and patient populations; and a compilation of redesign strategies and the relative effectiveness of the redesigned solution compared to the original EBPI or strategy.

Results

The UWAC received institutional review board approval for the three separate studies in March 2018 and was funded in May 2018.

Conclusions

The outcomes from this center will inform the implementation of EBPIs by identifying cross-cutting features of EBPIs and implementation strategies that influence the use and acceptability of these interventions, actively involving stakeholder clinicians and implementation practitioners in the design of the EBPI modification or implementation strategy solution and identifying the impact of HCD-informed modifications and solutions on intervention effectiveness and quality.

Trial Registration

ClinicalTrials.gov NCT03515226 (https://clinicaltrials.gov/ct2/show/NCT03515226), NCT03514394 (https://clinicaltrials.gov/ct2/show/NCT03514394), and NCT03516513 (https://clinicaltrials.gov/ct2/show/NCT03516513).

International Registered Report Identifier (IRRID)

DERR1-10.2196/14990

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

✪ The influence of a blended, theoretically-informed pre-implementation strategy on school-based clinician implementation of an evidence-based trauma intervention

Abstract:

Background

Individual-level implementation determinants, such as clinician attitudes, commonly influence the successful adoption of evidence-based practices, but few explicit strategies have been tested with regard to their ability to impact these key mechanisms of change. This paper reports on an initial test of a blended, theoretically informed pre-implementation strategy designed to target malleable individual-level determinants of behavior change. Beliefs and Attitudes for Successful Implementation in Schools (BASIS) is a brief and pragmatic pre-implementation strategy that uses strategic education, social influence techniques, and group-based motivational interviewing to target implementation attitudes, perceived social norms, perceived behavioral control, and behavioral intentions to implement among mental health clinicians working in the education sector.

Methods

As part of a pilot trial, 25 school mental health clinicians were randomized to BASIS (n = 12) or an attention control placebo (n = 13), with both conditions receiving training and consultation in an evidence-based intervention for youth experiencing trauma (the Cognitive Behavioral Intervention for Trauma in Schools). Theorized mechanisms of change (attitudes, perceived social norms, perceived behavioral control, and behavioral intentions) were assessed at baseline, post-training, and 4-month follow-up. Clinician participation in post-training consultation and intervention adoption were also tracked.

Results

A series of regression models and independent sample t tests indicated that BASIS had significant, medium to large effects on the majority of its proximal mechanisms from baseline to post-training. BASIS was also associated with a greater latency between initial training in the intervention and discontinuation of participation in post-training consultation, with clinicians in the BASIS condition persisting in consultation for an average of 134 days versus 32 days for controls, but this difference was not statistically significant. At 4-month follow-up, most differences in the theorized mechanisms had attenuated, and approximately the same small number of BASIS clinicians adopted the trauma intervention as controls.

Conclusion

Findings suggest that the brief BASIS pre-implementation strategy had a significant influence on its proximal mechanisms of change, but that these changes did not persist over time or translate into adoption of the trauma intervention. Implications for theory refinement, revisions to the BASIS protocol, and next steps for research surrounding individual-level implementation strategies are discussed.

Trial Registration

ClinicalTrials.gov Identifier: NCT03791281. Registered 31 December 2018—Retrospectively registered.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

Under the hood: lay counsellor element use in a modular multi-problem transdiagnostic intervention in lower resource countries

✪ Implementation Climate and Time Predict Intensity of Supervision Content Related to Evidence Based Treatment

Abstract:

Objective

Children infrequently receive evidence-based treatments (EBTs) for mental health problems due to a science-to-practice implementation gap. Workplace-based clinical supervision, in which supervisors provide oversight, feedback, and training on clinical practice, may be a method to support EBT implementation. Our prior research suggests that the intensity of supervisory focus on EBT (i.e., thoroughness of coverage) during workplace-based supervision varies. This study explores predictors of supervisory EBT intensity.

Methods

Participants were twenty-eight supervisors and 70 clinician supervisees. They completed a baseline survey, and audio recorded supervision sessions over 1 year. Four hundred and thirty eight recordings were coded for supervision content. We chose to explore predictors of two EBT content elements due to their strong evidence for effectiveness and sufficient variance to permit testing. These included a treatment technique (“exposure”) and a method to structure treatment (“assessment”). We also explored predictors of non-EBT content (“other topics”). Mixed-effects models explored predictors at organizational/supervisor, clinician, and session levels.

Results

Positive implementation climate predicted greater intensity of EBT content coverage for assessment (coefficient = 0.82, p = 0.004) and exposure (coefficient = 0.87, p = 0.001). Intensity of exposure coverage was also predicted by more time spent discussing each case (coefficient = 0.04, p < 0.001). Predictors of greater non-EBT content coverage included longer duration of supervision sessions (coefficient = 0.05, p < 0.001) and lower levels of supervisor EBT knowledge (coefficient = −0.17, p = 0.013). No other supervisor- or clinician-level variables were significant predictors in the mixed effects models.

Conclusion

This was the first study to explore multi-level predictors of objectively coded workplace-based supervision content. Results suggest that organizations that expect, support and reward EBT are more likely to have greater intensity of EBT supervision coverage, which in turn may positively impact clinician EBT fidelity and client outcomes. There was evidence that supervisor knowledge of the EBT contributes to greater coverage, although robust supervisor and clinician factors that drive supervision are yet to be identified. Findings highlight the potential effectiveness of implementation strategies that simultaneously address organizational implementation climate and supervisor practices. More research is needed to identify mechanisms that support integration of EBT into supervision.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

Supervision in Community Mental Health: Understanding Intensity of EBT Focus

✪ Objective coding of content and techniques in workplace-based supervision of an EBT in public mental health

Abstract:

Background

Workplace-based clinical supervision as an implementation strategy to support evidence-based treatment (EBT) in public mental health has received limited research attention. A commonly provided infrastructure support, it may offer a relatively cost-neutral implementation strategy for organizations. However, research has not objectively examined workplace-based supervision of EBT and specifically how it might differ from EBT supervision provided in efficacy and effectiveness trials.

Methods

Data come from a descriptive study of supervision in the context of a state-funded EBT implementation effort. Verbal interactions from audio recordings of 438 supervision sessions between 28 supervisors and 70 clinicians from 17 public mental health organizations (in 23 offices) were objectively coded for presence and intensity coverage of 29 supervision strategies (16 content and 13 technique items), duration, and temporal focus. Random effects mixed models estimated proportion of variance in content and techniques attributable to the supervisor and clinician levels.

Results

Interrater reliability among coders was excellent. EBT cases averaged 12.4 min of supervision per session. Intensity of coverage for EBT content varied, with some discussed frequently at medium or high intensity (exposure) and others infrequently discussed or discussed only at low intensity (behavior management; assigning/reviewing client homework). Other than fidelity assessment, supervision techniques common in treatment trials (e.g., reviewing actual practice, behavioral rehearsal) were used rarely or primarily at low intensity. In general, EBT content clustered more at the clinician level; different techniques clustered at either the clinician or supervisor level.

Conclusions

Workplace-based clinical supervision may be a feasible implementation strategy for supporting EBT implementation, yet it differs from supervision in treatment trials. Time allotted per case is limited, compressing time for EBT coverage. Techniques that involve observation of clinician skills are rarely used. Workplace-based supervision content appears to be tailored to individual clinicians and driven to some degree by the individual supervisor. Our findings point to areas for intervention to enhance the potential of workplace-based supervision for implementation effectiveness.

Trial registration

NCT01800266, Clinical Trials, Retrospectively Registered (for this descriptive study; registration prior to any intervention [part of phase II RCT, this manuscript is only phase I descriptive results])
^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

Authors:

✪ Open Access

Read the full text in the open access journal JMIR Research Protocols

Abstract:

Background

Objective

Methods

Results

Conclusions

Trial Registration

International Registered Report Identifier (IRRID)

Authors:

Read the full text in the open access journal Implementation Science

Abstract:

Background

Methods

Results

Conclusion

Trial Registration

Authors:

Read the full text in the subscription access journal The Cognitive Behaviour Therapist

Authors:

Read the full text in the open access journal Frontiers in Public Health

Abstract:

Objective

Methods

Results

Conclusion

Authors:

Read the full text in the subscription access journal Behavior Therapy

Authors:

✪ Open Access

Read the full text in the open access journal Implementation Science

Abstract:

Background

Methods

Results

Conclusions

Trial registration

Authors:

Subscription Required for Full Text Access

Read the full text in the subscription access journal Administration and Policy in Mental Health and Mental Health Services Research

Authors:

Subscription Required for Full Text Access

Read the full text in the subscription access journal Administration and Policy in Mental Health and Mental Health Services Research