Skip to content
Implementation Science at UW

Keywords: Implementation Climate

✪ Individual and organizational factors that affect implementation of evidence-based practices for children with autism in public schools: a cross-sectional observational study

Authors:

Jill Locke, Gwendolyn M. Lawson, Rinad S. Beidas, Gregory A. Aarons, Ming Xie, Aaron R. Lyon, Aubyn Stahmer, Max Seidman, Lindsay Frederick, Cristine Oh, Christine Spaulding, Shannon Dorsey & David S. Mandell

University of Washington affiliated authors are displayed in bold.

✪ Open Access

Published: March 2019

Read the full text in the open access journal Implementation Science

Abstract:

Background

Children with autism receive most of their intervention services in public schools, but implementation of evidence-based practices (EBPs) for autism varies. Studies suggest that individual (attitudes) and organizational characteristics (implementation leadership and climate) may influence providers’ use of EBPs, but research is relatively limited in this area. This study examined individual and organizational factors associated with implementation of three EBPs—discrete trial training, pivotal response training, and visual schedules—for children with autism in special education classrooms in public elementary schools.

Methods

Participants included 67 autism support teachers and 85 other classroom staff from 52 public elementary schools in the northeastern United States. Participants reported their attitudes toward EBPs (e.g., intuitive appeal, willingness if required, openness, and divergence), implementation leadership and climate of their school, and the frequency with which they deliver each of three EBPs. Linear regression was used to estimate the association of attitudes about EBPs with organizational characteristics and intensity of EBP use. Demographic covariates with a bivariate association with EBP use significant at p < .20 were entered into the adjusted models.

Results

There were significant findings for only one EBP, discrete trial training. Teachers who reported higher perceived divergence (perceived difference of usual practice with academically developed or research-based practices) between EBPs and current practices used less discrete trial training (ƒ² = .18), and teachers who reported higher appeal (willingness to adopt EBPs given their intuitive appeal) of EBPs used more discrete trial training (ƒ² = .22). No organizational factors were significantly associated with implementation with any of the three EBPs.

Conclusions

Attitudes toward EBPs may affect teachers’ decisions to use EBPs; however, implementation leadership and climate did not predict EBP use. Future implementation efforts ought to consider the type of EBP and its fit within the context in terms of the EBP’s similarities to and differences from existing practices and programs in the setting. Implementation strategies that target individual attitudes about EBPs may be warranted in public schools.

**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**

BACK TO THE LIBRARY

✪ Implementation Climate and Time Predict Intensity of Supervision Content Related to Evidence Based Treatment

Authors:

Michael D. Pullmann, Leah Lucid, Julie P. Harrison, Prerna Martin, Esther Deblinger, Katherine S. Benjamin and Shannon Dorsey
University of Washington affiliated authors are displayed in bold.
Published: October 2018

Read the full text in the open access journal Frontiers in Public Health

Abstract:

Objective

Children infrequently receive evidence-based treatments (EBTs) for mental health problems due to a science-to-practice implementation gap. Workplace-based clinical supervision, in which supervisors provide oversight, feedback, and training on clinical practice, may be a method to support EBT implementation. Our prior research suggests that the intensity of supervisory focus on EBT (i.e., thoroughness of coverage) during workplace-based supervision varies. This study explores predictors of supervisory EBT intensity.

Methods

Participants were twenty-eight supervisors and 70 clinician supervisees. They completed a baseline survey, and audio recorded supervision sessions over 1 year. Four hundred and thirty eight recordings were coded for supervision content. We chose to explore predictors of two EBT content elements due to their strong evidence for effectiveness and sufficient variance to permit testing. These included a treatment technique (“exposure”) and a method to structure treatment (“assessment”). We also explored predictors of non-EBT content (“other topics”). Mixed-effects models explored predictors at organizational/supervisor, clinician, and session levels.

Results

Positive implementation climate predicted greater intensity of EBT content coverage for assessment (coefficient = 0.82, p = 0.004) and exposure (coefficient = 0.87, p = 0.001). Intensity of exposure coverage was also predicted by more time spent discussing each case (coefficient = 0.04, p < 0.001). Predictors of greater non-EBT content coverage included longer duration of supervision sessions (coefficient = 0.05, p < 0.001) and lower levels of supervisor EBT knowledge (coefficient = −0.17, p = 0.013). No other supervisor- or clinician-level variables were significant predictors in the mixed effects models.

Conclusion

This was the first study to explore multi-level predictors of objectively coded workplace-based supervision content. Results suggest that organizations that expect, support and reward EBT are more likely to have greater intensity of EBT supervision coverage, which in turn may positively impact clinician EBT fidelity and client outcomes. There was evidence that supervisor knowledge of the EBT contributes to greater coverage, although robust supervisor and clinician factors that drive supervision are yet to be identified. Findings highlight the potential effectiveness of implementation strategies that simultaneously address organizational implementation climate and supervisor practices. More research is needed to identify mechanisms that support integration of EBT into supervision.

**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**

✪ Developing measures to assess constructs from the Inner Setting domain of the Consolidated Framework for Implementation Research

Authors:

Maria E. Fernandez, Timothy J. Walker, Bryan J. Weiner, William A. Calo, Shuting Liang, Betsy Risendal, Daniela B. Friedman, Shin Ping Tu, Rebecca S. Williams, Sara Jacobs, Alison K. Herrmann and Michelle C. Kegler

University of Washington affiliated authors are displayed in bold.

✪ Open Access

Published: March 2018

Read the full text in the open access journal Implementation Science

Abstract:

Background

Scientists and practitioners alike need reliable, valid measures of contextual factors that influence implementation. Yet, few existing measures demonstrate reliability or validity. To meet this need, we developed and assessed the psychometric properties of measures of several constructs within the Inner Setting domain of the Consolidated Framework for Implementation Research (CFIR).

Methods

We searched the literature for existing measures for the 7 Inner Setting domain constructs (Culture Overall, Culture Stress, Culture Effort, Implementation Climate, Learning Climate, Leadership Engagement, and Available Resources). We adapted items for the healthcare context, pilot-tested the adapted measures in 4 Federally Qualified Health Centers (FQHCs), and implemented the revised measures in 78 FQHCs in the 7 states (N = 327 respondents) with a focus on colorectal cancer (CRC) screening practices. To psychometrically assess our measures, we conducted confirmatory factor analysis models (CFA; structural validity), assessed inter-item consistency (reliability), computed scale correlations (discriminant validity), and calculated inter-rater reliability and agreement (organization-level construct reliability and validity).

Results

CFAs for most constructs exhibited good model fit (CFI > 0.90, TLI > 0.90, SRMR < 0.08, RMSEA < 0.08), with almost all factor loadings exceeding 0.40. Scale reliabilities ranged from good (0.7 ≤ α < 0.9) to excellent (α ≥ 0.9). Scale correlations fell below 0.90, indicating discriminant validity. Inter-rater reliability and agreement were sufficiently high to justify measuring constructs at the clinic-level.

Conclusions

Our findings provide psychometric evidence in support of the CFIR Inner Setting measures. Our findings also suggest the Inner Setting measures from individuals can be aggregated to represent the clinic-level. Measurement of the Inner Setting constructs can be useful in better understanding and predicting implementation in FQHCs and can be used to identify targets of strategies to accelerate and enhance implementation efforts in FQHCs.

**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**

The influence of formal and informal policies and practices on health care innovation implementation: A mixed-methods analysis

Authors:

Lisa D. DiMartino, Sarah A. Birken, Laura C. Hanson, Justin G. Trogdon, Alecia S. Clary, Morris Weinberger, Katherine Reeder-Hayes, & Bryan J. Weiner

University of Washington affiliated authors are displayed in bold.

Subscription Required for Full Text Access

Published: March 2018

Read the full text in the subscription access journal Health Care Management Review