Methodology Archives - Page 3 of 5 - Implementation Science at UW

✪ Mediators of measurement-based care implementation in community mental health settings: results from a mixed-methods evaluation

Abstract:

Background

Tailored implementation approaches are touted as superior to standardized ones with the reasoning that tailored approaches afford opportunities to select strategies to resolve determinants of the local context. However, results from implementation trials on this topic are equivocal. Therefore, it is important to explore relevant contextual factors that function as determinants to evaluate if they are improved by tailoring and subsequently associated with changes in implementation outcomes (i.e., via statistical mediation) to better understand how tailoring achieves (or does not achieve) its effects. The present study examined the association between a tailored and standardized implementation approach, contextual factors that might mediate change, and a target implementation outcome in an initiative to implement measurement-based care (specifically the clinical integration of the Patient Health Questionnaire [PHQ-9] for depression) in a community mental health organization.

Methods

Using a cluster randomized control design, twelve community-based mental health clinics were assigned to a tailored or standardized implementation group. Clinicians completed a self-report battery assessing contextual factors that served as candidate mediators informed by the Framework for Dissemination at three time points: baseline, 5 months after active implementation support, and 10 months after sustainment monitoring. A subset of clinicians also participated in focus groups at 5 months. The routine use of the PHQ-9 (implementation outcome) was monitored during the 10-month sustainment period. Multi-level mediation analyses assessed the association between the implementation group and contextual factors and the association between contextual factors and PHQ-9 completion. Quantitative results were then elaborated by analyzing qualitative data from exemplar sites

Results

Although tailored clinics outperformed standard clinics in terms of PHQ-9 completion at the end of active implementation, these group differences disappeared post sustainment monitoring. Perhaps related to this, no significant mediators emerged from our quantitative analyses. Exploratory qualitative analyses of focus group content emphasized the importance of support from colleagues, supervisors, and leadership when implementing clinical innovations in practice.

Conclusions

Although rates of PHQ-9 completion improved across the study, their sustained levels were roughly equivalent across groups and low overall. No mediators were established using quantitative methods; however, several partial quantitative pathways, as well as themes from the qualitative data, reveal fruitful areas for future research.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

✪ The Cognitive Walkthrough for Implementation Strategies (CWIS): a pragmatic method for assessing implementation strategy usability

Abstract:

Background

Implementation strategies have flourished in an effort to increase integration of research evidence into clinical practice. Most strategies are complex, socially mediated processes. Many are complicated, expensive, and ultimately impractical to deliver in real-world settings. The field lacks methods to assess the extent to which strategies are usable and aligned with the needs and constraints of the individuals and contexts who will deliver or receive them. Drawn from the field of human-centered design, cognitive walkthroughs are an efficient assessment method with potential to identify aspects of strategies that may inhibit their usability and, ultimately, effectiveness. This article presents a novel walkthrough methodology for evaluating strategy usability as well as an example application to a post-training consultation strategy to support school mental health clinicians to adopt measurement-based care.

Method

The Cognitive Walkthrough for Implementation Strategies (CWIS) is a pragmatic, mixed-methods approach for evaluating complex, socially mediated implementation strategies. CWIS includes six steps: (1) determine preconditions; (2) hierarchical task analysis; (3) task prioritization; (4) convert tasks to scenarios; (5) pragmatic group testing; and (6) usability issue identification, classification, and prioritization. A facilitator conducted two group testing sessions with clinician users (N = 10), guiding participants through 6 scenarios and 11 associated subtasks. Clinicians reported their anticipated likelihood of completing each subtask and provided qualitative justifications during group discussion. Following the walkthrough sessions, users completed an adapted quantitative assessment of strategy usability.

Results

Average anticipated success ratings indicated substantial variability across participants and subtasks. Usability ratings (scale 0–100) of the consultation protocol averaged 71.3 (SD = 10.6). Twenty-one usability problems were identified via qualitative content analysis with consensus coding, and classified by severity and problem type. High-severity problems included potential misalignment between consultation and clinical service timelines as well as digressions during consultation processes.

Conclusions

CWIS quantitative usability ratings indicated that the consultation protocol was at the low end of the “acceptable” range (based on norms from the unadapted scale). Collectively, the 21 resulting usability issues explained the quantitative usability data and provided specific direction for usability enhancements. The current study provides preliminary evidence for the utility of CWIS to assess strategy usability and generate a blueprint for redesign.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

✪ Harmonizing evidence-based practice, implementation context, and implementation strategies with user-centered design: a case example in young adult cancer care

Abstract:

Background

Attempting to implement evidence-based practices in contexts for which they are not well suited may compromise their fidelity and effectiveness or burden users (e.g., patients, providers, healthcare organizations) with elaborate strategies intended to force implementation. To improve the fit between evidence-based practices and contexts, implementation science experts have called for methods for adapting evidence-based practices and contexts and tailoring implementation strategies; yet, methods for considering the dynamic interplay among evidence-based practices, contexts, and implementation strategies remain lacking. We argue that harmonizing the three can be facilitated by user-centered design, an iterative and highly stakeholder-engaged set of principles and methods.

Methods

This paper presents a case example in which we used a three-phase user-centered design process to design and plan to implement a care coordination intervention for young adults with cancer. Specifically, we used usability testing to redesign and augment an existing patient-reported outcome measure that served as the basis for our intervention to optimize its usability and usefulness, ethnographic contextual inquiry to prepare the context (i.e., a comprehensive cancer center) to promote receptivity to implementation, and iterative prototyping workshops with a multidisciplinary design team to design the care coordination intervention and anticipate implementation strategies needed to enhance contextual fit.

Results

Our user-centered design process resulted in the Young Adult Needs Assessment and Service Bridge (NA-SB), including a patient-reported outcome measure and a collection of referral pathways that are triggered by the needs young adults report, as well as implementation guidance. By ensuring NA-SB directly responded to features of users and context, we designed NA-SB for implementation, potentially minimizing the strategies needed to address misalignment that may have otherwise existed. Furthermore, we designed NA-SB for scale-up; by engaging users from other cancer programs across the country to identify points of contextual variation which would require flexibility in delivery, we created a tool intended to accommodate diverse contexts.

Conclusions

User-centered design can help maximize usability and usefulness when designing evidence-based practices, preparing contexts, and informing implementation strategies—in effect, harmonizing evidence-based practices, contexts, and implementation strategies to promote implementation and effectiveness.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

✪ Measuring characteristics of individuals: An updated systematic review of instruments’ psychometric properties

Abstract:

Background

Identification of psychometrically strong implementation measures could (1) advance researchers’ understanding of how individual characteristics impact implementation processes and outcomes, and (2) promote the success of real-world implementation efforts. The current study advances the work that our team published in 2015 by providing an updated and enhanced systematic review that identifies and evaluates the psychometric properties of implementation measures that assess individual characteristics.

Methods

A full description of our systematic review methodology, which included three phases, is described in a previously published protocol paper. Phase I focused on data collection and involved search string generation, title and abstract screening, full-text review, construct assignment, and measure forward searches. During Phase II, we completed data extraction (i.e., coding psychometric information). Phase III involved data analysis, where two trained specialists independently rated each measurement tool using our psychometric rating criteria.

Results

Our team identified 124 measures of individual characteristics used in mental or behavioral health research, and 123 of those measures were deemed suitable for rating using Psychometric and Pragmatic Evidence Rating Scale. We identified measures of knowledge and beliefs about the intervention (n = 76), self-efficacy (n = 24), individual stage of change (n = 2), individual identification with organization (n = 7), and other personal attributes (n = 15). While psychometric information was unavailable and/or unreported for many measures, information about internal consistency and norms were the most commonly identified psychometric data across all individual characteristics’ constructs. Ratings for all psychometric properties predominantly ranged from “poor” to “good.”

Conclusion

The majority of research that develops, uses, or examines implementation measures that evaluate individual characteristics does not include the psychometric properties of those measures. The development and use of psychometric reporting standards could advance the use of valid and reliable tools within implementation research and practice, thereby enhancing the successful implementation and sustainment of evidence-based practice in community care.

Plain Language Summary

Measurement is the foundation for advancing practice in health care and other industries. In the field of implementation science, the state of measurement is only recently being targeted as an area for improvement, given that high-quality measures need to be identified and utilized in implementation work to avoid developing another research to practice gap. For the current study, we utilized the Consolidated Framework for Implementation Research to identify measures related to individual characteristics’ constructs, such as knowledge and beliefs about the intervention, self-efficacy, individual identification with the organization, individual stage of change, and other personal attributes. Our review showed that many measures exist for certain constructs (e.g., measures related to assessing providers’ attitudes and perceptions about evidence-based practice interventions), while others have very few (e.g., an individual’s stage of change). Also, we rated measures for their psychometric strength utilizing an anchored rating system and found that most measures assessing individual characteristics are in need of more research to establish their evidence of quality. It was also clear from our results that frequency of use/citations does not equate to high quality, psychometric strength. Ultimately, the state of the literature has demonstrated that assessing individual characteristics of implementation stakeholders is an area of strong interest in implementation work. It will be important for future research to focus on clearly delineating the psychometric properties of existing measures for saturated constructs, while for the others the emphasis should be on developing new, high-quality measures and make these available to stakeholders.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

✪ Using a Clinical Workflow Analysis to Enhance eHealth Implementation Planning: Tutorial and Case Study

Abstract:

eHealth apps often fail to improve clinical outcomes due to poor integration with clinical workflow—the sequence and personnel needed to undertake a series of tasks for clinical care. Our central thesis is that eHealth interventions will be more effective if the clinical workflow is studied and taken into consideration for intervention implementation. This paper aims to provide an introductory tutorial on when and how to use a clinical workflow analysis to guide the implementation of eHealth interventions. The tutorial includes a step-by-step guide to conducting a clinical workflow analysis in planning for eHealth implementation.

We began with a description of why a clinical workflow analysis is best completed before the implementation of eHealth interventions. Next, we described 4 steps needed to perform the clinical workflow analysis: the identification of discrete workflow components, workflow assessment, triangulation, and the stakeholder proposal of intervention implementation. Finally, we presented a case study of a clinical workflow analysis, which was conducted during patient visits of patients aged 11 or 12 years from 4 diverse pediatric or family medicine clinics to plan the implementation of a tablet-based app for adolescent vaccination.

Investigators planning the implementation of new eHealth interventions in health care settings can use the presented steps to assess clinical workflow, thereby maximizing the match of their intervention with the clinical workflow. Conducting a prospective workflow study allows for evidence-based planning, identifying potential pitfalls, and increasing stakeholder buy-in and engagement. This tutorial should aid investigators in increasing the successful implementation of eHealth interventions.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

✪ The role of dissemination and implementation science in global breast cancer control programs: Frameworks, methods, and examples

Abstract:

Global disparities in breast cancer outcomes are attributable to a sizable gap between evidence and practice in breast cancer control and management. Dissemination and implementation science (D&IS) seeks to understand how to promote the systematic uptake of evidence‐based interventions and/or practices into real‐world contexts. D&IS methods are useful for selecting strategies to implement evidence‐based interventions, adapting their implementation to new settings, and evaluating the implementation process as well as its outcomes to determine success and failure, and adjust accordingly.

Process models, explanatory theories, and evaluation frameworks are used in D&IS to develop implementation strategies, identify implementation outcomes, and design studies to evaluate these outcomes. In breast cancer control and management, research has been translated into evidence‐based, resource‐stratified guidelines by the Breast Health Global Initiative and others. D&IS should be leveraged to optimize the implementation of these guidelines, and other evidence‐based interventions, into practice across the breast cancer care continuum, from optimizing public education to promoting early detection, increasing guideline‐concordant clinical practice among providers, and analyzing and addressing barriers and facilitators in health care systems. Stakeholder engagement through processes such as co‐creation is critical.

In this article, the authors have provided a primer on the contribution of D&IS to phased implementation of global breast cancer control programs, provided 2 case examples of ongoing D&IS research projects in Tanzania, and concluded with recommendations for best practices for researchers undertaking this work.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

✪ The Breast Health Global Initiative 2018 Global Summit on Improving Breast Healthcare Through Resource‐Stratified Phased Implementation: Methods and overview

Abstract:

Background

The Breast Health Global Initiative (BHGI) established a series of resource‐stratified, evidence‐based guidelines to address breast cancer control in the context of available resources. Here, the authors describe methodologies and health system prerequisites to support the translation and implementation of these guidelines into practice.

Methods

In October 2018, the BHGI convened the Sixth Global Summit on Improving Breast Healthcare Through Resource‐Stratified Phased Implementation. The purpose of the summit was to define a stepwise methodology (phased implementation ) for guiding the translation of resource‐appropriate breast cancer control guidelines into real‐world practice. Three expert consensus panels developed stepwise, resource‐appropriate recommendations for implementing these guidelines in low‐income and middle‐income countries as well as underserved communities in high‐income countries. Each panel focused on 1 of 3 specific aspects of breast cancer care: 1) early detection, 2) treatment, and 3) health system strengthening.

Results

Key findings from the summit and subsequent article preparation included the identification of phased‐implementation prerequisites that were explored during consensus debates. These core issues and concepts are key components for implementing breast health care that consider real‐world resource constraints. Communication and engagement across all levels of care is vital to any effectively operating health care system, including effective communication with ministries of health and of finance, to demonstrate needs, outcomes, and cost benefits.

Conclusions

Underserved communities at all economic levels require effective strategies to deploy scarce resources to ensure access to timely, effective, and affordable health care. Systematically strategic approaches translating guidelines into practice are needed to build health system capacity to meet the current and anticipated global breast cancer burden.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

Development of a Wellness Committee Implementation Index for Workplace Health Promotion Programs in Small Businesses

Using implementation science to close the gap between the optimal and typical practice of quantitative methods in clinical science

Cascade Analysis: An Adaptable Implementation Strategy Across HIV and Non-HIV Delivery Platforms

Anjuli D. Wagner, Sarah Gimbel, Kristjana H. Ásbjörnsdóttir, Peter Cherutich, Joana Coutinho, Jonny Crocker, Emiliae Cruz, Fatima Cuembelo, Vasco Cumbe, McKenna Eastment, Jennifer Einberg, Florencia Floriano, Douglas Gaitho, Brandon L. Guthrie, Grace John-Stewart, Alex H. Kral, Barrot H. Lambdin, Shan Liu, Martin Bcom Maina, Nelia Manaca, Mika Matsuzaki, Loris Mattox, Nancy Mburu, R. Scott McClelland, Mark A. Micek, Ana Olga Mocumbi, Albertoe Muanido, Ruth Nduati, Irene N. Njuguna, Geoffrey Oluoch, Laura B. Oyiengo, Keshet Ronen, Caroline Soi, Bradley H. Wagenaar, George Wanje, Lynn D. Wenger, and Kenneth Sherr

^{University of Washington affiliated authors are displayed in bold.}

Published: December 2019

Authors:

✪ Open Access

Read the full text in the open access journal Implementation Science

Abstract:

Background

Methods

Results

Conclusions

Authors:

✪ Open Access

Read the full text in the open access journal Implementation Science Communications

Abstract:

Background

Method

Results

Conclusions

Authors:

✪ Open Access

Read the full text in the open access journal Implementation Science Communications

Abstract:

Background

Methods

Results

Conclusions

Authors:

✪ Open Access

Read the full text in the open access journal Implementation Research and Practice

Abstract:

Background

Methods

Results

Conclusion

Plain Language Summary

Authors:

✪ Open Access

Read the full text in the open access journal JMIR mHealth and uHealth

Abstract:

Authors:

✪ Open Access

Read the full text in the open access journal Cancer

Abstract:

Authors:

✪ Open Access

Read the full text in the open access journal Cancer

Abstract:

Background

Methods

Results

Conclusions

Authors:

Subscription Required for Full Text Access

Read the full text in the subscription access American Journal of Health Promotion

Authors:

Subscription Required for Full Text Access

Read the full text in the subscription access Journal of Abnormal Psychology

Read the full text in the subscription access JAIDS: Journal of Acquired Immune Deficiency Syndromes