Skip to content

✪ The Cognitive Walkthrough for Implementation Strategies (CWIS): a pragmatic method for assessing implementation strategy usability

Authors:

Aaron R. Lyon, Jessica Coifman, Heather Cook, Erin McRee, Freda F. Liu, Kristy Ludwig, Shannon Dorsey, Kelly Koerner, Sean A. Munson, & Elizabeth McCauley

University of Washington affiliated authors are displayed in bold.

✪ Open Access

Published: July 2021

Read the full text in the open access journal Implementation Science Communications

Abstract:

Background

Implementation strategies have flourished in an effort to increase integration of research evidence into clinical practice. Most strategies are complex, socially mediated processes. Many are complicated, expensive, and ultimately impractical to deliver in real-world settings. The field lacks methods to assess the extent to which strategies are usable and aligned with the needs and constraints of the individuals and contexts who will deliver or receive them. Drawn from the field of human-centered design, cognitive walkthroughs are an efficient assessment method with potential to identify aspects of strategies that may inhibit their usability and, ultimately, effectiveness. This article presents a novel walkthrough methodology for evaluating strategy usability as well as an example application to a post-training consultation strategy to support school mental health clinicians to adopt measurement-based care.

Method

The Cognitive Walkthrough for Implementation Strategies (CWIS) is a pragmatic, mixed-methods approach for evaluating complex, socially mediated implementation strategies. CWIS includes six steps: (1) determine preconditions; (2) hierarchical task analysis; (3) task prioritization; (4) convert tasks to scenarios; (5) pragmatic group testing; and (6) usability issue identification, classification, and prioritization. A facilitator conducted two group testing sessions with clinician users (N = 10), guiding participants through 6 scenarios and 11 associated subtasks. Clinicians reported their anticipated likelihood of completing each subtask and provided qualitative justifications during group discussion. Following the walkthrough sessions, users completed an adapted quantitative assessment of strategy usability.

Results

Average anticipated success ratings indicated substantial variability across participants and subtasks. Usability ratings (scale 0–100) of the consultation protocol averaged 71.3 (SD = 10.6). Twenty-one usability problems were identified via qualitative content analysis with consensus coding, and classified by severity and problem type. High-severity problems included potential misalignment between consultation and clinical service timelines as well as digressions during consultation processes.

Conclusions

CWIS quantitative usability ratings indicated that the consultation protocol was at the low end of the “acceptable” range (based on norms from the unadapted scale). Collectively, the 21 resulting usability issues explained the quantitative usability data and provided specific direction for usability enhancements. The current study provides preliminary evidence for the utility of CWIS to assess strategy usability and generate a blueprint for redesign.

**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**

✪ Building Implementation Science Capacity in Academic Pharmacy: Report of the 2020-2021 AACP Research and Graduate Affairs Committee

Authors:

Kimberly B. Garza, Ephrem Abebe, Jennifer L. Bacci, Nitesh K. Kunda, Kajua B. Lor, Nathaniel Rickles, Nathan Shankar, and Dorothy Farrell

University of Washington affiliated authors are displayed in bold.

✪ Open Access

Published: July 2021

Read the full text in the American Journal of Pharmaceutical Education

Abstract:

Executive Summary

The 2020-2021 AACP Research and Graduate Affairs Committee (RGAC) continued the work begun by the 2019-2020 RGAC to increase awareness of and capacity for implementation research to advance practice transformation in academic pharmacy. AACP President Anne Lin charged the RGAC with developing resources and programs for training faculty and graduate students in implementation science. The committee was further charged with developing a mechanism to pair pharmacy faculty and implementation experts on practice advancement projects. In its work, the committee focused on generating near-term opportunities for pharmacy practice faculty to pursue projects while developing programs that would support ongoing career development and future implementation practice and research by pharmacy faculty and trainees.

To support training, the RGAC developed a competency framework for graduate education that builds on the six domains of graduate education in pharmaceutical sciences put forward by the 2016-2017 RGAC. The new, specialized framework identifies and organizes those competencies that should be cultivated to successfully pursue implementation research in pharmacy, with utility beyond graduate education. The identified competencies could act as the base for pharmacy-specific implementation science training materials suitable for faculty development. The committee proposes a faculty development program that leverages the framework and existing implementation science training resources offered by federal agencies and universities to provide facilitated learning opportunities and peer support for faculty who wish to pursue implementation research.

The committee considered multiple approaches AACP could take to promote collaboration between pharmacy faculty and implementation experts. While recognizing the importance of individual development support for pharmacy faculty, the committee felt that a focus on supporting interdisciplinary teams that span practice and research and engage faculty, trainees and preceptors would have the greatest impact on enabling practice advancement projects in the near term while increasing capacity in the long-term. The committee envisions a program in which AACP creates opportunities for pharmacy faculty and implementation experts to come together with the purpose of matching researchers and projects. AACP support for these collaborations should begin prior to any such matching event, for example by helping pharmacy faculty articulate the scope and value of their proposed practice advancement projects, and extend beyond the initial match to include assisting nascent collaborations establish connections and secure institutional and external backing.

The RGAC made specific recommendations to AACP about how the framework and proposed programs could be used to further develop pharmacy faculty and graduate students as researchers while advancing the goal of sustainable transformation of pharmacy practice. A key recommendation is that AACP cultivate an inclusive community of interest in implementation science in pharmacy that will promote adoption of the framework, act as facilitators and coaches for training programs and assist in planning and supporting collaborative research projects.

Expected Organizational Costs for Inserting Prevalence Information Into Lumbar Spine Imaging Reports

Authors:

Brian W. Bresnahan, Jeffrey G. Jarvik, Eric N. Meier, Kathryn T. James, Laura S. Gold, Sean D. Rundell, Judith A. Turner, Pradeep Suri, Patrick H. Luetmer, Janna L. Friedly, Karen J. Sherman,
Patrick J. Heagerty, David F. Kallmes, Andrew L. Avins, Brent D. Griffith, & Larry G. Kessler

University of Washington affiliated authors are displayed in bold.

Subscription Required for Full Text Access

Published: June 2021

Read the full text in the subscription access Journal of the American College of Radiology

✪ Development of an Asthma Home-Visit Training Program for Community Health Workers and Their Supervisors in Washington State

Authors:

Nuha Elkugia, Mary E. Crocker, James W. Stout, Kaylin Bolt, Bryan J. Weiner, and C. Bradley Kramer

University of Washington affiliated authors are displayed in bold.

✪ Open Access

Published: June 2021

Read the full text in the open access journal Frontiers in Public Health

Abstract:

The community health worker (CHW) asthma home-visiting model developed by Public Health-Seattle & King County (PHSKC) is an evidence-based approach proven to improve health outcomes and quality of life. In addition, it has been shown to be an effective and culturally appropriate approach to helping people with asthma understand the environmental and behavioral causes of uncontrolled asthma, while acquiring the skills they need to control their asthma.

This paper describes the development and implementation of training curricula for CHWs and supervisors in the asthma home visiting program. To facilitate dissemination, this program took advantage of the current healthcare landscape in Washington State resulting from Centers for Medicare & Medicaid Services (CMS) approval of the 1115 Medicaid Waiver project. Key aspects of the training program development included: (1) Engagement: forming a Community Advisory Board with multiple stakeholders to help prioritize training content; (2) Curriculum Development: building the training on evidence-based home-visit protocols previously developed at PHSKC; (3) Implementation of the training program; (4) Evaluation of the training; and (5) Adaptation of the training based on lessons learned.

We describe key factors in the training program's improvement including the use of a community-based participatory approach to engage stakeholders at multiple phases of the project and ensure regional adaption; combining in-person and online modules for delivery; and holding learning collaboratives for post-training and technical support. We also outline our training program evaluation plan and the planned evaluation of the home visit program which the trainees will deliver, both of which follow the RE-AIM framework. However, because the COVID-19 pandemic has curtailed training activities and prohibited the trainees from implementation of these CHW home visit practices, our evaluation is currently incomplete. Therefore, this case study provides insight into the adaptation of the training program, but not the delivery of the home visit program, the outcomes of which remain to be seen.

Exploring expanded interdisciplinary roles in goals of care conversations in a national goals of care initiative: A qualitative approach

Authors:

Jessica E Ma, Marie Haverfield, Karl A Lorenz, David B Bekelman, Cati Brown-Johnson, Natalie Lo, Mary Beth Foglia, Jill S Lowery, Anne M Walling, & Karleen F Giannitrapani

University of Washington affiliated authors are displayed in bold.

Subscription Required for Full Text Access

Published: June 2021

Read the full text in the subscription access journal Palliative Medicine

✪ An implementation evaluation of “Zero Suicide” using normalization process theory to support high-quality care for patients at risk of suicide

Authors:

Julie E. Richards, Gregory E. Simon, Jennifer M. Boggs, Rinad Beidas, Bobbi Jo H. Yarborough, Karen J. Coleman, Stacy A. Sterling, Arne Beck, Jean P. Flores, Cambria Bruschke, Julie Goldstein Grumet, Christine C. Stewart, Michael Schoenbaum, Joslyn Westphal, & Brian K. Ahmedani

University of Washington affiliated authors are displayed in bold.

✪ Open Access

Published: May 2021

Read the full text in the open access journal Implementation Research and Practice

Abstract:

Background

Suicide rates continue to rise across the United States, galvanizing the need for increased suicide prevention and intervention efforts. The Zero Suicide (ZS) model was developed in response to this need and highlights four key clinical functions of high-quality health care for patients at risk of suicide. The goal of this quality improvement study was to understand how six large health care systems operationalized practices to support these functions—identification, engagement, treatment and care transitions.

Methods

Using a key informant interview guide and data collection template, researchers who were embedded in each health care system cataloged and summarized current and future practices supporting ZS, including, (1) the function addressed; (2) a description of practice intent and mechanism of intervention; (3) the target patient population and service setting; (4) when/how the practice was (or will be) implemented; and (5) whether/how the practice was documented and/or measured. Normalization process theory (NPT), an implementation evaluation framework, was applied to help understand how ZS had been operationalized in routine clinical practices and, specifically, what ZS practices were described by key informants (coherence), the current state of norms/conventions supporting these practices (cognitive participation), how health care teams performed these practices (collective action), and whether/how practices were measured when they occurred (reflexive monitoring).

Results

The most well-defined and consistently measured ZS practices (current and future) focused on the identification of patients at high risk of suicide. Stakeholders also described numerous engagement and treatment practices, and some practices intended to support care transitions. However, few engagement and transition practices were systematically measured, and few treatment practices were designed specifically for patients at risk of suicide.

Conclusions

The findings from this study will support large-scale evaluation of the effectiveness of ZS implementation and inform recommendations for implementation of high-quality suicide-related care in health care systems nationwide.

Plain Language Summary

Many individuals see a health care provider prior to death by suicide, therefore health care organizations have an important role to play in suicide prevention. The Zero Suicide model is designed to address four key functions of high-quality care for patients at risk of suicide: (1) identification of suicide risk via routine screening/assessment practices, (2) engagement of patients at risk in care, (3) effective treatment, and (4) care transition support, particularly after hospitalizations for suicide attempts. Researchers embedded in six large health care systems, together caring for nearly 11.5 million patients, are evaluating the effectiveness of the Zero Suicide model for suicide prevention. This evaluation focused on understanding how these systems had implemented clinical practices supporting Zero Suicide. Researchers collected qualitative data from providers, administrators, and support staff in each system who were responsible for implementation of practices supporting Zero Suicide. Normalization process theory, an implementation evaluation framework, was applied following data collection to: (A) help researchers catalog all Zero Suicide practices described, (B) describe the norms/conventions supporting these practices, (C) describe how health care teams were performing these practices, and (D) describe how practices were being measured. The findings from this evaluation will be vital for measuring the effectiveness of different Zero Suicide practices. This work will also provide a blueprint to help health care leaders, providers, and other stakeholders “normalize” new and existing suicide prevention practices in their own organizations.

**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**