Aaron Lyon, Sean A. Munson, Madhu Reddy, Stephen M. Schueller, Elena Agapie, Svetlana Yarosh, Alex Dopp, Ulrica von Thiele Schwarz, Gavin Doherty, Andrea K Graham, Kaylee Payne Kruzan & Rachel Kornfield
University of Washington affiliated authors are displayed in bold.
Published: April 2023
Samantha B Dolan, Rachel Wittenauer, Jessica C Shearer, Anne Njoroge, Penina Onyango, George Owisom, William B Lober, Shan Liu, Nancy Puttkammer, & Peter Rabinowitz
University of Washington affiliated authors are displayed in bold.
Published: March 2023
In an effort to increase vaccination coverage in low-resource settings, digital tools have been introduced to better track immunization records, improve data management practices, and provide improved access to vaccination coverage data for decision-making. Despite the potential of these electronic systems to improve the provision of health services, few digital health interventions have been institutionalized at scale in low- and middle-income countries.
In this paper, we aimed to describe how health care workers in Kenya had integrated an electronic immunization registry into their immunization clinic workflows and to use these findings to inform the development of a refined program theory on the registry’s usability.
Informed by realist methodology, we developed a program theory to explain usability of the electronic immunization registry. We designed a qualitative study based on our theory to describe the barriers and facilitators influencing data entry and use. Qualitative data were collected through semistructured interviews with users and workflow observations of immunization clinic sessions. Our findings were summarized by context-mechanism-outcome relationships formed after analyzing our key themes across interviews and workflow observations. Using these relationships, we were able to identify common rules for future implementers.
Across the 12 facilities included in our study, 19 health care workers were interviewed, and 58 workflow sessions were observed. The common rules developed from our qualitative findings are as follows: rule 1—ensure that the users complete training to build familiarity with the system, understand the value of the system and data, and know where to find support; rule 2—confirm that the system captures all data needed for users to provide routine health care services and is easy to navigate; rule 3—identify work-arounds for poor network, system performance, and too few staff or resources; and rule 4—make users aware of expected changes to their workflow, and how these changes might differ over time and by facility size or number of patients. Upon study completion, we revised the program theory to reflect the importance of the goals and workflows of electronic immunization registries aligning with reality.
We created a deeper understanding of the underlying mechanisms for usability of the registry. We found that the electronic immunization registry had high acceptability among users; however, there were numerous barriers to using the system, even under ideal conditions, causing a misalignment between the system and the reality of the users’ workflows and their environment. Human-centered design and human-factors methods can assist during pilot stages to better align systems with users’ needs and again after scale-up to ensure that interventions are suitable for all user settings."
**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**
Megan Moldestad, Valentina V Petrova, Katie Tirtanadi, Sonali R Mishra, Suparna Rajan, George Sayre, John C Fortney & Heather Schacht Reisinger
University of Washington affiliated authors are displayed in bold.
Published: March 2023
User modifications are common in evidence-based psychosocial interventions (EBPIs) for mental health disorders. Often, EBPIs fit poorly into clinical workflows, require extensive resources, or pose considerable burden to patients and therapists. Implementation science is increasingly researching ways to improve the usability of EBPIs before implementation. A user-centered design can be used to support implementation methods to prioritize user needs and solutions to improve EBPI usability.
Trauma-focused EBPIs are a first-line treatment for patients with posttraumatic stress disorder (PTSD) in the Department of Veterans Affairs. Written exposure therapy (WET) is a brief, trauma-focused EBPI wherein patients handwrite about trauma associated with their PTSD. Initially developed for in-person delivery, WET is increasingly being delivered remotely, and outcomes appear to be equivalent to in-person delivery. However, there are logistical issues in delivering WET via video. In this evaluation, we explored usability issues related to WET telehealth delivery via videoconferencing software and designed a solution for therapist-facing challenges to systematize WET telehealth delivery.
In the Discover phase, both groups had challenges delivering WET and other PTSD therapies via telehealth because of technology issues with videoconferencing software, environmental distractions, and workflow disruptions. Narrative transfer (ie, patients sending handwritten trauma accounts to therapists) was the first target for design solution development as it was deemed most critical to WET delivery. In the Design and Build phase, we identified design constraints and brainstormed solution ideas. This led to the development of 3 solution workflows that were presented to a subgroup of therapist users through cognitive walkthroughs. Meetings with this subgroup allowed workflow refinement to improve narrative transfers. Finally, to facilitate using these workflows, we developed PDF manuals that are being refined in subsequent phases of the implementation project (not mentioned in this paper).
In the Discover phase, both groups had challenges delivering WET and other PTSD therapies via telehealth because of technology issues with videoconferencing software, environmental distractions, and workflow disruptions. Narrative transfer (ie, patients sending handwritten trauma accounts to therapists) was the first target for design solution development as it was deemed most critical to WET delivery. In the Design and Build phase, we identified design constraints and brainstormed solution ideas. This led to the development of 3 solution workflows that were presented to a subgroup of therapist users through cognitive walkthroughs. Meetings with this subgroup allowed workflow refinement to improve narrative transfers. Finally, to facilitate using these workflows, we developed PDF manuals that are being refined in subsequent phases of the implementation project (not mentioned in this paper).
The Discover, Design and Build, and Test framework can be a useful tool for understanding user needs in complex EBPI interventions and designing solutions to user-identified usability issues. Building on this work, an iterative evaluation of the 3 solution workflows and accompanying manuals with therapists and patients is underway as part of a nationwide WET implementation in telehealth settings.
**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**
Aaron R. Lyon, Jessica Coifman, Heather Cook, Erin McRee, Freda F. Liu, Kristy Ludwig, Shannon Dorsey, Kelly Koerner, Sean A. Munson, & Elizabeth McCauley
University of Washington affiliated authors are displayed in bold.
Published: July 2021
Implementation strategies have flourished in an effort to increase integration of research evidence into clinical practice. Most strategies are complex, socially mediated processes. Many are complicated, expensive, and ultimately impractical to deliver in real-world settings. The field lacks methods to assess the extent to which strategies are usable and aligned with the needs and constraints of the individuals and contexts who will deliver or receive them. Drawn from the field of human-centered design, cognitive walkthroughs are an efficient assessment method with potential to identify aspects of strategies that may inhibit their usability and, ultimately, effectiveness. This article presents a novel walkthrough methodology for evaluating strategy usability as well as an example application to a post-training consultation strategy to support school mental health clinicians to adopt measurement-based care.
The Cognitive Walkthrough for Implementation Strategies (CWIS) is a pragmatic, mixed-methods approach for evaluating complex, socially mediated implementation strategies. CWIS includes six steps: (1) determine preconditions; (2) hierarchical task analysis; (3) task prioritization; (4) convert tasks to scenarios; (5) pragmatic group testing; and (6) usability issue identification, classification, and prioritization. A facilitator conducted two group testing sessions with clinician users (N = 10), guiding participants through 6 scenarios and 11 associated subtasks. Clinicians reported their anticipated likelihood of completing each subtask and provided qualitative justifications during group discussion. Following the walkthrough sessions, users completed an adapted quantitative assessment of strategy usability.
Average anticipated success ratings indicated substantial variability across participants and subtasks. Usability ratings (scale 0–100) of the consultation protocol averaged 71.3 (SD = 10.6). Twenty-one usability problems were identified via qualitative content analysis with consensus coding, and classified by severity and problem type. High-severity problems included potential misalignment between consultation and clinical service timelines as well as digressions during consultation processes.
CWIS quantitative usability ratings indicated that the consultation protocol was at the low end of the “acceptable” range (based on norms from the unadapted scale). Collectively, the 21 resulting usability issues explained the quantitative usability data and provided specific direction for usability enhancements. The current study provides preliminary evidence for the utility of CWIS to assess strategy usability and generate a blueprint for redesign.
**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**
Emily R. Haines, Alex Dopp, Aaron R. Lyon, Holly O. Witteman, Miriam Bender, Gratianne Vaisson, Danielle Hitch, & Sarah Birken
University of Washington affiliated authors are displayed in bold.
Published: April 2021
Attempting to implement evidence-based practices in contexts for which they are not well suited may compromise their fidelity and effectiveness or burden users (e.g., patients, providers, healthcare organizations) with elaborate strategies intended to force implementation. To improve the fit between evidence-based practices and contexts, implementation science experts have called for methods for adapting evidence-based practices and contexts and tailoring implementation strategies; yet, methods for considering the dynamic interplay among evidence-based practices, contexts, and implementation strategies remain lacking. We argue that harmonizing the three can be facilitated by user-centered design, an iterative and highly stakeholder-engaged set of principles and methods.
This paper presents a case example in which we used a three-phase user-centered design process to design and plan to implement a care coordination intervention for young adults with cancer. Specifically, we used usability testing to redesign and augment an existing patient-reported outcome measure that served as the basis for our intervention to optimize its usability and usefulness, ethnographic contextual inquiry to prepare the context (i.e., a comprehensive cancer center) to promote receptivity to implementation, and iterative prototyping workshops with a multidisciplinary design team to design the care coordination intervention and anticipate implementation strategies needed to enhance contextual fit.
Our user-centered design process resulted in the Young Adult Needs Assessment and Service Bridge (NA-SB), including a patient-reported outcome measure and a collection of referral pathways that are triggered by the needs young adults report, as well as implementation guidance. By ensuring NA-SB directly responded to features of users and context, we designed NA-SB for implementation, potentially minimizing the strategies needed to address misalignment that may have otherwise existed. Furthermore, we designed NA-SB for scale-up; by engaging users from other cancer programs across the country to identify points of contextual variation which would require flexibility in delivery, we created a tool intended to accommodate diverse contexts.
User-centered design can help maximize usability and usefulness when designing evidence-based practices, preparing contexts, and informing implementation strategies—in effect, harmonizing evidence-based practices, contexts, and implementation strategies to promote implementation and effectiveness.
**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**
Alex R. Dopp, Kathryn E. Parisi, Sean A. Munson, & Aaron R. Lyon
University of Washington affiliated authors are displayed in bold.
Published: February 2020
Innovative approaches are needed to maximize fit between the characteristics of evidence-based practices (EBPs), implementation strategies that support EBP use, and contexts in which EBPs are implemented. Standard approaches to implementation offer few ways to address such issues of fit. We characterized the potential for collaboration with experts from a relevant complementary approach, user-centered design (UCD), to increase successful implementation.
Using purposive and snowball sampling, we recruited 56 experts in implementation (n = 34) or UCD (n = 22). Participants had 5+ years of professional experience (M = 10.31), worked across many settings (e.g., healthcare, education, human services), and were mostly female (59%) and white (73%). Each participant completed a web-based concept mapping structured conceptualization task. They sorted strategies from established compilations for implementation (36 strategies) and UCD (30 strategies) into distinct clusters, then rated the importance and feasibility of each strategy.
We used multidimensional scaling techniques to examine patterns in the sorting of strategies. Based on conceptual clarity and fit with established implementation frameworks, we selected a final set of 10 clusters (i.e., groups of strategies), with five implementation-only clusters, two UCD-only clusters, and three trans-discipline clusters. The highest-priority activities (i.e., above-average importance and feasibility) were the trans-discipline clusters plus facilitate change and monitor change. Implementation and UCD experts sorted strategies into similar clusters, but each gave higher importance and feasibility ratings to strategies/clusters from their own discipline.
In this concept mapping study, experts in implementation and UCD had perspectives that both converged (e.g., trans-discipline clusters, which were all rated as high-priority) and diverged (e.g., in importance/feasibility ratings). The results provide a shared understanding of the alignment between implementation science and UCD, which can increase the impact and sustainability of EBP implementation efforts. Implications for improved collaboration among implementation and UCD experts are discussed.
**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**
Aaron R Lyon, Sean A Munson, Brenna N Renn, David C Atkins, Michael D Pullmann, Emily Friedman, & Patricia A Areán
University of Washington affiliated authors are displayed in bold.
Published: September 2019
This paper presents the protocol for the National Institute of Mental Health (NIMH)–funded University of Washington’s ALACRITY (Advanced Laboratories for Accelerating the Reach and Impact of Treatments for Youth and Adults with Mental Illness) Center (UWAC), which uses human-centered design (HCD) methods to improve the implementation of evidence-based psychosocial interventions (EBPIs). We propose that usability—the degree to which interventions and implementation strategies can be used with ease, efficiency, effectiveness, and satisfaction—is a fundamental, yet poorly understood determinant of implementation.
We present a novel Discover, Design/Build, and Test (DDBT) framework to study usability as an implementation determinant. DDBT will be applied across Center projects to develop scalable and efficient implementation strategies (eg, training tools), modify existing EBPIs to enhance usability, and create usable and nonburdensome decision support tools for quality delivery of EBPIs.
Stakeholder participants will be implementation practitioners/intermediaries, mental health clinicians, and patients with mental illness in nonspecialty mental health settings in underresourced communities. Three preplanned projects and 12 pilot studies will employ the DDBT model to (1) identify usability challenges in implementing EBPIs in underresourced settings; (2) iteratively design solutions to overcome these challenges; and (3) compare the solution to the original version of the EPBI or implementation strategy on usability, quality of care, and patient-reported outcomes. The final products from the center will be a streamlined modification and redesign model that will improve the usability of EBPIs and implementation strategies (eg, tools to support EBPI education and decision making); a matrix of modification targets (ie, usability issues) that are both common and unique to EBPIs, strategies, settings, and patient populations; and a compilation of redesign strategies and the relative effectiveness of the redesigned solution compared to the original EBPI or strategy.
The UWAC received institutional review board approval for the three separate studies in March 2018 and was funded in May 2018.
The outcomes from this center will inform the implementation of EBPIs by identifying cross-cutting features of EBPIs and implementation strategies that influence the use and acceptability of these interventions, actively involving stakeholder clinicians and implementation practitioners in the design of the EBPI modification or implementation strategy solution and identifying the impact of HCD-informed modifications and solutions on intervention effectiveness and quality.
ClinicalTrials.gov NCT03515226 (https://clinicaltrials.gov/ct2/show/NCT03515226), NCT03514394 (https://clinicaltrials.gov/ct2/show/NCT03514394), and NCT03516513 (https://clinicaltrials.gov/ct2/show/NCT03516513).
DERR1-10.2196/14990
**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**
Alex R. Dopp, Kathryn E. Parisi, Sean A. Munson, & Aaron R. Lyon
University of Washington affiliated authors are displayed in bold.
Published: December 2018