✪ School-Partnered Collaborative Care (SPACE) for Pediatric Type 1 Diabetes: Development and Usability Study of a Virtual Intervention With Multisystem Community Partners

Abstract:

Background

School-partnered interventions may improve health outcomes for children with type 1 diabetes, though there is limited evidence to support their effectiveness and sustainability. Family, school, or health system factors may interfere with intervention usability and implementation.

Objective

To identify and address potential implementation barriers during intervention development, we combined methods in user-centered design and implementation science to adapt an evidence-based psychosocial intervention, the collaborative care model, to a virtual school-partnered collaborative care (SPACE) model for type 1 diabetes between schools and diabetes medical teams.

Methods

We recruited patient, family, school, and health system partners (n=20) to cocreate SPACE through iterative, web-based design sessions using a digital whiteboard (phase 1). User-centered design methods included independent and group activities for idea generation, visual voting, and structured critique of the evolving SPACE prototype. In phase 2, the prototype was evaluated with the usability evaluation for evidence-based psychosocial interventions methods. School nurses reviewed the prototype and tasks in cognitive walkthroughs and completed the Intervention Usability Scale (IUS). Two members of the research team independently identified and prioritized (1-3 rating) discrete usability concerns. We evaluated the relationship between prioritization and the percentage of nurses reporting each usability issue with Spearman correlation. Differences in IUS scores by school nurse characteristics were assessed with ANOVA.

Results

In the design phase, the partners generated over 90 unique ideas for SPACE, prioritizing elements pertaining to intervention adaptability, team-based communication, and multidimensional outcome tracking. Following three iterations of prototype development, cognitive walkthroughs were completed with 10 school nurses (n=10, 100% female; mean age 48.5, SD 9.5 years) representing different districts and years of experience. Nurses identified 16 discrete usability issues (each reported by 10%-60% of participants). Two issues receiving the highest priority (3.0): ability to access a virtual platform (n=3, 30% of participants) and data-sharing mechanisms between nurses and providers (n=6, 60% of participants). There was a moderate correlation between priority rating and the percentage of nurses reporting each issue (ρ=0.63; P=.01). Average IUS ratings (77.8, SD 11.1; 100-point scale) indicated appropriate usability. There was no difference in IUS ratings by school nurse experience (P=.54), student caseload (P=.12), number of schools covered (P=.90), or prior experience with type 1 diabetes (P=.83), suggesting that other factors may influence usability. The design team recommended strategies for SPACE implementation to overcome high-priority issues, including training users on videoconferencing applications, establishing secure forms for school data reporting, and sharing glucose data in real-time during SPACE meetings.

Conclusions

Cross-sector interventions are complex, and perceived usability is a potential barrier to implementation. Using web-based cocreation methods with community partners promoted high-quality intervention design that is aligned with end-user priorities. Quantitative and qualitative assessments indicated appropriate degree of usability to move forward with pilot-testing.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

✪ Usability, Acceptability, and Barriers to Implementation of a Collaborative Agenda-Setting Intervention (CASI) to Promote Person-Centered Ovarian Cancer Care: Development Study

Abstract:

Background

People with advanced ovarian cancer and their caregivers report unmet supportive care needs. We developed a Collaborative Agenda-Setting Intervention (CASI) to elicit patients’ and caregivers’ needs through the patient portal before a clinic visit and to communicate these needs to clinicians using the electronic health record.

Objective

We aimed to assess the usability and acceptability of the CASI and identify barriers to and facilitators of its implementation.

Methods

We recruited English- and Spanish-speaking patients, caregivers, and clinicians from the gynecologic oncology program at a comprehensive cancer center. Participants used the CASI prototype and then completed individual cognitive interviews and surveys. We assessed usability with the System Usability Scale (scores range 0‐100, scores ≥70 indicate acceptable usability) and acceptability with the Acceptability of Intervention Measure and Intervention Appropriateness Measure (scores for both measures range from 1 to 5, higher scores indicate greater acceptability). Interviews were audio recorded, transcribed, and analyzed using directed content analysis. Domains and constructs from the Consolidated Framework for Implementation Research comprised the initial codebook. We analyzed survey data using descriptive statistics and compared usability and acceptability scores across patients, caregivers, and clinicians using analyses of variance.

Results

We enrolled 15 participants (5 patients, 5 caregivers, and 5 clinicians). The mean System Usability Scale score was 72 (SD 16). The mean Acceptability of Intervention Measure and Intervention Appropriateness Measure scores were 3.9 (SD 1.0) and 4.1 (SD 0.8), respectively. Participants viewed the CASI content and format positively overall. Several participants appreciated the CASI’s integration into the clinical workflow and its potential to increase attention to psychosocial concerns. Suggestions to refine the CASI included removing redundant items, simplifying item language, and adding options to request a conversation or opt out of supportive care referrals. Key barriers to implementing the CASI include its complexity and limited resources available to address patients’ and caregivers’ needs.

Conclusions

The CASI is usable and acceptable to patients with advanced ovarian cancer, caregivers, and clinicians. We identified several barriers to and facilitators of implementing the CASI. In future research, we will apply these insights to a pilot randomized controlled trial to assess the feasibility of comparing the CASI to usual care in a parallel group-randomized efficacy trial.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

✪ Usability of a cross-system housing stability program for juvenile courts: A multimethod study of probation counselor perspectives

Abstract:

Background

Youth in contact with the juvenile legal system experience disproportionate rates of homelessness. While system contact is a critical intervening point, juvenile courts do not typically offer housing services. One solution is to refer youth to evidence-based, community-based services to meet housing-related needs, but a myriad of individual and organizational factors often impede court staff from making such referrals. Housing Stability for Youth in Courts (HSYNC) is a novel, cross-system service linkage model for court-involved youth facing housing instability. HSYNC was developed using codesign as a strategy to improve usability and incorporated evidence on juvenile court linkage strategies effective at increasing service referrals and improving outcomes.

Method

The current study utilized a multimethod approach to evaluate the usability of HSYNC from the perspective of juvenile probation counselors (JPCs) as one group of end-users. As part of the pilot implementation study, survey (N = 15) and semi-structured interview (N = 20) data were obtained from JPCs from four juvenile courts in one northwestern state.

Results

Survey results indicated good usability for HSYNC (mean implementation strategy usability scale score of 80). Themes from the interviews complemented survey findings, pointing to HSYNC as a value-added program that integrated well with existing probation practices. The housing navigator's approach was highlighted as a strong facilitator of program usability. Themes also provided greater insight into implementation barriers (i.e., individual and contextual factors) and areas for improvement (e.g., eligibility screening process).

Conclusions

Drivers of program usability and implications for program improvement to address identified implementation challenges to ensure the HSYNC model is effective and sustainable long-term are discussed.

Plain Language Summary

A study examining the key factors impacting successful implementation of a novel housing stability program to address homelessness risk for court-involved youth and their families.
Court-involved youth experience disproportionate rates of homelessness, yet housing services are not traditionally offered by juvenile courts. Court staff, such as probation staff, can refer youth to community-based services to meet their housing needs. Unfortunately, many factors complicate whether these referrals are made, including staff feeling overburdened by other work obligations and uncertainty about whether referring youth to community services fits within their job responsibilities. In this study, we surveyed and interviewed probation staff who were responsible for implementing a new referral-based housing stability program—Housing Stability for Youth in Courts (HSYNC)—to understand whether the program addressed previously identified barriers to referral-making. A unique feature of HSYNC is that there is a dedicated housing navigator, who works as a “linkage specialist” to support probation staff in making referrals while working directly with youth and families to access housing-related services in the community. Our findings suggest that, from the perspective of probation staff, HSYNC is considered to be a user-friendly program that is well-aligned with existing court practices. The housing navigator and their unique approach to working with probation staff, youth, and families was perceived as the strongest facilitator of implementation success; meanwhile, screening for program eligibility was the biggest barrier, pointing to the need for improvement before HSYNC is widely implemented. The persistent intersection of youth housing instability and court involvement calls for better coordination between juvenile courts and community agencies providing housing-related services. From the probation perspective, HSYNC offers a viable solution to meeting youths’ housing needs without creating additional workload burden for court staff.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

✪ Human-Centered Design of an mHealth Tool for Optimizing HIV Index Testing in Wartime Ukraine: Formative Research Case Study

Abstract:

Background

Assisted partner services (APSs; sometimes called index testing) are now being brought to scale as a high-yield HIV testing strategy in many nations. However, the success of APSs is often hampered by low levels of partner elicitation. The Computer-Assisted Self-Interview (CASI)–Plus study sought to develop and test a mobile health (mHealth) tool to increase the elicitation of sexual and needle-sharing partners among persons with newly diagnosed HIV. CASI-Plus provides client-facing information on APS methods and uses a standardized, self-guided questionnaire with nonjudgmental language for clients to list partners who would benefit from HIV testing. The tool also enables health care workers (HCWs) to see summarized data to facilitate partner tracking.

Objective

The formative research phase of the CASI-Plus study aimed to gather client and HCW input on the design of the CASI-Plus tool to ensure its acceptability, feasibility, and usability.

Methods

This study gathered input to prioritize features and tested the usability of CASI-Plus with HCWs and clients receiving HIV services in public health clinics in wartime Ukraine. The CASI-Plus study’s formative phase, carried out from May 2023 to July 2024, adapted human-centered design (HCD) methods grounded in principles of empathy, iteration, and creative ideation. The study involved 3 steps: formative HCD, including in-depth individual interviews with clients, such as men who have sex with men and people who inject drugs, and internet-based design workshops with HCWs from rural and urban HIV clinics in Chernihiv and Dnipro; software platform assessment and heuristic evaluation, including assessment of open-source mHealth platforms against CASI-Plus requirements, prototype development, and testing of the REDCap (Research Electronic Data Capture) prototype based on usability heuristics; and usability walk-throughs, including simulated cases with HCWs and clients.

Results:

The formative phase of the CASI-Plus study included in-depth individual interviews with 10 clients and 3 workshops with 22 HCWs. This study demonstrated how simplified HCD methods, adapted to the wartime context, gathered rich input on prioritized features and tool design. The CASI-Plus design reflected features that are both culturally sensitive and in alignment with the constraints of Ukraine’s wartime setting. Prioritized features included information about the benefits of HIV index testing; a nonjudgmental, self-guided questionnaire to report partners; client stories; and bright images to accompany the text. Two-way SMS text messaging between clients and HCWs was deemed impractical based on risks of privacy breaches, national patient privacy regulations, and HCW workload./p>

Conclusions

It was feasible to conduct HCD research in Ukraine in a wartime setting. The CASI-Plus mHealth tool was acceptable to both HCWs and clients. The next step for this research is a randomized clinical trial of the effect of the REDCap-based CASI-Plus tool on the number of partners named and the rate of partners completing HIV testing.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

✪ Harnessing Human-Centered Design for Evidence-Based Psychosocial Interventions and Implementation Strategies in Community Settings: Protocol for Redesign to Improve Usability, Engagement, and Appropriateness

Authors:

Aaron R Lyon, Sean A Munson, Michael D Pullmann, Brittany Mosser, Tricia Aung , John Fortney, Alex Dopp, Katie P Osterhage, Helen G Haile, Kathryn E Bruzios, Brittany E Blanchard, Ryan Allred, Macey R Fuller, Patrick J Raue, Ian Bennett, Jill Locke, Karen Bearss, Denise Walker, Elizabeth Connors, Eric J Bruns, Jenna Van Draanen, Doyanne Darnell, Patricia A Areán

^{University of Washington affiliated authors are displayed in bold.}

✪ Open Access

Published: January 2025

Read the full text in the open access journal JMIR Research Protocols

Abstract:

Background

Although substantial progress has been made in establishing evidence-based psychosocial clinical interventions and implementation strategies for mental health, translating research into practice—particularly in more accessible, community settings—has been slow.

Objective

This protocol outlines the renewal of the National Institute of Mental Health–funded University of Washington Advanced Laboratories for Accelerating the Reach and Impact of Treatments for Youth and Adults with Mental Illness Center, which draws from human-centered design (HCD) and implementation science to improve clinical interventions and implementation strategies. The Center’s second round of funding (2023-2028) focuses on using the Discover, Design and Build, and Test (DDBT) framework to address 3 priority clinical intervention and implementation strategy mechanisms (ie, usability, engagement, and appropriateness), which we identified as challenges to implementation and scalability during the first iteration of the center. Local redesign teams work collaboratively and share decision-making to carry out DDBT.

Methods

All 4 core studies received institutional review board approval by June 2024, and each pilot project will pursue institutional review board approval when awarded. We will provide research infrastructure to 1 large effectiveness study and 3 exploratory pilot studies as part of the center grant. At least 4 additional small pilot studies will be solicited and funded by the center. All studies will explore the use of DDBT for clinical interventions and implementation strategies to identify modification targets to improve usability, engagement, and appropriateness in accessible nonspecialty settings (Discover phase); develop redesign solutions with local teams to address modification targets (Design and Build phase); and determine if redesign improves usability, engagement, and appropriateness (Test phase), as well as implementation outcomes. Center staff will collaborate with local redesign teams to develop and test clinical interventions and implementation strategies for community settings. We will collaborate with teams to use methods and centerwide measures that facilitate cross-project analysis of the effects of DDBT-driven redesign on outcomes of interest.

Results

As of January 2025, three of the 4 core studies are underway. We will generate additional evidence on the robustness of DDBT and whether combining HCD and implementation science is an asset for improving clinical interventions and implementation strategies.

Conclusions

During the first round of the center, we established that DDBT is a useful approach to systematically identify and address chronic challenges of implementing clinical interventions and implementation strategies. In this subsequent grant, we expect to increase evidence of DDBT’s impact on clinical interventions and implementation strategies by expanding a list of common challenges that could benefit from modification, a list of exemplary solutions to address these challenges, and guidance on using the DDBT framework. These resources will contribute to broader discourse on how to enhance implementation of clinical interventions and implementation strategies that integrate HCD and implementation science.

International Registered Report Identifier (IRRID)

PRR1-10.2196/65446

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

Human-Centered Design to Enhance Implementation and Impact in Health

✪ Applying the Cognitive Walkthrough for Implementation Strategies methodology to inform the redesign of a selection-quality implementation toolkit for use in schools

Abstract:

Background

Implementation strategies are key to enhancing the translation of new innovations but there is a need to systematically design and tailor strategies to match the targeted implementation context and address determinants. There are increasing methods to inform the development and tailoring of implementation strategies to maximize their usability, feasibility, and appropriateness in new settings such as the Cognitive Walkthrough for Implementation Strategies (CWIS) approach. The aim of the current project is to apply the CWIS approach to inform the redesign of a multifaceted selection-quality implementation toolkit entitled Adoption of Curricular supports Toolkit: Systematic Measurement of Appropriateness and Readiness for Translation in Schools (ACT SMARTS) for use in middle and high schools.

Methods

We systematically applied CWIS as the second part of a community-partnered iterative redesign of ACT SMARTS for schools to evaluate the usability and inform further toolkit redesign areas. We conducted three CWIS user testing sessions with key end users of school district administrators (n = 3), school principals (n = 6), and educators (n = 6).

Results

Our CWIS application revealed that end users found ACT SMARTS acceptable and relevant but anticipate usability issues engaging in the ACT SMARTS process. Results informed the identification of eleven usability issues and corresponding redesign solutions to enhance the usability of ACT SMARTS for use in middle and high schools.

Conclusions

Results indicated the utility of CWIS in assessing implementation strategy usability in service of informing strategy modification as part of our broader redesign to improve alignment with end user, end recipient, and setting needs. Recommendations regarding the use of this participatory approach are discussed.

Contributions to the literature

The current case study adds to the nascent literature applying the Cognitive Walkthrough for Implementation Strategies (CWIS) method as part of a larger implementation project, including one of the first independent applications outside of the CWIS method developers.
The current case study illustrates the impact and utility of the CWIS method for designing and tailoring implementation strategies.
This case study provides further guidance and recommendations for the application of CWIS to future implementation projects.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

✪ Implementation Strategies Used by Facilitators to Improve Control of Cardiovascular Risk Factors in Primary Care

✪ Helping Educational Leaders Mobilize Evidence (HELM): The iterative redesign of the Leadership and Organizational Change for Implementation (LOCI) intervention for use in schools

Abstract:

Background

Few “intervention agnostic” strategies have been developed that can be applied to the broad array of evidence-based practices (EBPs) in schools. This paper describes two studies that reflect the initial iterative redesign phases of an effective leadership-focused implementation strategy—Leadership and Organizational Change for Implementation (LOCI)—to ensure its acceptability, feasibility, contextual appropriateness, and usability when used in elementary schools. Our redesigned strategy—Helping Educational Leaders Mobilize Evidence (HELM)—is designed to improve principals’ use of strategic implementation leadership to support the adoption and high-fidelity delivery of a universal EBP to improve student outcomes.

Method

In Study 1, focus groups were conducted (n = 6) with 54 district administrators, principals, and teachers. Stakeholders provided input on the appropriateness of original LOCI components to maximize relevance and utility in schools. Transcripts were coded using conventional content analysis. Key themes referencing low appropriateness were summarized to inform LOCI adaptations. We then held a National Expert Summit (Study 2) with 15 research and practice experts. Participants provided feedback via a nominal group process (NGP; n = 6 groups) and hackathon (n = 4 groups). The research team rated each NGP suggestion for how actionable, impactful/effective, and feasible it was. We also coded hackathon notes for novel ideas or alignment with LOCI components.

Results

Study 1 suggestions included modifications to LOCI content and delivery. Study 2's NGP results revealed most recommendations to be actionable, impactful/effective, and feasible. Hackathon results surfaced two novel ideas (distributed leadership teams and leaders’ knowledge to support educators EBP use) and several areas of alignment with LOCI components.

Conclusion

Use of these iterative methods informed the redesign of LOCI and the development of HELM. Because it was collaboratively constructed, HELM has the potential to be an effective implementation strategy to support the use of universal EBP in schools.

Plain Language Summary

Our research team designed a strategy (HELM) for school principals to improve the support they provide to staff to implement practices proven to work in research for improving student outcomes. We designed HELM by conducting focus groups with school district administrators, principals, and teachers. Participants were asked for their feedback on how to adapt an existing leadership strategy (LOCI) to the school context. After collecting this feedback, we held a meeting with 15 research and practice experts. During this meeting, the group of experts reviewed the focus group feedback and decided how to incorporate it into the design of the HELM strategy. We believe that collecting this feedback and involving research and practice experts in interpreting and integrating participant feedback into the HELM strategy will make HELM a more effective strategy for supporting school principals’ in implementing supports in their schools.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

✪ Centering School Leaders’ Expertise: Usability Evaluation of a Leadership-Focused Implementation Strategy to Support Tier 1 Programs in Schools

Abstract:

Although there are a growing number of implementation strategies to increase the adoption, fidelity, and sustainment of evidence-based practices (EBP), they are often complex and bulky, which can interfere with their widespread application. To address these challenges, the Helping Educational Leaders Mobilize Evidence (HELM) strategy was created as an adaptation of the evidence-based Leadership and Organizational Change for Implementation (LOCI) implementation strategy to enhance elementary school principals’ use of strategic implementation leadership to support the adoption and delivery of Tier 1 (i.e., universal social, emotional, and behavioral) EBP. In service of its iterative development, a human-centered design methodology was employed to increase the successful uptake and use of HELM. The Cognitive Walkthrough for Implementation Strategies (CWIS), a novel mixed-methods approach to evaluate implementation strategy usability, was applied to identify and test HELM strategy tasks of critical importance. A sample of 15 elementary school principals participated in group cognitive walkthrough testing as either school principal recipients or HELM coaches. Both user types rated the strategy as acceptable (principal M = 77.8, SD = 15.5; coach M = 87.5, SD = 7.9). Five usability issues were identified using a highly structured common usability issue framework and provided direction for the generation of redesign solutions to be incorporated in a subsequent version of the HELM strategy. The evaluation of the strategy’s usability improved its alignment with user needs, expectations, and contextual constraints, rendering a more usable strategy and broadly applicable information surrounding the development of psychosocial implementation strategies in real-world settings.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

Authors:

✪ Open Access

Read the full text in the open access journal JMIR Diabetes

Abstract:

Background

Objective

Methods

Results

Conclusions

Authors:

✪ Open Access

Read the full text in the open access journal JMIR Cancer

Abstract:

Background

Objective

Methods

Results

Conclusions

Authors:

✪ Open Access

Read the full text in the open access journal Implementation Research and Practice

Abstract:

Background

Method

Results

Conclusions

Plain Language Summary

Authors:

✪ Open Access

Read the full text in the open access journal JMIR Formative Research

Abstract:

Background

Objective

Methods

Results:

Conclusions

Authors:

✪ Open Access

Read the full text in the open access journal JMIR Research Protocols

Abstract:

Background

Objective

Methods

Results

Conclusions

International Registered Report Identifier (IRRID)

Authors:

Subscription Required for Full Text Access

Read the full text in the subscription access journal Annual Review of Public Health

Authors:

✪ Open Access

Read the full text in the open access journal Implementation Science Communications

Abstract:

Background

Methods

Results

Conclusions

Contributions to the literature

Authors:

✪ Open Access

Read the full text in the open access journal Journal of the American Board of Family Medicine

Authors:

✪ Open Access

Read the full text in the open access journal Implementation Research and Practice

Abstract:

Background

Method

Results

Conclusion

Plain Language Summary

Authors:

✪ Open Access

Read the full text open access in the journal School Mental Health

Abstract: