Skip to content
Implementation Science at UW

Keywords: Case Studies

✪ A case study of pragmatic measure development of the Treatment Integrity for Elementary Settings self-report measure for teachers

Authors:

Emma R Dear, Bryce D McLeod, Nicole M Peterson, Kevin S Sutherland, Michael D Broda, Alex R Dopp & Aaron R Lyon

University of Washington affiliated authors are displayed in bold.

✪ Open Access

Published: January 2024

Read the full text in the open access journal Implementation Research and Practice

Abstract:

Introduction

Due to usability, feasibility, and acceptability concerns, observational treatment fidelity measures are often challenging to deploy in schools. Teacher self-report fidelity measures with specific design features might address some of these barriers. This case study outlines a community-engaged, iterative process to adapt the observational Treatment Integrity for Elementary Settings (TIES-O) to a teacher self-report version designed to assess the use of practices to support children's social-emotional competencies in elementary classrooms.

Method

Cognitive walkthrough interviews were conducted with teachers to improve the usability of the teacher self-report measure, called the Treatment Integrity for Elementary Schools–Teacher Report (TIES-T). Qualitative content analysis was used to extract themes from the interviews and inform changes to the measure.

Results

Increasing clarity and interactive elements in the measure training were the dominant themes, but suggestions for the measure format and jargon were also suggested.

Conclusion

The suggested changes resulted in a brief measure, training, and feedback system designed to support the teacher's use of practices to support children's social-emotional competencies in elementary classrooms. Future research with the TIES-T will examine the score reliability and validity of the measure.

**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**

✪ Participatory logic modeling in a multi-site initiative to advance implementation science

Authors:

Douglas V. Easterling, Rebekah R. Jacob, Ross C. Brownson, Debra Haire-Joshu, Daniel A. Gundersen, Heather Angier, Jennifer E. DeVoe, Sonja Likumahuwa-Ackman, Thuy Vu, Russell E. Glasgow, & Robert Schnoll

University of Washington affiliated authors are displayed in bold.

✪ Open Access

Published: August 2023

Read the full text in the open access journal Implementation Science Communications

Abstract:

Background

Logic models map the short-term and long-term outcomes that are expected to occur with a program, and thus are an essential tool for evaluation. Funding agencies, especially in the United States (US), have encouraged the use of logic models among their grantees. They also use logic models to clarify expectations for their own funding initiatives. It is increasingly recognized that logic models should be developed through a participatory approach which allows input from those who carry out the program being evaluated. While there are many positive examples of participatory logic modeling, funders have generally not engaged grantees in developing the logic model associated with their own initiatives. This article describes an instance where a US funder of a multi-site initiative fully engaged the funded organizations in developing the initiative logic model. The focus of the case study is Implementation Science Centers in Cancer Control (ISC3), a multi-year initiative funded by the National Cancer Institute.

Methods

The reflective case study was collectively constructed by representatives of the seven centers funded under ISC3. Members of the Cross-Center Evaluation (CCE) Work Group jointly articulated the process through which the logic model was developed and refined. Individual Work Group members contributed descriptions of how their respective centers reviewed and used the logic model. Cross-cutting themes and lessons emerged through CCE Work Group meetings and the writing process.

Results

The initial logic model for ISC3 changed in significant ways as a result of the input of the funded groups. Authentic participation in the development of the logic model led to strong buy-in among the centers, as evidenced by their utilization. The centers shifted both their evaluation design and their programmatic strategy to better accommodate the expectations reflected in the initiative logic model.

Conclusions

The ISC3 case study demonstrates how participatory logic modeling can be mutually beneficial to funders, grantees and evaluators of multi-site initiatives. Funded groups have important insights about what is feasible and what will be required to achieve the initiative’s stated objectives. They can also help identify the contextual factors that either inhibit or facilitate success, which can then be incorporated into both the logic model and the evaluation design. In addition, when grantees co-develop the logic model, they have a better understanding and appreciation of the funder’s expectations and thus are better positioned to meet those expectations.

**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**

✪ A 4D Theoretical Framework for Measuring Topic-Specific Influence on Twitter: Development and Usability Study on Dietary Sodium Tweets

Authors:

Lingchao Mao, Emily Chu, Jinghong Gu, Tao Hu, Bryan J Weiner, Yanfang Su

University of Washington affiliated authors are displayed in bold.

✪ Open Access

Published: June 2023

Read the full text in the open access journal Journal of Medical Internet Research

Abstract:

Background

Social media has emerged as a prominent approach for health education and promotion. However, it is challenging to understand how to best promote health-related information on social media platforms such as Twitter. Despite commercial tools and prior studies attempting to analyze influence, there is a gap to fill in developing a publicly accessible and consolidated framework to measure influence and analyze dissemination strategies.

Objective

We aimed to develop a theoretical framework to measure topic-specific user influence on Twitter and to examine its usability by analyzing dietary sodium tweets to support public health agencies in improving their dissemination strategies.

Methods

We designed a consolidated framework for measuring influence that can capture topic-specific tweeting behaviors. The core of the framework is a summary indicator of influence decomposable into 4 dimensions: activity, priority, originality, and popularity. These measures can be easily visualized and efficiently computed for any Twitter account without the need for private access. We demonstrated the proposed methods by using a case study on dietary sodium tweets with sampled stakeholders and then compared the framework with a traditional measure of influence.

Results

More than half a million dietary sodium tweets from 2006 to 2022 were retrieved for 16 US domestic and international stakeholders in 4 categories, that is, public agencies, academic institutions, professional associations, and experts. We discovered that World Health Organization, American Heart Association, Food and Agriculture Organization of the United Nations (UN-FAO), and World Action on Salt (WASH) were the top 4 sodium influencers in the sample. Each had different strengths and weaknesses in their dissemination strategies, and 2 stakeholders with similar overall influence, that is, UN-FAO and WASH, could have significantly different tweeting patterns. In addition, we identified exemplars in each dimension of influence. Regarding tweeting activity, a dedicated expert published more sodium tweets than any organization in the sample in the past 16 years. In terms of priority, WASH had more than half of its tweets dedicated to sodium. UN-FAO had both the highest proportion of original sodium tweets and posted the most popular sodium tweets among all sampled stakeholders. Regardless of excellence in 1 dimension, the 4 most influential stakeholders excelled in at least 2 out of 4 dimensions of influence.

Conclusions

Our findings demonstrate that our method not only aligned with a traditional measure of influence but also advanced influence analysis by analyzing the 4 dimensions that contribute to topic-specific influence. This consolidated framework provides quantifiable measures for public health entities to understand their bottleneck of influence and refine their social media campaign strategies. Our framework can be applied to improve the dissemination of other health topics as well as assist policy makers and public campaign experts to maximize population impact.

**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**

✪ Creating research-ready partnerships: the initial development of seven implementation laboratories to advance cancer control

Authors:

Gina R. Kruse, Erica Hale, Justin E. Bekelman, Jennifer E. DeVoe, Rachel Gold, Peggy A. Hannon, Thomas K. Houston, Aimee S. James, Ashley Johnson, Lisa M. Klesges & Andrea L. Nederveld

University of Washington affiliated authors are displayed in bold.

✪ Open Access

Published: February 2023

Read the full text in the open access journal BMC Health Services Research

Abstract:

Background

In 2019–2020, with National Cancer Institute funding, seven implementation laboratory (I-Lab) partnerships between scientists and stakeholders in ‘real-world’ settings working to implement evidence-based interventions were developed within the Implementation Science Centers in Cancer Control (ISC3) consortium. This paper describes and compares approaches to the initial development of seven I-Labs in order to gain an understanding of the development of research partnerships representing various implementation science designs.

Methods

In April-June 2021, members of the ISC3 Implementation Laboratories workgroup interviewed research teams involved in I-Lab development in each center. This cross-sectional study used semi-structured interviews and case-study-based methods to collect and analyze data about I-Lab designs and activities. Interview notes were analyzed to identify a set of comparable domains across sites. These domains served as the framework for seven case descriptions summarizing design decisions and partnership elements across sites.

Results

Domains identified from interviews as comparable across sites included engagement of community and clinical I-Lab members in research activities, data sources, engagement methods, dissemination strategies, and health equity. The I-Labs use a variety of research partnership designs to support engagement including participatory research, community-engaged research, and learning health systems of embedded research. Regarding data, I-Labs in which members use common electronic health records (EHRs) leverage these both as a data source and a digital implementation strategy. I-Labs without a shared EHR among partners also leverage other sources for research or surveillance, most commonly qualitative data, surveys, and public health data systems. All seven I-Labs use advisory boards or partnership meetings to engage with members; six use stakeholder interviews and regular communications. Most (70%) tools or methods used to engage I-Lab members such as advisory groups, coalitions, or regular communications, were pre-existing. Think tanks, which two I-Labs developed, represented novel engagement approaches. To disseminate research results, all centers developed web-based products, and most (n = 6) use publications, learning collaboratives, and community forums. Important variations emerged in approaches to health equity, ranging from partnering with members serving historically marginalized populations to the development of novel methods.

Conclusions

The development of the ISC3 implementation laboratories, which represented a variety of research partnership designs, offers the opportunity to advance understanding of how researchers developed and built partnerships to effectively engage stakeholders throughout the cancer control research lifecycle. In future years, we will be able to share lessons learned for the development and sustainment of implementation laboratories.

**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**

✪ “We are the sun for our community:” Partnering with community health workers/promotores to adapt, deliver and evaluate a home-based collaborative care model to improve equity in access to quality depression care for older U.S. Latino adults who are underserved

Authors:

Lesley E. Steinman, Amelia Gasca, Theresa J. Hoeft, Patrick J. Raue, Stuart Henderson, Rosa Perez, Alfredo Huerta, Alex Fajardo, Melinda A. Vredevoogd, Katherine James, Ladson Hinton, Laura Rath, and Jurgen Unützer.

University of Washington affiliated authors are displayed in bold.

✪ Open Access

Published: February 2023

Read the full text in the open access journal Frontiers in Public Health
BACK TO THE ARCHIVE

Abstract:

Background:

While depression is a leading cause of poor health, less than half of older adults receive adequate care. Inequities in both access and outcomes are even more pronounced for socially disadvantaged older adults. The collaborative care model (CCM) has potential to reduce this burden through community-based organizations (CBOs) who serve these populations. However, CCM has been understudied in diverse cultural and resource-constrained contexts. We evaluated the implementation and effectiveness of PEARLS, a home-based CCM adapted with and for community health workers/promotores (CHWs/Ps).

Methods:

We used an instrumental case study design. Our case definition is a community-academic partnership to build CHW/P capacity for evidence-based depression care for older U.S. Latino adults in the Inland Empire region of California (2017–2020). We aimed to understand adaptations to fit local context; acceptability, feasibility, and fidelity; clinical effectiveness; and contextual determinants of implementation success or failure. Data sources included quantitative and qualitative administrative and evaluation data from participants and providers. We used descriptive statistics and paired t-tests to characterize care delivery and evaluate effectiveness post-intervention, and deductive thematic analysis to answer other aims.

Findings:

This case study included 152 PEARLS participants and nine data sources (N = 67 documents). The CBO including their CHWs/Ps partnered with the external implementation team made adaptations to PEARLS content, context, and implementation strategies to support CHWs/Ps and older adults. PEARLS was acceptable, feasible and delivered with fidelity. Participants showed significant reductions in depression severity at 5 months (98% clinical response rate [mean (SD), 13.7 (3.9) drop in pre/post PHQ-9; p < 0.001] and received support for 2.6 social needs on average. PEARLS delivery was facilitated by its relative advantage, adaptability, and trialability; the team's collective efficacy, buy-in, alignment with organization mission, and ongoing reflection and evaluation during implementation. Delivery was challenged by weak partnerships with clinics for participant referral, engagement, reimbursement, and sustainability post-grant funding.

Discussion:

This case study used existing data to learn how home-based CCM was adapted by and for CHWs/Ps to reduce health inequities in late-life depression and depression care among older Latino immigrants. The CBOs and CHWs/Ps strong trust and rapport, addressing social and health needs alongside depression care, and regular internal and external coaching and consultation, appeared to drive successful implementation and effectiveness.

**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**

✪ Applying the Consolidated Framework for Implementation Research to Identify Implementation Determinants for the Integrated District Evidence-to-Action Program, Mozambique

Authors:

Celso Inguane, Caroline Soi, Sarah Gimbel, Nélia Manaca, Isaías Ramiro, Florência Floriano, Georgina de Castro, Orvalho Augusto, Stélio Tembe, James Pfeiffer, Quinhas Fernandes, and Kenneth Sherr

University of Washington affiliated authors are displayed in bold.

✪ Open Access

Published: September 2022

Read the full text in the open access journal Global Health: Science and Practice
BACK TO THE ARCHIVE

Abstract

Introduction

The Integrated District Evidence-to-Action program is an audit and feedback intervention introduced in 2017 in Manica and Sofala provinces, Mozambique, to reduce mortality in children younger than 5 years. We describe barriers and facilitators to early-stage effectiveness of that intervention.

Method

We embedded the Consolidated Framework for Implementation Research (CFIR) into an extended case study design to inform sampling, data collection, analysis, and interpretation. We collected data in 4 districts in Manica and Sofala Provinces in November 2018. Data collection included document review, 22 in-depth individual interviews, and 2 focus group discussions (FGDs) with 19 provincial, district, and facility managers and nurses. Most participants (70.2%) were nurses and facility managers and the majority were women (87.8%). We audio-recorded all but 2 interviews and FGDs and conducted a consensus-based iterative analysis.

Results

Facilitators of effective intervention implementation included: implementation of the core intervention components of audit and feedback meetings, supportive supervision and mentorship, and small grants as originally planned; positive pressure from district managers and study nurses on health facility staff to strive for excellence; and easy access to knowledge and information about the intervention. Implementation barriers were the intervention’s lack of compatibility in not addressing the scarcity of human and financial resources and inadequate infrastructures for maternal and child health services at district and facility levels and; the intervention’s lack of adaptability in having little flexibility in the design and decision making about the use of intervention funds and data collection tools.

Discussion

Our comprehensive and systematic use of the CFIR within an extended case study design generated granular evidence on CFIR’s contribution to implementation science efforts to describe determinants of early-stage intervention implementation. It also provided baseline findings to assess subsequent implementation phases, considering similarities and differences in barriers and facilitators across study districts and facilities. Sharing preliminary findings with stakeholders promoted timely decision making about intervention implementation.

**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**

✪ Adapting and Operationalizing the RE-AIM Framework for Implementation Science in Environmental Health: Clean Fuel Cooking Programs in Low Resource Countries

Authors:

Ashlinn K. Quinn, Gila Neta, Rachel Sturke, Christopher O. Olopade, Suzanne L. Pollard, Kenneth Sherr and Joshua P. Rosenthal

University of Washington affiliated authors are displayed in bold.

Published: December 2019

✪ Open Access
Read the full text in the open access journal Frontiers in Public Health

Abstract:

Introduction

The use of models and frameworks to design and evaluate strategies to improve delivery of evidence-based interventions is a foundational element of implementation science. To date, however, evaluative implementation science frameworks such as Reach, Effectiveness, Adoption, Implementation, Maintenance (RE-AIM) have not been widely employed to examine environmental health interventions. We take advantage of a unique opportunity to utilize and iteratively adapt the RE-AIM framework to guide NIH-funded case studies of the implementation of clean cooking fuel programs in eleven low- and middle-income countries.

Methods

We used existing literature and expert consultation to translate and iteratively adapt the RE-AIM framework across several stages of the NIH Clean Cooking Implementation Science case study project. Checklists and templates to guide investigators were developed at each stage.

Results

The RE-AIM framework facilitated identification of important emerging issues across this set of case studies, in particular highlighting the fact that data associated with certain important outcomes related to health and welfare are chronically lacking in clean fuel programs. Monitoring of these outcomes should be prioritized in future implementation efforts. As RE-AIM was not originally designed to evaluate household energy interventions, employing the framework required adaptation. Specific adaptations include the broadening of Effectiveness to encompass indicators of success toward any stated programmatic goal, and expansion of Adoption to include household-level uptake of technology.

Conclusions

The RE-AIM implementation science framework proved to be a useful organizing schema for 11 case studies of clean fuel cooking programs, in particular highlighting areas requiring emphasis in future research and evaluation efforts. The iterative approach used here to adapt an implementation science framework to a specific programmatic goal may be of value to other multi-country program efforts, such as those led by international development agencies. The checklists and templates developed for this project are publicly available for others to use and/or further modify.

**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**