Emily C. Williams, Madeline C. Frost, Olivia V. Fletcher, Carol E. Achtmeyer, Jessica A. Chen, Rachel L. Bachrach, E. Jennifer Edelman, Bryan J. Weiner, George N. Ioannou, and Judith I. Tsui
University of Washington affiliated authors are displayed in bold.
Published: August 2026
Julian Brunner, Ellen A. Ahlness, Ekaterina Anderson, Brianne K. Molloy-Paolillo, Alexandre Braga, Sarah L. Cutrona, Christian D. Helfrich, Deborah Levy, Erin Matteau, Edward Walton, George Sayre, Seppo T. Rinne
University of Washington affiliated authors are displayed in bold.
Published: October 2024
The Department of Veterans Affairs (VA) is undergoing an unprecedented electronic health record (EHR) transition, switching from its homegrown EHR to a commercial system. The transition affects nearly every clinical employee but is particularly disruptive to health professions trainees (HPTs)—an often-overlooked population in EHR transitions. To better understand and address trainee challenges with the EHR transition, we formed a multistakeholder learning community. In this study, we describe the findings of this learning community and the practices and policies developed in response.
In the qualitative study designed and executed by our learning community, we conducted 51 interviews with HPTs, program leaders, and preceptors before and multiple times after an EHR transition site's go-live (February 16, 2022 to April 7, 2023). We merged interview transcripts with 125 survey free-text responses from a survey conducted with preceptors 2 months post-go-live and conducted thematic analysis to identify key themes. To complement qualitative findings, we also include a quantitative survey finding, and, where applicable, we note policy and practice responses spurred by our learning community.
Interviews yielded six key themes: (1) High satisfaction with HPT programs, despite negative impacts of the EHR transition; (2) early delays, then substantial improvements, in HPTs' EHR access; (3) persistent challenges with HPTs' EHR training and support, mitigated by local and national efforts; (4) the challenge of learning to use a rapidly evolving EHR during clinical training; (5) reduced visit volume as a continuing barrier to education; and (6) an impression that HPTs' relative lack of exposure to the prior EHR facilitated their proficiency with the new EHR.
Findings highlighted challenges for HPT programs related to the EHR transition, which spurred important changes including the creation of a national VA council to represent the needs of HPTs in the EHR transition, and improvements to HPTs' EHR training and access.
**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**
Monisha Sharma, Barbara Wanjiku Mambo, Hanley Kingston, George Otieno, Sarah Masyuko, Harison Lagat, David A. Katz, Beatrice Wamuti, Paul Macharia, Rose Bosire , Mary Mugambi, Edward Kariithi, and Carey Farquhar
University of Washington affiliated authors are displayed in bold.
Published: July 2024
Assisted partner services (APS), or exposure notification and HIV testing for sexual partners of persons diagnosed HIV positive (index clients), is recommended by the World Health Organization. Most APS literature focuses on outcomes among index clients and their partners. There is little data on the benefits of providing APS to partners of partners diagnosed with HIV.
We utilized data from a large-scale APS implementation project across 31 facilities in western Kenya from 2018 to 2022. Females testing HIV positive at facilities were offered APS; those who consented provided contact information for all male sexual partners in the last 3 years. Male partners were notified of their potential HIV exposure and offered HIV testing services (HTS). Males newly testing positive were also offered APS and asked to provide contact information for their female partners in the last 3 years. Female partners of male partners (FPPs) were provided exposure notification and HTS. All participants with HIV were followed up at 12 months post-enrolment to assess linkage-to antiretroviral treatment (ART) and viral suppression. We compared HIV positivity, demographics and linkage outcomes among female index clients and FPPs.
Overall, 5708 FPPs were elicited from male partners, of whom 4951 received HTS through APS (87% coverage); 291 FPPs newly tested HIV positive (6% yield), an additional 1743 (35.2%) reported a prior HIV diagnosis, of whom 99% were on ART at baseline. At 12 months follow-up, most FPPs were taking ART (92%) with very few adverse events: <1% reported intimate partner violence or reported relationship dissolution. FPPs were more likely than female index clients to report HIV risk behaviours including no condom use at last sex (45% vs. 30%) and multiple partners (38% vs. 19%).
Providing HIV testing via APS to FPP is a safe and effective strategy to identify newly diagnosed females and achieve high linkage and retention to ART and can be an efficient means of identifying HIV cases in the era of declining HIV incidence. The high proportion of FPPs reporting HIV risk behaviours suggests APS may help interrupt community HIV transmission via increased knowledge of HIV status and linkage to treatment.
**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**
Vaughan K. Collins, Catherine M. Corbin, Jill J. Locke, Clayton R. Cook, Mark G. Ehrhart, Kurt D. Hatch & Aaron R. Lyon
University of Washington affiliated authors are displayed in bold.
Published: March 2024
Although there are a growing number of implementation strategies to increase the adoption, fidelity, and sustainment of evidence-based practices (EBP), they are often complex and bulky, which can interfere with their widespread application. To address these challenges, the Helping Educational Leaders Mobilize Evidence (HELM) strategy was created as an adaptation of the evidence-based Leadership and Organizational Change for Implementation (LOCI) implementation strategy to enhance elementary school principals’ use of strategic implementation leadership to support the adoption and delivery of Tier 1 (i.e., universal social, emotional, and behavioral) EBP. In service of its iterative development, a human-centered design methodology was employed to increase the successful uptake and use of HELM. The Cognitive Walkthrough for Implementation Strategies (CWIS), a novel mixed-methods approach to evaluate implementation strategy usability, was applied to identify and test HELM strategy tasks of critical importance. A sample of 15 elementary school principals participated in group cognitive walkthrough testing as either school principal recipients or HELM coaches. Both user types rated the strategy as acceptable (principal M = 77.8, SD = 15.5; coach M = 87.5, SD = 7.9). Five usability issues were identified using a highly structured common usability issue framework and provided direction for the generation of redesign solutions to be incorporated in a subsequent version of the HELM strategy. The evaluation of the strategy’s usability improved its alignment with user needs, expectations, and contextual constraints, rendering a more usable strategy and broadly applicable information surrounding the development of psychosocial implementation strategies in real-world settings.
**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**
Inge Petersen, Christopher G. Kemp, Deepa Rao, Bradley H. Wagenaar, Max Bachmann, Kenneth Sherr, Tasneem Kathree, Zamasomi Luvuno, André Van Rensburg, Sithabisile Gugulethu Gigaba, Londiwe Mthethwa, Merridy Grant, One Selohilwe, Nikiwe Hongo, Gillian Faris, Christy-Joy Ras, Lara Fairall, Sanah Bucibo, & Arvin Bhana
University of Washington affiliated authors are displayed in bold.
Published: November 2023
Integration of mental health into routine primary health care (PHC) services in low-and middle-income countries is globally accepted to improve health outcomes of other conditions and narrow the mental health treatment gap. Yet implementation remains a challenge. The aim of this study was to identify implementation strategies that improve implementation outcomes of an evidence-based depression care collaborative implementation model integrated with routine PHC clinic services in South Africa. An iterative, quasi-experimental, observational implementation research design, incorporating the Reach, Effectiveness, Adoption, Implementation and Maintenance (RE-AIM) framework, was applied to evaluate implementation outcomes of a strengthened package of implementation strategies (stage two) compared with an initial evaluation of the model (stage one). The first stage package was implemented and evaluated in 10 PHC clinics and the second stage strengthened package in 19 PHC clinics (inclusive of the initial 10 clinics) in one resource-scarce district in the province of KwaZulu-Natal, South Africa. Diagnosed service users were more likely to be referred for counselling treatment in the second stage compared with stage one (OR 23.15, SE = 18.03, z = 4.04, 95%CI [5.03–106.49], p < .001). Training in and use of a validated, mandated mental health screening tool, including on-site educational outreach and technical support visits, was an important promoter of nurse-level diagnosis rates (OR 3.75, 95% CI [1.19, 11.80], p = 0.02). Nurses who perceived the integrated care model as acceptable were also more likely to successfully diagnose patients (OR 2.57, 95% CI [1.03–6.40], p = 0.043). Consistent availability of a clinic counsellor was associated with a greater probability of referral (OR 5.9, 95%CI [1.29–27.75], p = 0.022). Treatment uptake among referred service users remained a concern across both stages, with inconsistent co-located counselling services associated with poor uptake. The importance of implementation research for strengthening implementation strategies along the cascade of care for integrating depression care within routine PHC services is highlighted.
**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**
Douglas V. Easterling, Rebekah R. Jacob, Ross C. Brownson, Debra Haire-Joshu, Daniel A. Gundersen, Heather Angier, Jennifer E. DeVoe, Sonja Likumahuwa-Ackman, Thuy Vu, Russell E. Glasgow, & Robert Schnoll
University of Washington affiliated authors are displayed in bold.
Published: August 2023
Logic models map the short-term and long-term outcomes that are expected to occur with a program, and thus are an essential tool for evaluation. Funding agencies, especially in the United States (US), have encouraged the use of logic models among their grantees. They also use logic models to clarify expectations for their own funding initiatives. It is increasingly recognized that logic models should be developed through a participatory approach which allows input from those who carry out the program being evaluated. While there are many positive examples of participatory logic modeling, funders have generally not engaged grantees in developing the logic model associated with their own initiatives. This article describes an instance where a US funder of a multi-site initiative fully engaged the funded organizations in developing the initiative logic model. The focus of the case study is Implementation Science Centers in Cancer Control (ISC3), a multi-year initiative funded by the National Cancer Institute.
The reflective case study was collectively constructed by representatives of the seven centers funded under ISC3. Members of the Cross-Center Evaluation (CCE) Work Group jointly articulated the process through which the logic model was developed and refined. Individual Work Group members contributed descriptions of how their respective centers reviewed and used the logic model. Cross-cutting themes and lessons emerged through CCE Work Group meetings and the writing process.
The initial logic model for ISC3 changed in significant ways as a result of the input of the funded groups. Authentic participation in the development of the logic model led to strong buy-in among the centers, as evidenced by their utilization. The centers shifted both their evaluation design and their programmatic strategy to better accommodate the expectations reflected in the initiative logic model.
The ISC3 case study demonstrates how participatory logic modeling can be mutually beneficial to funders, grantees and evaluators of multi-site initiatives. Funded groups have important insights about what is feasible and what will be required to achieve the initiative’s stated objectives. They can also help identify the contextual factors that either inhibit or facilitate success, which can then be incorporated into both the logic model and the evaluation design. In addition, when grantees co-develop the logic model, they have a better understanding and appreciation of the funder’s expectations and thus are better positioned to meet those expectations.
**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**
Lisa V. Rubenstein, Idamay Curtis, Chelle L. Wheat, David E. Grembowski, Susan E. Stockdale, Peter J. Kaboli, Jean Yoon, Bradford L. Felker, Ashok S. Reddy & Karin M. Nelson
University of Washington affiliated authors are displayed in bold.
Published:August 2023
The Veterans Affairs (VA) Clinical Resource Hub (CRH) program aims to improve patient access to care by implementing time-limited, regionally based primary or mental health staffing support to cover local staffing vacancies. VA’s Office of Primary Care (OPC) designed CRH to support more than 1000 geographically disparate VA outpatient sites, many of which are in rural areas, by providing virtual contingency clinical staffing for sites experiencing primary care and mental health staffing deficits. The subsequently funded CRH evaluation, carried out by the VA Primary Care Analytics Team (PCAT), partnered with CRH program leaders and evaluation stakeholders to develop a protocol for a six-year CRH evaluation.
The objectives for developing the CRH evaluation protocol were to prospectively: 1) identify the outcomes CRH aimed to achieve, and the key program elements designed to achieve them; 2) specify evaluation designs and data collection approaches for assessing CRH progress and success; and 3) guide the activities of five geographically dispersed evaluation teams.
The protocol documents a multi-method CRH program evaluation design with qualitative and quantitative elements. The evaluation’s overall goal is to assess CRH’s return on investment to the VA and Veterans at six years through synthesis of findings on program effectiveness. The evaluation includes both observational and quasi-experimental elements reflecting impacts at the national, regional, outpatient site, and patient levels. The protocol is based on program evaluation theory, implementation science frameworks, literature on contingency staffing, and iterative review and revision by both research and clinical operations partners.
Health systems increasingly seek to use data to guide management and decision-making for newly implemented clinical programs and policies. Approaches for planning evaluations to accomplish this goal, however, are not well-established. By publishing the protocol, we aim to increase the validity and usefulness of subsequent evaluation findings. We also aim to provide an example of a program evaluation protocol developed within a learning health systems partnership.
**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**
Linda K. Ko, Thuy Vu, Sonia Bishop, Jennifer Leeman, Cam Escoffery, Rachel L. Winer, Miriana C. Duran, Manal Masud & Yaniv Rait
University of Washington affiliated authors are displayed in bold.
Published:July 2023
Rural community-based organizations (CBOs) serving immigrant communities are critical settings for implementing evidence-based interventions (EBIs). The Implementation Studio is a training and consultation program focused on facilitating the selection, adaptation, and implementation of cancer prevention and control EBIs. This paper describes implementation and evaluation of the Implementation Studio on CBO’s capacity to implement EBIs and their clients’ knowledge of colorectal cancer (CRC) screening and intention to screen.
Thirteen community health educators (CHEs) from two CBOs participated in the Implementation Studio. Both CBOs selected CRC EBIs during the Studio. The evaluation included two steps. The first step assessed the CHEs’ capacity to select, adapt, and implement an EBI. The second step assessed the effect of the CHEs-delivered EBIs on clients’ knowledge of CRC and intention to screen (n = 44).
All CHEs were Hispanic and women. Pre/post-evaluation of the Studio showed an increase on CHEs knowledge about EBIs (pre: 23% to post: 75%; p < 0.001). CHEs’ ability to select, adapt, and implement EBIs also increased, respectively: select EBI (pre: 21% to post: 92%; p < 0.001), adapt EBI (pre: 21% to post: 92%; p < 0.001), and implement EBI (pre: 29% to post: 75%; p = 0.003). Pre/post-evaluation of the CHE-delivered EBI showed an increase on CRC screening knowledge (p < 0.5) and intention to screen for CRC by their clients.
Implementation Studio can address unique needs of low resource rural CBOs. An implementation support program with training and consultation has potential to build the capacity of rural CBOs serving immigrant communities to implementation of cancer prevention and control EBIs.
**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**
Monisha Sharma, Brienna Naughton, Harison Lagat, George Otieno, David A Katz, Beatrice M Wamuti, Sarah Masyuko, Rose Bosire, Mary Mugambi, Unmesha Roy Paladhi, Bryan J Weiner, Edward Kariithi & Carey Farquhar
University of Washington affiliated authors are displayed in bold.
Published: May 2023
Assisted partner services (APS), or exposure notification and HIV testing for sexual partners of individuals diagnosed with HIV (index clients), have been shown to be safe and effective in clinical trials. We assessed the real-world effectiveness of APS when integrated into HIV clinics in western Kenya.
In this single-arm, hybrid type 2 implementation science study, we facilitated APS implementation in 31 health facilities in Kenya by training existing health-care staff. We focused on male partner outcomes to assess the impact of APS in reaching male individuals in sub-Saharan Africa, who have lower rates of HIV testing than female individuals. Female individuals (aged ≥18 years or emancipated minor) who tested positive for HIV at participating facilities in Kenya were offered APS; consenting female participants provided contact information for all male sexual partners in the past 3 years. Male partners were notified of their potential HIV exposure and offered a choice of community-based or facility-based HIV testing services (HTS). Female index clients and male partners with HIV were followed up at 6 weeks, 6 months, and 12 months after enrolment, to assess linkage to antiretroviral treatment. Viral load was assessed at 12 months.
Between May 1, 2018, and March 31, 2020, 32 722 female individuals received HTS; 1910 (6%) tested positive for HIV, of whom 1724 (90%) received APS. Female index clients named 5137 male partners (median 3 per index [IQR 2–4]), of whom 4422 (86%) were reached with exposure notification and HTS. 524 (12%) of the male partners tested were newly diagnosed with HIV and 1292 (29%) reported a previous HIV diagnosis. At 12 months follow-up, 1512 (88%) female index clients and 1621 (89%) male partners with HIV were taking ART, with few adverse events: 25 (2%) female index clients and seven (<1%) male partners reported intimate partner violence, and 60 (3%) female index clients and ten (<1%) male partners reported relationship dissolution.
Evidence from this real-world APS scale-up project shows that APS is a safe, acceptable, and effective strategy to identify males with HIV and retain them in care.
**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**
Kristen Danforth, Ahsan Maqbool Ahmad, Karl Blanchet, Muhammad Khalid, Arianna Rubin Means, Solomon Tessema Memirie, Ala Alwan & David Watkins
University of Washington affiliated authors are displayed in bold.
Published: March 2023
Essential packages of health services (EPHS) are a critical tool for achieving universal health coverage, especially in low-income and lower middle-income countries. However, there is a lack of guidance and standards for monitoring and evaluation (M&E) of EPHS implementation. This paper is the final in a series of papers reviewing experiences using evidence from the Disease Control Priorities, third edition publications in EPHS reforms in seven countries. We assess current approaches to EPHS M&E, including case studies of M&E approaches in Ethiopia and Pakistan. We propose a step-by-step process for developing a national EPHS M&E framework. Such a framework would start with a theory of change that links to the specific health system reforms the EPHS is trying to accomplish, including explicit statements about the ‘what’ and ‘for whom’ of M&E efforts. Monitoring frameworks need to consider the additional demands that could be placed on weak and already overstretched data systems, and they must ensure that processes are put in place to act quickly on emergent implementation challenges. Evaluation frameworks could learn from the field of implementation science; for example, by adapting the Reach, Effectiveness, Adoption, Implementation and Maintenance framework to policy implementation. While each country will need to develop its own locally relevant M&E indicators, we encourage all countries to include a set of core indicators that are aligned with the Sustainable Development Goal 3 targets and indicators. Our paper concludes with a call to reprioritise M&E more generally and to use the EPHS process as an opportunity for strengthening national health information systems. We call for an international learning network on EPHS M&E to generate new evidence and exchange best practices.
**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**