Concept Mapping Archives - Implementation Science at UW

✪ Methods for community-engaged data collection and analysis in implementation research

Abstract:

Background

Community engagement is widely recognized as critical to successful and equitable implementation of evidence-based practices, programs, and policies. However, there are no clear guidelines for community involvement in data collection and analysis in implementation research.

Methods

We describe three specific methods for engaging community members in data collection and analysis: concept mapping, rapid ethnographic assessment, and Photovoice. Common elements are identified from a case study of each method: 1) selection and adaptation of evidence-based strategies for improving adolescent HPV vaccine initiation rates in disadvantaged communities, 2) strategies for implementing medication for opioid use disorders among low-income Medicaid enrollees during natural disasters, and 3) interventions to improve the physical health of adults with severe mental illness living in supportive housing.

Results

In all three cases, community members assisted in participant recruitment, provided data, and validated preliminary findings created by researchers. In the Photovoice case study, community members participated in both data collection and analysis, while in the concept mapping, community members also participated in the initial phase of organizing and prioritizing evidence-based strategies during the data analysis.

Conclusions

Community involvement in implementation research data collection and analysis contributes to greater engagement and empowerment of community members and validation of study findings. Use of methods that exhibit both scientific rigor and community relevance of implementation research also contributes to greater community investment in successful implementation outcomes. Nevertheless, the case studies point to the importance and efficiency of the division of labor embedded in community-engaged implementation research. Building capacity for community members to assume greater roles in obtaining and organizing data for preliminary analysis prior to interpretation is recommended.

Contributions to the literature

Guidelines exist for conducting community engaged research but provide no clear instructions for community involvement in data collection and analysis.
This paper examines the potential of community engaged data collection and analysis in implementation research through case studies of concept mapping, rapid ethnographic methods, and Photovoice.
Community involvement in implementation research data collection and analysis contributes to greater engagement and empowerment of community members, validation of study findings, and commitment to successful implementation outcomes.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

Employing user-centered design to develop a remote technology kit for studying young children’s social communication skills

✪ Use of concept mapping to inform a participatory engagement approach for implementation of evidence-based HPV vaccination strategies in safety-net clinics

Abstract:

Background

Multiple evidence-based strategies (EBS) for promoting HPV vaccination exist. However, adolescent HPV vaccination rates remain below target levels in communities at high risk for HPV-associated cancers and served by safety-net clinics. Participatory engaged approaches are needed to leverage the expertise of community and clinical partners in selecting EBS relevant to their local context. We engaged concept mapping as a method to inform the adoption and adaptation of EBS that seeks to empower implementation partners to prioritize, select, and ultimately implement context-relevant EBS for HPV vaccination.

Methods

Using 38 EBS statements generated from qualitative interviews and national HPV vaccine advocacy sources, we conducted a modified concept mapping activity with partners internal to safety-net clinics and external community members in two study sites of a larger implementation study (Greater Los Angeles and New Jersey), to sort EBS into clusters and rate each EBS by importance and feasibility for increasing HPV vaccination within safety-net clinics. Concept mapping findings (EBS statement ratings, ladder graphs and go-zones) were shared with leaders from a large federally qualified health center (FQHC) system (focusing on three clinic sites), to select and implement EBS over 12 months.

Results

Concept mapping participants (n=23) sorted and rated statements, resulting in an eight-cluster solution: 1) Community education and outreach; 2) Advocacy and policy; 3) Data access/quality improvement monitoring; 4) Provider tracking/audit and feedback; 5) Provider recommendation/communication; 6) Expanding vaccine access; 7) Reducing missed opportunities; and 8) Nurse/staff workflow and training. The FQHC partner then selected to intervene on eight of 17 EBS statements in the “go-zone” for action, with three from “reducing missed opportunities,” two from “nurse/staff workflow and training,” and one each from “provider tracking/audit and feedback,” “provider recommendation/communication,” and “expanding vaccine access,” which the research team addressed through the implementation of three multi-level intervention strategies (e.g., physician communication training, staff training and workflow assessment, audit and feedback of clinic processes).

Conclusions

Concept mapping provided a powerful participatory approach to identify multilevel EBS for HPV vaccination relevant to the local safety-net clinic context, particularly when several strategies exist, and prioritization is necessary. This study demonstrates how a clinic system benefited directly from the ratings and prioritization of EBS by multilevel clinic and community partners within the broader safety-net clinic context to identify and adapt prioritized solutions needed to advance HPV vaccine equity.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

Imagining and implementing healthy city interventions: Combined results from parallel concept mapping exercises in Montreal with community members and stakeholders

✪ A structured approach to applying systems analysis methods for examining implementation mechanisms

Abstract:

Background

It is challenging to identify and understand the specific mechanisms through which an implementation strategy affects implementation outcomes, as implementation happens in the context of complex, multi-level systems. These systems and the mechanisms within each level have their own dynamic environments that change frequently. For instance, sequencing may matter in that a mechanism may only be activated indirectly by a strategy through another mechanism. The dosage or strength of a mechanism may vary over time or across different health care system levels. To elucidate the mechanisms relevant to successful implementation amidst this complexity, systems analysis methods are needed to model and manage complexity.

Methods

The fields of systems engineering and systems science offer methods—which we refer to as systems analysis methods—to help explain the interdependent relationships between and within systems, as well as dynamic changes to systems over time. When applied to studying implementation mechanisms, systems analysis methods can help (i) better identify and manage unknown conditions that may or may not activate mechanisms (both expected mechanisms targeted by a strategy and unexpected mechanisms that the methods help detect) and (ii) flexibly guide strategy adaptations to address contextual influences that emerge after the strategy is selected and used.

Results

In this paper, we delineate a structured approach to applying systems analysis methods for examining implementation mechanisms. The approach includes explicit steps for selecting, tailoring, and evaluating an implementation strategy regarding the mechanisms that the strategy is initially hypothesized to activate, as well as additional mechanisms that are identified through the steps. We illustrate the approach using a case example. We then discuss the strengths and limitations of this approach, as well as when these steps might be most appropriate, and suggest work to further the contributions of systems analysis methods to implementation mechanisms research.

Conclusions

Our approach to applying systems analysis methods can encourage more mechanisms research efforts to consider these methods and in turn fuel both (i) rigorous comparisons of these methods to alternative mechanisms research approaches and (ii) an active discourse across the field to better delineate when these methods are appropriate for advancing mechanisms-related knowledge.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

✪ Toward a more comprehensive understanding of organizational influences on implementation: the organization theory for implementation science framework

Abstract:

Introduction

Implementation is influenced by factors beyond individual clinical settings. Nevertheless, implementation research often focuses on factors related to individual providers and practices, potentially due to limitations of available frameworks. Extant frameworks do not adequately capture the myriad organizational influences on implementation. Organization theories capture diverse organizational influences but remain underused in implementation science. To advance their use among implementation scientists, we distilled 70 constructs from nine organization theories identified in our previous work into theoretical domains in the Organization Theory for Implementation Science (OTIS) framework.

Methods

The process of distilling organization theory constructs into domains involved concept mapping and iterative consensus-building. First, we recruited organization and implementation scientists to participate in an online concept mapping exercise in which they sorted organization theory constructs into domains representing similar theoretical concepts. Multidimensional scaling and hierarchical cluster analyses were used to produce visual representations (clusters) of the relationships among constructs in concept maps. Second, to interpret concept maps, we engaged members of the Cancer Prevention and Control Research Network (CPCRN) OTIS workgroup in consensus-building discussions.

Results

Twenty-four experts participated in concept mapping. Based on resulting construct groupings' coherence, OTIS workgroup members selected the 10-cluster solution (from options of 7–13 clusters) and then reorganized clusters in consensus-building discussions to increase coherence. This process yielded six final OTIS domains: organizational characteristics (e.g., size; age); governance and operations (e.g., organizational and social subsystems); tasks and processes (e.g., technology cycles; excess capacity); knowledge and learning (e.g., tacit knowledge; sense making); characteristics of a population of organizations (e.g., isomorphism; selection pressure); and interorganizational relationships (e.g., dominance; interdependence).

Discussion

Organizational influences on implementation are poorly understood, in part due to the limitations of extant frameworks. To improve understanding of organizational influences on implementation, we distilled 70 constructs from nine organization theories into six domains. Applications of the OTIS framework will enhance understanding of organizational influences on implementation, promote theory-driven strategies for organizational change, improve understanding of mechanisms underlying relationships between OTIS constructs and implementation, and allow for framework refinement. Next steps include testing the OTIS framework in implementation research and adapting it for use among policymakers and practitioners.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

✪ Aligning implementation and user-centered design strategies to enhance the impact of health services: results from a concept mapping study

Abstract:

Background

Innovative approaches are needed to maximize fit between the characteristics of evidence-based practices (EBPs), implementation strategies that support EBP use, and contexts in which EBPs are implemented. Standard approaches to implementation offer few ways to address such issues of fit. We characterized the potential for collaboration with experts from a relevant complementary approach, user-centered design (UCD), to increase successful implementation.

Method

Using purposive and snowball sampling, we recruited 56 experts in implementation (n = 34) or UCD (n = 22). Participants had 5+ years of professional experience (M = 10.31), worked across many settings (e.g., healthcare, education, human services), and were mostly female (59%) and white (73%). Each participant completed a web-based concept mapping structured conceptualization task. They sorted strategies from established compilations for implementation (36 strategies) and UCD (30 strategies) into distinct clusters, then rated the importance and feasibility of each strategy.

Results

We used multidimensional scaling techniques to examine patterns in the sorting of strategies. Based on conceptual clarity and fit with established implementation frameworks, we selected a final set of 10 clusters (i.e., groups of strategies), with five implementation-only clusters, two UCD-only clusters, and three trans-discipline clusters. The highest-priority activities (i.e., above-average importance and feasibility) were the trans-discipline clusters plus facilitate change and monitor change. Implementation and UCD experts sorted strategies into similar clusters, but each gave higher importance and feasibility ratings to strategies/clusters from their own discipline.

Conclusions

In this concept mapping study, experts in implementation and UCD had perspectives that both converged (e.g., trans-discipline clusters, which were all rated as high-priority) and diverged (e.g., in importance/feasibility ratings). The results provide a shared understanding of the alignment between implementation science and UCD, which can increase the impact and sustainability of EBP implementation efforts. Implications for improved collaboration among implementation and UCD experts are discussed.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

✪ Toward criteria for pragmatic measurement in implementation research and practice: a stakeholder-driven approach using concept mapping

Abstract:

Background

Advancing implementation research and practice requires valid and reliable measures of implementation determinants, mechanisms, processes, strategies, and outcomes. However, researchers and implementation stakeholders are unlikely to use measures if they are not also pragmatic. The purpose of this study was to establish a stakeholder-driven conceptualization of the domains that comprise the pragmatic measure construct. It built upon a systematic review of the literature and semi-structured stakeholder interviews that generated 47 criteria for pragmatic measures, and aimed to further refine that set of criteria by identifying conceptually distinct categories of the pragmatic measure construct and providing quantitative ratings of the criteria’s clarity and importance.

Methods

Twenty-four stakeholders with expertise in implementation practice completed a concept mapping activity wherein they organized the initial list of 47 criteria into conceptually distinct categories and rated their clarity and importance. Multidimensional scaling, hierarchical cluster analysis, and descriptive statistics were used to analyze the data.

Findings

The 47 criteria were meaningfully grouped into four distinct categories: (1) acceptable, (2) compatible, (3) easy, and (4) useful. Average ratings of clarity and importance at the category and individual criteria level will be presented.

Conclusions

This study advances the field of implementation science and practice by providing clear and conceptually distinct domains of the pragmatic measure construct. Next steps will include a Delphi process to develop consensus on the most important criteria and the development of quantifiable pragmatic rating criteria that can be used to assess measures.

^{**This abstract is posted with permission under the Creative Commons Attribution 4.0 International License**}

Authors:

✪ Open Access

Read the full text in the open access journal Implementation Science Communications

Abstract:

Background

Methods

Results

Conclusions

Contributions to the literature

Authors:

Subscription Required for Full Text Access

Read the full text in the subscription access journal Journal of Pediatric Psychology

Authors:

✪ Open Access

Read the full text in the open access journal Implementation Science Communications

Abstract:

Background

Methods

Results

Conclusions

Authors:

Subscription Required for Full Text Access

Read the full text in the subscription access journal Canadian Journal of Public Health

Authors:

✪ Open Access

Read the full text in the open access journal Implementation Science Communication

Abstract:

Background

Methods

Results

Conclusions

Authors:

✪ Open Access

Read the full text in the open access journal Frontiers in Health Services

Abstract:

Introduction

Methods

Results

Discussion

Authors:

✪ Open Access

Read the full text in the open access journal Implementation Science Communications

Abstract:

Background

Method

Results

Conclusions

Authors:

✪ Open Access

Read the full text in the open access journal Implementation Science

Abstract:

Background

Methods

Findings

Conclusions