Skip to content
Implementation Science at UW

Author: Elspeth Nolen

An introduction to stakeholder and policy analysis

Authors: Dr. Arianna Rubin Means & Dr. Brad Wagenaar

It is sometimes difficult to understand the difference between policy and programmatic and/or intervention goals. Specifically, a policy is a plan for future action adopted by a governmental entity and formulated through the political process.

For example, a programmatic action that focuses on engaging with women’s groups to change local perceptions around HPV vaccination is not, by itself, a public policy change, as it does not necessarily have to be adopted by a governmental entity and does not touch the political process. Similarly, for road traffic accidents, training emergency providers in better treatment or transport of trauma patients also does not necessarily intersect with the government or need to be formulated through the political process.

In contrast, enacting a law to increase the drinking age from 18 to 21 to control road traffic injuries or instituting taxes on tobacco products for cardiovascular disease prevention are examples of policy changes as they necessitate engaging government structures within the political process. Similarly, for HPV vaccination campaigns, creating legislation that requires all girls who attend school to be vaccinated is also a policy change. However, even local efforts to work with women’s groups to change perceptions regarding HPV vaccines could be re-framed into a policy change. Perhaps the government could enact a guideline requiring that all public clinics hire community outreach workers focused on HPV vaccination messaging. While this might still mobilize women’s group to engage with HPV vaccination campaigns, it would do so via a government guideline or law.

Health leadership around the world frequently need to prioritize one potential policy change over another. Often “implementation feasibility” or “cost of enacting the policy change” are primary considerations, but there are many other factors to include in this decision making process. One available resource for weighing one intervention against another is the Disease Control Priorities Project, which provides a table of different common interventions and their cost-effectiveness.

During the comparison process, scoring proposed policy changes by developing standardized criteria within a matrix can be a great way to be open and transparent. A transparent policy selection process can be very compelling to stakeholders who want to understand why you are advocating for a certain policy change over another.

In stakeholder and policy analysis, it is critical to identify appropriate stakeholders and understand their influence on the policy process, their motivators given the proposed policy change, and strategies for engaging the stakeholder in the policy process. It can be tempting to focus on stakeholders who are champions of the proposed change and to frame their motivators in positive/supportive terms rather than identifying stakeholders who might be opposed to a given policy change. However, it is often the opposed stakeholders that you need to work hard to understand and reach out to. One successful strategy is to engage dissenting stakeholders in coalitions with other stakeholders who you have identified as supportive.

Lastly, be sure to think critically about the motivations for all stakeholders. For example, a policy change of increasing taxes on cigarettes or alcohol may address a public health issue by decreasing disease burden, but it is important to understand if the general population (which includes both people who smoke and/or drink) would be supportive of the policy change. Many people in the community might be opposed to this policy because they might see themselves as at low risk of associated diseases, but impacted by the increased financial burden. Individuals who are alcohol dependent or addicted to cigarettes will see the price of their addiction increase, and while many of them may want to quit, there are systemic and structural issues that can make this difficult. Be sure you consider all stakeholders and how you could develop strategies to engage broad groups with diverse interests, many of which may not be explicitly or publicly stated.

An introduction to quality improvement

The goal of quality improvement (QI) is to test multiple rapid PDSA (Plan-Do-Study-Act) cycles, so QI is often implementing changes to existing processes that are practical and, ideally, simple. Identifying appropriate change concepts, change innovations, tests of change, and selecting an appropriate outcome variable are all necessary aspects of PDSA, but doing so can be challenging.

When identifying a change concept to target, it can be tempting to identify those more akin to traditional research study questions, such as exploring how to design valid diabetes clinical screening guidelines. Instead, a QI approach change concept could be focused on optimizing a specific aspect of the delivery of these evidence-based diabetes guidelines.

There is a bit of an art to defining a change concept that is neither too general nor too specific. A change concept such as “Increase HIV testing” is probably too general, while a change concept such as “Increase HIV testing by providing incentives in the form of branded t-shirts” is probably too specific. You want an overall change concept that is specific enough to guide your tests of change, but also general enough to allow multiple different approaches to be tested if your first test of change fails to have the desired impact. One possibility for a change concept would be to increase the number of people consenting to be tested for HIV in health facilities.

When determining change innovations that build upon your change concept, you may want to avoid defining a multi-component innovation. Whether or not you observe improvements during your QI, you may face difficulty untangling which part of your innovation contributed to the result you observed. One example of a testable innovation linked to the change concept above is improve patient satisfaction with HIV testing processes. All of your tests of change, therefore, would be focused on increasing the number of people who consent to be tested (change concept) by improving satisfaction with HIV testing processes (change innovation).

When designing tests of change, it is important to think about how such tests are linked to their stated change concept. For example, if your change concept is to increase voluntary participation in HIV testing at a health facility, it likely would not be helpful for your test of change to involve monitoring antiretroviral adherence among HIV-infected populations. Rather you might want to focus on tests of change such as launching new sensitization messaging to target individuals in the waiting room, or renovating HIV testing spaces to make them more comfortable or private.

It is also critical keep in mind the scale of your proposed test of change. PDSA tests of change are typically quick, and do not need to be on a large scale with a very large sample size. Your tests of change can even target a few patients being seen by one provider, and then progressively be scaled-out to larger sample sizes if the results are promising. Initial ideas that are too large in scale for PDSA (i.e. to test a community-wide sensitization strategy), can often be brought down to a much smaller, more PDSA-appropriate scale (i.e. test the use of sensitization meetings with 2-3 religious leaders in the community).

Another exciting aspect of QI is that frontline staff play a key role in determining the tests of change that are appropriate for addressing a specific outcome. These are the staff most likely to understand how their particular system and context operate, and therefore are best placed to make or suggest necessary adaptations to an intervention. You should spend time thinking about how you will meaningfully involve frontline staff in identifying and evaluating tests of change in your QI work.

Identifying an appropriate outcome for your QI work is important so that you have a helpful benchmark of success. With QI, you want a clear relationship between the change you are testing and the metric you are using to measure results so that you can rapidly evaluate whether your change should be continued as-is, scaled up, modified, or discontinued. Given the example above, a possible outcome metric is increase the number of adult patients who consent to be HIV tested from 12% (baseline) to 25% in 3 months.

Additionally, there are clinical outcomes that are either expensive to measure, or could take years to observe. Measuring morbidity and mortality is often too distal or difficult to rapidly and repeatedly measure in QI (unless perhaps you are talking about QI process improvement in a high-volume trauma care center where mortality would be frequently recorded). Thus, QI is often best positioned to maximize program outputs or outcomes rather than program impact.

The main point? With PDSA cycles, simple metrics can be most helpful given that you will evaluate the outcome frequently and will want the outcome to be inexpensive and straightforward to measure.