Mitigating climate change, avoiding the use of nuclear weapons, and preventing mass violence and atrocities are fundamental to ensuring a peaceful and secure world, but they require collective action and good governance to get there. In service of this goal, the Stanley Center for Peace and Security brings together governments at all levels, businesses, journalists, civil society organizations, and researchers to collectively identify and advocate for policy solutions that move us closer to a healthy planet and lasting peace.

The Stanley Center works in a complex, global ecosystem of NGOs, government entities, research institutions, and media outlets. Compounding this complexity are the myriad contexts and variables affecting conditions that either promote or threaten peace and security. When sizable and effective policy gains are infrequent and often unpredictable, the goal of evaluation is not solely to measure against a set of predetermined, long-term outcomes, but to instead understand the extent to which a player or network of players can remain nimble and responsive to changing policy landscapes.

Evaluation of advocacy and progress toward policy change goals is notoriously challenging, since such progress is seldom linear and differently situated individuals in a field or community often hold competing perspectives on what strategies will work and how to define success.

To support the Center’s learning in such rapidly changing spaces, we prioritized a developmental approach to evaluation. Developmental evaluations are very much alive and growing with the initiatives they assess; as the intervention evolves, so too does the evaluation, such that it informs real-time strategy decisions as well as measuring progress toward longer-term outcomes. In the meantime, we oriented our inquiries around a sequence of progressive interim outcomes based on prevailing social research on how policy change happens.

Conducting a developmental evaluation for the Stanley Center meant designing unique cycles of planning, data collection, analysis, and reporting for each program area—nuclear weapons, climate change, and mass violence and atrocities. The Stanley Center organizes its policy advocacy work in roughly 24-month increments. We timed our evaluation cycles to correspond with significant inflection points in each program area’s strategy design. With some of the research, we were exploring the effects of specific activities, events, or publications; in others we solicited from participants their assessments of how the Center was implementing its programs (for example, Policy Lab convenings); in all cases we sought evidence of incremental progress toward interim outcomes one or two steps removed from the Center’s work.  To do so, we tapped tools particularly suited to developmental and advocacy evaluation approaches.

One such tool was Ripple Effect Mapping, a participatory group process designed to capture the results of policy change advocacy efforts. With the Center’s climate change team, we brought together stakeholders from across the globe ahead of in-person events they were attending in Paris. Using elements of Appreciative Inquiry (questions that are designed to elicit observations on particularly synergistic or energy-producing ideas, activities, or interventions), mind mapping, and qualitative data analysis, climate change field leaders came together to consider the different pathways for change originating with, or stimulated by, the Stanley Center’s work.

Ripple Effect Mapping was a powerful data collection and analysis tool for understanding the ways the Center has facilitated shifts in the climate change policy landscape. It was also an opportunity to bring stakeholders together to pause and reflect on their work and identify alignment and synergy they might not have otherwise noticed. Again and again, we have seen that the very act of pausing, asking questions, and bringing people together in a research-oriented conversation functions itself to enhance or strengthen the intervention about which we’re inquiring.

For the Stanley Center’s nuclear weapons and their mass violence and atrocities program areas, we used Bellwether interviews to better understand how pervasively the Center’s policy message or “frame” was traveling across varying spheres of influence. For these same programs at different points in time, we fielded collaboration-focused surveys to explore different ways in which the Stanley Center’s work was contributing to conversations and cooperative efforts among experts from different sectors to tackle these complex and multifaceted challenges. Analyses from these and other data gathering methods deepened the Center’s understanding of its role in the complex policy ecosystems of climate change, nuclear weapons, and mass violence and atrocities. Our learning conversations shed new light on the problems the Center is trying to solve, which in turn sharpened its strategy and messaging and informed the evolution of its identity as an influential cross-sector convener.

To view one of the learning memos from our work with Stanley Center, drop us a line.