California’s clinicians have a unique understanding of how to deliver health care, making them prime change agents for identifying, testing, and sharing improvements at their own organizations and throughout the state. To be effective, however, they require strong leadership, management, and business skills, which are often missing in clinical education programs.

The Health Care Leadership Program, funded by the California Health Care Foundation, aims to fill that gap. Implemented by Healthforce Center at UCSF, the Health Care Leadership Program works to equip clinicians with skills they might not otherwise learn in order to strengthen community health care organizations and ultimately spark field-level changes that increase access to care and drive equity in health outcomes.

Evaluations of leadership development programs often struggle to assess organization- and field-level effects in addition to changes for individuals. The change for the individual may be immediate; we can easily track how a program participant gains knowledge, skills, confidence, and a network of supportive peers. But how they wield those resources and to what end often remains to be seen on a timeline that extends far beyond the intervention itself.

To address this challenge, we developed an evaluation that tapped quantitative data for easier-to-measure changes for individuals, which we triangulated with creative and adaptive qualitative data collection methods for the harder-to-detect shifts at the organizational and field level.

A survey that built on our decade-long exploration of health care leadership development. Having evaluated ten cohorts of the Clinic Leadership Program, a program for emerging leaders in community health centers implemented by Healthforce UCSF and funded by Blue Shield of California Foundation, we had a deep understanding of the leadership development theory of change and a cache of survey tools to draw on for the Health Care Leadership Program. Our survey to Program alumni revealed insights that showed the rippling effects of the individual investment on organizations: 95% of respondents reported leading or directly influencing change within their organizations, and of those alumni, 83% said the Program was necessary or influential to these changes.

“Since the beginning when we started the work with Blue Shield, the partnership has been a thoughtful one with lots of attention to detail and methodologically sound. The work with Informing Change reinforced my confidence in what we were collecting and how we were doing it.”

– Sunita Mutha, Healthforce Center at UCSF

Research methods that brought alumni together, supporting a growing network and surfacing field-level shifts. To get at this question of how an investment in individual leaders can affect the broader health care field in California, one of our key data collection activities brought alumni together for virtual and in-person outcome mapping sessions. We drew on three key methods for these sessions—most significant change, outcome harvesting, and ripple effect mapping. Our questions prompted alumni to riff off of one another’s contributions in identifying how the Program supported their ability to improve and change health care organizations, systems, and policies. We learned more about how Program alumni scaled innovations, led collaboratives and coalitions, and informed and contributed to policy and advocacy efforts focused on the health care issues facing Californians.

“When the Informing Change team wanted to try something new methodologically for CHCF, I was on board because I trusted we could ask the hard questions and get a response from alumni.”

– Sunita Mutha, Healthforce Center at UCSF

Grounding systems-level change in alumni stories. The mapping sessions illuminated for us, Program implementers, and Alumni themselves a broad landscape of change, but we knew there was more to learn about how, exactly, the Program contributed to these changes. Using what we learned in the mapping sessions, we conducted deep-dive interviews with individual Program alumni. In these one-on-one conversations, we dove deep with Program alumni on their experience with the Program, what they learned, and how they’ve applied it throughout their careers.

One alum, Dr. Jocelyn Freeman Garrick, used the business acumen and political savvy she gained through the Program to launch her own nonprofit, which works to diversify the health care workforce. Another alum reported that deeper connections to the network has enabled her to advocate for legislation on medically-assisted treatment for opioid use disorder. Still another is using the support of his cohort and the resources provided by the Program to transform how health care is delivered through culturally responsive care. Each alum described different pathways to different field-level shifts, yet all were bolstered in some way by the support of the Program.

“I am more in the forefront of speaking out about policy changes to better serve families that are defenseless and vulnerable.”

– Alum (Cohort 6)

These individual cases highlighted the richness of the Program’s rippling effects, while also reinforcing the common themes we heard in outcome mapping sessions: providing mutually reinforcing emotional, social, and technical resources strengthens networks and deepens leadership.

By simultaneously building participants’ intellectual, social, and emotional resources, the Program leaves a lasting influence on their capacities to effect change.