Turning a theory of change into practice and impact

“Education can lead to impact when learning leaders connect the dots between the intervention, the individuals and their contexts.”

Karen Watkins,
Co-founder of the Geneva Learning Foundation

FROM RESEARCH TO RESULTS IN THREE YEARS

International organizations and governments face ‘wicked problems’ that resist conventional approaches.

Too often, the default response to such seemingly intractable problems remains investment in conventional methods of learning, educational and training – whether face-to-face workshops or digital e-learning modules.

Three years ago, we asked: do such methods lead to the change that is needed?

This  report, our  first Impact  Review, tells the story of how we developed the ‘Scholar package’, a set of interventions to apply our new theory of change to complex challenges.

Within three months of creating the Foundation, we kicked off  three pilots.

Each pilot immediately went to scale: we convened hundreds and then thousands of participants from all over the world.

And they stayed on, learning to lead change together.

#DigitalScholar

Our own #DigitalScholar course, a six-week journey for learning leaders to adapt their practices in the Digital Age, recruited over 1,000 learning leaders to explore the Foundation’s approach to digital learning. Some programme graduates with no prior experience have since become digital learning leaders.

“It’s been exactly 3 years since I took the #DigitalScholar programme and here I am today in charge of digital learning and education with an ambitious start up… Who would have known!”

Nadene Canning, VP Education and Learning, EDACY

#Ambulance!

Over 5,000 front-line, community staff and volunteers from 110 countries, including 200 in Syria, joined the #AMBULANCE! exercises developed in partnership with the Norwegian Red Cross and the International Federation of Red Cross and Red Crescent Societies (IFRC) to document incidents of violence faced by health workers. We observed change in preparedness, thinking, and practice to face the threat of violence. By the third iteration, this learning initiative was led by frontline practitioners themselves.

“I thought I already knew how to face violence. Then I heard how they do things in other parts of the world. I learned how I can do my work differently. I became mindful in new ways.”

Cristina Guerrero, leader of a helicopter rescue team in Cadiz (Spain) who became a facilitator in the #Ambulance! exercises

WHO immunization scholar Programme

We started with a six-week pilot course to help country staff use global guidance to improve routine immunization.

Three years on, WHO has invested in 23 cohorts reaching 90 countries, leading to effective adoption and application of immunization guidelines by programme participants.

“After completing the Survey Scholar programme, I felt empowered to support the change process in how countries are planning vaccination coverage surveys.”

Joseph Nsiari-Muzeyi Biey, IST focal point for West Africa, WHO

“Attempting to do more of what has been done in the past is not the answer.
We need to do new things in new ways.”

George Siemens, Trustee of the Geneva Learning Foundation

How can a new organization make bold claims about its ability to produce change?

To launch these and other pioneering programmes presented in this Review, we asked our first partners to make a leap of faith. In most cases, they were willing to experiment because they had observed a lack of measurable impact using conventional methods.

Three years on and over fifty cohorts later, our theory of change is now robust enough that our partners no longer need to take such a leap of faith.

We have continued to innovate, adding to what we call the “Scholar package” of interventions.

We now stand ready to offer the Scholar package to any organization or network that needs to mobilize people at scale for change.

Growing programmes and partnerships to lead change

Our programme partnerships cut across disciplines and silos to contribute to tackling ‘wicked problems’ in any area of work where the future of our societies is at stake. Although, in our first three years, we have focused on global health and humanitarian response, our research and development has led to a generalizable model for change applicable to most if not all complex problems.

Global Health

Teach to Reach, Bill & Melinda Gates Foundation

Humanitarian response

Safer access for Emergency Response Units (ERUs), International Federation of Red Cross and Red Crescent Societies (IFRC)
Health Care in Danger (2016-2019); Humanitarian protection (2018), Norwegian Red Cross (in cooperation with the International Committee of the Red Cross)
Gender Scholar, CARE International
Duty of care, Duty of Care International

Leading change

Remote Partnering, Partnership Brokers Association (PBA)
Global prosperity, UCL Institute for Global Prosperity
Collective impact, Khurram Hassan

The Scholar theory of change

“These are more than just courses. These are interventions designed to foster and improve practice at every level.”

Conclusion of the WHO Scholar programme evaluation (2016-2018)

Why do we need a new theory for digital learning?

Digital learning often replicates the worst of face-to-face training, typically relying on top-down, one-way transmission of knowledge often too abstract to be useful – and fails to recognize local expertise.
Poor-quality training can now be transmitted to thousands of individuals at marginal cost, rather than being limited by the economy of effort required to carry out face to face training. Few results, little change, and no impact are likely to be observed.

LEARN TO CONNECT

Practitioners learn by supporting each other, engaging in deep collaboration and reflection.

It may sound abstract to affirm that our programmes are rooted in a participatory process in which a global community creates knowledge together, inspiring discovery through exploration and invention.
Yet this is exactly what consistently stirs learners in profound ways.
Our programmes consistently foster transformative growth and “coming to consciousness” in individuals who become Scholars, whether they are government civil servants or community-based volunteers.

NETWORK

CONNECT TO LEARN

Practitioners experience new ways to learn, using digital means that are not limited to knowledge  acquisition.

More importantly, they discover new ways to ‘be human’ with each other, defying distance and contexts to make diversity productive.
They see that the new digital economy of effort creates the conditions for new ways of thinking and doing.
Giving and receiving feedback becomes a “super power” that comes from within, and that depends on others.
These are the conditions for change.

DIGITAL

LEAD TO IMPACT

Post-course support networks accelerate progress toward implementation, results, and – ultimately – impact.

Exceptional, driven individuals emerge, showing high potential to lead the change that is needed.
Participants defy distance to genuinely care about each other’s problems, to commit to overcoming challenges together, and to celebrate each other’s successes.
This collaborative, flexible, motivating, participatory and supportive approach is not simply a kinder and gentler form of learning.
Rather, the pedagogical patterns of our approach closely emulate the core leadership competencies of humanitarian, global health, and development workers, who are expected to work in networked rather than hierarchical configurations and to use participatory methodologies to partner with affected populations.
The learning system fosters the emergence of actions networks and new forms of leadership.

ACTION

How do we measure impact?

Impact measurement of learning and capacity building initiatives is notoriously difficult, given the complex chain between an intervention and the results that are sought. In July 2019, we built the first Impact Accelerator, a radically-new system for continuous measurement and double-loop feedback. It continuously provides both quantitative and qualitative insights into progress made and to what extent this progress may be attributed to the intervention.

Value for money

While digital formats lower cost and enable scale, they do not, on their own, lead to strategic relevance, sustainability, or results. We scale without compromising quality.

In three days, we build a new programme from scratch.
We use only digital means. There is no per diem or travel.
Participants can continue to perform their daily work while in the programme (opportunity cost).
1,000 or more participants from across the world can join, for the cost of a single face-to-face workshop or e-learning module.
Our completion rates are at least ten times higher than higher-volume MOOCs (Massive Open Online Courses).
Value comes from the explicit pathways that lead from our programmes to impact on the ground.
We combine high volume at low cost, achieving learning outcomes beyond those achievable through low-volume, high-cost face-to-face approaches.
These outcomes take participants beyond knowledge retention to practice the digital, analytical, and leadership capabilities that are the most difficult to achieve in any medium.
In addition to learning outcomes, our programmes support the development of context-specific projects and their field implementation to exercise these capabilities, supported by a global network of peers.

3 break-throughs for the Foundation to make change, faster and better

1

We built an amazing team with the right mix of capabilities needed to design, build, and deliver more than ever before, while continuously increasing the quality.

2

We delivered the first significant research and evaluation to substantiate our observations of remarkable outcomes.

3

We invented a new system, the ‘Impact Accelerator’, to support Scholars to transform what they gain from a course into action, results and impact.

The first two break-throughs have given us confidence to offer the Scholar package of interventions to any organization tackling a ‘wicked problem’.

Furthermore, we are now equipped to prove that learning-driven change initiatives do not need to be a ‘sunk cost’: new business models can ensure financial sustainability and allow for investment into further development and growth.

This rapid growth has led to inevitable growing pains, as our budget, staff, and costs have multiplied. Nevertheless, thanks to our digital economy of effort, they remain tiny compared to their global reach and value for money.

The Impact Accelerator has opened a brand-new area for our research and development efforts: how do we sustain our programmes’ outcomes in ways that will catalyze new forms of connected, adaptive leadership for the Digital Age?

“Our theory of change grew from intersecting the science of learning with culture, reframing education as a philosophy for change in the Digital Age. That, and a lot of elbow grease.”

Reda Sadki, President of the Geneva Learning Foundation

What does the Scholar package allow organizations to do?

Adapt global recommendations to tackle problems faced in their local settings, from the district level up.

Innovate and share discoveries
and successes to inspire others.

Augment country problem-solving
through peer networks.

Scale peer support in ways that are low‑cost and community-driven.

Strengthen individual leadership capacities and informal shared leadership within systems.

Forge a shared identity among individuals focused on impact across different countries and organizations.

Theory of change in action:

the WHO immunization Scholar programme

“Just producing a guideline is not enough. We need Scholars to reflect. How can they bring innovation into the current system to offer better immunization services in the communities they serve and reach all children, be it in remote areas, urban slums, or other hard to reach situations?”

Dr Katherine O’Brien, Director, Department of Immunization, Vaccines & Biologicals, WHO

How WHO turns guidelines into action

The World Health Organization’s technical experts produces global guidance to support countries.

Using the Scholar package, we supported immunization staff in English, French, and Spanish – from the districts to the national team in the capital – to take this guidance and turn it into action. The Scholar team manages the entire process, keeping it simple for the subject matter experts and partners.

The World Health Organization’s Expanded Programme on Immunization (EPI) has used the Scholar package since 2016.

Scholars adapt and apply WHO’s global guidelines to their own contexts, with the structured input of both global experts and peers.

Successful participants leave the course with their own context-specific, peer-reviewed action plans.

In the first two years of the programme, some participants reported implementing their course projects without any post-course support.

A NETWORK 15,000 STRONG AND GROWING FAST

90 countries 23 cohorts, up to 400 per cohort, 1/3 from the district level

Completion rates up to 95%

Connecting the dots between a course, action, and results

Using the WHO guidance for routine immunization, this Scholar, an immunization officer for a global partner of a country in the Pacific, identified a need to develop a culture of supportive supervision, as training alone was not achieving the desired outcomes. He developed a series of tailored strategies to identify and reach under-vaccinated and unvaccinated persons and to regularly and systematically build capacity in vaccinators and district managers. He then presented this course project to the Minister of Health. When we spoke to him two years later, he was able to describe how implementation of his project was one significant action that contributed to an improvement in both quality of service and immunization coverage.

“I was skeptical that a digital intervention could actually help practitioners become fit for purpose. Seeing Scholars dedicate inordinate amounts of time to learning and collaboration has made me a convert – and convinced me to partner with the Scholar programme.”

David Koffi, ADS Conseil, GAVI Survey Scholar field-based programme

Working with civil society to find children who have been lost to follow-up

This Scholar, a technical support officer in a West African country, sought to increase immunization performance by finding children lost to follow up, focusing on lower performing districts. The project focused on developing community linkages, working with civil society organizations to outreach communities and to implement household visits by health community workers. Two years on, the Scholar reported that community health workers were now able to locate and vaccinate 99% of the children who were previously lost to follow up.

“You are doing magic”

Molly Abbruzzese, Program Officer, Bill & Melinda Gates Foundation

Pakistan

Muhammed Imran Qureshi advises the Pakistani immunization programme for WHO.

States within the province experience wide disparities in coverage. In his first Scholar project, Dr Qureshi identified a transformative investment to build the capacity of EPI supervisors to develop management skills covering training, culture and processes, and staff motivation, with individualized, specific goals for each State.

He then contributed to the plan’s implementation over a period of 3 years in which time Fully Immunized Child coverage in the PDHS Survey increased from 67% (2012-13) to 80% (2017) and DTP containing coverage increased from 76% (PDHS 2012-13) to 89% (PDHS 2017-18).

Muhammad Imran Qureshi receives a Survey Scholar certificate from (left to right) statistician Dale Rhoda, WHO scientist Carolina Danovaro, and survey specialist John Wagai.

“I was very happy that we achieved these wonderful results, increasing coverage from 76% to 89%.”

Muhammed Imran Qureshi, Scholar Alumni leader for Pakistan

Nigeria

“After my first Scholar course, I reviewed the data from a measles campaign I had participated in. I could see new problems as well as potential solutions. I want to use my new capabilities to improve my country’s immunization programme.”

Gift (Ettentuk) Aniyom,
Scholar Alumni leader for Nigeria

Defying distance

to tap into passion and commitment to change

By observing new forms of leadership that first emerged organically, we co-created and nurtured the Accompanists and the Impact Accelerator to strengthen support to Scholars, during and after courses.

Accompanists

In our first cohorts, we observed that experienced participants offered support to new Scholars.

In response, the Scholar Team invited these volunteers to become ‘Accompanists’, creating a powerful, scalable volunteer web of support.

Half of each new cohort commits to ‘accompany’ others. Initially strangers, Accompanists and Scholars may be halfway across the world from each other, but quickly form a close-knit team.

Furthermore, Scholars who receive Accompanist support are more likely – if their organization supports them – to apply what they gained from the programme, even when faced with significant challenges. This finding surprised us, given that Accompanist support is offered only during the course.

Impact Accelerator

“Learners became leaders and facilitators. In some countries, they are taking on the issues that matter to them, turning course projects into an agenda for change.”

Ian Steed, volunteering adviser

In a few countries, Scholars self-initiated informal, motivated groups of professionals operating across agencies, to transform ideas into collaborative project implementation.

Observing these voluntary dynamics, we saw potential for systemic change – and immediately convened Scholars to develop a new system to support this across the network, leading less than six months later to the creation of the Impact Accelerator in 55 countries and of country groups in 24 of them.

“When we ran the first pilot in 2016, I was surprised by the high level of engagement from country-based participants – and hoped that this would lead to more than training.”

Jhilmil Bahl, Capacity Building Officer, WHO