Outputs, outcomes and impacts
Using Theory of Change to plan, develop and evaluate your initiative
Aims of the session
• To make explicit the link between vision, planning and
evaluation
• To introduce you to Theory of Change as a tool for
understanding, developing and evaluating your initiative
• To help you prepare for your team planning time this
afternoon
2
Introduction to Theory of Change
Theory of Change is…
• “an outcomes-based, participatory method…for planning,
evaluation, and organisational capacity-building”; it
“defines all building blocks required to bring about a
given long-term goal”(www.theoryofchange.org)
• a development of ‘programme theory’ and ‘logic models’,
adapted to complex change where both the path and the
destination are evolving (Gamble 2008)
• useful as a “bridging tool” that creates “provisional
stability”, linking planning and evaluation (Saunders et al
2005)
3
Purpose of Theory of Change
Plan
Formulate your
Theory of
Change
Improve
Evaluate
Develop your
initiative
Test your Theory
of Change
4
Developing your Theory of Change
1. Identify your long-term goal.
2. Conduct ‘backwards mapping’ to identify the
intermediate outcomes (or preconditions) necessary to
achieve that goal.
3. Identify the activities that your initiative will undertake to
create these outcomes.
At the same time consider contextual factors, assumptions,
resources needed etc…
5
Developing your Theory of Change
A good Theory of Change should be:
• Plausible – the logic of the theory is credible
• Doable – achievable with the resources (and time)
available
• Meaningful – stakeholders see the goals as important
and worthwhile
• Testable – there are credible ways of discovering
whether the predicted results occur (and how/why)
6
Activity
• In teams, develop a simple Theory of Change for
your initiative using the template provided (40 mins)
• Consider:
• Is your goal meaningful?
• Does the logic of your theory make sense?
• What assumptions are you making?
• What resources will you need?
• How will you test your theory?
7
Testing your Theory of Change
Impacts:
Sphere of interest
Outcomes:
Sphere of influence
Ouputs:
Sphere of control
• 'complex'
• long-term evaluation
• quantitative and qualitative
data, synthetic methods,
multiple perspectives
• 'complicated'
• medium-term evaluation
• mainly qualitative
data/methods (‘how’ & ‘why’)
• 'simple'
• short-term evaluation
• mainly quantitative
data/methods
8
Sources of data
1. Data from evaluation activities
• surveys, focus groups, interviews, case studies etc
• time-consuming to collect and limited in scope
2. Data generated by initiative itself
• planning documents, decision logs, network maps,
meeting notes, emails etc
• freely available but of limited worth externally
3. ‘Naturally occurring’ institutional data
• NSS, module feedback, retention/achievement data etc
• valued as impact data but difficult to show causality
9
Developing your evaluation strategy
1. What aspects of your initiative will you evaluate?
• Not everything can or should be evaluated – what is both
important and testable?
2. Why are you evaluating?
• To meet performance indicators? To show impact? To
understand, inform and develop your initiative?
3. Who are you evaluating for?
• Senior managers? Funders? Staff? Students? The sector?
Yourselves?
4. How will you evaluate?
• What sources of data will you use? What methods will you
use to collect and analyse the data?
10
Summary and next steps
• Theory of Change makes explicit what is assumed or
tacitly understood about your initiative
• Theory of Change is a planning tool that helps you evaluate
and an evaluation tool that helps you plan
• Your Theory of Change is just that – a theory: it should be
revisited and revised throughout your initiative
• Use the team planning time this afternoon to develop your
ToC further and consider your evaluation strategy
(purposes, audiences, data and methods)
11
References and further reading
Dozois, E, Langlois, M & Blanchet-Cohen, H (2010) DE201: A practitioner’s
guide to developmental evaluation, J W McConnell Family Foundation.
Gamble, J (2008) A developmental evaluation primer, J W McConnell Family
Foundation.
Rogers, P J (2008) Using Programme Theory for Complicated and
Complex Programmes, Evaluation, 14 (1), 29-48.
Saunders, M, Charlier B, Bonamy, J (2005) Using evaluation to create
‘provisional stabilities’: Bridging innovation in higher education change
processes, Evaluation, 11 (1), 37-54.
Theory of Change Community, http://www.theoryofchange.org/
12