Introduction to Evaluation
Tammy Horne, Ph.D.
WellQuest Consulting Ltd.
(780) 451-6145
[email protected]
www.wellquestconsulting.com
What is Evaluation?

“Program evaluation is the systematic
collection of information about the
activities, characteristics, and outcomes of
programs to make judgments about the
program, improve program effectiveness,
and/or inform decisions about future
program development…
7/13/2017
2
What is Evaluation?

…Program evaluation does not occur in a
vacuum and is influenced by real-world
constraints. Evaluation should be practical
and feasible and must be conducted
within the confines of resources, time, and
political context…
7/13/2017
3
What is Evaluation?

…Moreover, evaluation should serve a
useful purpose, be conducted in an ethical
manner, and produce accurate findings.
Evaluation findings should be used to
make decisions about program
implementation and to improve program
effectiveness.” (Centers for Disease
Control, 2001, p. 6)
www.cdc.gov/tobacco/evaluation_manual/Evaluation.pdf
7/13/2017
4
What Is Evaluation?

Formative evaluation – improve existing
program, ‘stabilize’ it for summative
evaluation – focus is usually processes and
sometimes early outcomes

Summative evaluation – judgment of merit
or worth of program; make decisions
about to keep, change, expand, cut –
often emphasizes longer-term outcomes
7/13/2017
5
Why Do We Evaluate?

To learn what is happening in our programs
(Processes - what we do and how people
respond)

To learn what is changing, what differences we
are making (Outcomes - desired or expected
results; the changes we hope to see.)

To learn how processes and outcomes are
related (Theory of change/program theory)
7/13/2017
6
Why Do We Evaluate?

To learn about how our program is
influenced by other factors in our
environments (Context)

The above kinds of learning help us to:
 Be accountable to funders
 Continue to improve our programs
 Share our knowledge with others doing
similar work
7/13/2017
7
Some Example Process Questions

What activities do we offer? How often?
For whom?

What types of people do we reach, and
how?

Who expresses interest in our program?
7/13/2017
8
Some Example Process Questions
• How do we respect diversity – e.g., be
inclusive of/responsive to people from
particular cultural backgrounds?

How do people participate? (e.g., recipients
of a service; partners in program planning
and/or delivery, community builders or
leaders, advocates for healthy
policies/practices)
7/13/2017
9
Some Example Process Questions

What makes it easy for people to
participate? What makes it hard for people
to participate? (barriers – if any)

Who stays? Who drops out and why?

What is running smoothly?

What ‘kinks’ still need to be worked out?
7/13/2017
10
Some Example Process Questions

What do participants like best about the
program?

What changes would they like to see?

What would they tell others about your
program?
7/13/2017
11
Some Example Outcome Questions:
Individual Level
• What new knowledge do participants learn?
• What new skills do they develop or
strengthen?
• What changes do they make in their
behaviour? (e.g., changes in personal lifestyle,
relationships, community engagement)
7/13/2017
12
Some Example Outcome Questions:
Organizational Level
 Do
organizations you are working with get
involved in new collaborations or
partnerships,?
 Do
organizations develop more culturally
competent practices that are responsive to
the cultural groups with whom they work?
7/13/2017
13
Some Example Outcome Questions:
Community Level

How has community capacity been
enhanced, such as:




People involved in community leadership?
New community structures/resources?
(e.g., community gardens, walking trails)
People participating, with others, in
community-building activities?
People sharing knowledge or skills?
7/13/2017
14
Some Example Outcome Questions:
Community Level

How is the quality of community life
improving? (e.g., positive relationships
between youth & seniors, neighbours
getting to know each other, better access
to local food, people feel safe on streets or
in parks)
7/13/2017
15
Some Example Outcome Questions:
Policy/System Level

What policy changes are happening in your
school district in areas like: availability of
healthy food, opportunities for physical
activity, prevention of bullying and
harassment?

How well are public non-smoking policies
being enforced?
7/13/2017
16
Some Example Outcome Questions:
Policy/System Level

How receptive are municipal councils to
community members’ requests for
recreation /parks opportunities, land for
community gardens?
7/13/2017
17