Muskie School of Public Service
Institute for Public Sector Innovation
Where we are trying to go?
• Common outcome measures
• Quality data collection for outcomes
• A clear understanding of individual
program process measurement needs
• Program evaluation practiced locally
Muskie School of Public Service
Institute for Public Sector Innovation
Purpose of Trainings
• To build capacity to move from reporting
inputs/outputs at program level (the
current state) to data-driven
performance measurement and
management linked to organizational
goals and priorities (the future state).
Muskie School of Public Service
Institute for Public Sector Innovation
Training Overview
• Four Logic Model sessions starting in
November 2008 and running through February
2009. These sessions will be 3-4 hours each.
• Grantees will complete “Evaluation Planning”
Worksheets.
• Grantees will complete and submit a data
collection methodology.
• Grantees will submit data to Muskie by summer
2009.
Muskie School of Public Service
Institute for Public Sector Innovation
Training Objectives - Overall
• By early 2009, all JJAG grantees will have
developed a project-specific Logic Model(s)
and evaluation planning worksheets
• By mid 2009, grantees will have
developed/modified existing data collection
tools, developed an evaluation methodology,
and started to collect data.
Muskie School of Public Service
Institute for Public Sector Innovation
Training Objectives
• Differentiate the 3 levels of evaluation
• Understand the Steps in the Evaluation
Process
• Recognize the advantages of Logic
Models
• Complete a Basic Logic Model
Muskie School of Public Service
Institute for Public Sector Innovation
Program Evaluation Can Answer:
• What have we done?
• How well have we done it?
• How much have we done?
• How effective has our program been?
• What could we do better or differently?
• Was it worthwhile?
• Was it important?
Muskie School of Public Service
Institute for Public Sector Innovation
Use Three Levels of Evaluation
to help answer the evaluation questions
• Process: (Interventions, Activities or
Strategies)
• Impact: (Initial and Intermediate outcomes)
• Outcome: (Long-term outcomes)
Steps in the Evaluation Process
START HERE
1. Engage
Stakeholders
6. Ensure Use
and Share
Lessons
Learned
2. Describe
Program
5. Analyze
and
Interpret
Data
3. Determine
Evaluation
Design
4. Collect
Data
Muskie School of Public Service
Institute for Public Sector Innovation
Program Overview Logic Model
Program
Components
Initial
Outcomes
Intermediate
Outcomes
Long-Term
Outcomes
Interventions
Process
Impact
Outcome
Muskie School of Public Service
Institute for Public Sector Innovation
HIV Program Logic Model Template
(for providers)
Program Components
(priority populations)
Unsafe Sex,
Both Partners
Male
Intervention
Group Level
Six session
HIV
prevention
education
group in
Correctional
settings.
Education will
include
condom use
negotiation
with sex
partners.
Initial Outcomes
(KANS)
Persons
attending the
groups will
report an
increase in the
intent to
negotiate
condom use
with sex
partners.
*remember YOUTH,
RACIAL and ETHNIC
MINORITIES and
PEOPLE with HIV throughout
*KANS=Knowledge, Attitudes, Norms, Skills; **B=Behavior
Intermediate
Outcomes
(B)
Persons
attending
the groups
will
negotiate
condom
use with
sex
partners.
Persons
attending
the group
will use
condoms
with their
sex
partners.
Long-Term
Outcome
(Health)
Decreased
HIV
transmission
Improved
well-being
of HIV
infected
people
Muskie School of Public Service
Institute for Public Sector Innovation
Data Sources Exercise
LongTerm
Outcome
Intermediate
Outcome
Initial
Outcome
Interventions
Evaluation
Questions
Performance
Indicators
Measures
Data
Sources
Timing
Muskie School of Public Service
Institute for Public Sector Innovation
Expectations:
1. Grantees will be responsible for
completing multiple worksheets
between sessions.
2. We will monitor your progress and
report any delays in submitting
assignments.
3. Muskie will provide ample TA.
You can call on us if you have any
questions.