Evaluation of
PHO’s KT
Project
Mel Brockman
June 17, 2014
Overview
 CIHR
supplement grant obtained.
 Time
period: June 2012 to June 2013.
 External
evaluation arranged in late 2013.
Project Overview
 Two


target audiences:
Community/general public.
Decision-makers.
 Goals:


Promote awareness.
Encourage action.
 Integrated
approach
knowledge translation (iKT)
Key Activities
Purpose of Evaluation
 Assess



whether…
awareness was generated;
action was prompted;
and iKT was achieved.
 Improvement-focused.
Type of Evaluation
 Process

Project operations and implementation.
 Outcome


Short-term (learning-based).
Intermediate (action-based).
Methodology
 Records
 Key
informant interviews (N = 17).
 Online


review.
surveys:
Summit participants (N = 20).
PHO network (N = 7).
Key Findings – Positives

Della video a big hit.

Summit was a success despite not arriving at
a national agenda.

Public opinion survey a valuable KT research
activity.

Partners happy to see the Region getting
involved.
Key Findings – Areas for
Improvement
 Planned
 Some
activities with school community.
groups not effectively engaged.
 Process-tracing
 iKT
issues.
not really achieved.
Recommendations
 Create
a comprehensive conceptual
table at the outset; preferably, prior to
submitting the grant application.
KT
Rationale
activity
Aim
Target
audience(s)
Target
subgroup(s)
Goal(s)
activity will
contribute to
Recommendations
 Provide
potential partners with enough
information to make an informed decision
about whether they are willing and able,
time- and resource-wise, to sign on to the
grant.
Recommendations
 Involve
committed partners and other
stakeholders in the conceptualization of
the project; preferably, prior to submitting
the grant application.
Recommendations
 Collaboratively
create a “game plan”
and clearly articulate project team
members’ roles and responsibilities at the
outset.
Recommendations
 Establish
process-tracing procedures at
the outset.
Recommendations
 Continue
to strive toward better internal
communication and coordination.
Recommendations
 Be
more specific in the identification of
target audiences and target sub-groups
within them.
Recommendations
 Focus
on the quality of the activities, not
the quantity.
Recommendations
 As
a team, develop a project plan that is
proportionate to the time and resources
that will be available across all team
members.
Recommendations
 Continue
to explore creative communication
and dissemination strategies to engage the
general public in order to go beyond
“preaching to the choir.”
Recommendations
 Give
people specific options for actions
they can take to make a difference.
Recommendations
 Invest
the most time and resources in
activities aimed at increasing health
equity locally and provincially.
Recommendations
 Articulate
the desired actions that will be
prompted by the KT activities.
Recommendations
 Identify
at the outset what will be
measured for evaluation purposes and
design measures accordingly.
Recommendations
 Strive
to achieve higher response rates
with data collection activities.
Activity
Level of Fit
Della video
High
Computer game for youth
High
School presentations and student forums
High
Interactive social media tools (e.g., CVC)
High
Health Equity Summit
High
Mobilize! forum
High
Dissemination of SHS report
Medium
Conference presentations, journal publications
Medium
Replication/public opinion survey
Low
Follow-up questionnaire with those who received SHS report
Low