KENTUCKY CANCER CONSORTIUM
EVALUATION PLAN RESULTS
FOR JULY 2011-JUNE 2012
AND NEXT STEPS
PRESENTED BY JESSICA JONES, MSW
CDC PROGRAM
EVALUATION
FRAMEWORK
1) Engage Stakeholders and
Primary Intended Users
2) Describe the Program/
Partnership/Plan
3) Focus the evaluation design
i.
Develop evaluation
questions and design of the
evaluation- Based on ‘need
to know’ information
4) Gather credible evidence
i.
ii.
Develop indicators
Data collection methods
i.
Analyze and interpret the
evidence
5) Justify Conclusions
6) Ensure use and share
lessons learned
EVALUATION PLAN
• Required part of our CDC funding
• Consists of 3 sections:
• Program
• Partnership
• Plan
1. PROGRAM
• Defined as:
Evaluating selected strategies in the
Cancer Action Plan (CAP)
HOW HAVE WE MEASURED THE
“PROGRAM”?
• 2009-2010: Coordination between the Kentucky
Cancer Program and the Kentucky Cancer
Consortium
• Joint projects and products
• 2010-2011: Offered technical assistance to the
Kentucky Cancer Program (KCP)
1) Examined the “success rate” of the Cooper/Clayton
Method to Stop Smoking, a state-wide smoking cessation
program
2) Obtained baseline data on the pilot project involving
worksites and colon cancer screening education, “Colon
Cancer Prevention is Good Business”
HOW HAVE WE MEASURED THE
“PROGRAM”?
• 2011-2012:
• “Program Completion” rate of the Cooper/Clayton
program
• Obtained second year data on the KCP worksite wellness
campaign, “Colon Cancer Prevention is Good Business”
RELATING IT TO THE CAP
Goal 1: Reduce incidence/mortality
from tobacco-related cancer
• Objective 1.5- Objective
1.10: Focus on decreasing
adult prevalence rates
• Evaluate effectiveness of
the Cooper/Clayton
Method to Stop Smoking
program
Goal 7: Reduce incidence/mortality
from colon cancer
• Objective 7.1- Objective 7.3:
Focus on increasing colon
cancer screening rates
• Encourage worksite
cultures that promote
and incorporate
appropriate colon
cancer screening
behaviors
PROGRAM- RESULTS
• C/C Evaluation:
Conducted same
evaluation in Jan. 2012July 2012:
• 43.5% program completion
rate
• Female (71%)
• Caucasian (94%)
• Between the ages of 4564 (54%)
• High School Diploma or
GED Only (56%)
• Had a smoking history of
16-25 years (25%) followed
by 26-35 years (24.6%)
• Worksite Wellness
Evaluation:
• Expanded to include more
worksites in 2011-2012
• Evaluation results are still
being analyzed
FEEDBACK
• The evaluation study on both the Cooper/Clayton
and Worksite Wellness programs will continue for
2012-2013. Based on the small teams, we will find
other areas to evaluate which include
Policy/Systems/Environmental changes.
• Comments/Suggestions
• Is this information useful to you?
• Do you want to know more?
2. PARTNERSHIP
• Defined as:
“the quality, contributions, and impacts
of the Kentucky Cancer Consortium”
• Through Partnership, KCCProvides a neutral forum for cancer control
partners to network, coordinate and
collaborate to address Policy/Systems/
Environmental changes
HOW HAVE WE MEASURED
COLLABORATION OR THE “PARTNERSHIP”?
• 2009-2010:
Partnership
Satisfaction Survey
• 2010-2011: Cancer
Action Plan Survey
• 2011-2012: PARTNER
TOOL survey
PARTNER TOOL
• Partner= “Program to
Analyze, Record, and
Track Networks to
Enhance Relationships”
• Robert Wood Johnson
funded initiative for
measuring collaboration
• Has a uniform survey with
network analysis
questions designed into
the survey tool
• Analysis tool is built into
the Partner website,
based on Excel program
Website: http://www.partnertool.net/
PARTNERTOOL
• What did we hope to measure
by using PartnerTool?
• Density/Cohesiveness
• How connected
Consortium members are
to each other
• Degree of Centralization/
Connectivity
• Who are the central
partners
• Trust
• Key to a well-functioning
coalition
PARTNERSHIP- RESULTS
• Email with survey link was sent to 44
organizations of the Consortium
• 17 organizations attempted the survey
yielding a 39% participation rate
• After question #9, participation
dropped off to 31%.
PARTNERSHIP-RESULTS
A Word of Caution about interpreting these
results…only 14 organizations responded…
• Density/Cohesiveness: 14.60%
• Members are not necessarily connected to each other
• Not a bad thing with a diverse coalition
• Our coalition has more reach and diversity
• Degree of Centralization/Connectivity: 57.80%
• There are three key partners with whom many partners
collaborate/connect
• Trust: 70.9%
• Indicates level of trust
• Want this number to be higher- ideally between 80%-85%
FEEDBACK ON THE
PARTNERTOOL SURVEY
• Are these results useful to you?
• Should we conduct this survey again or use some
other way to measure our “Partnership”?
• What action steps should KCC staff take to improve
the scores?
• What actions should you as a Consortium member
take to improve these scores?
3. PLAN
• Defined as:
“the quality and implementation of the
statewide CCC plan” or our Kentucky
Cancer Action Plan (CAP)
• Through Partnership, KCC• Maintains, implements, and evaluates the Cancer
Action Plan
• Addresses Policy/Systems/Environmental Changes
HOW HAVE WE MEASURED
THE “PLAN”?
• 2009-2010:
• Looked at two specific
objectives of the PLAN
related to CRC screening
and updated with data
• CRC screening for adults
ages 50 and older
• Percentage of
Kentuckians diagnosed at
an early stage
• 2010-2011:
• Priority Areas being
implemented
• Do you USE the CAP to
select strategies to
implement? Not really…
MEASURING PLAN
• 2011-2012: Use of Cancer Action Plan
• Added to the Partner Tool Survey (at the end)
1) Use of the CAP as a reference: How often do you refer to
the Cancer Action Plan or CAP as a guide or reference to
plan and implement your organization’s work plan activities?
2) Choosing CAP Strategies: How often does your organization
choose strategies to implement because they are suggested
in the Cancer Action Plan?
• Response categories for these questions:
•
•
•
•
Not at all
A small amount
A fair amount
A great deal
PLAN-RESULTS
• Use of the CAP: How often do
you refer to the Cancer Action
Plan or CAP as a guide or
reference to plan and
implement your organization’s
work plan activities?
• Not at all: n=3 or 21%
• A small amount: n=7 or 50%
• A fair amount: n=2 or 14%
• A great deal: n=1 or 7%
• Missing: n=1 or 7%
• Choosing CAP Strategies: How
often does your organization
choose strategies to implement
because they are suggested in
the Cancer Action Plan?
• Not at all: n=4 or 29%
• A small amount: n=6 or 43%
• A fair amount: n=3 or 21%
• A great deal: n= 1 or 7%
FEEDBACK
• Why do you think organizations use the CAP
only “a small amount”?
• Why do you think organizations don’t use
the CAP to choose strategies?
• Is there a way to make evaluating the
“Plan” more useful to your organization?
EVALUATION COMMITTEE
• Would you like to work more closely on
evaluation efforts with the Kentucky Cancer
Consortium/Jessica?
• Most work is done in small groups
• Reviewing documents, offering feedback,
conference calls
THANK YOU
• Thank you to all who helped with the 20112012 evaluation plan!
• A BIG thank you to all the Consortium
members for their help with our evaluation
efforts!
• For more information, please contact
Jessica Jones at [email protected]