Class 3
Jeff Driskell, MSW, PhD
Today’s Class
 Check-in/announcements
 Lecture/activities
 Evaluation
Program Lifecycle
Breaking it Down…
 Program Development- An ongoing, comprehensive
planning process used to establish programs supported by well thought
out and documented plan of action
 Program Implementation- The stage in the planning
process where you develop procedures for carrying out program.
 Program Evaluation- Designed to verify, document, quantify
activities and efforts. Process and outcome evaluation.
 Replication- Replicating evidenced-based programs.
Development
1.
2.
3.
4.
5.
Needs assessment
Clearly defined audience/target population
Identify behavioral theory
Identify behavioral domains
Identify behavioral/program goals and objectives
Culturally Competent Program Development
1. Program Development
1.
2.
3.
4.
5.
6.
7.
8.
9.
Assessing needs and assets
Defining the problem
Clearly defined audience/target population
Identification/Application of behavioral domains
(theoretical concepts)
Identifying goals and objectives
Planning program components
Using a program model/framework
Staffing/training
Setting
Implementation
 Select and implement behavioral interventions
 Variety of teaching methods
 Information personalized
 Realistic schedule
 Repetition of the message
 Evaluation
Example #1- Project Enhance
 Overall Goal- To reduce rates of HIV infection by 25%






among HIV negative and unknown status MSM by
2010 in Boston, MA.
Randomized control trial targeting HIV positive MSM
involved in health care (setting)
Secondary prevention (level of prevention)
IMB Model (primary theoretical framework)
Facilitated by Medical Social Worker (staffing)
Utilizes a manual treatment approach (delivery method)
Consists of 8 counseling sessions- over a 1 year period
of time (Dose)
 Change in sexual risk behavior (outcome evaluation)
Project Enhance
randomized to
intervention
Consent,
Assessment
Randomization &
Baseline PCM
PCM follow-up &
Intervention
Session 1
randomized
to PCM
PCM follow-up
PCM follow-up &
Intervention
Sessions 2,3 & 4
Month 3:
Booster, PCM &
Assessment
PCM
& Assessment
Month 6:
Booster, PCM &
Assessment
PCM
& Assessment
Month 9:
Booster, PCM &
Assessment
PCM
& Assessment
Booster, PCM &
Assessment
PCM
& Assessment
Month 12:
Culturally Competent Program Implementation
1. Program Implementation
1.
2.
3.
4.
5.
6.
7.
Involvement of community leaders, consumers
Staff recruitment and training
Responding to target population needs
Addressing potential language barriers
Retaining program staff
Participant recruitment
Monitoring of program
Why Conduct Evaluation?
Types of Evaluation
 Evaluation of Need
 Evaluation of Process
 Evaluation of Efficiency- cost/benefit
 Evaluation of Outcome
Evaluation Categories:
Matching Activity
 Evaluation of Need
Formative
Summative
 Evaluation of
Process
 Evaluation of
Efficiencycost/benefit
 Evaluation of
Outcome
Mapping it Out…
Milestone Program
 Target Population- HIV infected men
 Service- Prevention Case Management
 Evaluation type- Process
 Community Advisory Board (CAB)- consumer voice
and input
 Internal vs external evaluators- Conducted by
University of Washington
Experience with Evaluation
 Evaluation at past or currently place of
employment/field.
Activity 1: Weekly Comments
Based on the Olver (2009) article, answer the
following questions:
 What problem is being addressed?
 What type of evaluation?
 Describe the intervention
 Identify data collection strategies
 What are the implications for practice? Policy?
Research?
Practice Relevance: Application
 Scenario: A male client in his mid 30’s approaches
you as his therapist and informs you that he is
interested in participating in a workshop called the
Mankind Project. He is a bit ambivalent and asks what
you know about it. What might you do in supporting
this client? Think in terms of Evidenced based
practice and evaluation.
Mankind Project
What is a Logic Model?
 Based on the reading for this week… Rubin and Babbie
Logical Model Overview
 http://www.youtube.com/watch?v=OZ2oBVCDU9I
Framework for Evaluation
 Systems approach
 Inputs- What you invest
 Outputs- What you do and who you reach


Activities
Participation
 Outcomes/impact
Simplest Form
INPUTS
Program
investments
What
we
invest
OUTPUTS
Activities
What
we do
Participation
Who
we
reach
OUTCOMES
Short
Medium
Longterm
What are the results
Logical Model Example
 https://www.youtube.com/watch?v=KEV2Km23MFk
Fully Detailed Logic Model
Activity - Logical Model
 http://www.uwex.edu/ces/pdande/evaluation/evallogi
cmodelexamples.html
 Break it down- What elements are associated with each
stage of the logic model example?

Select the tobacco prevention for youth
Activity- Evaluation and EBP
 http://nrepp.samhsa.gov/AdvancedSearch.aspx
 Step 1


What is NREPP?
What does NREPP provide in relation to interventions?
 Step 2



Identify 1 program of interest using advanced search
Briefly describe the program
Identify 2 key findings based on targeted outcomes (key
findings)
Activity – Student Articles
 Identify the type of research, i.e.
 Outcome evaluation
 Process evaluation
 Needs assessment
 Questions What question(s) were the researchers trying to answer
with the evaluative research?
 What are the study or program goals/objectives?
 Where the goals/objectives achieved?
 What are the implications for practice? Policy?
Application: Logic Model
Weekly Comments
 Describe what is mean by a literature review.
 What is the purpose of a literature review?
 What strategies do Rudestam & Newton (2001) give for
organizing and conducting a literature review?
 What is meant by the concept mixed methods design?