Measuring the Impact of
Environmental
Strategies in Prevention
Jeremy T. Goldbach, PhD, LMSW
C.A.R.E. Consulting Group
This product is supported by Florida Department of Children and Families Office
of Substance Abuse and Mental Health funding.
LEARNING OBJECTIVES
After participation in this workshop, participants will be able to :
1. Explain the advantages of environmental strategies
2. Use data to identify environmental strategies that may be a good fit in their
own communities
3. Measure the impact of environmental strategies in prevention
WHAT’S THE POINT?!
SO WHY ENVIRONMENTAL
STRATEGIES?
•
A significant portion of youth ages 12 to 17 in the United
States have used alcohol and other drugs.
•
Rates of binge drinking (five or more drinks) in underage
youth are a concern, with large increases between 7th grade
(1.6% of 12 and 13 year olds) and high school graduation
(34.7% of 18 to 20 year olds).
•
Economically, the costs of alcohol use and its related
consequences in U.S. communities add up to more than
$235 billion annually (Rehm, et al, 2009).
(NSDUH, 2009; Rehm et al. 2009)
WHY ENVIRONMENTAL
STRATEGIES CONTINUED
• Addressing alcohol abuse through only one strategy (such
as a single direct service prevention intervention only) is
less effective than taking a broader approach that includes
the larger community.
• multi-component strategies promote a systemic approach
to solving issues recognizing that individuals operate within,
and are affected by, an “environment”.
(Hawley, 1950; Rousseau & House, 1994; Stokols, 1996).
BREAKING IT DOWN
What are Environmental Strategies?
Making flowers bloom?
Ending Wars?
Getting rid of pollution?
NO…well, not for us anyway!
WHAT ARE ENVIRONMENTAL
STRATEGIES?
Environmental
strategies do not
have to be
complex
ENVIRONMENTAL
STRATEGIES
Environmental strategies focus on the following
areas:
• Access & availability
• Policy/laws and their enforcement
• Community norms/media messages
ACCESS & AVAILABILITY
Are the amount of time, energy, and money that must be
expended to obtain a substance (alcohol, tobacco, drugs).
The more resources it takes to get the substance
(harder), the less the availability (access).
POLICIES/REGULATIONS/LAWS
They control availability, support norms, and specify
sanctions for violations.
Norms
The rightness or wrongness, the
acceptability or unacceptability,
and/or deviance of specific
behaviors for a specific group of
individuals.
WHY ENVIRONMENTAL STRATEGIES?
• Broad reach – universal population
• Substantial effects
• Immediate and enduring effects
• Cost effectiveness
• Ease of maintenance
SHARE YOUR EXPERIENCE
• Discuss successes you’ve had in using
environmental strategies
• Discuss barriers/struggles in using environmental
strategies
SELECTING THE RIGHT
ENVIRONMENTAL STRATEGY
• How do you know which type of environmental
strategy is best for your state or community?
• What do you need to know to decide which is
best?
• Can you do just one or should you do more than
just one?
THE SPF FRAMEWORK
Assessment
Evaluation
Monitor, evaluate,
sustain, and improve
or replace those that
fail
Profile population
needs, resources,
and readiness to
address needs and
gaps
Sustainability &
Cultural Competence
Implement
evidence-based
prevention programs
and activities
Implementation
Capacity
Mobilize and/or build
capacity to address
needs
Develop a
Comprehensive
Strategic Plan
Planning
WHY USE THE SPF?
• Effective capacity building process
• Required by SAMHSA and used by many others
• Gives a roadmap based on research, data, and evidence
• Decisions can be evaluated through data-driven
measurement
STEP 1: NEEDS ASSESSMENTS
• Why do we assess?
• To guide the prevention effort
• To provide information about
the target population and their
corresponding substance abuse
problem
KEY ELEMENTS
Substance
Abuse and
Related
Consequences
Risk and
Protective
Factors
EvidenceBased
Prevention
Strategies
Types of Risk/Protective Factors: “Why?”
• Retail access
• Social access
• Pricing
• Norms around Alcohol Issues
• Enforcement
• Promotion
RISK FACTORS FOR BINGE DRINKING
Substance Abuse
& Related
Consequences
Risk/Protective
Factor
Social Access/
Availability
Binge Drinking
Retail Access
Strategies
Risk Factor Assessment: “But Why Here?”
• When does this happen?
• Where does this happen?
• How does this happen?
• Who is affected?
• Who allows this to happen?
BUT WHY HERE?
Substance Abuse
& Related
Consequences
Binge Drinking
Risk/Protective
Factor
Social Access/
Availability:
Parents in
community
host underage
drinking parties
Strategies
STEP 2: CAPACITY BUILDING
• To find out what we already
have and what we need
• Different types of capacity
(fiscal, physical, staff, etc)
- So we don’t duplicate efforts
or pick much-needed, but
unachievable strategies
STEP 3: PLANNING
• What kind of planning is done?
• Identifying the most powerful factors
(intervening variables) contributing to the
target behavior
• Investigating appropriate evidence-based
strategies to match population’s needs
• Planning expected outcomes (both short and
long-term)
TYPES OF ENVIRONMENTAL
STRATEGIES
• Policy
• Enforcement
• Communication
SELECTION OF EVIDENCEBASED STRATEGIES
Relevant?
Appropriate?
Effective?
Demonstrates
“Conceptual Fit”
Demonstrates
“Practical Fit”
Demonstrates
“Evidence of
Effectiveness”
Best Fit
Prevention
Intervention
Evidence-Based
Environmental Strategies
Substance Abuse
& Related
Consequences
Risk/Protective
Factor
Strategies
Enforcement
Social Host
Ordinances
Binge Drinking
Social Access
Communication
Social Marketing
“Parents Who
Host Lose the Most”
KEY ELEMENTS FOR SUCCESS
OF ENVIRONMENTAL
STRATEGIES
• Use data for decision-making
• Fit: relevant, appropriate and effective
• Good mix of strategies
• Monitor implementation and outcomes
STEP 4: IMPLEMENTING
• What do we do?
• Develop an action plan for implementing the
strategic plan
• Take fidelity of implementation into account
• Write detailed evaluation plan that includes
process and outcome measurements
STEP 5: EVALUATING
• What do we evaluate?
• The effectiveness of the selected evidencebased program, policy, or practice (EBP)
• How?
• By collecting required outcome data
• Recommendations made for quality
improvement
WHAT IS AN EVALUATION?
“The most serious mistakes are not being made as a result of the wrong answers…
The truly dangerous thing is asking the wrong question.” – Peter Drucker
• Asks questions to understand Who, What, When, Where,
Why, and How things are being done
• Identifies strengths and barriers to the coalition process
• Identifies strengths in the implementation of your strategies
• Identifies gaps and areas for improvement
• Tells you about your PROCESS as well as your
OUTCOMES
Evaluation is Important Because It…
(I promise, there are real answers!)
“One of the great mistakes is to judge policies and programs by their intentions
rather than their results” – Milton Friedman
• Helps to improve program planning
• Monitors the effectiveness of strategies
• Advances knowledge in the prevention sciences
• Serves as a mechanism to build capacity by involving community in
evaluation
• Can encourage the community to publicize success stories from the
prevention effort
PROCESS EVALUATION
• What is process evaluation?
EXAMPLES:
• What is it like to be a new member of the coalition?
• To what extent is our strategy being implemented as planned?
• What obstacles were encountered during implementation?
OUTCOME EVALUATION
Over the duration of the program, to what extent has:
• Coalition membership increased?
• Community-wide prevention awareness activities changed adult
norms about substance use?
• Youth alcohol use decreased?
HOW DO WE EXPLORE THESE
QUESTIONS EFFECTIVELY?
• Each activity that the coalition does should have a
purpose
• Each activity should be measureable and link
logically to an outcome of interest
THE LOGIC MODEL
GOALS
30 DAY
ALCOHOL
USE IN 8TH
GRADERS
IS AT 20%
AS
MEASURE
D BY THE
FLORIDA
SCHOOL
SURVEY
INTERVENING
VARIABLES
RETAIL
ACCESS
STRATEGIES
RETAILER
BEVERAGE
SERVICE
(RBS)
TRAINING
ACTIVITIES
SHORT-TERM
OUTCOMES
INTERMEDIAT LONG-TERM
E OUTCOMES OUTCOMES
IDENTIFY
AREAS OF
HIGHEST
OFFENSE
RETAILERS
SELL LESS
FREQUENTLY
TO MINORS
COMPLETE RBS
TRAINING OF
TRAINERS
COMPLIANCE
CHECKS FIND
FEWER NONCOMPLIANT
RETAILERS
RECRUIT
COMMUNITY
PARTNERS
CONDUCT
RBS TRAINING
WITH 50
RETAILERS PER
MONTH
30-DAY
ALCOHOL
USE GOES
DOWN IN 8TH
GRADERS
FROM 20% TO
15% FROM
2012-2015
WHAT LESSONS DID WE
LEARN?
“The only man who behaves sensibly is my tailor; he
takes my measurements anew every time he sees me,
while all the rest go on with their old measurements
and expect me to fit them!”
- George Bernard Shaw
SUSTAINABILITY AND
CULTURAL COMPETENCE
• What are we sustaining, the prevention programs or the
prevention effort?
• Why is cultural competence important?
• How do we create prevention systems with culturally
responsive behaviors, attitudes, and practices?
QUESTIONS AND COMMENTS
RESOURCES
• SAMHSA Center for Substance Abuse Prevention:
• www.samhsa.gov
• http://www.samhsa.gov/about-us/who-we-are/officescenters/csap
• National Institute on Drug Abuse:
• https://www.drugabuse.gov/related-topics/prevention
• C.A.R.E. Consulting, Online Training Resources:
• http://www.careconsultinggroup.com/training/