Evidence Based Prevention Assessment & Application
Purpose
EC-319 - Evidence Based Prevention
Assessment
The purpose of this Assessment is to provide a consistent process to determine which state
funded or contracted prevention programs are “evidenced based” as defined per SAMHSA
guidelines.
Deadline: July 15, 2016
Overview of why you need to complete this form:
The SAMSHA Center for Substance Abuse Prevention requires the implementation of the
Strategic Prevention Framework (SPF) to identify and select evidenced-based interventions. The
Regional Behavioral Health Authority (RBHA) is required to develop an evidenced-based process
and annually report the results.
To meet the definition for evidence-based practice, a program/strategy must meet one of the
following three definitions below:
Evidence-based practice definitions:
1.
Included on Federal Lists or Registries of evidence-based interventions; OR
2.
Reported (with positive effects) in peer-reviewed journals; OR
3.
Documented effectiveness based on all three guidelines for evidence:
Guideline 1:
The intervention is based on a theory of change that is documented in a clear logic or conceptual
model;
AND
Guideline 2:
The intervention is similar in content and structure to interventions that appear in registries and/or
the peer-reviewed literature;
AND
Guideline 3:
The intervention is supported by documentation that it has been effectively implemented in the
past, and multiple times, in a manner attentive to scientific standards of evidence and with results
that show a consistent pattern of credible and positive effects.
AND
Guideline 4:
Page 0 of 6
July 13, 2017
Evidence Based Prevention Assessment & Application
The intervention is reviewed and deemed appropriate by a panel of informed prevention experts
that includes: well-qualified prevention researchers who are experienced in evaluating
prevention interventions similar to those under review; local prevention practitioners; and key
community leaders as appropriate, e.g., officials from law enforcement and education sectors or
elders within indigenous cultures.
For more information on ‘evidence based practices’ please refer to “Identifying and Selecting
Evidence-Based Interventions - Guidance Document for Strategic Prevention Framework State
Incentive Grant Program”, Substance Abuse and Mental Health Services Administration Center
for Substance Abuse Prevention, https://store.samhsa.gov/shin/content/SMA09-4205/SMA094205.pdf
Deadline for submission: applicants will be required to submit their application and approved logic
model no later than July 15, 2016.
Evidence-Based Assessment Application
Please note: Applications must be no longer than 4 pages. Answers to the questions should be
brief. The application may include one attached journal article and attached evaluation instrument
used.
SECTION I
1. Program contact information
Program Name:
Program Director:
Address:
County:
Phone:
Fax:
E-mail:
* If a program/strategy has multiple sites, please attach a list of all sites including counties.
2. Please submit an e-version no later than JULY 15, 2016 to:
[email protected] labeled RF-1018 in the subject line and a copy to Linda
Weinberg at: ([email protected]).
Page 1 of 6
July 13, 2017
Evidence Based Prevention Assessment & Application
3. Program/strategy description: include description of a) strategies and b) the involvement
of your target population.
a)
b)
4. Has the program/strategy been adapted to fit your target population? If yes, please
describe the theory of why this adaptation should work.
5. Briefly describe how the program/strategy takes into consideration culture and tradition.
6. Did you involve the local community in the planning of your program/ strategies?
Please circle: YES/NO
If you answered yes to question #6, please circle the tool used to assess your community’s
needs / concerns (see glossary for terms).
a) Key informant interviews
b) Collected epidemiological data
c) Survey
d) Community planning
e) Community forum/ focus groups
7. Approximate number of participants served annually:
8. List the most relevant goals/objectives for your program/strategy:
9. Describe most recent evaluation outcomes (Please indicate year):
Page 2 of 6
July 13, 2017
Evidence Based Prevention Assessment & Application
10. List any curricula used in program/strategy implementation, including copyright date and
web site of the curricula:
SECTION II
According to SAMHSA’s Guidance document for the Strategic Prevention Framework,
‘Evidence-based’ is established on either one of the following:
1. Inclusion in a Federal List of Registry of evidence based interventions;
Main curriculum name:
Name of the federal list or registry:
Program curriculum web site address/ copyright date/ published date:
Please circle one: Model, effective, promising
If you answered question #1 stop here, if not continue to question #2.
2. Being reported (with positive effects) in peer-reviewed journal;
a) Reference for the peer reviewed journal:
b) Name of the journal article attached (copy):
If you answered question #2 stop here, if not continue to question #3.
3. Documentation of effectiveness based on the guidelines listed below:
a) What is the theory or theoretical perspective that informs this prevention program
through logical analysis and/or past research?
b) How is the program similar in theory, content, and structure to interventions that are
considered evidence based?
c) What research has validated this theory?
d) Describe the evidence which indicates this intervention is effective in preventing
substance abuse (i.e. previous evaluation, data collected, and research articles on
similar interventions).
Page 3 of 6
July 13, 2017
Evidence Based Prevention Assessment & Application
SECTION III
1. Please attach evaluation instrument(s) used.
2. Please check model(s) of evaluation used (see glossary):
Pre / post (3)
Post / retro (4)
Observation (5)
3. How many participants were surveyed to measure the program / strategy outcomes?
4. Describe how the data collection method accounts for missing data:
5. Explain how the study adequately rules out competing explanations for the findings:
6. Describe how the study design demonstrates effectiveness of program/ strategy
addressing collected data:
7. How many times has your agency implemented/replicated the program/strategy?
8. Has your agency adapted the program/strategy at different sites? If yes, please describe:
9. Describe how the program/strategy theory is linked to its expected outcome:
10. Did the program/strategy work? Why or why not?
Page 4 of 6
July 13, 2017
Evidence Based Prevention Assessment & Application
SECTION IV
According to SAMHSA’s Guidance, documented effectiveness must be supported by other
sources of information and the consensus judgment of informed experts.
If you answered “No” stop here, if not continue to question #2.
1. Have you have been unable to locate the intervention on a Federal list of Registry or
Peer review journal?
2. SAMHSA allows the intervention to be reviewed and deemed appropriate by a panel of
informed prevention experts (please read qualifications). Has your organization obtained
a panel of experts to review the intervention?
If you answered No stop here, if not continue to question #3.
3. Panel members must be familiar with interventions similar to those under review. List the
names, afflication and title of the member in the expert panel to include years of
prevention service.
4. Provide a detail review on the supposition: Does the intervention according to the panel
provide the list below, please provide details
a)
Evidence of positive measurable outcomes. Please attach evaluation instrument (s).
b) Evidence that the intervention is suitable for the community in which it will be
implemented.
c) Implementation process to ensure fidelity :
5. Has the intervention been utlized in other counities similar to the organizations service
area? If yes, please provide details to include outcomes.
Page 5 of 6
July 13, 2017
Evidence Based Prevention Assessment & Application
GLOSSARY
Adaptation - Modification made to a chosen program (e.g., qualitative and/or quantitative
changes to components); changes in audience, setting, and/or intensity of program delivery.
Research indicates that adaptations are more effective when (a) underlying program theory is
understood; (b) core program components have been identified; and (c) both the community and
the needs of the population of interest have been carefully defined.
Culture - The shared values, beliefs, views, traditions, norms, customs, arts, folklore, history and
institutions of a group of people.
Documented Effectiveness - Defined under the SPF SIG Program by guidelines for evidence to
demonstrate intervention effectiveness. These guidelines include grounding in solid theory, a
positive empirical track record, and the consensus judgment of informed experts and community
prevention leaders.
Evidence-Based Interventions - Interventions based on a strong theory or conceptual
framework that comprises activities grounded in that theory or framework and that produce
empirically verifiable positive outcomes when well implemented.
Fidelity - In operational terms, the rigor with which a program adheres to the developer’s model.
Fidelity/Adaptation Balance - A dynamic process that addresses both the need for fidelity to the
original program model and the demonstrable need for local adaptation.
Focus Groups - A group selected for its relevance to an evaluation that is engaged by a trained
facilitator in a series of discussions designed for sharing insights, ideas, and observations on a
topic of concern
Key Informant Interviews - Obtaining information from a community resident who is in a position
to know the community as a whole, or the particular portion you are interested in
Pre-Post Test - Evaluation instruments that assess change by comparing the baseline
measurements taken before a program begins to measurements taken after a program has
ended.
Program - A set of prevention strategies, which address a common set of goals and objectives
for a common target audience in one county (DBHS Framework).
Post-Retro Test - Evaluation instruments that assess change by measuring the change in a
participant’s knowledge, skills and attitudes after a program has ended.
Strategies/Interventions - Interventions encompass programs, practices, policies, and
strategies that affect individuals, groups of individuals, or entire communities.
Page 6 of 6
July 13, 2017