Evaluation of drug policy
2005-2009
Activity 26.3 of Drug Action Plan 2007-2009
Reitox Academy, June 2010
Object of evaluation
• Drug policy formulated in strategic documents:
– National Drug Strategy 2005 - 2009
– (second) Action plan 2007 - 2009
2
National Drug Strategy
Main goals
• Decrease the level of any drug use •To combat organized crime
prevalence and the potential risks
Specific goals:
I. Stabilize / decrease the no. of
problem drug users
II. Stop the increase of
experimental drug use
III. Stabilize / decrease drug
consumption
IV. Decrease the risks of drug use
V. Guarantee availability of drug
services
VI. Decrease the accessibility of
drugs
Technical-organizational goals:
VII. Effective financing
VIII. Coordination
IX. Information to general public
X. International cooperation
XI. Evaluation of activities
3
Action plan
7 drug areas:
• Prevention
• Treatment
• Harm reduction
• Law enforcement
• Coordination and
financing
• Information, research,
evaluation
• International cooperation
Action plan 2007-2009:
• 43 objectives
• 170 activities
4
Action plan
5
Aim of evaluation
To learn :
• The extend to which strategic goals where achieved
(How has the drug situation changed)
• What were the main achievements and failures of
drug policy in 5 years
6
Organization
• 05/2009 – 03/2010
• internal evaluation – NDC
secretariat
• 1,5 employee
• no budget
• Established the Board of
Drug Strategy Evaluation
(5 experts):
– methodological
guidance of the
evaluation process and
evaluation tools
– Supervision of final
outcome/findings and
final version of
evaluation report
7
Concept
Based on the (simplified) policy cycle model
8
Formulation
Q1: Have changes occurred within the formulation
process?
Have strategic documents changed compared to their
planed design?
Q2: To what extend is the Action plan and Strategy
cohesive/linkedup?
How many activities really lead to strategic goals?
Q3: What is the character/type of activities planed in
Action plan?
from the point of their formulation, potential to solve a
problem, significance for drug situation
9
Formulation
Q1: Have changes occurred within the formulation
process?
• Comparison of official (and internal) documents
(what was planed in 2004
what was approved in 2005)
• Semi-structure interview with the former head of
secretariat NDC
10
Formulation
• Original purpose / structure and content of action plan
has been affected/changed
Abandoned area Use of
legal drugs
Changed area Public
relation for Information,
research, evaluation
?
Alcohol and tobacco is not
integrated part of drug policy
– failure of drug policy
?
no link exists between
activities and one specific
strategic goal
11
Formulation
Q2: To what extend is the Action plan and Strategy
cohesive/linkedup?
Q3: What is the character/type of activities planed in
Action plan?
• Analysis of documents based on the idea of logframe
analysis
helps to link and clarify mutual relationship among goals,
objectives, outputs and activities
12
Formulation
• internal working group (5 people) assessed each activity
of AP 2007-2009 in 5 different criteria
- clarity in formulation
- factuality/concreteness in formulation
- context of problem solving
- significance for drug situation
- relevance to strategic goals
13
Incidention matrix – all activities
14
Incidention matrix – key activities
15
Implementation
Q1: How successful was the implementation of Action
Plan 2007-2009?
Q2: What factors influenced the process of
implementation (positively and negatively)?
Q3: What are the conditions for implementation of
drug policy on the regional level? Development in
the area of drug policy in the last 5 years.
16
Implementation
Q1: How successful was the implementation of Action
Plan?
• Top-down approach
Objectives set on the central level are assessed from
the point of – how these have been achieved, what
hindered their achievement, and what is the
consistency of outputs and outcomes with set
objectives.
17
Implementation
• Questionnaire survey
– 36 subject involved in drug policy implementation
• 16 directly responsible for activity implementation
• 6 collaborating subjects – professional
organizations (new)
• 14 regions (new)
• 14 question for each activity, eg.:
–
–
–
–
Process of activity implementation (how)
Problems in implementation/reasons for not implementing
Outcome of activity implemented
Submission of outputs to secretariat NDC
18
Implementation
• Working group of the NDC secretariat (5 people)
– Assessed the information submitted
– Compared submitted outputs
with objective/activities planed in Action plan
This way we decreased the formality in fulfillment of activities
(eg. in prevention declared implemented/fulfilled activities
decreased from cca 45 % down to 35 %)
19
Implementation
Q2: What factors influenced the process of
implementation (positively and negatively)?
• on-line anonymous questionnaire (35 respondents) –
state organizations, local government, professional
organizations
– Marked the level of influence of listed factors
– Open questions
• Comparison of SWOT analysis in 2004 and 2009
(opportunities and threads)
– expert groups for 7 drug areas (cca. 60 people)
20
Implementation
Q3: What are the conditions for implementation of
drug policy on the regional level? Development in
the last 5 years?
• On-line questionnaire among regional drug coordinators (14
regions)
– Extend to which goals of national drug strategy has been
introduced into regional strategic documents
– Cooperation with state organizations
– Position of regional drug coordinator/possibilities to influence
drug policy implementation
– Development of conditions for implementation of regional drug
policy (personal, political, financial, institutional)
21
Outcome evaluation
Q1: What is the level of achievement of specific
strategic goals? (focused on the change of drug
situation)
• Analysis of quantitative data/indicators of drug situation
(NFP)
22
Outcome evaluation
Q2: Development in each drug area in the last 5
years?
• Expert group for each drug area (7 groups), 60 experts
• SWOT analysis
• Assessing each element of SWOT 2004
– Is the strength still strong point or has it become week point,
have the week points been eliminated, have we used identified
opportunities or have the threads occurred or not…
– Identify new elements of SWOT for 2009
23
Outcome evaluation
Q3: What is the level of achievement of technicalorganizational strategic goals?
• Combination of results and findings from expert working
groups and results from evaluation of implementation
24
Findings
25
Findings
The character and potential of planned activities (defined
in Action plan 2007-2009) and also the level of
successful implementation corresponds with the success
in achieving strategic goals and areas
26
eg. - prevention
Prevention in Action plan 2007-2009
• No activity leaded directly to specific
• High level of
strategic goals
experimental use –
• High % (70) of activities focused on
cannabis, alcohol …
organizational-coordination frame –
only 30 % of activities were focused on
solving a problem
• The consumption of
illegal drugs increased, of • Fulfilled/implemented 58 % of activities
legal drug stabilized on a
in 2005-2006, and 35 % in 2007-2009
high level
• Implemented 1 activity out of 6
characterized as those with higher
potential of changing a drug situation
• No improvement was noticed/declared
in this area; what more number of
strong points from 2004 were
weakened (SWOT)
27
eg. – harm reduction
• Infection diseases and
other health
consequences on a low
level
• Stabilization of problem
drug users
• Relatively stable
network of drug services
Harm-reduction in Action plan 20072009
• Had the highest no. of activities
leading directly to achieving most
of strategic goals
• Highest % (65) activities, which
were specifically focused on
problem solving,
• Fulfilled/implemented 78 % of
activities 2005-2006 (most from
all 4 pillars), 41 % in 2007-2009
• Implemented 8 key activities
(from 16) important for strategic
goals achievement
28
Thank you
[email protected]
www.vlada.cz
www.drogy-info.cz
29
Findings
• According to how activities were formulated in action plan we
can say that:
• Drug policy 2007-2009 was mainly focused on the
coordination, legislative, organizational frame and on the
evaluation of interventions
• Area of information, research, evaluation – has most activities
of strategic implication – important for key decisions
• Area of harm-reduction – had the greatest potential of
changing drug situation
• Area of prevention – had the lowest potential of changing drug
situation
30
Recommendations- Challenges
• Better linking up of activities and strategic goals
• Limit the no. of objectives, clearly state the priorities
• Prevention – its scope/intention, the way of defining
activities and support of their implementation
• Dealing with alcohol and tobacco use –
coordination, availability of information, services
• Law enforcement especially concerning legal drugs
• Maintain the network of drug services and
necessary interventions
31
Findings - formulation
• Incidention matrix.
• Typical problem of vague and unclear definition of activities or
goals
• Activity unclearly formulated – variety of interpretation, it
complicated implementation and evaluation
• Positive correlation was find between clear formulation of
activity and the level of achievement (this does not apply for
concreteness of activity)
• The use (the role) of indicators for activities was not clear
• Overlapping of strategic goals - goal III. Stabilize / decrease
drug consumption … was seemed to be redundant
32
Přehled – specifické cíle
I. Stabilizovat / snížit počet
problémových uživatelů
stabilizace, O.K.
II. Zastavit nárůst
experimentálního užívání
mírný nárůst. stabilizace na
vysoké úrovni
III. Stabilizovat / snížit spotřebu
drog
stabilizace na vysoké úrovni
(legální drogy), nárůst (nelegální
drogy)
IV. Snížit rizika užívání drog
stabilizace, O.K.
V. Dostupnosti služeb léčby a
resocializace
Stabilizace (kromě legálních), ale
v ohrožení
VI. Snížit dostupnost drog
vysoká dostupnost
33
Přehled – technicko-org. cíle
V II. Efektivní financování
částečně
V III. Koordinace
Částečně (vedlo k němu nejvíc
aktivit)
IX. Informování veřejnosti
nehodnotitelné, oblast -
X. Mezinárodní spolupráce
O.K.
XI. Evaluace aktivit
nehodnocené
34
Incidention matrix – all activities
35
Výsledky - implementace
• Akční plán 2005-2006 – splněno 73 % aktivit
• Akční plán 2007-2009 – splněno 52 % aktivit
36
Výsledky - implementace
Pozitivní faktory:
• Odborní potenciál
expertů
• Dostupnost relevantních
informací
• Institucionální
zabezpečení
protidrogové politiky
• Úroveň koordinace a
vzájemné spolupráce –
pozitivní i negativní
faktor implementace
Negativní faktory:
• Rozdílné výchozí přístupy
resortů k protidrogové
politice / prohlubování
resortizmu
• Formálnost ve stanovení i
plnění aktivit
• Nedostatek finančních zdrojů
• Snižující se zájmem o
problematiku ze strany
politiků
• Vysoká společenská
tolerance legálních drog
• Podceňování rizik
návykových látek
37