What makes a difference?
Research on the impact of good
practice and how services can achieve
excellence
Donald Forrester
Professor of Child and Family Social Work
CASCADE Centre for Children’s Social Care
Cardiff University
[email protected]
S
Overview
S What IS good social work practice?
S What difference does it make?
S And how can we create great practice –
the sort of practice that changes lives?
Child and family social work
Context
Practice
Outcomes
Overview of Studies
• RCT of
training in
MI in 1 LA
• 134
families T1
• 104 T2
• Systemic
Units
• 5 LAs
• 75 families
• Move to MSW
• Impact of feedback for
individuals and service
• 2/300 families T1
• Complex whole system
change
• T1/T2 Before AND
After
• 75 in each
Common data collection for c.500 families
Referral
and
Assessment
Children
in Need
team
S Follow up with families
S Observation (2nd visit : T1)
S Audio recording of meeting
S Interview (T1 and 20wks T2)
2nd or
3rd
SW
visit
First
SW
visit
SW asks
parent
about
observation
Family
enters
study
S Outcome measures: to be discussed! Including engagement,
achievement of goals and standardised instruments
S Primary focus today the 134/102 from the RCT… with some info from
others
S In addition over 250 qualitative interviews about change in LAs
What IS good social work
practice?
What IS good social work
practice?
S Hugely neglected in social work research
S But perhaps also in leadership and practice? Is there a clear
vision of great practice in your organisation? Can you tell
the person next to you what it is?
What IS good social work
practice?
S We need to spend far more time and effort exploring
what great practice IS and…
S Whether it actually makes a difference...
S And then reviewing what our vision of great practice is
We started with Motivational
Interviewing (MI)
S MI is just a handy (evidence and value based) description of
key elements of good practice
S This allows exploration of what good practice is and issues
around behaviour change in social workers and in those we
work with
S MI is a directive, client-centred approach to communication
that attempts to elicit intrinsic motivation
Coding for Motivational
Interviewing (MITI)
S MI Treatment Integrity (MITI) rates for:
S Collaboration
S Autonomy
S Evocation
S Empathy
S The first 3 are averaged for a MITI score
S But what is missed by this – what else is good practice?
Additional skill descriptions
1. Purposefulness
2. Clarity about concerns
3. Focus on child
Our Practice Skill Coding
1.
Collaboration
2.
Autonomy
3.
Evocation
4.
Empathy
5.
Purposefulness
6.
Clarity about concerns
7.
Focus on child
Summary of coding
1. Very poor practice
2. Low levels of skill
3. Adequate level of skill – ANCHOR POINT
4. Good level of skill
5. Excellence
What were skills of workers?
5
4.5
4
What were skills of workers?
(n=134)
3.5
3
2.5
2
2.57
1.5
1
2.69
2.78
2.67
2.64
3.01
2.88
2.88
5
4.5
4
What were skills of workers?
(n=134)
3.5
3
2.5
2
2.57
1.5
1
2.69
2.78
2.67
2.64
3.01
2.88
2.88
How were skills related to
each other?
S Guesses?
S Empathy vs Focus on child?
S Collaboration vs Clarity
about Concerns?
Correlations between coded skills in practice
How were skills related?
Autonomy
Evocation
.813**
.519**
.733**
.307**
.471**
.300**
Sig.
<.001
<.001
<.001
.002
<.001
.002
1
.665**
.697**
.308**
.453**
.381**
<.001
<.001
<.001
<.001
<.001
1
.433**
.138
.282**
.285**
<.001
.122
.001
.001
1
.372**
.485**
.341**
<.001
<.001
<.001
1
.434**
.238**
<.001
.007
1
.438**
Collaboration
Sig.
Evocation
.813**
<.001
Autonomy
Sig.
Empathy
Sig.
Purposefulness
Sig.
Clarity of
concerns
Sig.
Child focus
Sig.
Empathy
Child Focus
Collaborat
ion
(N=127) for all but
evocation (n=101)
Purposefu
lness
Clarity of
concerns
.519**
.665**
<.001
<.001
.733**
.697**
.433**
<.001
<.001
<.001
.307**
.308**
.138
.372**
.002
<.001
.122
<.001
.471**
.453**
.282**
.485**
.434**
<.001
<.001
.001
<.001
<.001
.300**
.381**
.285**
.341**
.238**
.438**
.002
<.001
.001
<.001
.007
<.001
<.001
Relationship between skills
S The “care” elements were strongly related
S The elements were ALL positively related to one
another
S … so overall SKILL is more important than care vs
control
S Put another way: good workers combine care and
control…
Does worker skill predict
parental engagement?
S Used Working Alliance Inventory (WAI):
S Involves three elements:
S
S
S
Quality of relationship
Agreement on goals
Whether tasks are carried out
S Used widely in counselling – linked consistently to outcomes
S So do skills predict engagement at T1 – and at 20 weeks?
S And if so – which ones?
Engagement
(WAI) T2
Engagement
(WAI) T1
Correlation
Does
worker skill predict
Sig. (2-tailed)
parental
engagement?
N
MI Spirit total
Empathy
Correlation
Sig. (2-tailed)
N
Purposefulness
Correlation
Sig. (2-tailed)
N
Clarity of concerns Correlation
Sig. (2-tailed)
N
Child focus
Correlation
Sig. (2-tailed)
N
Engagement
(WAI) T2
.367**
Engagement
(WAI) T1
.344**
Correlation
.357**
.303**
Sig. (2-tailed)
.002
.003
71
92
Correlation
.183
.174
Sig. (2-tailed)
.129
.099
70
91
.263*
.225*
.028
.032
70
91
Correlation
.197
.022
Sig. (2-tailed)
.102
.839
70
91
Correlation
Does
worker skill predict
Sig. (2-tailed)
.007
.004
parental
engagement?
N
52
70
MI Spirit total
Empathy
N
Purposefulness
N
Clarity of concerns Correlation
Sig. (2-tailed)
N
Child focus
N
How did skills relate to
outcomes at 20 weeks?
S What ARE “outcomes” in child and family work?
S What measures might we use?
S What measures do YOU use?
Outcomes Measures: Tricky,
Very Tricky
S The sheer range of issues – from unborn babies to 17 year olds etc
S Can we impose outcomes or should they be negotiated?
S Are the measures sensitive to change?
S Is change even necessarily the goal? What about maintenance or
quality in its own right?
Outcomes Measures: What We
Use
S Goal Attainment Scaling (GAS)
S Life Rating (10-points)
S Parent rating of worker and service
S Standardised instruments and z-scores:
S Parental anxiety/stress (GHQ)
S Child’s emotions and behaviour (SDQ)
S Drink and drugs (MAP)
S Social isolation (SSQ)
S Family functioning (SCORE-15) (includes arguments)
S DV (behavioural measures)
Overall how
satisfied are
you with your
SW (T2)
.825**
Rate life now
.035
Parents score
(+3 to -3)
-.005
.000
.729
.963
97
98
95
.292**
-.072
-.056
.003
.475
.583
N
100
102
99
Pearson
Correlation
Sig. (2-tailed)
.188
.171
.112
.174
.216
.426
54
54
53
.247*
.135
.152
.034
.248
.197
74
75
74
.172
.227
.106
.147
.051
.371
73
74
73
Engagement T2 Pearson
(WAI)
Correlation
Sig. (2-tailed)
N
Engagement T1 Pearson
(WAI)
Correlation
Sig. (2-tailed)
MI Spirit total
N
Empathy
Pearson
Correlation
Sig. (2-tailed)
N
Clarity of
concerns
Pearson
Correlation
Sig. (2-tailed)
N
How to make sense of
these findings?
How did skills relate to outcomes?
S These relationships are relatively weak. However, they
consistently point to a positive correlation between skill
and outcomes – as strong as most “evidence based”
methods
S They DO raise some issues about engagement… does it
matter?
S But the link between skills and outcomes is mediated by a
key variable… Which is obvious (when you think about
it…)
8 or more visits only
MI Spirit total
Pearson
Correlation
Sig. (2-tailed)
N
Empathy
Pearson
Correlation
Sig. (2-tailed)
N
Purposefulness
Pearson
Correlation
Sig. (2-tailed)
N
Clarity of
concerns
Pearson
Correlation
Sig. (2-tailed)
N
Rate life now
.216
Parents score
(+3 to -3)
.129
.421
.648
16
15
.360
.315
.091
.153
23
22
.674**
.155
.000
.491
23
22
.368
-.059
.084
.793
23
22
Some research issues with
implications for the findings
S The quality of practice was tested in ONE interview
S There will be variation between this and the quality across
all the work with a family – which will reduce the statistical
relationships
S Direct work is only one part of good social work – work
outside and assessment/ decision-making also crucial
Impact of good practice
S We need to think carefully about the impact of practice
on outcomes generally – relatively small overall…
S Reasons for limited impact on outcomes:
S Most families did not have big problems…
S LIFE – other things happen in families
S People are active agents – trying to sort out problems
S How big a difference do specific interventions make?
S Nonetheless – encouraging signs that where serious
issues then certain skills make a big difference
If P = R then
O
=
Q
I
S
If Problem = Real then
O
utcomes
=
Q
uality of
practice
S
I
intensity
If Problem = Real then
O
utcomes
=
Q
I
uality of
practice
S
tuff
ntensity
Implications
S What have learnt about good practice?
S What have we learnt about impact on families?
S What have we learnt about how to create good
practice?
Work in Islington…
1. Practice without training (c.60)
2. Intensive 4 days plus 8 weeks of coaching (c.70)
3. Embedded observation and feedback (c.100)
5
What impact on practice?
4.5
4
3.37
3.5
2.92
3
2.5
2.49
2.91
3.02
3.13
3.1
3.17
MI Spirit
Empathy
Purposefulness
2.37
2
1.5
1
Pre-Training
Post Training
Coaching and
feedback
Are there differences
between LAs?
Are there differences
between LAs?
And if there are – why?
5
Some comparisons…
4.5
4
3.75
3.5
3
3
2.9
2.76
2.6
2.9
2.83
2.79
2.73
2.6
2.59
2.5
2.39
2.35
2.28
1.97
2
1.5
1
LA1
LA2
LA3
LA4
LA5
MI Spirit Score
Empathy
Purposefulness
Developing a theory about how
to support excellent practice…
Developing a theory about how
to support excellent practice…
Key: A Vision of
Practice Excellence
Areas that create practice
Developing a theory about how
to support excellent practice…
Key: A Vision of
Practice Excellence
Areas that create practice
Developing a theory about how
to support excellent practice…
Key: A Vision of
Practice Excellence
Training
Supervision
Selection
Meetings
Etc…
Values
Areas that create practice
Developing a theory about how
to support excellent practice…
Key: A Vision of
Practice Excellence
Training
Supervision
Selection
Meetings
Etc…
Values
Effective Initial Assessment
Areas that create practice
Developing a theory about how
to support excellent practice…
Key: A Vision of
Practice Excellence
Training
Supervision
Selection
Meetings
Etc…
Values
Effective Initial Assessment
Quality of Practice
Areas that create practice
Developing a theory about how
to support excellent practice…
Key: A Vision of
Practice Excellence
Training
Supervision
Selection
Meetings
Etc…
Values
Effective Initial Assessment
Quality of Practice
Outcomes
Areas that create practice
Developing a theory about how
to support excellent practice…
Key: A Vision of
Practice Excellence
Training
Supervision
Selection
Meetings
Etc…
Values
Effective Initial Assessment
Quality of Practice
Outcomes
Areas that create practice
Developing a theory about how
to support excellent practice…
Key: A Vision of
Practice Excellence
Training
Supervision
Selection
Meetings
Etc…
Values
Effective Initial Assessment
Quality of Practice
Outcomes
Towards practice centred social
work
S Practice is not primarily produced by
individual workers being skilled or by
training
S It is produced by organisations that set
themselves to produce certain types of
practice
S Local authorities therefore need a vision
for the practice they want and then to
create the organisation that delivers it
Towards practice centred social
work
S I have very few answers.. But I think the
most important thing is to be asking the
right questions, such as:
S What IS great social work?
S What difference does it make?
S How can we make it happen?