Evaluation Update
PSSRU
Personal Social Services Research Unit
at the University of Manchester
1
Introduction
1.
2.
3.
4.
The task set
Our approach
Progress
Next steps
PSSRU
Personal Social Services Research Unit
at the University of Manchester
2
The task set by DH
 Map the different approaches to developing CAF
 Provide evidence about the quality of outcomes for
service users
 Provide evidence about workforce experience
 Provide evidence on the cost, effectiveness and costeffectiveness
 Provide evidence about the usability of the CAF IT
systems for practitioners and service users
 Disseminate and share widely the learning
PSSRU
Personal Social Services Research Unit
at the University of Manchester
Identify impact of Improving information communication
4
Key issues
Issues
 Providing evidence from each
site that relates to the study
objectives set in the brief while
being sensitive to the different
approaches being adopted in
each area
 Developing a Routine Data Set
that is relevant to each site and
which can be enhanced by
additional local material that is
specific to particular sites
 Recognition of the different
target groups under
consideration in each site
Response
 Overarching evaluation
framework – covering both
a) process of development and
b) service delivery and outcomes
 Multiple case study design, not
a single approach
 … informed and agreed by
discussion with each site
 Ultimately signed off by the
steering group and National
Research Ethics Committee
PSSRU
Personal Social Services Research Unit
at the University of Manchester
5
Two broad strands
 process and implementation – the process of setting-up
and executing the programme and the subsequent
cultural change that occurs as a result
 Led by SQW
 Intended to be fieldwork at regular intervals
 service delivery and outcomes – the resultant service
user, carer, and workforce engagement and subsequent
outcomes that are experienced as a result of
participating in the programme
 Led by PSSRU, with support from GfK
 Tailored approaches to reflect different user groups and timings
PSSRU
Personal Social Services Research Unit
at the University of Manchester
6
Key tasks undertaken to date
 Introductory visits to sites
 Refinement of research framework and design of
research tools
 Ethical approval and research governance arrangements
PSSRU
Personal Social Services Research Unit
at the University of Manchester
7
From site visits
 The wide range of activities being pursued
 Some areas much clearer in their approaches than others
 Conceptualising what they are doing that we can evaluate
 Focussing on where IT is used (not paper) and users/ carers benefit
 … and on the scale of activity
 But generally it appears lower than we anticipated
 Especially in the time period
 The challenge in defining what a ‘go live’ date actually was
 Often incremental development (paper, local IT, national IT)
 New systems rolled out to staff before users/carers
 Go live dates subject to change
PSSRU
Personal Social Services Research Unit
at the University of Manchester
8
Research tools
 Process and implementation
 Individual site case study
report outline
 Topic guide for project leader
consultations
 Topic guide for staff focus
groups
 Topic guide for user
representative consultation
 Topic guide for consultation
with IT professionals
 Implementation costs – staff
and money
 Service delivery and outcomes
 Service users routine data set
 Staff well being and time use
 Service user assessment
satisfaction survey
 Service user and carer
outcomes
 Carer outcomes
 Engagement with new
technology
 Assessment document
analysis
 Assessment pathway
PSSRU
Personal Social Services Research Unit
at the University of Manchester
Several Hoops to pass through
10
Ethics and research governance
 National application submitted in September
 Attended committee in October
 Formal clearance November
 Now working through local arrangements
 Require both Local Authority and NHS clearance
 As yet not fully formally agreed in any site
 We would really welcome your help in speeding things along
PSSRU
Personal Social Services Research Unit
at the University of Manchester
11
Local discussion – working towards
 Identify settings where CAF influences assessment and
care pathway for study
 Identify agreed local NHS contact/collaborator for
research (required for Res Gov)
 Research Governance in place LA and NHS (within
agreed National REC parameters)
 Agreed routine data set collection process
 Local plan for data collection from users/carers
experiencing CAF processes and similar people not
doing so
 Local agreement for access to a range of staff for
interviews in NHS and LA
PSSRU
Personal Social Services Research Unit
at the University of Manchester
12
So that we can…
 Begin collecting user/carer data as soon as possible
 Especially as we know in some cases users are beginning to
experience the new approaches
 Map out the delivery experience and lessons
 Describe and evidence the legacy of the CAF
programme
 Ensure the lessons can be shared widely with interested
parties
PSSRU
Personal Social Services Research Unit
at the University of Manchester
One CAF experience
14
Contact
Graham Thom
Associate Director
SQW Consulting
t. 0131 225 4007
e. [email protected]
w. www.sqw.co.uk
Prof David Challis
Director
PSSRU
University of Manchester
t. 0161 275 5250
e. [email protected]
Claire Ivins
Divisional Director
GfK NOP Social Research
t. 020 7890 9783
e. [email protected]
PSSRU
Personal Social Services Research Unit
at the University of Manchester