[email protected]
Therapy Outcome Measures
• Background
• Context
• Purpose
• Use of the tool
2
Data to support quality
assurance Donabedian (1980)
Structure
Process
Staff grades, costs of
assessment tools,
quality of
accommodation
No. of interventions
provided; no of
patients seen
3
Outcome
Changes in patients
communication,
wellbeing
Outcomes
‘address the effects, not the process, of
particular interventions’
(Hesketh & Sage, 1999)
“ results or visible effects of interventions….
forms part of the quality cycle….. provides
information on the impact of interventions….
identifies the effectiveness of practices….”
(Enderby, John & Petheram, 2006)
How can we improve our therapy?
•
•
•
•
•
•
•
Reflection
Learning from research
Learning from others
Learning from experience
Data collection
Reflection
Reflection
5
Why has outcome measurement got to
the top of the agenda?
• Increasing demands on the health service
• Knowledge of variation in provision
• Financial constraints
NHS Atlas of variation February 2016
• NHS patients are suffering from "unwarranted
variation" in their care, which cannot be linked to
levels of illness or patient-preference, a joint
report from leading health bodies shows.
• “Our challenge now is to consider how we can
better understand and tackle the underlying
causes. This is not a straightforward task, but
exploring the data that lies behind these
variations will be an important starting point,”
Chief Medical Officer
National Director for Commissioning Development
emphasises the importance of good governance for
CCGs.
• ' Where CCGs
wish to make changes to their
commissioning support arrangements, it is
critical that the rationale behind these
decisions is transparent and properly
documented.’
Commissioning Guidance for Rehabilitation 2016
Good rehabilitation services will:
1.Optimise physical, mental and social wellbeing and have a close working
partnership with people to support their needs
2.Recognise people and those who are important to them, including carers, as
a critical part of the interdisciplinary team
3. Instil hope, support ambition and balance risk to maximise outcome and
independence
4.Use an individualised, goal-based approach, informed by evidence and best
practice which focuses on people’s role in society
5. Require early and ongoing assessment and identification of rehabilitation
needs to support timely planning and interventions to improve outcomes
and ensure seamless transition
9
Commissioning Guidance for Rehabilitation 2016
Good rehabilitation services will:
6.Support self-management through education and information to maintain
health and wellbeing to achieve maximum potential
7.Make use of a wide variety of new and established interventions to improve
outcomes e.g. exercise, technology, Cognitive Behavioural Therapy
8. Deliver efficient and effective rehabilitation using integrated
multi-agency pathways including, where appropriate, seven days a week
9.Have strong leadership and accountability at all levels – with effective
communication
10.Share good practice, collect data and contribute to the evidence base by
undertaking evaluation/audit/research
10
Commissioning Guidance for Rehabilitation 2016
Good rehabilitation services will:
6.Support self-management through education and information to maintain
health and wellbeing to achieve maximum potential
7.Make use of a wide variety of new and established interventions to improve
outcomes e.g. exercise, technology, Cognitive Behavioural Therapy
8. Deliver efficient and effective rehabilitation using integrated
multi-agency pathways including, where appropriate, seven days a week
9.Have strong leadership and accountability at all levels – with effective
communication
10.Share good practice, collect data and contribute to the evidence base by
undertaking evaluation/audit/research
11
Commissioning Guidance for Rehabilitation
When considering what outcome data to request
from providers, the following should be considered:
• ‘what outcome data is already collected locally
(by the team managers and clinicians)?
• what outcome measurement tools are
appropriate for the client group, health condition
and method of service delivery?
• will it enable benchmarking with other services?
• will it show how existing inequalities have been
reduced in terms of access to services,
experiences of services and outcomes achieved?’
12
How can outcome measurement help
you with your service?
•
•
•
•
•
Examine changes over time
Investigate particular issues e.g. intensity of therapy
Identifying areas of strength
Identifying areas of weakness
Communicate with the client, other professionals
and commissioners
13
14
Rehabilitation/Habilitation/Enablement
• The process of trying to help people who have
suffered some injury/disease or developmental
delay to maximise psychological well being,
functional ability and social integration (Wade,
1992)
• An often complex process which enables individuals
after impairment by illness, developmental delay or
injury to regain as far as possible control over their
own lives (King’s Fund, 1999)
15
What For?
• Impairment/disorder reduction
• Improved Function
• Psycho social gain
• Wellbeing
16
Measures of Performance
PROMs
PREMs
TOMs
17
Therapy treatment goals
• to identify and reduce the disorder/ dysfunction
• to improve or maintain the function and ability
• to assist to achieve potential or integration
• and to alleviate anxiety or frustration.
18
Choosing an Outcome Measure
• Relevance
• Validity
• Reliability
Other considerations
• Ease of use
• Communication
19
Different outcome measures
•
•
•
•
Patient Reported Outcome Measures
Goal Attainment Scales
Clinical Assessments
Generic measures e.g. SF36, Teller,
COPM etc
• Therapy Outcome Measure
• Patient Experience Measures---- not an
‘outcome measure’!
The Therapy Outcome Measure
Note
21
Therapy Outcome Measures for
Rehabilitation Professionals
Pamela Enderby and Alexandra John (2015) ISBN 978-1-90782629-0 PUBLISHED BY J&R
22
“So when do we do it?”
23
Referral /case history/ assessment
Aim/Goal
Intervention
End of episode of care
24
Now for something different!
25
What is the International Classification of
Function, Disability and Health (ICF)?
An international classification of function and disability and
its effects on the individual:
•
•
•
•
Classifies body structure and function
Classifies activity/independence
Classifies social participation
Classifies how the environment impacts upon the
disabled individual
We have added wellbeing as a domain
26
Therapy Outcome Measures
The Dimensions
Impairment
Disease/Disorder/Developmental Delay
Physical
Mental/ cognitive
An injury, illness, or congenital condition that causes or is likely to cause a
loss or difference of physiological or psychological function as compared
to those without such.
Activity
Limitation
Abilities
Person
Difficulties an individual may have in the performance of
activities/level of independence.
Participation
Disadvantages
Circumstances
Society
Disadvantages an individual may have in the manner or extent
of involvement in life situations.
Well-being
Upset
Feelings
Satisfaction
Degree of upset, distress, or satisfaction with status
Incorporates frequency and severity
Carer Well-being
Upset
Feelings
Satisfaction
Emotional effect resulting in an upset, distress, or
satisfaction with status
THERAPY OUTCOME MEASURE
(TOM)
• Based on:
World Health Organisation Classification - ICF
• 11 point ordinal scale with 6 defined points
35
The Core Scale
36
Impairment ----TOM
0
.5
1
.5
2
.5
3
.5
4
.5
5
The most severe presentation of this impairment
Severe presentation of this impairment
Severe/moderate presentation
Moderate presentation
Just below normal/mild presentation
No impairment
37
Activity--- TOM
0 Totally dependant/unable to function
.5
1 Assists/co-operates but burden of task/achievement falls on professional
carer
.5
2 Can undertake some part of task /needs a high level of support to complete
.5
3 Can undertake task/function in familiar situation but required some
verbal/physical assistance
.5
4 Requires some minor assistance occasionally/or extra time to complete task
.5
5 Independent/able to function
38
Participation TOM
0 No autonomy, isolated, no social/family role
.5
1 Very limited choices, contact mainly with professionals, no social / family role, little control over life
.5
2
Some integration, value and autonomy in one setting
.5
3
Integrated, valued and autonomous in limited number of settings
.5
4
Occasionally some restriction in autonomy, integration, or role
.5
5
Integrated, valued, occupies appropriate role
39
Well-being TOM
0 Severe constant: High and constant levels of distress/ upset/ concern/ frustration/ anger/
1
2
3
4
5
distress/ embarrassment/ withdrawal/ severe depression/ or apathy, unable to express or
control emotions appropriately.
Frequently severe: Moderate distress/ upset/ concern/ frustration/ anger/ distress/
embarrassment/ withdrawal/ severe depression/ or apathy. Becomes concerned easily,
requires constant reassurance/support, needs clear/ tight limits and structure, loses
emotional control easily.
Moderate consistent: Distress/ upset/ concern/ frustration/ anger/ distress/
embarrassment/ withdrawal/ severe depression/ or apathy in unfamiliar situations,
frequent emotional encouragement and support required.
Moderate frequent: Distress/ upset/ concern/ frustration/ anger/ distress/
embarrassment/ withdrawal/ severe depression/ or apathy. Controls emotions with
assistance, emotionally dependant on some occasions, vulnerable to change in routine,
etc., spontaneously uses methods to assist emotional control.
Mild occasional: Distress/ upset/ concern/ frustration/ anger/ distress/ embarrassment/
withdrawal/ severe depression/ or apathy. Able to control feelings in most situations,
generally well adjusted/stable (most of the time/most situations), occasional emotional
support/encouragement needed.
No inappropriate: Distress/ upset/ concern/ frustration/ anger/ distress/
embarrassment/ withdrawal/ severe depression/ or apathy. Well adjusted, stable and
able to cope emotionally with most situations, good insight, accepts and understands
own limitations.
40
Adapted scales
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
Anorexia Nervosa and Bulimia Nervosa-scale under development
Augmentative and Alternative Communication (AAC)
Autistic Spectrum Disorder
Cardiac Rehabilitation
Cerebral Palsy
Child Language Impairment
Challenging Behaviour and Forensic Mental Health
Chronic Pain
Cleft Lip and Palate
Cognition
Complex and Multiple Difficulty
Dementia
Diabetes
Dietetic Intervention for the Prevention of Cardiovascular Disease
Dietetic intervention for Enteral Feeding – Paediatrics
Adapted scales 2
16.
17.
18.
19.
20.
21.
22.
23.
24.
25.
26.
27.
28.
29.
30.
Dietetic intervention for Home Enteral Feeding – Adult
Dietetic intervention for Irritable Bowel Syndrome
Dietetic intervention for Obesity – Paediatric
Dietetic intervention for Obesity – Adult
Dietetic intervention for Undernutrition – Paediatrics
Dietetic intervention for Undernutrition – Adults
Dysarthria
Dysfluency
Dysphagia
Dysphasia
Dysphonia
Dyspraxia –Developmental Co-Ordination Difficulties
Equipment Services
Head Injury
Hearing Therapy/ Aural Rehabilitation
Adapted scales 3
32.
33.
34.
35.
36.
37.
38.
39.
40.
41.
42.
43.
44.
45.
46.
47.
Laryngectomy
Learning Disability – Communication
Mental Health
Mental Health – Anxiety
Multi-Factorial Conditions
Musculo-Skeletal
Neurological Disorders (Including Progressive Neurological Disorders)
Palliative Care
Phonological Disorder
Podiatric Conditions - scale under development
Post Natal Depression
Respiratory Care- Chronic Obstructive Pulmonary Disease (COPD)
Schizophrenia
Stroke
Tracheostomy
Wound Care
44
SUMMARY
A cross-disciplinary method of gathering information on a broad spectrum of
issues associated with therapy/enablement/rehabilitation
It allows description of the abilities of a patient/client in four
ways:
Impairment
Activity
Participation
Wellbeing
(problems in body structure or function
(performance of activities/independence)
(disadvantages experienced in living)
(emotional level of upset or distress)
11 point ordinal scale:
0 = severe, 3 = moderate and 5 = normal---+ 1 /2 points
Administered at the beginning and again at the end of
episode of care.
Impairment Mrs PR has had multiple sclerosis for 15 years. She is
severely ataxic and has increased tone in all limbs. Her sitting balance
is poor.
Activity Mrs PR uses an adapted wheelchair and all aids and
appliances in the home eff ectively. She is in an adapted
accommodation and can get to the local shops. She is able to care for
the house, provide meals for the family and communicate effectively.
Participation Mrs PR plays an active social role: she is a school
governor as well as a volunteer for a local charity. She enjoys her
garden and wheelchair dancing.
Wellbeing/distress Mrs PR is a determined, resourceful lady who, not
surprisingly, becomes concerned and frustrated on occasion, but is
generally positive and uses good emotional support strategies.
45
Mrs PR
• Summary Mrs PR has a severe level of
impairment but overcomes most functional
restrictions by being resourceful and using
appropriate aids. Thus, she is only partially
limited in activity and is not socially
disadvantaged in any specific way.
46
How many scores?
• Impairment
Impairment 1
Impairment 2
Not impairment 3 (use multifactorial or
multiple difficulties)
47
How many scores?
• Disability/Activity
Activity 1--- is the most usual
• Participation
Participation 1
• Well-being
Well-Being- patient /client
Well-Being—carer (only if involved)
48
AAC -Impairments
•
•
•
•
•
Physical
Cognitive
Sensory
Speech and language
Comprehension
49
Palliative care
Palliative Scale - Mean Scores
4
3.5
3
2.5
2
1.5
1
0.5
0
50
Percentage of patients showing
change (in amount) impairment
% of patients showing change (by amount) in impairment scores preto post- PR
5% 1%
5%
18%
38%
33%
-1
-0.5
0
0.5
1
1.5
51
Different Reporting Structure
Number of patients showing each improvement
Changes in Activity scores pre- to post- Pulmonary Rehabilitation
35
30
25
20
15
10
5
0
Number o f P atients
-1
-0.5
0
0.5
1
1.5
2
1
1
26
29
20
6
1
Difference in Activity Scores
52
Therapy Outcome Measure
Ordinal Rating Scale
Profound
0 0.5
Severe Severe/ Moderate Moderate
1 1.5 2 2.5 3 3.5
Mild
4
Normal limits
4.5
5
53
Participation
Impairment
Activity
Wellbeing
Stephen Hawking
54
Conducting a Benchmarking Study
• Internal Benchmarking:
Assess own performance
• External Benchmarking:
Assess performance against benchmarking partners
Results
data on 8070 patient admissions to
intermediate care.
 from 32 IC teams across England
 provided details of the service context, costs,
staffing / skill mix (800 staff), patient health
status and outcomes.
There was a 2.9% improvement in Therapy Outcome Measure score impairment
scores for each additional discipline in the team.
1.0
0.5
0.0
-0.5
-1.0
0
15
10
5
Number of clinical support staff in team
20
There was a 1% improvement in TOMS impairment scores for each additional
clinical support staff member in the team
1.0
0.5
0.0
-0.5
-1.0
0
.
5
10
15
Number of clinical support staff in team
20
Royal College of Speech and Language
Therapists Outcomes Project
Selecting an overall outcome measure which
was:
• psychometrically robust,
• easy-to-use,
• covers all the domains associated with the
aims of therapy
• easy to communicate
Charts showing the service
user’s TOMs scores across an
episode of care
60
Time series graph showing the
service user’s TOMs scores
across an episode of care
61
Sample data report R01: Change in TOMs scores
between initial and final ratings across each domain
Data from your team/service
Comparison data from other teams/services involved
in the pilot (matched for parameters applied)
Sample data report R03: Change in TOMs scores
between initial and final ratings across each domain
Sample data report R05: Average change in TOMs scores
between initial and final rating across each domain
Bev Curtis, Pam Enderby, Alex John April 2017
TOMs Carer Confidence Scale
I am confident that I understand and I know how to help in
all situations
Confident
5
4.5
I am mostly confident that I understand but have occasional
difficulties in some settings. I mostly know how to help.
4
3.5
I am fairly confident that I understand what is helpful and
know how to try different things but have frequent doubts. I
usually know how to help.
3
2.5
I have some confidence in one setting (e.g. home) that I
understand what to do but do not know what to do if that
doesn’t work. I know a little about how to help.
2
1.5
I have a little confidence but I am often worried that I am
not doing things right. I sometimes know how to help.
I am not confident that I understand what to do. I don’t
know how to help.
1
0.5
0
No Confidence
67
Beware of causality
68