Phys Occup Ther Pediatr Downloaded from informahealthcare.com by University of British Columbia on 09/15/10
For personal use only.
Goal Attainment Scaling:
Its Use in Evaluating
Pediatric Therapy Programs
Gillian A. King
Janette McDougall
Robert J. Palisano
Janet Gritzan
Mary Ann Tucker
ABSTRACT. Goal attainment scaling is becoming an increasingly
popular technique for evaluating the functional goal attainment of children receiving pediatric therapy services. This article reports on the
experiences of the authors in conducting formal program evaluations
using this individualized measurement approach. Goal attainment scaling is described, its utility is assessed, and issues in its use are identified. The article considers the pros and cons of the technique, highlights
the key decisions required to use goal attainment scaling effectively,
and provides standard criteria and procedures for its use in pediatric
settings. [Article copies available for a fee from The Haworth Document DelivGillian A. King, PhD, is Investigator, CanChild Centre for Childhood Disability
Research, McMaster University and Research Program Manager, Thames Valley
Children’s Centre. Janette McDougall, MA, is Research Associate, Thames Valley
Children’s Centre. Robert J. Palisano, ScD, PT, is Professor and Director, Program in
Movement Science, Medical College of Pennsylvania, Hahnemann University, and
Co-Investigator with CanChild. Janet Gritzan, MClSc, is Speech-Language Pathologist and Mary Ann Tucker, BSc, is Manager of School-Age and Adolescent Services,
both at Thames Valley Children’s Centre.
The authors sincerely thank the service providers, parents, teachers, and children
who assisted with this research. The authors also thank Paul Stolee for his useful
feedback on this article.
This research has been made possible through funding provided by CanChild and
Thames Valley Children’s Centre.
Physical & Occupational Therapy in Pediatrics, Vol. 19(2) 1999
E 1999 by The Haworth Press, Inc. All rights reserved.
31
Phys Occup Ther Pediatr Downloaded from informahealthcare.com by University of British Columbia on 09/15/10
For personal use only.
32
PHYSICAL & OCCUPATIONAL THERAPY IN PEDIATRICS
ery Service: 1-800-342-9678. E-mail address: [email protected]
<Website: http://www.haworthpressinc.com>]
KEYWORDS. Goal attainment scaling, therapy, program evaluation,
functional outcomes, children with special needs, rehabilitation
Many questions need to be considered when designing outcome
evaluation studies for children receiving occupational, physical, or
speech-language therapy, either in the community or in a health care
centre. One of the fundamental questions is whether to use a standardized or individualized measurement approach–or both.1,2 Individualized methods indicate whether single individuals have achieved the
goals of intervention. These methods also provide clear goals and
priorities for intervention, ensure the ongoing relevance of the child’s
goals, and reflect a client-centered perspective to service delivery.1,3
One of the most widely-used individualized approaches is goal attainment scaling (GAS),1 which provides an individualized, criterionreferenced measure of change. The GAS procedure involves: (a) defining a unique set of goals for each child, (b) specifying a range of
possible outcomes for each goal (on a scale recommended to contain
five levels, from *2 to +2),4 and (c) using the scale to evaluate the
child’s functional change after a specified intervention period. As we
have used it, a score of *2 represents the child’s baseline level before
intervention, *1 represents improvement that is less than the expected level of attainment after intervention, 0 represents the expected
level of attainment after intervention, and +1 and +2 represent levels
of attainment that exceed expectations but represent outcomes that the
child is thought to be capable of achieving under favorable conditions.
(Examples of these five-point GAS scales written for children receiving occupational, physical, and speech-language therapy are presented
in Tables 1 to 3. The examples reflect subcategories of goals for three
areas (productivity, mobility, and communication), which are often
targeted for intervention in the school setting.)
GAS was initially used to measure the impact of intervention in the
mental health field.5 Since then, it has been used widely to evaluate
health services, educational programs, and social services.6,7 In 1979,
GAS was considered the most popular outcome evaluation technique
in the human sciences.8
There are two main reasons for measuring outcomes in the field of
Phys Occup Ther Pediatr Downloaded from informahealthcare.com by University of British Columbia on 09/15/10
For personal use only.
King et al.
33
TABLE 1. Examples of Goal Attainment Scales for Children Receiving Occupational Therapy Services to Address Classroom Productivity
Goal Subcategory
Attainment Level
Score
Written Communication
Functional Fine
Motor Skills
Organizational Skills
Baseline
*2
Writes some letters in
isolation (i,e,u,l,t) with
verbal and visual cueing
Cuts within a 1/4I wide
boundary with verbal
cues to turn page with
nonĆdominant hand at
corners
Organizes desk (all notes
and books stacked neatly)
with physical assistance,
standby supervision,
scheduling, and a
checklist
Less than expected
outcome
*1
Writes all letters of the
alphabet in isolation with
verbal and visual cueing
Cuts within a 1/8I wide
boundary with verbal
cues to turn page with
nonĆdominant hand at
corners
Organizes desk (all notes
and books stacked neatly)
with standby supervision,
scheduling, and a
checklist
Expected outcome
O
Writes all letters of the
alphabet in isolation with
visual cueing
Cuts within a 1/16I wide
boundary with verbal
cues to turn page with
nonĆdominant hand at
corners
Organizes desk (all notes
and books stacked neatly)
with standby supervision
and a checklist
Greater than
expected outcome
+1
Writes all letters of the
alphabet with grouping of
2 to 3 letters with visual
cueing
Cuts within a 1/16I wide
boundary and
independently turns page
with nonĆdominant hand
at corners
Organizes desk (all notes
and books stacked neatly)
with standby supervision
Much greater than
expected outcome
+2
Writes all letters of the
alphabet with grouping of
3 to 4 letters
independently
Cuts on a regular penĆ
width line and
independently turns page
with nonĆdominant hand
at corners
Organizes desk (all notes
and books stacked neatly)
independently (general
instructions from teacher)
pediatric therapy: (1) to evaluate outcomes for a specific child (to
improve services to that child), and (2) to determine the effectiveness
of a service or program as a whole.9 GAS can be used for both
purposes–to document therapeutic change in individual children or to
examine change in groups of children. The focus of this article is on
the latter–the use of GAS in program evaluation studies.
GAS has been used in a number of studies evaluating pediatric
therapy services and programs, most fairly recent.10-17 Four studies
took place in early intervention settings and focused on the attainment
of motor development goals by infants or preschool children.14-17
Three other studies focused on the attainment of rehabilitation therapy
Phys Occup Ther Pediatr Downloaded from informahealthcare.com by University of British Columbia on 09/15/10
For personal use only.
34
PHYSICAL & OCCUPATIONAL THERAPY IN PEDIATRICS
TABLE 2. Examples of Goal Attainment Scales for Children Receiving Physical
Therapy Services to Address Mobility
Goal Subcategory
Attainment Level
Score
Functional Gross
Motor Skills
Ambulation
Transitions
Baseline
*2
Descends 6 stairs,
holding the handrail,
utilizing a stepĆtoĆstep
pattern, with one hand
held
Ambulates (with walker)
from the resource room to
classroom in 8.5 minutes
with supervision and verbal
cueing
Transfers self from walker
to desk chair with verbal
assistance and physical
assistance (other person
holding trunk and placing
feet on/off footrest with
weight supported)
Less than expected
outcome
*1
Descends 6 stairs,
holding the handrail,
utilizing a stepĆtoĆstep
pattern, with standby
assistance
Ambulates (with walker)
from the resource room to
classroom in 6 to 8 minutes
with supervision and verbal
cueing
Transfers self from walker
to desk chair with verbal
cueing and physical
assistance (holding trunk
and placing feet on/off
footrest)
Expected outcome
0
Descends 6 stairs,
holding the handrail,
utilizing a reciprocating
pattern, with one hand
held
Ambulates (with walker)
from the resource room to
classroom in 5 minutes or
less with supervision and
verbal cueing
Transfers self from walker
to desk chair with verbal
cueing and physical
assistance (placing feet
on/off footrest)
Greater than
expected outcome
+1
Descends 6 stairs,
holding the handrail,
utilizing a reciprocating
pattern, with standby
assistance
Ambulates (with walker)
from the resource room to
classroom in 5 minutes or
less with supervision and no
verbal cueing
Transfers self from walker
to desk chair with verbal
cueing and no physical
assistance
Much greater than
expected outcome
+2
Descends 6 stairs,
holding the handrail,
utilizing a reciprocating
pattern, independently
Ambulates (with walker)
from the resource room to
classroom in 5 minutes or
less independently (no
supervision/verbal cueing)
Transfers self from walker
to desk chair independently
goals by school-aged children.10,11,13 A recent study by Brown et al.12
examined the effects of physical therapy intervention on the attainment of gross motor goals in individuals ranging from 3 to 30 years
with severely limited physical and cognitive abilities. Thus, there is
growing evidence that GAS is a useful way to measure therapeutic
change. GAS has shown that children receiving pediatric therapy intervention achieve goals in a variety of spheres–classroom productivity, mobility, and communication. Still, questions remain about how to
properly conduct GAS and concerns are often raised about its reliabil-
Phys Occup Ther Pediatr Downloaded from informahealthcare.com by University of British Columbia on 09/15/10
For personal use only.
King et al.
35
TABLE 3. Examples of Goal Attainment Scales for Children Receiving
Speech-Language Therapy Services to Address Communication
Goal Subcategory
Attainment Level
Score
Speech Sound Production
of Blends
Speech Sound Production
in Final Word Position
Speech Sound Production
in Initial Word Position
Baseline
*2
Produces ``l'' blends with
85% accuracy in
structured phrases (i.e.,
17 of 20 trials correct)
Produces ``m'' sound in
final word position with
80% accuracy at the
spontaneous phrase level
(i.e., 8 of 10 trials
correct)
Produces ``f'' sound in
initial word position in
imitated phrases with
85% accuracy (i.e., 17 of
20 trials correct)
Less than expected
outcome
*1
Produces ``l'' blends with
85% accuracy in imitated
phrases
Produces ``m'' sound in
final word position with
80% accuracy at the
imitated sentence level
Produces ``f'' sound in
initial word position in
imitated sentences with
85% accuracy
Expected outcome
0
Produces ``l'' blends with
85% accuracy in
structured sentences
Produces ``m'' sound in
final word position with
80% accuracy at the
structured sentence level
Produces ``f'' sound in
initial word position in
structured sentences with
85% accuracy
Greater than
expected outcome
+1
Produces ``l'' blends with
85% accuracy in
spontaneous sentences
Produces ``m'' sound in
final word position with
80% accuracy at the
spontaneous sentence
level
Produces ``f'' sound in
initial word position in
spontaneous sentences
with 85% accuracy
Much greater than
expected outcome
+2
Produces ``l'' blends with
85% accuracy at the
spontaneous
conversational level
Produces ``m'' sound in
final word position with
80% accuracy at the
conversational level
Produces ``f'' sound in
initial word position
when describing a picture
with 85% accuracy
ity and validity.8,18 The appropriate use of GAS depends on a clear
understanding of its strengths and weaknesses19 and thoughtful decision-making concerning the issues that arise when using GAS in pediatric settings.
The aim of this article is to assist potential users to decide whether
or not to use GAS and to provide information about how to implement
GAS with a minimum of bias. Practical tips and guidelines are presented based on our combined experience with five studies that used
GAS in three types of pediatric settings: (a) a multidisciplinary,
school-based therapy program encompassing occupational, physical,
and speech-language therapy services,10,11 (b) physical therapy services for infants addressing mobility and other developmental
goals,15,16 and (c) a residential setting for children and adults with
Phys Occup Ther Pediatr Downloaded from informahealthcare.com by University of British Columbia on 09/15/10
For personal use only.
36
PHYSICAL & OCCUPATIONAL THERAPY IN PEDIATRICS
severely limited physical and cognitive abilities.12 These studies were
conducted in the United States or in Canada. Based on feedback from
therapists involved in these studies, our own observations, and guidelines in the general GAS literature, we have devised standard criteria
and procedures for the application of GAS in the formal evaluation of
pediatric therapy programs.
Other articles have provided information on the technical aspects
of using GAS, specifically the steps involved in setting the scales and
rating outcomes.13-23 Most of these articles have dealt with the application of GAS in non-therapeutic settings, such as special education services,20,22 human services programs,21 and mental health
services.23 On a practical level, the most useful publications are
those by Kiresuk and Lund,24 who discuss typical errors in creating
GAS scales and answer commonly-asked questions about biases in
goal setting and rating, and Smith,23 who provides guidelines around
who should set goals and who should rate goal attainment.
To date, publications have not addressed the specific issues that
arise in using GAS in pediatric therapy: Issues such as how to approach goal selection and goal definition for children with special
needs, and where to set the baseline on the GAS scale when a child
has a progressive or non-progressive condition. General guidelines
for the use of GAS (such as those laid out by Kiresuk et al.6) must be
adapted to particular intervention settings and populations because of
the different issues that arise in each setting.25 The present article
identifies key decision points and provides criteria and guidelines for
the systematic use of GAS in pediatric therapy settings, including
school-based therapy programs and early intervention programs.
DECIDING WHETHER OR NOT TO USE GAS
TO EVALUATE PEDIATRIC THERAPY PROGRAMS
A number of organizational conditions are necessary for the successful implementation of a program evaluation study using GAS.
They include: (a) a motivated team whose members are committed to
the evaluation and who share a common drive toward improvement of
therapy services, (b) adequate orientation and training of therapists,
(c) the availability of people to coach therapists in the proper application of GAS, so that both technical and practical issues are addressed
in an integrated fashion, and (d) sufficient resources allocated to do the
Phys Occup Ther Pediatr Downloaded from informahealthcare.com by University of British Columbia on 09/15/10
For personal use only.
King et al.
37
job.24 We often have employed a study coordinator to oversee the data
collection and training and to ensure the technical quality of the GAS
scaling and rating procedures.
The pros and cons of using GAS have been outlined in many articles.1,6,8,13,15,16,18,19,23,26 A primary strength of GAS is its ability to
measure change in performance, whereas most standardized measures
are discriminative tools designed to measure post-intervention status
(based on norms for children without special needs) and have not been
validated as responsive to clinically significant change.6,16,19 Clinical
significance refers to the magnitude of an effect in real-world terms.27
GAS is criterion-referenced, rather than norm-referenced, making it
potentially responsive to small changes that are perceived by children,
families, and teachers as important for daily function. GAS may be
particularly useful for children with low cognitive functioning, since
standardized measures may not be sensitive to the small but meaningful changes targeted for these individuals.
Relatively few standardized measures address functional outcomes
that are appropriate for children with special needs within a context
such as school (e.g., children’s ability to walk from the bus to the
school classroom). Standardized assessments of function often are
designed to measure a broad range of abilities. Some of these areas
may not reflect therapy goals and not be relevant to particular children. Pediatric studies provide a fair amount of evidence that GAS and
parallel standardized measures provide scores that are only moderately correlated with one another.10,11,15-17 For formal program evaluation purposes, we recommend the use of both GAS and standardized
measures.10,11
In addition to the ability to measure change in the performance of
individual children, GAS has other advantages: clinical utility, relevance, client involvement, and acceptability.1,23 GAS is ideally suited
to collaborative goal setting between a therapist, child, parent, and
other professionals (such as teachers). Its collaborative use reflects a
client- or family-centered approach to service delivery.1,3,28 Other
potential advantages of GAS include: (a) improved clarity of therapy
objectives for both therapists and clients, (b) improved conceptualization and delivery of the intervention, (c) more realistic client and
therapist expectations of therapy, (d) increased client satisfaction, and
(e) increased motivation of the client toward improvement, provided
by the very existence of goals.1,6,13,26
Phys Occup Ther Pediatr Downloaded from informahealthcare.com by University of British Columbia on 09/15/10
For personal use only.
38
PHYSICAL & OCCUPATIONAL THERAPY IN PEDIATRICS
There are a number of potential limitations in using GAS in a
program evaluation study: (a) biases in goal scaling and rating can
occur, (b) training and standardized implementation procedures are
required, which are time-consuming (when therapists are unfamiliar
with GAS), and (c) GAS can interfere with day-to-day practice because, when conducting a program evaluation, therapists should not
modify a goal in the course of the intervention. There are two reasons
for this. First, the study intervention period may not be sufficiently
long for change to be expected on a new or modified goal. Secondly,
therapists may elect to change goals they discover that they cannot
meet, which undermines the utility of GAS.
The major drawback to GAS is the possibility of bias in the use of
the tool, which can affect its validity.8,13,16,18 Unintentional bias can
occur in goal scaling (so goals are overly easy to attain) or in goal
rating (showing children make improvements that are not in fact
real). Reliability and validity can be improved, however, by comprehensive training of raters, adequate definitions of the levels of goal
attainment, and the use of multiple raters.6,18 As well, there are
aspects of how services are delivered in pediatric settings that naturally reduce the possibility and extent of bias in goal scaling and
rating. A collaborative goal setting model (a common feature of a
multidisciplinary, family-centered approach to service delivery)
helps to ensure that goal levels are meaningful and ratings are valid
because both are based on a consensus involving several individuals
who are knowledgeable about the child and invested in ensuring that
the child makes real gains.26,29 Thus, collaborative goal setting helps
to ensure that therapy goals are meaningful to the child and family
and not simply easy goals that therapists set on their own and can be
sure of attaining, which is a criticism raised by many.13,25
We have observed differences between the rehabilitation disciplines
with respect to the ease of writing appropriate functional goals. The
GAS scaling format appears easiest to apply for speech therapy goals
and harder to apply for physical therapy and occupational therapy
goals. As well, in King et al.,11 speech-language pathologists found it
easier to set goals that could be integrated into the child’s function in
the school setting. We speculate that the established hierarchy of the
development of speech sounds may assist speech-language pathologists in setting goals in the area of articulation. Physical and occupational therapists needed to give more thought to the process of writing
Phys Occup Ther Pediatr Downloaded from informahealthcare.com by University of British Columbia on 09/15/10
For personal use only.
King et al.
39
the various goal attainment levels. For instance, physical therapists
found it hard to set equal intervals between goal levels for goals
targeting unique mobility difficulties and found it hard to establish
relevant goal levels for high functioning children. Interestingly, there
has been relatively little application of GAS in the speech-language
area. The majority of applications have dealt with physical and occupational therapy goals.12,14-17,29
HOW TO USE GAS APPROPRIATELY AND EFFECTIVELY
IN A PEDIATRIC CONTEXT
Overview of GAS Procedure
We recommend that the child’s treating therapist participate in the
goal setting process and that a therapist not providing services to the
child do the goal rating. For each goal, the treating therapist provides a
written description of the child’s baseline level of performance. In a
family-centered service delivery model, this baseline level of performance would be vetted with clients (i.e., teachers, parents, and often
the children themselves). The treating therapist, in conjunction with
clients, also provides a written description of the expected level of
performance for the child at the end of intervention (corresponding to
the 0 rating). We have found it most effective for the other levels of the
scale ( *1, +1, and +2) then to be written by the treating therapist in
conjunction with a person trained to oversee the quality of the GAS
scales (i.e., a person who understands the steps of the scaling procedure, the necessary criteria, and pitfalls to avoid). The final step is a
peer review of the GAS scales by the therapists for each discipline
who do the ratings of goal attainment at the end of the intervention
(with the assistance of a person well-versed in the GAS scaling methodology). If a formal program evaluation is being done, a research
assistant may be involved. However, any well-trained person could
help identify the goal attainment levels and assist in the peer review of
the GAS scales.
Ensuring the Technical Quality of the GAS Procedure
One needs to ensure that the goal scales are reliable (i.e., that a
rating made by one therapist observing the child’s performance is
Phys Occup Ther Pediatr Downloaded from informahealthcare.com by University of British Columbia on 09/15/10
For personal use only.
40
PHYSICAL & OCCUPATIONAL THERAPY IN PEDIATRICS
comparable to the rating made by a different therapist) and that bias
does not occur in goal scaling (overly easy goals) or goal rating (improvements that are not real). The reliability and validity of GAS
scales can be improved by various procedures: ensuring that treating
therapists have a minimum level of experience so they can set realistic
goals in conjunction with children and parents; providing comprehensive training to therapists; using collaborative goal setting and peer
review in the goal selection phase; ensuring well-written goals
through training, peer review, and use of a standard procedure and
checklist; and using independent raters (i.e., raters who do not have a
personal investment in the outcome).
Table 4 outlines the questions that need to be considered in using
GAS effectively, criteria that should be met to ensure reliability and
validity of the procedure, and procedures and tools that can be used to
meet the criteria. These questions and criteria are based on the authors’ experiences in using GAS and on recommendations in the literature.
How Much Clinical Experience Is Necessary to Set Appropriate
Goals? One year of full-time clinical experience in the setting of
interest (a school-based therapy program or an early intervention
program) is ideal. This amount of time provides therapists with
enough exposure to different types of goals and different children so
that they can estimate the performance levels that children will most
likely attain. We did not meet this criterion in our own study11 (only
83% of therapists had over a year experience in providing schoolbased intervention) and realize that the experience level and number
of staff in the program implementing the evaluation are limiting
factors.
The ability to decide on the key variables that must change for goal
attainment is a skill that improves with experience. Experienced therapists develop competency in assessing child and environmental factors
affecting performance, are able to generate various ideas about possible variables to change, and are able to focus in on the variable they
believe is the most amenable to change (based on their past knowledge
and experience). For example, speech-language pathologists working
on communication goals in the area of articulation select particular
target sounds based on knowledge of the developmental hierarchy in
attainment of speech sounds, the child’s stimulation potential for that
sound in isolation, the importance of that sound in increasing the
Phys Occup Ther Pediatr Downloaded from informahealthcare.com by University of British Columbia on 09/15/10
For personal use only.
King et al.
41
TABLE 4. How to Use Goal Attainment Scaling Effectively
Questions to Consider
Criteria to Be Met
Procedures and Tools
to Help Ensure Criteria
Are Met
How much clinical
experience is necessary to set
appropriate goals?
Therapists should have a minimum of 1
year of pediatric experience.
scaling.
Involve only experienced
therapists in goal setting and
How much instruction and
training is necessary for
therapists?
Therapists should have approximately 7
hours of specific training in the use of
GAS.
See Table 5 for
recommended orientation
and training steps.
How can one ensure adequate
goal selection?
The process of selecting goal areas should
ensure that:
1. Goals are meaningful and relevant to
others.
2. Goals make sense from a conceptual
point of view (e.g., if the goals of the
program are to improve dayĆtoĆday
function, then the majority of goals set
should be functional in nature, rather
than impairmentĆbased).
How can one ensure adequate
goal scaling (i.e., adequately
written goal levels on the
fiveĆpoint scale)?
Each of the levels on the scale should:
1. be written as clearly as possible, in
concrete behavioral terms
2. specify an observable behavior of the
child
3. be written in the present tense
4. be achievable or realistically possible
The scale as a whole should:
1. have levels that reflect clinically
meaningful gradations of
improvement
2. have approximately equal intervals
between the goal attainment levels
(i.e., the change from +1 to + 2 should
be similar to that between *2 to *1, etc.)
3. specify a time period for achievement
4. reflect a single dimension of change
(as long as a goal remains meaningful),
keeping other variables constant
5. not reflect attainment that is
dependent on the therapist's physical
assistance (unless the assistance of
others is a written part of the goal)
Employ collaborative
goal setting (therapists
select broad goal areas in
conjunction with
knowledgeable others
such as the teacher,
parent, and/or child).
Use peer review of goal
content.
Each criterion can be
assured through the use of
three interĆconnected
procedures and tools:
Therapist training
Peer review to ensure the
adequacy of the goal
scales
Use of a standard
procedure and checklist
to review the technical
adequacy of written goal
scales (see Figure 1)
Phys Occup Ther Pediatr Downloaded from informahealthcare.com by University of British Columbia on 09/15/10
For personal use only.
42
PHYSICAL & OCCUPATIONAL THERAPY IN PEDIATRICS
TABLE 4 (continued)
Questions to Consider
Criteria to Be Met
Procedures and Tools
to Help Ensure Criteria
Are Met
How can one ensure adequate
goal rating?
1. Ratings should be done by therapists
āĂnot involved in providing the intervention.
2. Reduce the ``performance demands" of
ĂāĂĂ the visiting independent rater.
Use independent raters.
How is the summary score
determined?
Clients' individual outcome scores need to
be aggregated in some way, preferably
using TĆscores.
statistical software package.
Ensure that you have
thought of the data analysis
stage and have access to a
child’s intelligibility, and the consensus of the caregiver. Determining
a child’s potential for change during the intervention period is based
on clinical judgment (i.e., experience, the underlying cause of the
child’s articulation disorder, the type of error with the targeted sounds,
the number of other sounds that are in error).
How Much Instruction and Training Is Necessary for Therapists?
Orientation and training of therapists is necessary for the successful
use of GAS.13 The steps and time requirements included in Table 5 are
based on Kiresuk et al.’s6 recommendations and our own experiences.
Many therapists play the dual role of treating therapist and rating
therapist (only for children to whom they do not provide intervention),
which requires approximately 12 hours of training.
The training procedures in Table 5 incorporate the recommendations of therapists involved in our evaluation studies. For instance,
therapists recommended that a list of common errors in creating GAS
scales (and solutions) would be useful (Table 6), as would an inventory of potential goals.
To develop skill in writing the scales, we recommend the use of
small group sessions in which therapists practice putting goals into
GAS format (i.e., identifying five levels of possible goal attainment).
A series of case scenarios can be used for practice and discussion.
Therapists also can practise putting goals in GAS format for two or
three children from their caseload. At this point, it is useful for them to
have access to a resource binder with examples of the five-point rating
Phys Occup Ther Pediatr Downloaded from informahealthcare.com by University of British Columbia on 09/15/10
For personal use only.
King et al.
43
TABLE 5. Orientation and Training of Therapists to the Goal Attainment Scaling
Procedure
All Therapists
General Orientation (2 hours)
S Orientation session
S Handout
S examples of goals
S specific guidelines
S list of common errors
Treating Therapists-Goal Setting
Skill Acquisition (3 hours)
S Small group practice
S reviewing types of goals
S setting goal levels
S use of actual case material
S OneĆtoĆone guidance with trainer
Skill Maintenance (2 hours)
S Continued monitoring of goal setting
S Question and answer sessions
Rating Therapists-Goal Rating
Skill Acquisition (4 hours)
S Review goals with trainer
S peer review of goal writing
S familiarizing raters with the goals that they will
be rating
S Review of goal rating procedure
S Handout
Skill Maintenance (1 hour)
S Question and answer sessions
scale applied to goals in particular areas (e.g., mobility goals, communication goals).
How Can One Ensure Adequate Goal Selection? It is important that
the selected goals are meaningful to the child and family and reflect
the primary focus of therapy for the child. We therefore recommend
that treating therapists be involved in goal selection rather than independent goal setters.30
How Can One Ensure Adequate Goal Scaling? The literature refers
to six criteria for good goal writing: Goals should be relevant, understandable, measurable, behavioral, attainable, and time-limited.13,18
We have gone beyond these general criteria to specify criteria relevant
to each of the goal levels and those dealing with the properties of the
scale as a whole (see Table 4).
Three criteria concerning the scale as a whole require more explanation. First, although the GAS scale examples in Tables 1 through
3 did not specify a time period, it should be noted that a standard time
Phys Occup Ther Pediatr Downloaded from informahealthcare.com by University of British Columbia on 09/15/10
For personal use only.
44
PHYSICAL & OCCUPATIONAL THERAPY IN PEDIATRICS
TABLE 6. Common Errors in Creating GAS Scales
Error
Description
Overly Generalized
Goals
If the expected level (i.e., 0 level) of a scale
is written in very general terms (e.g., ``walks
a greater distance in a set period with
assistance"), it will be difficult or
impossible to create the remaining scale
points, therefore making the goal unmeasurable.
The expected level of a scale
should be written as clearly as
possible (e.g., ``walks with
platform walker 100 metres in six
minutes with two hands on walker
to assist with steering").
Solution
Overly Technical
Goals
A goal setter may use terms specific to
his/her profession in creating a scale that the
goal rater is not familiar with.
Write goals in common terms,
especially if the goal rater is not of
the same professional background
as the goal setter.
Multiple Variables
of Change
A scale may include two or more variables
of change. This could be problematic if the
scale is written so that change is expected to
occur simultaneously on these variables.
Decide on one variable by which to
measure change in performance
and hold others constant. If in
doing so, the goal does not remain
meaningful, two variables could
change within in a single scale,
provided each scale level differs on
only one variable.
Unequal Scale
Intervals
A scale may be created where the amount of
clinical change is greater between, say, the
+1 and +2 levels than the amount of change
between the *2 and*1 levels.
Aim for clinically equal intervals
between all levels of the scale.
Clinically Irrelevant
or Unrealistic Scale
Levels
A scale may be created where one or more
of the levels represents an amount of change
that would not be clinically relevant to the
child (i.e., the amount of change is too small
to matter) or the amount of change is
unrealistic for the child (i.e., the amount of
change is too great).
The amount of change between all
scale levels needs to be clinically
relevant and all levels should be
achievable for the child.
Using Different
Tenses (i.e., Past,
Present, Future)
When Writing Scale
Levels
A GAS scale may be written with the *2
level written in one tense and all other levels
in another tense, which could be confusing
and bias the goal rater.
All scale levels should be phrased
in the present tense, in order for
evaluation to make sense at
different time points (i.e., ``walks . . .'').
Redundant or
Incomplete Scale
Levels
A scale may be written where a child could
be scored on two levels at the same time
(e.g., the +1 level has walking distances
specified between ``40 and 50 metres'' and the
+2 level specifies distances between ``50 and
metres"). If a child walks exactly 50 metres,
both the +1 and the +2 level would be
correct. On the other hand, a gap could be
present in the scale where a child could not
be scored on any level (e.g., the +1
specifies walking distances between ``40 and
50 metres" and the +2 specifies distance
between ``60 and 70 metres"; if a child
walks 55 metres, neither the +1 nor the +2
level is correct).
Be careful not to create scale levels
that are redundant or incomplete.
Careful wording (e.g., +1 would be
``more than 40 metres and up to 50
metres" and +2 would be ``more than
50 metres and up to 60 metres")
or specific instructions to the
rater (e.g., if a child obtains a
midway point between two levels,
score the child at the lower level)
will be of benefit.
Blank Scale Levels
It may be difficult to write the more extreme
levels of a scale, tempting the goal setter to
leave these levels blank. If a child happens to
achieve an upper or lower extreme, it would
be impossible to rate the child's performance.
Be careful to set goals where it is
possible to complete all scale
levels.
Phys Occup Ther Pediatr Downloaded from informahealthcare.com by University of British Columbia on 09/15/10
For personal use only.
King et al.
45
period (the length of the intervention period as prescribed by the study
design) was set for all goals.
An example may clarify criterion 4. The following goal includes
three possible variables of change, namely distance, time, and level of
assistance: ‘‘Walks 100m with platform walker in 8 minutes with two
hands on walker to assist with steering.’’ In order to write an appropriate scale, the therapist must decide which is the most important variable of change. That variable would then be altered in each of the
written goal levels, with the other variables held constant.
With respect to criterion 5, if physical assistance is required, this
should be explicitly stated in the written goal, either as a constant
factor or as the variable that changes over time. The key idea is that
goal attainment levels should reflect change in the child’s behavior,
not unacknowledged variations in the therapist’s physical assistance. It
is permissible to write goals where the physical assistance of someone
else is explicitly stated as the variable that changes over time, so that
the goal scale shows meaningful changes in the level of assistance the
child requires to perform a task. An example of the gradations that
could be included in a scale focusing on changes in level of assistance
is: physical and verbal assistance required (*2), verbal assistance
with checklist required (*1), checklist and verbal cueing or prompting required (0), verbal cueing/prompting required (+1), and completely independent (+2).
According to strict research methodology,8 the therapist who sets
the levels of goal attainment should not be the same therapist who
provides the treatment. This is a difficult criterion to meet since, in
actual clinical practice, the treating therapist is involved in goal setting, often in conjunction with the client.10,11,23 This is defensible
when one involves an independent rating therapist and a trainer/study
coordinator in the review of the goal attainment levels (who follows a
standard procedure with set criteria). In our experience, even highly
experienced therapists have some difficulty identifying the baseline
and expected levels of the goals. Assistance and review by others is
useful, appreciated, and necessary. Figure 1 provides a checklist that
can be used in the review of written goals by a trainer and independent
rating therapist.
How Can One Ensure Adequate Goal Rating? Cardillo30 addresses
the selection of raters and the decision about the timing of the goal
rating session, but provides little information on how to conduct the
Phys Occup Ther Pediatr Downloaded from informahealthcare.com by University of British Columbia on 09/15/10
For personal use only.
46
PHYSICAL & OCCUPATIONAL THERAPY IN PEDIATRICS
FIGURE 1. Goal Attainment Scaling Checklist–Goal Review Procedure
Name of Participant:
S Therapy Goal: Expected Outcome (i.e., a score of 0)
As a whole, the scale must meet the following criteria:
Criteria
Criterion
Met
Criterion
Not Met
Comments
Criterion
Not Met
Comments
The amount of change between the levels is
clinically important.
There are approximately equal intervals
between the goal attainment levels.
There is a time period for achievement of the
goal.
The scale reflects a single dimension of
change (or, if not feasible, each scale level
reflects a single dimension of change).
Each level on the scale must meet the following criteria:
Criteria
Criterion
Met
Be written in concrete behavioral terms
Specify an observable behavior of the child
Be written in the present tense
Be achievable or realistically possible
actual goal rating session. Our experiences have led to some recommendations. First, it is important to consider the child’s view of the
rating situation. When children are aware that a new person is coming
to watch a session, they may be very motivated to perform well for
this visiting person. The treating therapist can reduce the ‘‘hype’’
regarding the rater’s visit by informing the child in advance of the visit
and assuring the child that his/her regular performance is what is
called for. The rating therapist should minimize the effect of his/her
presence by sitting quietly in the back of the room and making notes
discreetly.
Phys Occup Ther Pediatr Downloaded from informahealthcare.com by University of British Columbia on 09/15/10
For personal use only.
King et al.
47
A second issue concerns the behavior of the treating therapist in
the session. For goals whose attainment cannot be observed under
naturally occurring circumstances at school, we have found it most
appropriate for the treating therapist to interact with the child and
request performance of the behavior outlined in the goal. (Some
goals require that the child be set to the task because of infrequent
naturally occurring opportunities to display the behavior.) The treating therapist begins by orienting the child to the task. If prompting,
cueing or some other support is required, the therapist starts with the
0 level of the scale and prompts for performance up or down the scale
depending on the child’s success at the 0 level. Brown et al.12 also
used a prompting procedure and allowed up to three trials per goal
(for individuals with severely limited physical and cognitive abilities). Similarly, we have found that children may need more than one
attempt to demonstrate their true ability. Some children react to the
presence of the rating therapist by showing silly behavior and two or
three attempts are required before they calm down and demonstrate
true performance.
How Is the Summary Score Determined? For program evaluation
purposes, users need to calculate an appropriate summary score to
reflect the overall goal attainment of children in the therapy or early
intervention program. The recommended procedure is to convert children’s outcome scores into aggregate T-scores (see Cardillo and
Smith,4 for a complete discussion of T-scores and other summary
scores). T-scores can be calculated using a statistical computer software package such as the Statistical Package for the Social Sciences
(SPSS). Mean aggregate T-scores facilitate reliability analyses and
comparisons across children, and provide an overall evaluation of
children’s performance. Another advantage of using T-scores is that
they can be compared to other standardized scores.
T-scores can be computed using the formula developed by Kiresuk
and Sherman:5
(10 WiXi)
T = 50 + p
(1 * r) Wi 2 + r (Wi 2)
In this formula, 50 represents the mean and 10 is the standard
deviation. Wi represents the weighing for a particular goal and Xi
Phys Occup Ther Pediatr Downloaded from informahealthcare.com by University of British Columbia on 09/15/10
For personal use only.
48
PHYSICAL & OCCUPATIONAL THERAPY IN PEDIATRICS
represents the score for each goal. The r represents the expected overall intercorrelation among outcome scores. The formula for computing
the T-score assumes a relatively low correlation among goals of .30.
Kiresuk and Sherman5 found this correlation useful because it yields a
standard deviation of 10 units. Cardillo and Smith4 strongly recommend against the use of differential weighing of GAS goals. The use
of this formula may appear time-consuming and difficult, but the need
for manual computation is rare. If goals are not weighted and the
suggested intercorrelation of .30 is used, tables are available that allow
the quick and easy conversion of outcome scores into T-scores for
clients with up to eight scored scales (see Kiresuk et al.6).
Conceptual Issues Arising in the Use of GAS
Validity Issues: How to Ensure that Goal Attainment Is Real. Two
basic strategies can be used to address the issue of validity:
(1) supplement the use of GAS with measures that provide more
conventional estimates of post-treatment status (i.e., standardized
measures) to provide a more comprehensive assessment of outcome,6
or (2) employ randomly selected control goals (after Brown et al.12).
In Brown et al., therapists created GAS scales both for treatment
goals that were practiced in the physical therapy setting and for
control goals that were set but not addressed in practice. The subjects’ progress on treatment goals was significantly greater than their
progress on the control goals, suggesting that the therapy intervention was the factor contributing to improved goal attainment and that
the goal attainment was real.
How to Set the Baseline in the GAS Scale. This is an important
consideration. When no deterioration is expected in children’s performance, such as when the child has a non-progressive, chronic condition, *2 can be defensibly used as the child’s baseline.10,11 When
evaluating goal attainment of individuals with severely limited physical and cognitive abilities, Brown et al.12 used *2 as a baseline but
captured lower performance on a test day by using a score of *3.
When evaluating the performance of children with progressive conditions who may deteriorate in function over time, it would make sense
to set the baseline at *1. If the expectation is that decreased performance will occur and the goal is to minimize performance loss, it
would make sense to write goal levels in terms of gradations of diminished performance.
Phys Occup Ther Pediatr Downloaded from informahealthcare.com by University of British Columbia on 09/15/10
For personal use only.
King et al.
49
How to Best Demonstrate Reliability. It is important to report the
reliability of the goal ratings. There are various types of reliability
including inter-rater stability (independent raters’ agreement over
time) and inter-rater reliability (independent raters’ agreement at
the same point in time).8 In King et al.,10 we computed a measure of
inter-rater stability, which involved correlating T-scores based on
the ratings of the treating and rating therapists obtained on separate
occasions. We found a correlation of .6, which indicates a moderate
level of agreement between the rating therapists and the treating therapists on the amount of improvement the children made on their goals,
and is typical of estimates of stability obtained when ratings by different individuals are obtained on different occasions.6 In our second
study, we felt it was more appropriate to obtain a measure of interrater reliability from two independent rating therapists who rated
the goals on the same occasion.8 Using this procedure, the inter-rater reliability (Intraclass Correlation Coefficient) was .98. To demonstrate that the GAS scales are reliable, we recommend that others
examine and report inter-rater reliability estimates from two independent raters.
Brown et al.12 computed measures of inter-rater reliability between
a treating therapist who directly observed levels of goal attainment in a
physical therapy setting and an independent rater who scored goal
attainment based on videotapes of the same sessions. This is a good
way to reduce performance demands or reactive behavior created by
the presence of an unfamiliar, independent rater.
What Does It Mean When the Change Exceeds Therapists’ Expectations? Studies have found that GAS ratings are often higher than the
expected level of 0 on the 5-point scale.10,11,15,16 Since the ratings
were reliable, it appears that finding change exceeding therapists’
expectations is not due to biases in the rating procedure, but rather in
the goal setting. There is therefore fairly strong evidence that expected
outcomes may be underestimated systematically by therapists. The
multiple reasons for this are hard to disentangle. Therapists may set
low levels of expected goal attainment to be cautious and ensure
success for the child. It is also possible that therapists underestimate
the therapy gains that children can make, which may be due to not
fully taking children’s motivation into account. The underestimation
of expected outcomes should be discussed with therapists during training in future studies.
Phys Occup Ther Pediatr Downloaded from informahealthcare.com by University of British Columbia on 09/15/10
For personal use only.
50
PHYSICAL & OCCUPATIONAL THERAPY IN PEDIATRICS
How to Approach Goal Selection and Definition. When selecting a
goal, therapists need to be aware of the underlying factor that is being
targeted for change–whether the major limiting factor is the child’s
endurance or motivation, for example, or aspects of the physical environment. Goal definition therefore is based on the therapist’s working
hypothesis of the best way to assist the child to perform the task.
The therapist’s hypothesis about the major factor limiting the child’s
successful performance is sometimes evident from looking at the goal
scale that is set. We have found that when the variable being changed
deals with a quality (such as accuracy) or a quantity (such as number of
letters formed correctly), then the working hypothesis seems to deal with
some aspect internal to the child. On the other hand, when the variable
being changed deals with the level of physical or verbal assistance provided, or levels of equipment support, then the variable of change corresponds to the factor of critical importance in the therapist’s view. Under
these conditions, the written goal levels provide a window on the therapist’s view of the factors limiting the child’s goal achievement.
Implications for Therapists and Managers
This article should help therapists and managers make informed
decisions about whether or not to use GAS, based on an understanding
of the questions they need to ask and the requirements for using GAS
appropriately and effectively in pediatric program evaluation studies.
Moreover, the criteria and procedural guidelines presented here should
provide useful assistance to those using GAS in pediatric settings.
Information from GAS can to be used to improve programs, to assist
in meeting accreditation requirements, and to demonstrate accountability to governing bodies and funders.
GAS has been criticized due to its potential for bias when implemented without thought and care.18 We have provided a more hopeful
or balanced perspective here. GAS can be implemented appropriately
when guidelines and standard procedures are used. Under these conditions, it is well worth the effort to use GAS to evaluate pediatric
therapy programs. Service providers, managers, parents, and children
themselves benefit from the knowledge provided by goal attainment
scaling.
Phys Occup Ther Pediatr Downloaded from informahealthcare.com by University of British Columbia on 09/15/10
For personal use only.
King et al.
51
REFERENCES
1. Russell D, King G, Palisano R, Law M. Measuring individualized outcomes
(Research Report No. 95-1). Hamilton, ON: McMaster University and ChedokeMcMaster Hospitals, CanChild Centre for Childhood Disability Research; 1995.
2. Russell D, King G, Palisano R, Law M. Measuring individualized outcomes.
Proceedings of the American Academy for Cerebral Palsy and Developmental Medicine. 1996; 25.
3. Zaza C, Stolee P, Prkachin K. The application of goal attainment scaling in
chronic pain settings. Journal of Pain and Symptom Management. 1999; 55.
4. Cardillo JE, Smith A. Psychometric issues. In: Kiresuk TJ, Smith A, Cardillo
JE, eds. Goal attainment scaling: Applications, theory, and measurement. Hillsdale,
NJ: Lawrence Erlbaum Associates; 1994; 173-212.
5. Kiresuk TJ, Sherman RE. Goal attainment scaling: A general method for evaluating comprehensive community mental health programs. Community Ment Health
J. 1968; 4:443-453.
6. Kiresuk TJ, Smith A, Cardillo JE. Goal attainment scaling: Applications,
theory, and measurement. Hillsdale, NJ: Lawrence Erlbaum Associates; 1994.
7. MacKay G, Somerville W, Lundie J. Reflections on goal attainment scaling
(GAS): Cautionary notes and proposals for development. Educational Research.
1996; 38:161-172.
8. Cytrynbaum S, Ginath Y, Birdwell J, Brandt L. Goal attainment scaling: A
critical review. Eval Q. 1979; 3:5-40.
9. Law M, King GA, MacKinnon E, Russell DJ. Quality performance: Designing clinical services around person-centered outcomes. In: Gardner JF, Nudler S, eds.
Quality performance in human services: Leadership, values, and vision. Baltimore,
MD: Paul H. Brookes; 1999; 81-106.
10. King G, Tucker M, Alambets P, Gritzan J, McDougall J, Ogilvie A, Husted K,
O’Grady S, Brine M, Malloy-Miller T. The evaluation of functional, school-based
therapy services for children with special needs. A feasibility study. Phys Occup Ther
Pediatr. 1998; 18: 1-27.
11. King G, McDougall J, Tucker MA, Gritzan J, Malloy-Miller T, Alambets P,
Cunning D, Thomas K, Gregory K. An evaluation of functional, school-based therapy services for children with special needs. Manuscript submitted for publication;
1999.
12. Brown DA, Effgen SK, Palisano RJ. Performance following ability-focused
physical therapy intervention in individuals with severely limited physical and cognitive abilities. Phys Ther. 1998; 78:934-947.
13. Clark MS, Caudrey DJ. Evaluation of rehabilitation services: The use of goal
attainment scaling. Int Rehabil Med. 1983; 5: 41-45.
14. Maloney FP, Mirrett P, Brooks C, Johannes K. Use of goal attainment scaling
in the treatment and ongoing evaluation of neurologically handicapped children. Am
J Occup Ther. 1978; 32: 505-510.
15. Palisano RJ, Haley SM, Brown DA. Goal attainment scaling as a measure of
change in infants with motor delays. Phys Ther. 1992; 72: 432-437.
16. Palisano RJ. Validity of goal attainment scaling with infants with motor delays. Phys Ther. 1993; 73: 651-658.
Phys Occup Ther Pediatr Downloaded from informahealthcare.com by University of British Columbia on 09/15/10
For personal use only.
52
PHYSICAL & OCCUPATIONAL THERAPY IN PEDIATRICS
17. Stephens TE, Haley SM. Comparison of two methods for determining change
in motorically handicapped children. Phys Occup Ther in Pediatr. 1991; 11(1): 1-17.
18. Ottenbacher KJ, Cusick, A. Goal attainment scaling as a method of clinical
service evaluation. Am J Occup Ther. 1990; 44: 519-525.
19. Ottenbacher KJ, Cusick A. Discriminative versus evaluative assessment:
Some observations on goal attainment scaling. Am J Occup Ther. 1993; 47: 349-354.
20. Carr RA. Goal attainment scaling as a useful tool for evaluating progress in
special education. Except Child. 1979; October: 88-95.
21. Kiresuk TJ, Lund SH. Goal attainment scaling. In: Atkisson C, ed. Evaluation
of human service programs. New York, NY: Academic Press; 1978; 341-370.
22. Shuster SK, Fitzgerald N, Shelton G, Barber P, Desch S. Goal attainment scaling with moderately and severely handicapped preschool children. Journal of the Division for Early Childhood. 1984; Winter: 26-37.
23. Smith A. Introduction and overview. In: Kiresuk TJ, Smith A, Cardillo JE,
eds. Goal attainment scaling: Applications, theory, and measurement. Hillsdale, NJ:
Lawrence Erlbaum Associates; 1994; 1-14.
24. Kiresuk TJ, Lund SH. Implementing goal attainment scaling. In: Kiresuk TJ,
Smith A, Cardillo JE, eds. Goal attainment scaling: Applications, theory, and measurement. Hillsdale, NJ: Lawrence Erlbaum Associates; 1994; 119-134.
25. Lewis A, Spencer JH, Haas GL, DiVittis A. Goal attainment scaling: Relevance and replicability in follow-up of inpatients. J Nerv Ment. 1987; 175: 408-417.
26. Mitchell T, Cusick A. Evaluation of a client-centered paediatric rehabilitation
program using goal attainment scaling. Australian Occupational Therapy Journal.
1998; 45: 7-17.
27. Bain BA, Dollaghan CA. Clinical forum: Treatment efficacy. The notion of
clinically significant change. Language, Speech, and Hearing Services in Schools.
1991; 22: 264-270.
28. Rosenbaum P, King S, Law M, King G, Evans J. Family-centered service: A
conceptual framework and research review. Phys Occup Ther Pediatr. 1998; 18:
1-20.
29. Stolee P, Zaza C, Pedlar A, Myers AM. Clinical experience with goal attainment scaling in geriatric care. Journal of Aging and Health. 1999; 11: 96-124.
30. Cardillo JE. Goal setting, follow-up, and goal monitoring. In: Kiresuk TJ,
Smith A, Cardillo JE, eds. Goal attainment scaling: Applications, theory, and measurement. Hillsdale, NJ: Lawrence Erlbaum Associates; 1994; 39-59.