This is an enhanced PDF from The Journal of Bone and Joint Surgery
The PDF of the article you requested follows this cover page.
Measurement of the Cobb angle on radiographs of patients who have
scoliosis. Evaluation of intrinsic error
RT Morrissy, GS Goldsmith, EC Hall, D Kehl and GH Cowie
J Bone Joint Surg Am. 1990;72:320-327.
This information is current as of July 10, 2008
Reprints and Permissions
Click here to order reprints or request permission to use material from this
article, or locate the article citation on jbjs.org and click on the [Reprints and
Permissions] link.
Publisher Information
The Journal of Bone and Joint Surgery
20 Pickering Street, Needham, MA 02492-3157
www.jbjs.org
(‘opynght
990
by The Journal
of the
Measurement
Cobb
Who
of Patients
RAYMOND
T.
MORRISSY,
DOUGLAS
KEHL,
M.D.t,
M.D.t,
From
GREGORY
AND
the Scottish
G.
Incorporated
Angle
on Radiographs
Scoliosis
INTRINSIC
ERROR*
S. GOLDSMITH,
HENRY
Rite
Surgery.
Have
OF
EVALUATION
BY
of Bone and Joint
Children’s
M.D.t,
COWIE,
M.D.t,
Medical
ELMER
Center,
certainty.
In most scientific
is a need for accurate
first when the same
Most
change
investigators
or more
including
medicine,
there
For patients who have
scoliosis,
the amount
of lateral
bend is the most important
and distinguishing
feature
of the radiographic
examination.
The
Cobb
method4
is the standard
method
clinically
important,
to support
the use
demiological
radiographs
of
to be
there is no firm evidence
figure38.
studies.
stimulation
was used
as indicating
for inclusion
common
for
cerning
bracing,
reports
on the evaluation
progression
of scoliosis,
and thus
on the basis
was
the criterion
of an increase
it is
con-
in the curve
between
two successive
radiographs.
An in20 to 25 degrees
may be a reason
to prescribe
and an increase
a recommendation
from 40 to 45 degrees
for operative
of this study
was to determine
observer
contribution
of each
may prompt
treatment.
and
then
when
servers
measured
the same radiographs.
attempted
to identify the possible
sources
relative
of
S degrees
in the study”’2.
In the clinical
setting,
practitioners
to make recommendations
treatment
of S degrees
crease
from
in recent
in the treatment
source
the error,
different
ob-
in addition,
we
of error and the
to the amount
of error.
of quantitating
Materials
liosis of between
files of the Scottish
about the progression
and the effectiveness
of
treatment.
Fifty
This
is little
5 degrees
considered
successive
even though
of this
electrical
formity,
it is used to make decisions
of a curve,
the need for treatment,
there
two
the Cobb angles
on two successive
radiographs
as the critenon
of progression.
Brooks
et al. and Rogala
et al. also
used 5 degrees
as the criterion
of progression
in their epi-
tients
its importance,
PH.D.,
their study of the natural progression
of scoliosis,
and Carlson
used a 5-degree
difference
between
In
Lonstein
this measurement.
Although
the Cobb angle is recognized
as being a measure
of the amount
of tilt of the end-vertebrae
rather
than an objective
measure
of all aspects
of the de-
Despite
have
between
The purpose
endeavors,
measurement.
HALL,
GEORGIA
Atlanta
To quantitate
the intrinsic error in meafifty anteroposterior
radiographs
of patients
who had scoliosis
were each measured
on six separate
occasions
by four orthopaedic
surgeons
using the Cobb
method.
For the first two measurements
(Set I), each
observer
selected
the end-vertebrae
of the curve; for the
next two measurements (Set II), the end-vertebrae
were
pre-selected
and constant.
The last two measurements
(Set Ill) were obtained
in the same manner
as Set II,
except
that each examiner
used the same protractor
rather than the one that he carried with him. The pooled
results
of all four observers
suggested
that the 95 per
cent confidence limit for intraobserver
variability
was
4.9 degrees
for Set I, 3.8 degrees
for Set II, and 2.8
degrees for Set III. The interobserver
variability
was 7.2
degrees
for Set I and 6.3 degrees for Sets II and III. The
mean angles differed significantly
between
observers,
but the difference
was smaller
when the observers
used
the same protractor.
ABSTRACT:
surement,
C.
ATLANTA,
information
about
good-quality
who
narrow
a curve
had
and Methods
anteroposterior
radiographs
a thoracic,
range
thoracolumbar,
20 and 40 degrees
were
Rite Medical
Center,
of magnitude
is in this range
certain
was
Atlanta,
selected
decisions
of pa-
or lumbar
scoselected
from the
Georgia.
because
regarding
when
treatment
the degree
of certainty
of changes
in the Cobb angle from
one radiograph
to the next. One study”
showed
the difference between
the findings
of two examiners
to be 3. 12 ±
are made and a few degrees
of variation
in the range are of
relatively
greater
magnitude
than in higher
ranges
of curvature.
Two radiographs
could not be used because
coded
0.48 degrees,
the true angle of measurement
numbers
8.8
instead
was
estimated
*
No benefits
and in another9,
to be within
in any form
±
have
been
degrees,
received
with
95 per cent
or will be received
from
a commercial
party related directly
or indirectly
to the subject ofthis article.
No funds were received
in support
of this study.
t Scottish
Rite Children’s
Medical
Center,
lOOl Johnson
Ferry Road,
N.E.
Atlanta,
Georgia
30363.
: Emory
University.
1365 Clifton
Road,
N.E.
Atlanta,
Georgia
30322.
,
320
,
the two
viations
on the radiograph
of the measurements
measurements
were
from their respective
apparently
can result from
All radiographs
angle.
been
recorded
In addition,
more than seven
standard
demeans
and therefore
qualified
as statistical
outliers.
The original
radiographs
of quality
that
was numbered.
had
of the Cobb
were
used
to avoid
the loss
duplication.
Each radiograph
were marked
for measure-
THE JOURNAL
OF BONE
AND
JOINT
SURGERY
MEASUREMENT
OF
THE
COBB
TABLE
SET I: DIFFERENCES
WITH
BETWEEN
ANGLES
THE END-VERTEBRAE
(FREQUENCY
Ex aminer
Difference
Measured
No.
Twice
of
Per
Curves
CUMULATIVE
Exa miner
Cumulative
between
Angles
I
AND
Cent
Curves
ON
321
RADIOGRAPHS
I
MEASURED
SELECTED
TWICE
FOR EACH
BY EACH EXAMINER
CENT
PER
II
Ill
Exa miner
Cumulative
Per Cent of
Curves
No. of
Curves
EXAMINER,
DISTRIBUTIONS)
Exa miner
Cumulative
Per Cent of
Curves
No. of
Curves
of
ANGLE
IV
Total
Cumulative
Per Cent
Curves
No. of
Curves
of
0 degrees
7
15
9
19
11
23
13
27
40
20.8
I degree
17
50
13
46
16
56
17
63
63
53.6
2degrees
II
73
5
56
13
83
15
94
44
76.6
3degrees
9
92
9
75
2
88
3
100
23
88.5
4 degrees
2
96
5
85
3
94
10
93.8
Sdegrees
I
98
2
90
1
96
4
95.8
6 degrees
1
100
2
94
1
98
4
97.9
7degrees
I
96
1
100
2
99.0
8 degrees
1
98
1
99.5
I
100
1
100.0
9 degrees
Total
48
48
48
ment with a soft-lead
pencil,
and after the measurements
had been recorded
the markings
were erased with trichloracetone.
The radiographs
were measured
by four examiners
who
had various levels of experience
in managing
patients who
have scoliosis.
The participants
included
two orthopaedic
48
to analysis
and interpretation
was
descriptive
were used. The first approach
that is, it was a simple enumeration of intraobserver
differences
on successive
measurements of the Cobb angle on the forty-eight
radiographs.
The
purely
-
involved
the use of summary
calculations
to express
surgeons
who had several
years of experience,
a fellow
in
pediatric
orthopaedics,
and a senior
resident
who was rotating
through
the pediatric
orthopaedic
service.
Each examiner
measured
the radiographs
four times,
with at least
and interobserver
prediction
limits
variability
for the errors
one week between
each session.
The first set of data (Set
1) was derived from forty-eight
radiographs
that were mea-
error index, as defined by Oda et al.
each examiner
for the first set of forty-eight
sured
surements.
twice
by the four
his choices
Each
of the end-vertebrae
measurement
he had made
same
examiners.
and the Cobb
without
knowledge
one week previously.
examiner
examiner
to select
recorded
angle
for each
of the measurements
It was thus possible
different
end-vertebrae
that
for
as an estimate
tebrae
to compare
provided
the variability
vertebrae
the larger
the number,
The
second
aminer
chose
measurements
measurements
individual
Each
set of data
the same
forty-eight
the end-vertebrae
of the Cobb
were obtained
physician
examiner
the study.
(Set
carried
that
angle.
with
similarly
except
were
Both
to be used
Set-I and
the instrument
in his pocket
kept the same
instrument
and
for all
Set-Il
that
used
was used
the
daily.
for the duration
third
of the protractor,
Jntraobserver
derived
of
the
all
inthe
for the measurement.
Originally,
the study was to include
only Set-I data.
However,
after analyzing
the data and sharing
the results
with the examiners,
Set II was designed
to assess
the ob-
served problems
III was suggested
sessing
the amount
Statistical
VOL.
with the Set-I data. The inclusion
of Set
by a reviewer
as a means of further as-
72.A,
NO.
of variation
methods:
3. MARCH
Two
1990
in the
measuring
complementary
devices.
approaches
duplicate
means
ofthe
the more
mea-
with
which
in selecting
forty-eight
end-
radiographs:
the variability.
Results
that one cx-
set of data (Set III) was obtained
in
same manner as the second
set (Set II), except
that
examiners
used the same protractor.
The base-line
was
scribed
on the protractor,
and this base-line,
rather than
edge
The
II) was
radiographs,
a simple
of the ver-
angle,
a so-called
was computed
for
of each observer
for the two readings
on the
95 per cent
In addition,
in the selection
of the Cobb
,
The number
statistics
from
intraobserver
and to provide
in measurement.
of the variability
for the measurements
same radiograph.
from
of
192
second approach
analysis-of-variance
the
Cumulative
Per Cent
Curves
No. of
Curves
Variability
Table
I shows
the pairs
of angles
the
examiners.
seven
four
of the
the distribution
that
were
For
forty-eight
of differences
measured
example,
curves
in Set
Examiner
with
perfect
between
I for each
of
I measured
agreement
(0
degrees
of difference
between
successive
measurements)
and measured
seventeen
with 1 degree
of difference,
and
so on. The column
for cumulative
percentage
in Table
I
indicates that, for Examiner
I, 92 per cent of the differences
were 3 degrees or less and 98 per cent were 5 degrees
or
less. For Examiners
II, III, and IV, 75, 88, and 100 per
cent, respectively,
of all pairs differed by 3 degrees or less.
Over-all,
88.5 per cent of all pairs differed
by 3 degrees or
less and 95.8 per cent differed
by S degrees or less. Conversely,
1 1 .5 per cent of all pairs differed
by more than 3
degrees
and 4.2 per cent differed
by more than 5 degrees.
Differences
were as large as 9 degrees.
The Set-Il data (Table
Ii), for which
the selections
of
the cephalad
and caudad
vertebrae
were fixed,
showed
some
322
R. 1.
MORRISSY
TABLE
l1 DIFFERENCES
SET
WITH
BETWEEN
THE
Examiner
Difference
between
Angles
Measured
Twice
No. of
Curves
Odegrees
I
AND
MEASURED
PER
11
FOR EACH
BY ONE
CENT
EXAMINER,
EXAMINER
DISTRIBUTIONS)
Examiner
Cumulative
Per Cent of
Curves
No. of
Curves
TWICE
SELECTED
CUMULATIVE
Examiner
Cumulative
Per Cent of
Curves
II
ANGLES
END-VERTEBRAE
(FREQUENCY
ET AL.
III
Examiner
Cumulative
Per Cent of
Curves
No. of
Curves
IV
Total
Cumulative
Per Cent of
Curves
No. of
Curves
Cumulative
Per Cent of
Curves
No. of
Curves
6
17
9
19
13
27
24
50
54
28.1
I degree
15
48
19
58
21
71
18
88
73
66.1
2 degrees
10
69
15
90
7
85
5
98
37
85.4
3 degrees
10
90
4
98
5
96
1
100
20
95.8
4 degrees
2
94
1
100
3
97.4
5 degrees
2
98
3
99.0
1
99
1
100
6 degrees
7 degrees
8 degrees
9
1
degrees
Total
48
in intraobserver
variability;
48
95.8 per
over-all,
by more than 5 degrees.
Again,
as 9 degrees.
Except
for Examiner
differed
as large
server
variability,
as expressed
by such
were
differences
I, intraob-
percentile
calcula-
UsIN;
THE
DIFFERENCES
SAME
BETWEEN
PROTRACTOR,
(FREQUENCY
Examiner
Difference
Angles
between
I
AND
Examiner
Measured
Twice
No. of
Curves
0 degrees
Per
Cent
Curves
of
No. of
Curves
IV indicated
similar.
The
a more
smaller
consistent
measurements
for Examiner
indices
that
of Examiners
index
choice
I,
for Ex-
of end-ver-
of the Cobb
IV is also
angle.
This
reflected
in
the four examiners,
III
END-VERTEBRAE
PER
II
TWICE
FOR EACH
SELECTED
CENT
Ill
EXAMINER
Examiner
Cumulative
Per Cent
Curves
No. of
Curves
EXAMINER,
BY ONE
DISTRIBUTIONS)
Examiner
of
error
I and II, which show that, among
THE
of
No. of
Curves
IV
Total
Cumulative
Per Cent
Curves
of
Cumulative
Per Cent
Curves
No. of
Curves
9
19
24
50
9
19
19
40
61
69
17
85
23
67
23
88
87
77
2degrees
7
83
7
100
8
83
6
100
28
92
3degrecs
7
98
98
4 degrees
5 degrees
I
Total
tions,
caudad
between
data
agreement
Set-I
(Table
vertebrae
used,
48
were
showed
and
Ill),
Set-Il
measurements.
for which
the cephalad
pre-selected
that,
between
and the same
compared
the duplicate
iners
I and II was improved,
IV was not changed
much.
13
100
2
99
1
100
with
Set-Il
measurements
the
(see Appendix).
was the choice
is a method
end-vertebrae
192
variability
in the measurements
angle.
protractor
data,
the
Mean
Ill and
Angle
The mean
measured
angle
was the mean
by one examiner
of the angles
that were
on all of the radiographs
for one
set of data. The over-all
mean angle differed
(p < 0.0001)
among
the examiners
for Sets
dicating
The error index
IV had the least
of the Cobb
for Exam-
but that for Examiners
48
Examiner
and
Index
in choosing
96
2
48
ofexpressing
on
two
The larger the error index,
of end-vertebrae
between
the variability
different
of
32
6
100
48
improved
Set-Ill
Error
quite
The
measurements
24
I degree
was
aminer
MEASURED
Cumulative
Per Cent
Curves
Cumulative
III were
Tables
CUMULATIVE
radiograph.
for the Set-I
II, and
ANG[.ES
WITH
192
of the same
calculated
tebrae
for replicate
apparent
consistency
TABLE
l11
SET
48
surements
cent of all pairs differed
by 3 degrees
or less and 99 per
cent differed
by 5 degrees or less. Conversely,
4.2 per cent
of all pairs differed
by more than 3 degrees
and only 1 per
cent
were
99.5
100.0
100
48
improvement
1
1
occasions
the more variable
successive
mea-
that some
curves
as being
iners.
The
different
was
that
only
examiners
larger
pair
consistently
or smaller
of
means
than
that
tended
did
was
significantly
I and II, into measure
the other
not
exam-
significantly
(0.05, pairwise
Tukey
multiple-comparison
tests)
of Examiner
II and Examiner
III in Set I. The four
ThE JOURNAL
OF BONE
AND
JOINT
SURGERY
MEASUREMENT
OF THE COBB
TABLE
1 ERROR
SET
INDICES,
NINETY-FIVE
WITH
Characteristics
THE
PER
Estimated*
PREDICTION
SELECTED
Examiner
Cobb
angle
5: standard
deviation
of the Cobb
angle2
(degrees)
of a measurement
(degrees)
deviation
of the difference
two successive
replicate
measurements
of the Cobb
angl&
(degrees)
95% prediction
between
two
measurements
See
limit for the difference
successive
replicate
of the Cobb angle’
(degrees)
Appendix
t Based
examiners
I recorded
Examiner
In the
for
footnotes
on a pooled
analysis
and
additional
of variance
angles
were
and
VI).
four
more
standard
For example,
II: OVER-ALL
Characteristics
Over-all
mean
Cobb
in the Set-I
STANDARD
1 .4
2.4
4.5
6.7
4.6
3.0
4.9
examiners.
similar,
al-
of a mea-
data,
for which
THE
END-VERTEBRAE
5: standard
deviation
of the Cobb
angle2
DEVIATIONS,
Estimated8
angle
two
measurements
See
t Based
Examiner
I
(degrees)
30.6
of a measurement
(degrees)
VOL.
72-A,
limit for the difference
successive
replicate
of the Cobb
angle5 (degrees)
Appendix
for
footnotes
on a pooled
analysis
NO.
each examiner
selected
surement
of the curve,
ference
3, MARCH
1990
and
additional
of variance
Examiner
four
the
II would
the end-vertebrae
it could
successive
exceed
for each
mea-
be predicted
that the
replicate
measurements
6.7
degrees
time due to an error in measurement.
S per cent
However,
difof
of the
in the Set-
II data, for which the end-vertebrae
were pre-selected,
the
95 per cent prediction
limit was reduced to 3.4 degrees
for
Examiner
II.
Except for Examiner
I, all examiners
had some improvement
(reduction)
in the prediction
limit between
Set I
Set
II. Errors
graphs
yielded
9 degrees.
in the
measurement
differences
Over-all,
of the
in the Cobb
approximately
angle
5
same
radio-
of as much
per cent
as
of the dif-
ferences
exceeded
4.9 degrees
when each examiner
selected
the end-vertebrae
for the measurements
of the curve
and
exceeded
3 .8 degrees
when the same end-vertebrae
were
used by all. In Set III, the prediction
limit improved
examiners
and, over-all,
approximately
5 per cent
differences
exceeded
Intraobserver
pressed
ments
2.8
degrees.
variability
could
in terms
of the percentage
that differed
by S degrees
an assumption
that error
for all
of the
be
alternatively
cx-
of duplicate
measureor more,
on the basis of
in the measurements
has a normal
V
PER
EXAMINER
Examiner
26.8
CENT
AND
II
USED
PREDICTION
LIMITS
FOR ALL
Examiner
27.9
FOR
EACH
EXAMINER,
MEASUREMENTS
III
Examiner
29. 1
IV
Alit
28.6
1.8
1.2
1.3
0.7
1.3
2.5
1 .7
1 .9
1 .0
1.9
5. 1
3.4
3.9
2.0
3.8
explanation.
of the
between
NINETY-FIVE
BY ONE
deviation
of the difference
two successive
replicate
measurements
of the Cobb
angle3 (degrees)
between
AND
SELECTED
Sd: standard
between
95% prediction
28.6
2.3
deviation
MEANS,
WITH
0.54
28.9
3.3
TABLE
SET
0.39
27.6
2.2
difference
between
two successive
replicate
of the Cobb
angle
that would
be exceeded
5 per cent of the time due to an error in
measurement.
AlIt
1 .7
surement
of the Cobb angle and the standard
deviation
of
the difference
between two measurements
of the Cobb angle
are described
in the Appendix.
The 95 per cent prediction
limit for each examiner
indicated
the
measurements
approximately
0.56
IV
explanation.
of the
considerably
The
Examiner
1 .0
and
V,
III
1 .6
The intraobserver
variability
in the measurement
of the
Cobb angle (Tables
I, II, and III) can also be expressed
in
statistical
terms involving
three closely
related quantities
IV,
Examiner
2.3
Limit
(Tables
II
MEASUREMENT
1 .6
sets.
Prediction
FOR EACH
0.63
though
they were still significantly
different
(p < 0.001).
Examiner
I recorded
the largest
mean angle in all three sets.
The other three examiners
were in relatively
close agreement
in all three
AND
EXAMINER,
27.4
had the same rank order in both sets; Examiner
the largest over-all
mean angle, followed
by
1V, Examiner
Ill, and Examiner
II.
Set-Ill data, for which the same protractor
was
the mean
DEVIATIONS,
EACH
FOR
Examiner
30.7
Sd: standard
between
*
STANDARD
LIMITS
INDEPENDENTLY
I
323
ON RADIOGRAPHS
IV
MEANS,
0.56
mean
used,
CENT
END-VERTEBRAE
Errorindex’
Over-all
*
OVER-ALL
ANGLE
examiners.
324
R. 1.
MORRISSY
TABLE
Ill:
SET
WiTH
Characteristics
Over-all
mean
Cobb
USE
OVER-ALL
OF THE
SAME
MEANS,
Estimated*
angle
S: standard
deviation
of the Cobb angle2
WITH
Examiner
(degrees)
VI
STANDARD
PROTRACTOR,
ET AL.
DEVIATIONS,
THE
I
Sd: standard
deviation
of the difference
two successive
replicate
measurements
of the Cobb
angle3 (degrees)
PREDICTION
LIMITS,
SELECTED
II
Examiner
29.8
of a measurement
(degrees)
AND
END-VERTEBRAE
BY ONE
Examiner
29.0
EXAMINER
III
Examiner
29.2
IV
Alit
28.5
29.1
1.2
0.9
1.2
0.7
1.0
1 .7
1 .0
1 .7
1 .0
1.4
3.4
2.0
3.4
2.0
2.8
between
95% prediction
between
two
measurements
*
See
limit for the difference
successive
replicate
of the Cobb
angle’
(degrees)
Appendix
t Based
(gaussian)
for footnotes
on a pooled
distribution.
was
that 4 per cent
differ
by 5 degrees
and
analysis
additional
of variance
Since
or more,
due
measuring
estimate
would
measure-
Cobb angle would differ by more than 6.3 degrees,
whether
or not the examiners
used the same protractor,
approximately 5 per cent of the time. The largest
difference
was
and from
14 degrees;
measurements
to errors
was
in the
1 per cent
Variability
each
examiner
implying
the sizes
several
Again,
distributed
in terms
measured
in each set, analysis-of-variance
vide estimates
of interobserver
pointed
out that the examiners’
significantly,
examiners.
From Set I, the gaussian
of the duplicate
ment. From Set II, the estimate
Set III, 0.04 per cent.
Interobserver
explanation.
of the four
each
radiograph
examiner-to-examiner
differences
These
differences
exceeded
that the errors
10 degrees.
in measurement
normally,
interobserver
error
of the percentage
of the angles,
could
each
were
be expressed
measured
by
two examiners,
that would differ by 5 degrees
or more.
From the Set-I data, the gaussian
estimate
was that 16 per
cent of such measurements
would differ by 5 degrees
or
more and from Set-tI and III data, that 1 1 per cent would.
twice
calculations
could also provariation.
It has already
been
over-all
mean angles differed
of the angles.
differences
assuming
in
Identifiable
could
Causes
of Error
In each instance
in which the measurements
of two
examiners
differed by more than 6 degrees,
the radiographs
were evaluated
qualitatively.
All of the curves were either
thoracolumbar
or lumbar. Also, the end-plate
of one or both
of the end-vertebrae
appeared
indistinct,
cup-shaped,
or biconcave,
due to the tilt of a vertebral
body that was in
be summarized
with calculations
that were similar to those
of Tables IV, V, and VI, but that included examiners
as an
additional
source
of variation
affecting
the 95 per cent prediction
limit.
The 95 per cent prediction
limit,
including
interobserver
variation,
was 7.2 degrees
for Set I and 6.3
degrees
for Sets II and III. Therefore,
from the Set-I data,
it could
be predicted
that if two different
examiners
each
lordosis
or kyphosis.
measured
the angle on the same radiograph
and each selected
the end-vertebrae,
the measurements
would differ by more
than 7.2 degrees
approximately
5 per cent of the time. If,
In an effort to determine
mean angles differed among
why the measurements
of the
the examiners,
the protractors
were
II were
however,
Examiner
IV, who used the protractor
on a Pedroli ruler (a
ruler on which the base-line
is inscribed),
the examiners
the same
examiners
(Set-Il
end-vertebrae
were
and
the measurements
III data),
pre-selected
,iiiiiii
for both
of the
,iiI,iIIisIIIii
iiiiiii IIIIII;
studied
iiiiiiiIiiIiIii
after
TiIjt,,
Sets
IIIIIII!j1I1IIII1 IiIItI1
lUI::i
.
.
..
‘
1.
_,g,R_
_.B
,sns,t*onaI
e
..
_____
:
Inssue*t.ssd
:rc1.
fl
.
vsOsbs
AIoiiotw#{227}
on’
.
..i’;-
I and
for
: H8RLR:
*
s
/
.
.
1
r,’
,
,1
i3
II, superimposed.
The
bottom
.
8t
8
.
FIG.
protractors
of Examiners
the difference.
Except
III11lI
‘
,
,-
The
completed.
.8’,
1*0’
#{149}1:3=!2
greater
I and
edge
ilS
S
I
of each
ruler
differs
in relation
THE
to the
JOURNAL
angles.
The
OF BONE
smaller
AND
the
JOINT
angle,
the
SURGERY
MEASUREMENT
used
standard
The
instruments
protractors,
tractors
that were
all measurements
smaller
the angle,
ofthe
for this
appears
is the
ruler
angle.
of the
COBB
at 90 degrees
The reason
protractors,
rather
than
the base-
an absolute
line
to the angles.
Therefore,
if more than the
of plastic is removed
from the bottom of the
the
ruler
is stamped,
it will
measure
ANGLE
a larger
Discussion
terpreting
have attempted
measurement.
the data
in these
to determine
Understanding
reports
is difficult,
the
and
since
acinit is
often not clear how the numbers
were derived9,
what the
numbers
refer to (standard
error of the mean or standard
deviation)’
or how the authors’
interpretation
relates
to
intraobserver
or interobserver
differences6.
,
Sevastikoglou
error
and
for the Cobb
tween
server
Bergquist
method
two examiners.
error.
Beckman
found
to be 3. 12
They
and
±
the interobserver
0.48 degrees
be-
of the two
examiners
was 4.2 ± 0.95 degrees, and the range was 1 to 10 degrees.
Beckman
and Hall also did not study the intraobserver
error.
Gross et al. studied
the errors
in measurement
of three
examiners
who
measured
each
magnitude
of the curves
ranged
radiograph
five
times.
The
by the four
the end-vertebrae
measurements
tebrae
be seen
can
pre-selected
Oda
et al.
error among
radiographs
vertebrae
studied
the
intraobserver
five orthopaedic
surgeons
of scoliotic
curves twice,
in the manner
that
was
used
the Set-I data. The intraobserver
surements
of the same radiograph
The 95 per cent confidence
interval
ranged
selection
from
10.0 to 30.2
of the end-vertebrae
error in repeated
interobserver
20. 18 degrees.
and
be within
Our study
interobserver
reality.
should
angle
VOL.
degrees.
was
measurements
error among
The authors
per cent certainty
would
est source
of error.
The
examiners
the
the study
same
was
degrees
who
3, MARCH
1990
to obtain
error for duplicate
meaaveraged
12.61 degrees.
for the five orthopaedists
The variability
the largest
factor
reading
sees
in the
in the
The
averaged
was a 95
by any orthopaedist
of the true
designed
to determine
error under
circumstances
A physician
NO.
in this study
ranged
in our study
a patient
reproducibility
Similarly,
the intraobserver
that mirrored
who
has scoliosis
of the Cobb
if different
phy-
by comparing
vertebrae
of all Cobb
5 degrees.
1 per cent of
than
only
angles
varied
by more
the same
the
results
98 per cent
in our study
end-vertebrae
between
and
reason
et al. The
than
end-ver-
of Examiners
of the measurements
much
were
and 0.63
better
the
is reflected
0.39
between
for this
were
than
at se-
examiners
by the error
for the four
in
index,
examiners
0.8 and 2.4 in the study by Oda
large discrepancy
is unclear.
All
but one of the examiners
in the study of Oda et al. used a
grease marker, and the one examiner
who used a pencil had
the smallest
standard
our study
marking
were
deviation.
done
with
All of the measurements
a soft-lead
pencil
in
designed
for
radiographs.
When
the variability
Examiner
of selection
the amount
the examiners
I appeared
degrees.
than
When
of the end-vertebrae
of actual error
was relatively
to be the
least
only five of forty-eight
3 degrees
precise
replicate
and
the end-vertebrae
in the measuresmall.
Although
only
one,
were
with
the
pro-
measurements
by more
constant
varthan
5
for all cx-
aminers,
the error was 6.3 degrees,
but when each examiner
was permitted
to select the end-vertebrae,
the error was 7.2
degrees.
Both of these
intraobserver
error. This
vertebrae
is the single
and
multiple
that
error
in addition
values
are larger
than the largest
confirms
that the selection
of end-
largest
examiners
source
of intraobserver
introduce
to the intraobserver
another
error
error,
source
of
and the selection
of end-vertebrae.
In actual clinical
practice,
curves may change,
and
additional
vertebrae
may be added to the curve. However,
this
study
lishing
angle.
by more
constant,
of selecting
by Oda et al. This
ied by more
who measured
fifty
selecting
the end-
of the same radiograph.
that a single
8.8
interobserver
the five orthopaedists
calculated
that there
know the precision
and
that he or she measures.
72-A,
and
importance
and constant,
tractor,
1 .0 degrees.
succes-
of Examiner
II were within 3 degrees
of each other, illustrating that selection
of different
end-vertebrae
was the larg-
used
±
be measuring
II and IV. When each examiner selected the end-vertebrae,
only 75 per cent of the duplicate
measurements
of Examiner
II were within 3 degrees of each other, while for Examiner
IV the value was 100 per cent. When the end-vertebrae
were
was eliminated,
ments among
was 2.3
varied
were
of all Cobb
The
from 4.5 to 106.5
.
examiners
5 degrees.
degrees.
The 95 per cent confidence
limit for the intraobserver
error
of the three examiners
was reported
to be between
± 2 and
± 4 degrees
The interobserver
error when each examiner
five measurements
will
I and II. When selection
ofdifferent
4.2 per cent of the measurements
When
which
the measurements
residents)
angles
measurements
of radiographs
of curves
that were between
25 degrees.
Two physicians
each selected
the endvertebrae
and measured
the radiographs
once.
The mean
between
example,
paring Tables
was permitted,
lecting
difference
(for
radiographs
of different
patients.
In the current study, the improvement
that was obtained
when the end-vertebrae
were constant
can be seen by com-
did not quantitate
the intraobHall studied
a series
of single
5 and
325
RADIOGRAPHS
sive radiographs
of a patient,
this interobserver
error should
be known.
In research
studies,
it is important
to know how
much error is introduced
when different
examiners
measure
the
Several
authors
curacy
of the Cobb
ON
sicians
the pro-
I and II, whose
overdiffered
the most. The
in some
ruler
THE
companies.
1 shows
are superimposed.
to be that,
bottom
when
Figure
the larger the error, while
two protractors
that is related
correct amount
by various
widely.
used by Examiners
of the mean angles
the lines
line
distributed
varied
OF
ducibility
of
intraobserver
the true severity
of the examiners’
error
was
of the curve,
measurements.
not
aimed
at estab-
but only
the repro-
It is of interest
that the selection
of end-vertebrae,
something
that might be
assumed to need clinical judgment,
was not correlated
with
the level of experience
of the examiner.
This was true also
for the intraobserver
error.
326
R. T. MORRISSY
All
of these
end-vertebrae
radiographs
findings
of the curve
again
if the
changed,
and to have
radiographs
emphasize
the
accurately,
end-vertebrae
to measure
appear
one person
whenever
need
measure
to note
the
previous
to have
all of a patient’s
possible.
One additional
factor must be considered
when evaluating each set of data and the improvement
over the three
sets of data. All examiners
were aware
of the purpose
their measurements
and of the results
of their performance
after each set and before
performing
the measurements
the next set. The conversation
among
the examiners
each set was analyzed,
and there is no doubt that some
sensitized
to their
among
themselves
the study does
that the results
also
shows
performance
to improve
that
precision
precision.
can
by the protractor,
the interobserver
has a clearly
cut
base-line
edge
is used
Protractors
curate, vary from one to another,
and result
measurements,
as was the case for Examiner
It is important
only
intrinsic
made
to emphasize
error.
at different
introduction
sources
times
are
of extrinsic
are
the
radiographic
this
measured.
of the
patient,
the
high
allows
for the
the three
main
of the
the radiograph
less than
will
3 degrees.
be worse,
precision.
Some
and
In critical
measurements
with no extrinsic
error,
the error in measurement
examiners
each
may
situations,
on each
radiograph
and some
his or her own
the average
of two
is a method
or more
of reducing
In patients
who have scoliosis,
decisions
for treatment
especially
those involving
an operation
are usually
not urgent.
This provides
the opportunity
to determine
the
change
in the Cobb angle among
three radiographs
that have
been made three to four months
apart,
instead
of between
only
two radiographs.
Three
successive
radiographs
on
which
-
different
the Cobb
meaning
angle
than
is 23, 26, and 29 degrees
three
in which
it is 23,
have a far
28,
1/2
and
end-vertebrae
angle.
Note
cephalad
choices
for successive
that an examiner
and caudad
of
meawho
vertebrae
for
are independently
U2)2
-
+
selected,
the quantity
(L1L2)2]112
distance
between
the two points (U1,L,)
and
(U2,L2) plotted on a plane with “upper”
(U) and “lower”
(L) axes.
2. 5 is the square root of the within-pairs
mean square
from a one-way
forty-eight
pairs
standard
deviation
analysis
of variance
of each examiner’s
of measurements.
It is an estimate
of the
of replicate
measurements
of the Cobb
expresses
the
examiner.
measurements
of a given
timated
standard
deviation
largest
standard
deviation
was
associated
with
intrinsic
error
For example,
the
in the
measure-
in Set I, multiple
angle by Examiner
II had an esof 2.3 degrees.
This was the
among
the four examiners,
and
largest
error
index.
Examiner
IV
had the smallest
standard
deviation
and the smallest
error
index. Except for Examiner
I, the standard
deviations
were
smaller in Set-Il measurements
than in Set-I measurements.
Set-Ill standard
deviations
were the smallest
for all examiners.
3. For two independent
successive
measurements
of
the Cobb
angle
difference
of the same
radiograph,
the variance
of the
between
the two measurements
is twice the vanance of a single measurement
that is, if S2 is the estimated
variance of a single measurement,
2S2 is the estimated
vari-
error.
-
L2)2]
-
is the euclidian
it
there is a
will be
will be better
determine
(L,
+
and L2 are the first and second
[(U
which
Our data indicate that, under the best of circumstances,
a careful examiner
uses a lead pencil and measures
end-vertebrae,
cent chance
that
U2)2
the same
the end-vertebrae
for each
when
same
may not be
the instruments
equals
-
,
selected
ment
position
radiographs
Finally,
will
of
successive
measurements
would have an error index of zero.
For any given pair of successive
measurements
for which
angle,
made5”’2.
95 per
U1 ,U,,L,
quantitated
and the time of day when
index
.
two radiographs
This
for which
in
was
the
1 The error
the cephalad
and caudad
surements
of the Cobb
be mac-
in falsely
I.
study
setting,
error,
position
tube,
that
In the clinical
successive
progression.
angles
5 degrees
of
that are used for drawing
on the radiographs
and for measurement
should be accurate
and, in multicenter
studies,
standardized.
where
aware-
may
criterion
sense,
be taken to select
A protractor
that
base-line
of the measured
usual
48
was at first thought
is essential.
as the
The
[(U,
however,
was not sufficient
to
error when the second decimal was
inscribed
the
between
two
to determine
in
it is likely
Our study
through
rounded
off. Nevertheless,
care should
an accurate
instrument
for measurement.
degrees.
after
were
idiosyncrasies
in the techniques
for
it is likely that it was due to the
the protractors.
The error that was intro-
among
change
adequate
always
angles
in 5 per cent
6.3
competed
In this
be improved
practice.
in the mean
the difference
exceed
Appendix
However,
variations
which
they
AL.
of
to consistent
measurement.
duced
change
their
that
not simulate
a real-life
situation;
in clinical
practice
are less precise.
ness and, perhaps,
The variation
to be due
and
ET
25
degrees.
If more than one examiner
make the measurements
in
a clinical
study,
the interobserver
error
should
be determined,
because
even under
the most ideal circumstances
ance
of the difference
the standard
between
two
measurements.
Hence,
deviation
of the difference
between
two successive
measurements
is (2S2)#{189} or 5d’ which
expresses
the
variability
of differences
between
two successive
measurements of the Cobb angle on the same radiograph.
In other
words,
if the directional
differences
(first
measurement
minus
second
measurement)
are recorded
for all pairs of
measurements
of the Cobb angle,
these quantities
provide
an estimate
of the standard
deviations
4. Ninety-five
per cent prediction
from
of such
limits
differences.
are obtained
the formula
THE JOURNAL
OF BONE
AND
JOINT
SURGERY
MEASUREMENT
I
±
P’d
where
P’d
between
ments
5
known
t1
measurement,
a2.n
Cobb
pairs
(that
first
ON
difference
replicate
minus
of pairs
measuresecond
of mea-
freedom
particular
(for a =
example,
the formula
successive
grees
327
RADIOGRAPHS
-
is, the average
number
ANGLE
surements
(forty-eight
I a/2 fractile of a
]
measurement
n is the
COBB
1 1 1/2
+
of successive
angle,
is zero),
L1
Sd
-
to be zero
all possible
of the
-
OF THE
reduces
replicate
approximately
in this
t
0.05
study),
distribution
and n
and
t,
with
=
48,
t
n
=
-
a2n
‘
1 degrees
2.01).
For
the
of
this
symbol
in
to 2.03 5d#{149}Hence, differences
between
measurements
will exceed
2.03 5d dethe
expression
S per cent
after
the
±
of the time.
References
,
,
I . BEEKMAN,
C. E. and HALL, VIVIAN:
Variability
of Scoliosis
Measurement
from
Spinal
Roentgenograms.
Phys.
Ther.
59: 764-765,
1979.
2. BROOKS, H. L.; AZEN,
S. P.: GERBERG,
E.; BROOKS, R.; and CHAN, L.: Scoliosis:
A Prospective
Epidemiological
Study.
J. Bone and Joint Surg.,
57-A: 968-972, Oct. 1975.
3. BROWN,
J. C.; AXELGAARD,
JENS;
and HOWSON,
D. C.: Multicenter
Trial of a Noninvasive
Stimulation
Method
for Idiopathic
Scoliosis:
A
Summary
of Early Treatment
Results.
Spine,
9: 382-387,
1984.
4. COBB,
J. R. : Outline
for the Study of Scoliosis.
In Instructional
Course
Lectures,
The American
Academy
of Orthopaedic
Surgeons.
Vol. 5, pp.
261-275.
Ann Arbor,
J. W. Edwards,
1948.
5. DAWSON,
E. G. ; SMITH,
R. K. ; and MCNIECE,
G. M. : Radiographic
Evaluation
of Scoliosis.
A Reassessment
and Introduction
of the Scoliosis
Chariot. Clin. Orthop., 131: 151-155, 1978.
6. GROSS, CLIFFORD;
GROSS,
MICHAEL;
and KUSCHNER,
STUART:
Error Analysis
of Scoliosis
Curvature
Measurement.
Bull. Hosp.
Joint Dis. Orthop.
Inst.,
43: 171-177,
1983.
7. LONSTEIN,
J. E. and CARLSON,
J. M.: The Prediction
of Curve Progression
in Untreated
Idiopathic
Scoliosis
during Growth.
J. Bone and Joint
Surg.,
66.A: 1061-1071,
Sept.
1984.
8. MCCOLLOUGH.
N. C., III: Nonoperative
Treatment
of Idiopathic
Scoliosis
Using Surface
Electrical
Stimulation.
Spine,
II: 802-804,
1986.
9. ODA, MARJORIE;
RAUH,
STEPHEN:
GREGORY,
P. B. ; SILVERMAN,
F. N. ; and BLECK,
E. E. : The Significance
of Roentgenographic
Measurement
in Scoliosis.
J. Pediat.
Orthop.
2: 378-382,
1982.
10. ROGALA,
E. J.; DRUMMOND,
D. S.; and GURR.
JEAN:
Scoliosis:
Incidence
and Natural
History.
A Prospective
Epidemiological
Study.
J. Bone
and Joint
Surg.
60-A:
173-176,
March
1978.
1 1 . SEVAST1KOGLOU,
J. A. and BERGQUIST,
E. : Evaluation
of the Reliability
of Radiological
Methods
for Registration
of Scoliosis.
Acta Orthop.
Scandinavica,
40: 608-613,
1969.
12. ZETTERBERG.
C.; HANSSON,
T.; LIOSTROM,
J.: IRSTRAM,
L.; and ANDERSSON,
G.: Daytime
Postural
Changes
ofthe
Scoliotic
Spine.
Orthop.
Trans.,
7: 7-8, 1983.
,
,
,
VOL.
72.A,
NO.
3. MARCH
,
1990