DEFINITION OF IMPROVEMENT IN JA
$57+5,7,65+(80$7,60
9RO1R-XO\SS
‹$PHULFDQ&ROOHJHRI5KHXPDWRORJ\
PRELIMINARY DEFINITION OF IMPROVEMENT IN
JUVENILE ARTHRITIS
(':$5'+*,$11,1,1,&2/,12583(572$1*(/25$9(//,
'$1,(/-/29(//'$9,'7)(/621DQG$/%(5720$57,1,
Objective. To identify a core set of outcome
variables for the assessment of children with juvenile
arthritis (JA), to use the core set to develop a
definition of improvement to determine whether
individual patients demonstrate clinically important
improvement, and to promote this definition as a
single efficacy measure in JA clinical trials by the
kappa statistic.
Methods. A core set of outcome variables
wasestablished using a combination of statistical and
consensus formation techniques. Variables in the core
set consisted of 1) physician global assessment of
disease activity; 2) parent/patient assessment of overall
well-being; 3) functional ability; 4) number of joints
with active arthritis; 5) number of joints with limited
range of motion; and 6) erythrocyte sedimentation
rate. To establish a definition of improvement using
this core set, 21 pediatric rheumatologists from 14
countries met, and, using consensus formation
techniques, scored each of 72 patient profiles as
improved or not improved. Using the physicians'
consensus as the gold standard, the chi-square,
sensitivity, and specificity were calculated for each of
240 possible definitions of improve____________
3UHVHQWHGDWWKH6L[WLHWK1DWLRQDO6FLHQWLILF0HHWLQJRIWKH
$PHULFDQ&ROOHJHRI5KHXPDWRORJ\2UODQGR)/2FWREHU
6XSSRUWHGE\D&OLQLFDO6FLHQFH*UDQWIURPWKH$UWKULWLV
)RXQGDWLRQE\WKH&KLOGUHQ
V+RVSLWDO5HVHDUFK)RXQGDWLRQDQGE\
&OLQLFD3HGLDWULFD,VWLWXWRGL5LFRYHURD&XUDD&DUDWWHUH6FLHQWLILFR
3ROLFOLQLFR60DWWHR3DYLD,WDO\6XSSRUWIRUWKHPHHWLQJLQ3DYLD
,WDO\ZDVSURYLGHGE\&HQWDFRU86&LED*HQHYD3KDUPDFHXWLFDO
86*HQ'HUP&RUSRUDWLRQ86/HSHWLW,WDO\1RUGPDUN,WDO\
3IL]HU,QF866DQGR],WDO\8QLYHUVLWDGHJOL6WXGLGL3DYLD,WDO\
DQG:\HWK$\HUVW/DEV86 (GZDUG+*LDQQLQL06F'U3+'DQLHO-/RYHOO0'
03+&KLOGUHQ
V+RVSLWDO0HGLFDO&HQWHU&LQFLQQDWL2KLR1LFROLQR
5XSHUWR0'$QJHOR5DYHOOL0'$OEHUWR0DUWLQL0'8QLYHUVLW\GL
3DYLD&OLQLFD3HGLDWULFD,VWLWXWRGL5LFRYHURD&XUDD&DUDWWHUH
6FLHQWLILFR3ROLFOLQLFR60DWWHR3DYLD,WDO\'DYLG7)HOVRQ0'
03+%RVWRQ8QLYHUVLW\$UWKULWLV&HQWHU%RVWRQ0DVVDFKXVHWWV
$GGUHVVUHSULQWUHTXHVWVWR(GZDUG+*LDQQLQL06F'U3+
5KHXPDWRORJ\&KLOGUHQ
V+RVSLWDO0HGLFDO&HQWHU3DYLOLRQ%XLOGLQJ
%XUQHW$YHQXH&LQFLQQDWL2+
6XEPLWWHGIRUSXEOLFDWLRQ$XJXVWDFFHSWHGLQ
UHYLVHGIRUP)HEUXDU\
ment. Definitions with sensitivity or specificity of
<80% were eliminated. The ability of the
remaining definitions to discriminate between the
effects of active agent and those of placebo, using
actual trial data, was then observed. Each
definition was also ranked for face validity, and
the sum of the ranks was then multiplied
Results. The definition of improvement
with the highest final score was as follows: at least
30% improvement from baseline in 3 of any 6
variables in the core set, with no more than 1 of
the remaining variables worsening by >30%. The
second highest scoring definition was closely
related to the first; the third highest was similar to
the Paulus criteria used in adult rheumatoid
arthritis trials, except with different variables.
Thisindicates convergent validity of the process
used.
Conclusion. We propose a definition of
improvement for JA. Use of a uniform definition
will help standardize the conduct and reporting of
clinical trials, and should help practitioners decide
if a child with JA has responded adequately to
therapy. We are in the process of prospectively
validating this definition and several others that
scored highly.
7KH DVVHVVPHQW RI FOLQLFDO UHVSRQVH LQ
MXYHQLOH DUWKULWLV -$ FOLQLFDO WULDOV LV QRW
VWDQGDUGL]HG0XOWLSOHPHDVXUHVRIRXWFRPHDUHLQ
XVH DQG GLIIHUHQW WULDOV PD\ XVH GLIIHUHQW HQG
SRLQWV6RPHRIWKHVHHQGSRLQWVKDYHORZYDOLGLW\
FKDUDFWHULVWLFV DQG DUH LQVHQVLWLYH WR FKDQJH VRPH DUH UHGXQGDQW DQG VRPH DUH QRQUHOLDEOH
SRRU UHSURGXFLELOLW\ $GGLWLRQDOO\ WKHUH LV
OLWWOHFRQVHQVXVDERXWWKHDPRXQWRIFKDQJHLQHQG
SRLQWV ZKLFK VLJQLILHV FOLQLFDOO\ LPSRUWDQW
LPSURYHPHQW RU ZRUVHQLQJ 7KLV ODFN RI
VWDQGDUGL]DWLRQ PD\ OHDG WR LQHIILFLHQW WULDOV WKDW
UHTXLUH ODUJHUWKDQQHFHVVDU\ VDPSOH VL]HV DQ
LQFUHDVHGULVNRIVWDWLVWLFDOHUURUSRVVLEOHUHSRUWLQJ
ELDV PXOWLSOH RU DPELJXRXV LQWHUSUHWDWLRQV RI
UHVXOWV DQG DQ LQDELOLW\ WR FRPSDUH PXOWLSOH
WKHUDSLHVXVLQJPHWDDQDO\VLVWHFKQLTXHV
DEFINITION OF IMPROVEMENT IN JA
$ VLPLODU VLWXDWLRQ LQ DGXOW UKHXPDWRLG DUWKULWLV
5$ OHG WR WKH GHYHORSPHQW RI D FRUH VHW RI RXWFRPH
YDULDEOHV DQG D SUHOLPLQDU\ GHILQLWLRQ RI LPSURYHPHQW
7KLV GHILQLWLRQ RI LPSURYHPHQW LQ DGXOW 5$ LV QRW
DSSURSULDWH IRU XVH LQ FKLOGUHQ ZLWK -$ EHFDXVH RI
VHYHUDO IDFWRUV -$ LV FRQVLGHUHG D GLIIHUHQW GLVHDVH
HQWLW\ VRPH FRUH YDULDEOHV DUH OHVV RIWHQ DEQRUPDO RU
KDYH ORZHU VFRUHV LQ FKLOGUHQ WKDQ LQ DGXOWV DQG WKHLU
PHDVXUHPHQW LV FRPSURPLVHG GXH WR DJHUHODWHG FRJQL
WLYHSUREOHPVHJVHOIUHSRUWHGSDLQ
7KH SXUSRVH RI WKLV SURMHFW ZDV WR GHYHORS DQG
SURPXOJDWH D FRUH VHW RI HQG SRLQWV WKDW FDQ EH XVHG LQ
IXWXUH FOLQLFDO WULDOV LQ FKLOGUHQ ZLWK -$ WR GHVFULEH WKH
DPRXQW RI FKDQJH LQ HDFK YDULDEOH WKDW LV FRQVLGHUHG
FOLQLFDOO\ LPSRUWDQW DQG WR XVH WKH HQWLUH FRUH VHW WR
GHYHORS D GHILQLWLRQ RI LPSURYHPHQW WR DLG LQ WKH
FODVVLILFDWLRQRILQGLYLGXDOSDWLHQWVDVHLWKHULPSURYHGRU
QRW LPSURYHG 7KH ORQJWHUP JRDOV DUH WR LQFUHDVH WKH
HIILFDF\ RI -$ FOLQLFDO WULDOV DQG IDFLOLWDWH IXWXUH
PHWDDQDO\VHV RI WKHUDSLHV IRU -$ :H DQWLFLSDWHG WKDW
WKH GHILQLWLRQ RI LPSURYHPHQW PLJKW DOVR EH XVHIXO WR
SK\VLFLDQV ZKHQ DVVHVVLQJ SDWLHQW LPSURYHPHQW LQ URX
WLQH SUDFWLFH :H XVHG DQ DSSURDFK VLPLODU WR WKDW
GHYHORSHG E\ WKH 2XWFRPH 0HDVXUHV LQ 5KHXPDWRLG
$UWKULWLV &OLQLFDO 7ULDOV 20(5$&7 SURMHFW WKDW OHG WR
WKH GHYHORSPHQW RI WKH $PHULFDQ &ROOHJH RI 5KHXPD
WRORJ\ $&5 FRUH VHW DQG GHILQLWLRQ RI LPSURYHPHQW
XVHG LQ DGXOW 5$ 7KH 20(5$&7 FRUH VHW KDV
QRZ EHHQ HQGRUVHG E\ WKH :RUOG +HDOWK 2UJDQL]DWLRQ
DQG,QWHUQDWLRQDO/HDJXH$JDLQVW5KHXPDWLVP
PATIENTS AND METHODS
$ PXOWLVWHS SURFHVV ZDV XVHG LQ GHYHORSLQJ WKH -$
FRUH VHW RI YDULDEOHV DQG GHILQLWLRQ RI LPSURYHPHQW 7KLV
SURFHVVLVGHVFULEHGEHORZDQGLVVXPPDUL]HGLQ)LJXUH
Selection of the preliminary core set of response
variables. ,QDPHPEHU$GYLVRU\&RXQFLOZDVIRUPHG
FRQVLVWLQJ RI PHPEHUV RI WKH 5KHXPDWRORJ\ 6HFWLRQ RI WKH
$PHULFDQ $FDGHP\ RI 3HGLDWULFV WKH 3HGLDWULF 6HFWLRQ RI WKH
$&5 DQG WKH $UWKULWLV )RXQGDWLRQ 20(5$&7 SDUWLFL
SDQWV DQG SULYDWH DQG DFDGHPLF SUDFWLWLRQHUV 3ULRU WR
FRQYHQLQJ DV PHHWLQJ RI WKLV FRPPLWWHH D EULHI TXHVWLRQQDLUH
ZDV PDLOHG WR HDFK PHPEHU DVNLQJ DERXW UHVSRQVH YDULDEOHV
XVHG ZKHQ DVVHVVLQJ FOLQLFDO UHVSRQVH LQ SDWLHQWV ZLWK $ 7KH
TXHVWLRQQDLUH OLVWHG YDULDEOHV WKDW KDG EHHQ XVHG LQ WKH
UHSRUWLQJ RI -$& FOLQLFDO WULDOV DQG DVNHG WKH SK\VLFLDQV WR
UDQNRUGHU WKHLU WRS FKRLFHV $Q ³RWKHU´ FDWHJRU\ ZDV
SURYLGHG WR DGG YDULDEOHV QRW LQFOXGHG LQ WKH OLVW 9DULDEOHV
ZHUH UDQNHG LQ RUGHU RI SULRULW\ YRWHV UHFHLYHG $ WRWDO RI YDULDEOHV UHFHLYHG YRWHV DQG WKHVH EHFDPH NQRZQ DV WKH
FDQGLGDWHYDULDEOHVIRULQFOXVLRQLQWKHFRUHVHW
7KHTXHVWLRQQDLUHDOVRDVNHGLILWZRXOGEHDFFHSWDEOH
WR FRPELQH D FRUHV VHW RI YDULDEOHV LQWR D VLQJOH GHILQLWLRQ RI
Creation of Advisory Council and
preliminary survey questionnaire
about useful end points.
16 ”candidate”
variables identified
of the literature and
Research
to determine the
databases
performance
characteristics of the
16 candidate
variables.
Florida conference of Advisory
Council. Presentation of data,
nominal group
technique to identify
preliminary
core set of endpoints.
Outcome Variables
Core Set of
assessment of
1. MD Global
overall disease
activity
2. Parent or patient global
assessment
of overall well-being
3. Functional
ability
4. No. of joints
with active arthritis
5. No. of joints with limited range
of motion
6. ESR Larger, international survey on the
core set and its use n=140
Development of definitions of
improvement for testing, using
survey results as a guide
International Consensus
Conference on Defining
Improvement in Juvenile Arthritis,
Pavia, Italy May 1996
Testing of definitions of
improvement for sensitivity,
specificity, false positive rates, and
face validity.
Identification of the definition of
improvement with the best
performance characteristics
At least 30% improvement in at
least 3 of the 6 core set variables
with no more than 1 remaining
variable worsening by >30%
Figure 13URFHVVRIFKRRVLQJDFRUHVHWRIRXWFRPHYDULDEOHVDQG
XVLQJWKHFRUHVHWDGHILQLWLRQRILPSURYHPHQWIRUXVHLQMXYHQLOH
DUWKULWLV(65 HU\WKURF\WHVHGLPHQWDWLRQUDWH
LPSURYHPHQW LQ RUGHU WR GLFKRWRPRXVO\ GLYLGH SDWLHQWV LQWR
WKRVH ZKR LPSURYH E\ D FOLQLFDOO\ LPSRUWDQW DPRXQW DQG WKRVH
ZKR GR QRW $ WRWDO RI RI WKH UHVSRQGHQWV LQGLFDWHG WKDW
D YDOLG GHILQLWLRQ IRU GHVLJQDWLQJ LPSURYHPHQW ZRXOG EH
DFFHSWDEOHWRWKHP
7KH SHUIRUPDQFH FKDUDFWHULVWLFV YDOLGLW\ UHOLDELOLW\
VHQVLWLYLW\WRFKDQJHUHGXQGDQF\RIDOOYDULDEOHVWKDWUHFHLYHG
YRWHV LQ WKH VXUYH\ TXHVWLRQQDLUH ZHUH LQYHVWLJDWHG XVLQJ WKH
OLWHUDWXUH DQG WKH FRUH GDWD EDQN RI WKH 3HGLDWULF 5KHXPDWRO
RJ\ &ROODERUDWLYH 6WXG\ *URXS 35&6* GHVFULEHG SUHYLRXVO\
%ULHIO\ WKLV GDWDEDQN FRQWDLQV DOO GDWD IURP DOO WULDOV RI
VHFRQGOLQHDJHQWVVWXGLHGE\WKH35&6*>Q @
,Q D FRQIHUHQFH RI WKH $GYLVRU\ &RXQFLO ZDV
FRQYHQHG LQ )ORULGD DQG WKH SHUIRUPDQFH FKDUDFWHULVWLFV RI
HDFK FDQGLGDWH YDULDEOH ZHUH UHYLHZHG IRU DWWHQGHHV E\ WKH
FRQIHUHQFH RUJDQL]HU (+* 1H[W XVLQJ FRQVHQVXV IRUPDWLRQ
WHFKQLTXHV QRPLQDO JURXS WHFKQLTXH>@ DWWHQGHHV GHYHO
RSHGDSUHOLPLQDU\FRUHVHWRIUHVSRQVHYDULDEOHV7KHFRUHVVHW
LQFOXGHG WKH IROORZLQJ HQG SRLQWV SK\VLFLDQV¶ JOREDO
DVVHVVPHQW RI RYHUDOO GLVHDVH DFWLYLW\ PHDVXUHG RQ D FP
YLVXDODQDORJVFDOH>9$6@SDUHQWRULIDSSURSULDWHLQDJH
SDWLHQW JOREDO DVVHVVPHQW RI RYHUDOO ZHOOEHLQJ PHDVXUHG RQ
D FP 9$6 IXQFWLRQDO DELOLW\ QXPEHU RI MRLQWV ZLWK
DFWLYH DUWKULWLV DV GHILQHG E\ WKH $&5 FULWHULD SUHVHQFH RI
VZHOOLQJ >QRW GXH WR FXUUHQWO\ LQDFWLYH V\QRYLWLV RU WR ERQ\
DEFINITION OF IMPROVEMENT IN JA
Table 1.&RUUHODWLRQRIGLIIHUHQWMXYHQLOHDUWKULWLVFRUHVHWYDULDEOHVFURVVVHFWLRQDOO\
2YHUDOO
)XQFWLRQDO
1RRIDFWLYH
1RRIMRLQWV
0'JOREDO
ZHOOEHLQJ
DELOLW\
MRLQWV
ZLWK/520
2YHUDOOZHOOEHLQJ
)XQFWLRQDODELOLW\W
1RRIDFWLYHMRLQWV
1RRIMRLQWVZLWK/520
(65
&RUUHODWLRQVUYDOXHV"DUHFRQVLGHUHGHYLGHQFHRIUHGXQGDQF\/520 OLPLWHGUDQJHRIPRWLRQ
(65 HU\WKURF\WHVHGLPHQWDWLRQUDWH
‚'HULYHGWKURXJKUHJUHVVLRQDQDO\VLV
HQODUJHPHQW@RULIQRVZHOOLQJLVSUHVHQWOLPLWDWLRQRIPRWLRQ
DFFRPSDQLHGE\KHDWSDLQRUWHQGHUQHVV>@QXPEHU
RI MRLQWV ZLWK OLPLWHG UDQJH RI PRWLRQ DQG HU\WKURF\WH
VHGLPHQWDWLRQUDWH(65
Ascertainment of international consensus on the
core set of variables. )ROORZLQJ WKH FRQIHUHQFH LQ )ORULGD
VHYHUDO LVVXHV DQG TXHVWLRQV UHPDLQHG WR EH DGGUHVVHG $
EURDGHU FRQVHQVXV DERXW WKH FRUH VHW DQG LWV XVH WR GHILQH
LPSURYHPHQWKDGWREHREWDLQHGIURPRWKHU1RUWK$PHULFDQ
DQG LQWHUQDWLRQDO SUDFWLWLRQHUV ,I WKH FRUH VHW ZDV WR EH
XVHGWRGHILQHLPSURYHPHQWGHFLVLRQVZRXOGKDYHWREHPDGH
UHJDUGLQJKRZPDQ\RIWKHYDULDEOHVZRXOGKDYHWRLPSURYH
DQG E\ KRZ PXFK EHIRUH SUDFWLWLRQHUV FRXOG FODVVLI\ WKH
SDWLHQWDVLPSURYHG6LQFHQRWDOOYDULDEOHVFDQEHH[SHFWHG
WRLPSURYHGHFLVLRQVZRXOGKDYHWREHPDGHUHJDUGLQJKRZ
PDQ\ YDULDEOHV FRXOG ZRUVHQ DQG E\ KRZ PXFK ZKLOH VWLOO
DOORZLQJWKHSDWLHQWWREHFODVVLILHGDVLPSURYHG
7R DGGUHVV WKHVH LVVXHV D TXHVWLRQQDLUH ZDV
GHVLJQHG DQG ZDV PDLOHG WR D ODUJHU PRUH LQWHUQDWLRQDO
VDPSOHRISUDFWLWLRQHUVWRREWDLQWKHLUUHDFWLRQWRWKHFRUHVHW
DQG LWV SURSRVHG XVH 7KH TXHVWLRQQDLUH ZDV GHVLJQHG WR
SURYLGHSUHOLPLQDU\GDWDIRURUJDQL]DWLRQRIWKH,QWHUQDWLRQDO
&RQVHQVXV &RQIHUHQFH GHVFULEHG EHORZ 7KH IROORZLQJ
PDLOLQJ OLVWV ZHUH XVHG WR VDPSOH SUDFWLWLRQHUV DOO
PHPEHUV RI WKH 35&6* DQG 3HGLDWULF 5KHXPDWRORJ\
'DWDEDVH 5HVHDUFK *URXS D FRQVRUWLXP RI SHGLDWULF
UKHXPDWRORJ\ FHQWHUV WKDW FROOHFWV GDWD RQ WKH GLDJQRVHV RI
SDWLHQWV YLVLWLQJ WKHLU FOLQLFV DOO DWWHQGHHV RI WKH 6HFRQG
(XURSHDQ,QWHUQDWLRQDO&RQJUHVVRQ3HGLDWULF5KHXPDWRORJ\
DQGRWKHUSUDFWLWLRQHUVRISHGLDWULFUKHXPDWRORJ\NQRZQWR
XVZKRZHUHQRWRWKHUZLVHRQWKHDERYHPDLOLQJOLVWV
$WRWDORITXHVWLRQQDLUHVZHUHPDLOHGDQG
ZHUHUHWXUQHGIURP(XURSHIURP1RUWK$PHU
LFD 5HVXOWV YDOLGDWHG WKH FRQFOXVLRQV UHDFKHG GXULQJ WKH
)ORULGDFRQIHUHQFH:KHQYDULDEOHVZHUHUDQNHGDFFRUGLQJWR
WKHLUSULRULW\VFRUHWKHVDPHFKRVHQLQ)ORULGDZHUHVFRUHG
KLJKHVWLQWKHODUJHUVXUYH\%DVHGRQWKHUHVXOWVLWZDVDOVR
GHWHUPLQHG WKDW WKH PHGLDQ LPSURYHPHQW WKDW VKRXOG EH
REVHUYHG LQ RUGHU WR FODVVLI\ D SDWLHQW DV LPSURYHG ZLWKLQ D
JLYHQYDULDEOHZDVIRUDOOYDULDEOHVH[FHSWIRUIXQFWLRQDO
DELOLW\ ZKLFK ZDV ,W ZDV IXUWKHU GHWHUPLQHG WKDW WKH
QXPEHU RI YDULDEOHV WKDW VKRXOG LPSURYH E\ WKH VSHFLILHG
DPRXQWLQRUGHUWRFODVVLI\DSDWLHQWDVFOLQLFDOO\LPSRUWDQWO\
LPSURYHGZDVDQGWKHQXPEHUWKDWFRXOGEHLJQRUHGLIWKH\
ZRUVHQHG ZDV!2QHKXQGUHGWZHQW\ IRXURI
WKHUHVSRQGHQWVVDLGWKDWWKH\ZRXOGEHZLOOLQJWRXVHWKH
FRUHVHWLQFRPELQDWLRQWRGHILQHLPSURYHPHQWLQLQGLYLGXDO
SDWLHQWV 'LIIHUHQFHV EHWZHHQ UHVXOWV IURP WKH 86 DQG WKH
LQWHUQDWLRQDOUHVSRQGHQWVZHUHLQFRQVHTXHQWLDO
Assessment of multicollinearity. +DYLQJ YHULILHG
WKDWWKHHQGSRLQWVLQWKHFRUHVHWZHUHDFFHSWDEOHWRDODUJH
SURSRUWLRQ RI SUDFWLWLRQHUV WKH DPRXQW RI PXOWLFROOLQHDULW\
H[LVWLQJ EHWZHHQ WKH YDULDEOHV KDG WR EH REVHUYHG LQ ODUJHU
DQG PRUH GLYHUVH GDWD VHWV )RU WKLV H[HUFLVH ZH XVHG WKH
35&6*FRUHGDWDEDQNQ DQLQFHSWLRQFRKRUWRI
FKLOGUHQ ZLWK -5$ IURP WKH &LQFLQQDWL 6SHFLDO 7UHDWPHQW
&HQWHUDQGIURP3DYLD,WDO\ZKRKDGWDNHQSDUWLQDVWXG\RI
SUHGLFWRUV RI RXWFRPH DQG D FRKRUW RI SDWLHQWV
IURP 3DYLD ,WDO\ ZKR KDG EHHQ IROORZHG XS E\ RQH RI XV
$5 DQG DVVHVVHG IRU HDFK YDULDEOH LQ WKH FRUH VHW U
YDOXHVJUHDWHUWKDQZHUHWDNHQDVHYLGHQFHRIFROOLQHDULW\
2QO\ WKH QXPEHU RI MRLQWV ZLWK OLPLWHG UDQJH RI PRWLRQ DQG
WKHQXPEHURIDFWLYHMRLQWVVKRZHGHYLGHQFHRIFROOLQHDULW\U
7DEOH 7KH GDWD EDQNV XVHG IRU WKLV H[HUFLVH KDG
UHSHDWHGPHDVXUHVIRUWKHVHYDULDEOHVDQGWKHUYDOXHIRUWKH
FKDQJH LQ WKH QXPEHU RI MRLQWVZLWK OLPLWHG UDQJH RI PRWLRQ
YHUVXVWKHFKDQJHLQWKHQXPEHURIMRLQWVZLWKDFWLYHDUWKULWLV
ZDV7KXVZHFRQFOXGHGIURPWKLVH[HUFLVHWKDWWKH
FRUH VHW YDULDEOHV ZHUH FRUUHODWHG EXW QRW H[FHVVLYHO\
UHGXQGDQW
Development and selection of a definition of
improvement%HFDXVHWKHOLWHUDWXUHFRQWDLQHGQRGHILQLWLRQV
RI LPSURYHPHQW XVLQJ FRPELQDWLRQV RI WKH YDULDEOHV LQ WKH
FRUH VHW ZH GHYHORSHG IRU WHVWLQJ D VHW RI GHILQLWLRQV
WKDWVHHPHGUHDVRQDEOHLQFRQVLGHUDWLRQRIWKHGDWD IURPWKH
LQWHUQDWLRQDOVXUYH\TXHVWLRQQDLUHGLVFXVVHGDERYH$VHFRQG
FRQIHUHQFH HQWLWOHG ,QWHUQDWLRQDO &RQVHQVXV &RQIHUHQFH RQ
'HILQLQJ ,PSURYHPHQW LQ -XYHQLOH $UWKULWLV ZDV KHOG LQ
3DYLD ,WDO\ LQ 0D\ 7KH PHHWLQJ ZDV DWWHQGHG E\ SHGLDWULF UKHXPDWRORJLVWV IURP GLIIHUHQW FRXQWULHV VHH
$FNQRZOHGJPHQWV IRU OLVW RI WKH DWWHQGHHV ZKR DUH QRW
DXWKRUVRI WKLV UHSRUW DQG ZDV IDFLOLWDWHG E\ RI XV (+*
15 ZLWK H[SHUWLVH LQ QRPLQDO JURXS SURFHVV 7KH RYHUDOO
JRDO RI WKH PHHWLQJ ZDV WR GHFLGH XSRQ D SUHOLPLQDU\
GHILQLWLRQRILPSURYHPHQWEDVHGRQWKHFRUHVHWRIHQGSRLQWV
XVLQJ D FRPELQDWLRQ RI VWDWLVWLFDO DQG FRQVHQVXV IRUPDWLRQ
WHFKQLTXHV ,Q RUGHU WR DFKLHYH WKLV WKHUH ZHUH REMHFWLYHV
ZKLFKDUHGHVFULEHGLQFRQVHFXWLYHRUGHUEHORZ
5DWH HDFK RI SDSHU SDWLHQW SURILOHV DV
FOLQLFDOO\ LPSRUWDQWO\ LPSURYHG RU QRW LPSURYHG XVLQJ
QRPLQDO JURXS WHFKQLTXH ([LVWLQJ FOLQLFDO WULDO GDWD ZHUH
XVHGIRUSDWLHQWSURILOHVSUHVHQWHGWRFRQIHUHQFHDWWHQGHHVIRU
HYDOXDWLRQ RI UHVSRQVH 7KH ,WDOLDQ 3HGLDWULF 5KHXPDWRORJ\
6WXG\ *URXSKDG FRQGXFWHG DQRSHQODEHOXQFRQWUROOHGWULDO
RIPHWKRWUH[DWH07;PJPZHHNQ 7KLVWULDO
ZDV
VHOHFWHG
DV
DEFINITION OF IMPROVEMENT IN JA
DVRXUFHRISDWLHQWVEHFDXVHLWZDVWKHRQO\RQHNQRZQWRXV
WKDWKDGPHDVXUHGHDFKRIWKHFRUHVHWYDULDEOHV7KHSURILOHV
HOHFWHGZHUHWKRVHQHDUWKHWKUHVKROGOHYHORILPSURYHPHQWDV
GHWHUPLQHGE\WKHLQWHUQDWLRQDOVXUYH\GHVFULEHGDERYHHJ
SDWLHQWV ZKR VKRZHG LPSURYHPHQW LQ DOO RXWFRPH
YDULDEOHV ZHUH QRW JRRG FDQGLGDWHV IRU LQFOXVLRQ EHFDXVH
HYHU\RQHZRXOGDJUHHWKDWWKHSDWLHQWKDGLPSURYHGDQGDOO
WKH GHILQLWLRQV RI LPSURYHPHQW ZRXOG FDWHJRUL]H WKH SDWLHQW
DV LPSURYHG 7KH -XYHQLOH $UWKULWLV )XQFWLRQDO $VVHVVPHQW
5HSRUW KDG EHHQ XVHG WR DVVHVV IXQFWLRQDO DELOLW\ )RU
HDFK FRUH VHW YDULDEOH DEVROXWH YDOXHV DW EDVHOLQH DQG PRQWKV ZHUH VKRZQ DV ZHOO DV WKH DEVROXWH GLIIHUHQFH DQG
SHUFHQW FKDQJH IURP EDVHOLQH 3DUWLFLSDQWV ZHUH UDQGRPL]HG
LQWR HTXDOO\ VL]HG QRPLQDO JURXSV DQG DVNHG WR VLOHQWO\ UDWH
HDFKRISDWLHQWSURILOHVDVFOLQLFDOO\LPSRUWDQWO\LPSURYHG
RUQRWLPSURYHG7KHPRGHUDWRUWKHQDVNHGHDFKPHPEHUKRZ
KH RU VKH KDG YRWHG RQ HDFK SDWLHQW ,I DQ FRQVHQVXV
DERXWZKHWKHUWKHSDWLHQWZDVLPSURYHGRUQRWLPSURYHGZDV
QRW DFKLHYHG WKH FDVH ZDV GLVFXVVHG LQ URXQGURELQ IDVKLRQ
DQG D VHFRQG YRWH WDNHQ ,I FRQVHQVXV ZDV VWLOO QRW
DWWDLQHG WKH SDWLHQW SURILOH ZDV GHFODUHG XQLQWHUSUHWDEOH DQG
QRWXVHGIXUWKHULQWKHQRPLQDOJURXS$SOHQDU\VHVVLRQZDV
WKHQ KHOG LQ DQ DWWHPSW WR UHVROYH WKRVH FDVHV VFRUHG
GLVFRUGDQWO\E\WKHJURXSV
2. Using the physicians' consensus judgment as the
gold standard, calculate the percent false positive and
false-negative rates, chi-square, sensitivity, and specificity for
each definition of improvement. :H HYDOXDWHG WKH DELOLW\ RI
WKH FDQGLGDWH GHILQLWLRQV RI LPSURYHPHQW WR FODVVLI\
LQGLYLGXDO SDWLHQWV DV LPSURYHG RU QRW LPSURYHG DQG WKHQ
DVVHVVHGWKHDJUHHPHQWEHWZHHQWKHGHFLVLRQRIWKHFULWHULD
DQG WKH FRQVHQVXV RI WKH SK\VLFLDQV :H XVHG RQO\ SDWLHQW
SURILOHV IRU ZKLFK SK\VLFLDQ FRQVHQVXV ZDV DFKLHYHG )RU
HDFK GHILQLWLRQ ZH FDOFXODWHG WKH FKLVTXDUH GHJUHH RI
IUHHGRPDQGWKHFRUUHVSRQGLQJ3YDOXHVHQVLWLYLW\DELOLW\RI
WKHGHILQLWLRQWRLGHQWLI\DSDWLHQWDVLPSURYHGZKRKDGEHHQ
FODVVLILHG DV LPSURYHG E\ WKH SK\VLFLDQV VSHFLILFLW\ DELOLW\
RIWKHGHILQLWLRQWRLGHQWLI\DSDWLHQWDVQRWLPSURYHGZKRKDG
EHHQ FODVVLILHG DV QRW LPSURYHG E\ WKH SK\VLFLDQV UDWH RI
IDOVHSRVLWLYLW\ >QXPEHU IDOVHO\ LGHQWLILHG DV LPSURYHG E\
FULWHULDDOOSDWLHQWVLGHQWLILHGDVLPSURYHG@[DQGUDWHRI
IDOVHQHJDWLYLW\ >QXPEHU IDOVHO\ LGHQWLILHG DV QRW LPSURYHG
E\WKHFULWHULDDOO3DWLHQWVLGHQWLILHGDVQRWLPSURYHG@[
7KRVHGHILQLWLRQVRILPSURYHPHQWVKRZLQJHLWKHUDVHQVLWLYLW\
RU VSHFLILFLW\ RI ZHUH HOLPLQDWHG IURP IXUWKHU
FRQVLGHUDWLRQ 2Q WKH QH[W GD\ WKH UHVXOWV RI WKH VWDWLVWLFDO
H[HUFLVHVZHUHSUHVHQWHGWRWKHJURXSRISK\VLFLDQV
3.Observe the ability of the remaining definitions of
improvement to discriminate between active agent and
placebo using existing trial data. 7KLV SKDVH RI WKH H[HUFLVH
ZDVYHU\OLPLWHGLQVFRSHEHFDXVHSODFHERFRQWUROOHGFOLQLFDO
WULDOGDWDEDVHVLQZKLFKDOORIWKHFRUHVHWYDULDEOHVKDGEHHQ
PHDVXUHG ZHUH QRW LQ H[LVWHQFH :H XVHG GDWD IURP WKH PJP ZHHN 07;YHUVXVSODFHER WULDO SXEOLVKHG SUHYLRXVO\
IRUUHDVRQVWKHDFWLYHWUHDWPHQWDUP07;SURGXFHG
ODUJHHIIHFWVL]HVLQWKHRULJLQDOWULDODQDO\VLVDQGPDQ\EXW
QRW DOO RI WKH FRUH VHW YDULDEOHV KDG EHHQ PHDVXUHG 7KRVH
FRUH VHW YDULDEOHV WKDW KDG QRW EHHQ PHDVXUHG ZHUH GHULYHG
WKURXJK UHJUHVVLRQ DQDO\VLV XVLQJ YDULDEOHV WKDW KDG EHHQ
DVVHVVHG DQG WKDW FRUUHODWH ZLWK FRUH VHW YDULDEOHV DV
GLVFXVVHGLQJUHDWHUGHWDLOEHORZ
Using nominal group technique, decide upon
which of the remaining definitions of improvement is easiest
to use and most credible (highest face validity). 7KHDWWHQGHHV
ZHUH DJDLQ VSOLW LQWR JURXSV DQG XVLQJ QRPLQDO JURXS
WHFKQLTXHZHUHDVNHGWRGHFLGHXSRQZKLFKRIWKHGHILQLWLRQV
RI LPSURYHPHQW WKDW SHUIRUPHG EHVW ZHUH HDVLHVW WR XVH DQG
PRVW FUHGLEOH IDFH YDOLGLW\ UDQNLQJ WKH EHVW IURP KLJKHVWIDFHYDOLGLW\WRORZHVW
Multiply the face validity score by the kappa
values to obtain the "best" definition. :H XVHG WKH NDSSD
VWDWLVWLF DV DQ DGGLWLRQDO PHDVXUH RI DJUHHPHQW EHWZHHQ WKH
SK\VLFLDQV
HYDOXDWLRQ DQG WKH GHILQLWLRQV Ȁ YDOXHV •
ZHUH FRQVLGHUHG WR EH HYLGHQFH RI DJUHHPHQW )LQDOO\ ZH
FRPELQHGWKHIDFHYDOLGLW\UDQNLQJVE\WKHQRPLQDOJURXSV
DQG PXOWLSOLHG WKLV VXP E\ LWV NDSSD VWDWLVWLF WR REWDLQ WKH
EHVWGHILQLWLRQ
RESULTS
Results of scoring the patient profiles. 7KH
SK\VLFLDQV VFRUHG RI WKH SDWLHQW SURILOHV DV
FOLQLFDOO\ LPSRUWDQWO\ LPSURYHG DV QRW LPSURYHG
DQG DV XQLQWHUSUHWDEOH ,Q QR FDVH GLG RQH QRPLQDO
JURXS UDWH D SDWLHQW DV LPSURYHG DQG WKH RWKHU JURXS
UDWHWKHVDPHSDWLHQWDVQRWLPSURYHG
Identification
of
9
definitions
of
improvement as the best performers. 1LQHRIWKH
GHILQLWLRQV RI LPSURYHPHQW VKRZHG D VHQVLWLYLW\ DQG
VSHFLILFLW\ • 7KHVH GHILQLWLRQV WKHLU
FRUUHVSRQGLQJ FKLVTXDUH YDOXHV P YDOXHV SHUFHQW
IDOVHSRVLWLYH DQG IDOVHQHJDWLYH UDWHV DQG NDSSD
VWDWLVWLFVDUHVKRZQLQ7DEOH
Face validity of the 9 definitions of
improvement, and final resolution. $IWHUSUHVHQWDWLRQ
RI WKH DERYH GDWD WKH DWWHQGHHV XVLQJ QRPLQDO JURXS
WHFKQLTXHUDQNHGWKHGHILQLWLRQVIRUIDFHYDOLGLW\RQD
VFDOH ZLWK EHLQJ WKH KLJKHVW $QRWKHU GHILQLWLRQVZHUHDGGHGE\WKHSDUWLFLSDQWVWRWKHOLVWWR
EH UDQNHG IRU IDFH YDOLGLW\ EXW DOO UHFHLYHG D ORZ
UDQNLQJLQWKHILQDOYRWHGDWDQRWVKRZQ7KHVXPRI
WKH FRPELQHG UDQNV IURP WKH JURXSV LV SUHVHQWHG LQ
7DEOH UDQJH )LQDOO\ WKH VXP RI WKH UDQNLQJ
ZDVPXOWLSOLHGE\LWVNDSSDVWDWLVWLFVWRREWDLQWKHILQDO
VFRUH UDQJH , DQG WKH GHILQLWLRQV RI
LPSURYHPHQW ZLWK WKH KLJKHVW ILQDO VFRUH ZHUH
LGHQWLILHG 7KH GHILQLWLRQ RI LPSURYHPHQW WKDW VFRUHG
KLJKHVWZDVDVIROORZVDWOHDVWLPSURYHPHQWLQDW
OHDVW FRUH VHW YDULDEOHV ZLWK QR PRUH WKDQ RI WKH
UHPDLQLQJYDULDEOHVGHWHULRUDWLQJE\PRUHWKDQ
$VFDQEHVHHQLQ7DEOHWKHGHILQLWLRQVWKDW
VFRUHGVHFRQGDQGWKLUGKLJKHVWZHUHYHU\VLPLODUWRWKH
JLVW7KHWKLUGGHILQLWLRQLVVLPLODUWRWKH3DXOXVFULWHULD
XVHGLQDGXOW5$7KHVLPLODULW\RIWKHWRSUDQNLQJ
GHILQLWLRQVLQGLFDWHVFRQYHUJHQWYDOLGLW\RIWKHSURFHVV
DEFINITION OF IMPROVEMENT IN JA
Table 2. )LQDOUHVXOWVIRUWKHEHVWGHILQLWLRQVRILPSURYHPHQW
)DOVH
6HQVLWLYLW\
6SHFLILFLW\
SRVLWLYH
'HILQLWLRQRILPSURYHPHQW
x2†
RIDQ\LPSURYHGE\•QRPRUH WKDQZRUVHE\!
RIDQ\LPSURYHGE\•RQR
PRUHWKDQZRUVHE\!
RIDQ\LPSURYHGE\•QRPRUH WKDQZRUVHE\!
0'JOREDOLPSURYHGE\•RIDQ\ UHPDLQLQJLPSURYHGE\•QR
PRUHWKDQZRUVHE\!
RIDQ\LPSURYHGE\•QRPRUH WKDQZRUVHE\!
RIDQ\LPSURYHGE\•QRQH
ZRUVHE\!
0'JOREDOLPSURYHGE\•RIDQ\ UHPDLQLQJLPSURYHGE\•QR
PRUHWKDQZRUVHE\!
RIDQ\LPSURYHGE\•QRPRUH WKDQZRUVHE\!
0'JOREDOLPSURYHGE\•RIDQ\ UHPDLQLQJLPSURYHGE\•QR
PRUHWKDQZRUVHE\!
6HHWH[WIRUGHILQLWLRQVRIIDOVHSRVLWLYHDQGIDOVHQHJDWLYHUDWHVDQGIRURWKHUGHWDLOV
†3YDOXHV
Discriminant validity of the 9 definitions
of improvement. $V GLVFXVVHG DERYH RXU DELOLW\ WR
DVVHVV WKH GLVFULPLQDQW DELOLW\ RI WKHVH GHILQLWLRQV
RILPSURYHPHQWZDVFRPSURPLVHGGXHWRWKHODFNRI
SODFHERFRQWUROOHGFOLQLFDOWULDOGDWDVHWVWKDWXVHWKH
FRUHVHWRIYDULDEOHV
)DOVH
QHJDWLYH
K
VWDWLVWLF
6XPRIIDFH
YDOLGLW\
VFRUHV
)LQDO
VFRUH
7R PDNH VRPH DWWHPSW DW GHWHUPLQDWLRQ RI
GLVFULPLQDQW YDOLGLW\ ZH XVHG WKH PJPZHHN
07; FOLQLFDO WULDO GDWD VHW RI WKH 35&6* 7KH
UHVXOWVDUHSUHVHQWHGLQ7DEOH:KLOHWKHSHUFHQWRI
07;WUHDWHGSDWLHQWVZKRLPSURYHGZDVKLJKUDQJH
IRU
Table 3.'LVFULPLQDQWYDOLGLW\RIWKHGHILQLWLRQVRILPSURYHPHQWWKDW\LHOGHGVHQVLWLYLW\DQGVSHFLILFLW\
•
RI07;WUHDWHG
RISODFHERWUHDWHG
SDWLHQWVLPSURYHG
SDWLHQWVLPSURYHG
'HILQLWLRQRILPSURYHPHQW
Q Q RIDQ\LPSURYHGE\•QRPRUH
WKDQZRUVHE\!
RIDQ\LPSURYHGE\•QRPRUH
WKDQZRUVHE\!
RIDQ\LPSURYHGE\•QRPRUH
WKDQZRUVHE\!
0'JOREDOLPSURYHGE\•RIDQ\
UHPDLQLQJLPSURYHGE\"QRPRUH
WKDQZRUVHE\!
RIDQ\LPSURYHGE\•QRPRUH
WKDQZRUVHE\!
RIDQ\LPSURYHGE\•QRQH
ZRUVHE\!
0'JOREDOLPSURYHGE\•RIDQ\
UHPDLQLQJLPSURYHGE\•QRPRUH
WKDQZRUVHE\!
RIDQ\LPSURYHGE\•QRPRUH
WKDQZRUVHE\!
0'JOREDOLPSURYHGE\•RIDQ\
UHPDLQLQJLPSURYHGE\•QRPRUH
WKDQZRUVHE\!
'DWDIURPDFOLQLFDOWULDORIPHWKRWUH[DWH07;PJPZHHNYHUVXVSODFHERLQMXYHQLOHDUWKULWLV
ZHUHXVHGIRUWKLVDQDO\VLV
DEFINITION OF IMPROVEMENT IN JA
DOO GHILQLWLRQV WKH SHUFHQW RI SODFHERWUHDWHG SDWLHQWV
VDPHZKRLPSURYHGDFFRUGLQJWRWKHVHGHILQLWLRQVZDV
DOVRKLJK7KHVHGDWDPXVWEHLQWHUSUHWHG
ZLWK FRQVLGHUDEOH FDXWLRQ 6LQFH WKLV GDWD VHW GRHV QRW
FRQWDLQ DOO RI WKH FRUH VHW YDULDEOHV VRPH KDG WR EH
GHULYHG DQG FRQYHUWHG IURP RWKHU VFDOHV RI PHDVXUH
7KH SK\VLFLDQ JOREDO HYDOXDWLRQ DQG SDUHQWSDWLHQW
JOREDO HYDOXDWLRQ KDG WR EH GHULYHG IURP D VFDOH WKDW
FRQWDLQHGRQO\WKHFDWHJRULHVPXFKEHWWHUEHWWHUVDPH
ZRUVH DQG PXFK ZRUVH )XQFWLRQDO DELOLW\ KDG WR EH
GHULYHGWKURXJKUHJUHVVLRQDQDO\VLVXVLQJWKHQXPEHURI
MRLQWV ZLWK OLPLWHG UDQJH RI PRWLRQ ZLWK ZKLFK LW KDV
VKRZQJRRGFRUUHODWLRQLQRWKHUGDWDVHWV
DISCUSSION
8VLQJDFRQVHQVXVIRUPDWLRQDQGVWDWLVWLFDODS
SURDFKRXUUHVXOWVVXJJHVWWKDWLPSURYHPHQWLQSDWLHQWV
ZLWK-$FDQEHGHILQHGDVIROORZVRIDQ\FRUHVHW
YDULDEOHVLPSURYHGE\DWOHDVWZLWKQRPRUHWKDQ
RI WKH UHPDLQLQJ YDULDEOHV ZRUVHQHG E\ PRUH WKDQ
7KH YDULDEOHV LQFOXGHG LQ WKH FRUH VHW DUH SK\VLFLDQ JOREDO DVVHVVPHQW RI GLVHDVH DFWLYLW\ SDUHQWSDWLHQW JOREDO DVVHVVPHQW RI RYHUDOO ZHOOEHLQJ
HDFKVFRUHGRQDFP9$6IXQFWLRQDODELOLW\
QXPEHU RI MRLQWV ZLWK DFWLYH DUWKULWLV QXPEHU RI
MRLQWV ZLWK OLPLWHG UDQJH RI PRWLRQ DQG (65 7KLV
VHWRIHQGSRLQWVKDVDQLQWXLWLYHDSSHDOWRWKHFOLQLFLDQ
LQWKDWLWFRPELQHVDVSHFWVRIWKHDUWLFXODUH[DPLQDWLRQ
ZLWKWUXHRXWFRPHIXQFWLRQDODELOLW\DQGSDUHQWSDWLHQW
DVVHVVPHQWRIRYHUDOOZHOOEHLQJ
:H SURSRVH WKHVH HQG SRLQWV DV D FRUH VHW
RQO\LQYHVWLJDWRUVFDQPHDVXUHDVPDQ\RWKHUYDULDEOHV
DVWKH\GHHPDSSURSULDWH,QGHHGZHEHOLHYHRWKHUYDUL
DEOHV VKRXOG VWLOO EH PHDVXUHG DQG UHSRUWHG )XUWKHU
PRUHWKHFRUHVHWGRHVQRWKDYHWRVHUYHDVWKHEDVLVIRU
WKHSULPDU\RXWFRPHEXWWKHFRUHVHW YDULDEOHVVKRXOG
DOZD\V EH PHDVXUHG 3DWLHQWV VKRXOG EH HYDOXDWHG DV
LPSURYHG RU QRW LPSURYHG E\ FRPSDULQJ WKH YDOXHV RI
WKH FRUH HQG SRLQWV DW WKH HQG RI WKH WULDO RU DW
ZLWKGUDZDO IURP WULDO LQWHQWWRWUHDW DSSURDFK ZLWK
EDVHOLQHYDOXHV
7KHGHILQLWLRQRILPSURYHPHQWGHYHORSHGKHUH
VKRZV KLJK VHQVLWLYLW\ DQG VSHFLILFLW\ DQG ORZ IDOVH
SRVLWLYH DQG IDOVHQHJDWLYH UDWHV 0RUHRYHU WKH WRS GHILQLWLRQVZLWKWKHKLJKHVWILQDOVFRUHVKRZHGFRQYHU
JHQW YDOLGLW\ DOO DUH YHU\ VLPLODU WR HDFK RWKHU WKH
WKLUG KLJKHVW LV VLPLODU WR WKH 3DXOXV FULWHULD ZLWK
VOLJKWO\GLIIHUHQWYDULDEOHVWKDWKDYHEHHQXVHGE\DGXOW
UKHXPDWRORJLVWV IRU PDQ\ \HDUV DQG WKH LQWHUQDWLRQDO
TXHVWLRQQDLUH VXUYH\ GRQH LQ \LHOGHG QHDUO\ WKH
GHILQLWLRQ RI LPSURYHPHQW LQ WHUPV RI WKH QXPEHU RI
YDULDEOHV WKDW PXVW LPSURYH WKH SHUFHQW FKDQJH WKDW
PXVWEHDWWDLQHGLQRUGHUWRFDOOWKHYDULDEOHVLPSURYHG
DQGWKHQXPEHURIYDULDEOHVWKDWFRXOGZRUVHQDQGWKH
SDWLHQW VWLOO EH FODVVLILHG DV LPSURYHG %HFDXVH RI WKH
UHODWHGQHVV DQG VLPLODULW\ LQ SHUIRUPDQFH RI WKH WRS GHILQLWLRQV LGHQWLILHG KHUH ZH LQWHQG WR REVHUYH WKH
UHODWLYH SHUIRUPDQFH RI HDFK LQ SURVSHFWLYH YDOLGDWLRQ
VWXGLHV
6HYHUDO LVVXHV UHPDLQ XQUHVROYHG %HFDXVH RI
D ODFN RI DGHTXDWH GDWD VHWV ZH ZHUH QRW DEOH WR
SURYLGHILUPFRQFOXVLRQVDERXWWKHGLVFULPLQDQWDELOLW\
RI WKH GHILQLWLRQV XQGHU SODFHERFRQWUROOHG WULDO
3URVSHFWLYH YDOLGDWLRQ RI WKH GHILQLWLRQ ZLOO EH D
QHFHVVDU\QH[WVWHS
:H GLG QRW VSHFLI\ ZKLFK RI WKH IXQFWLRQDO
DELOLW\ WRROV FXUUHQWO\ DYDLODEOH RU XQGHU GHYHORSPHQW
VKRXOG EH XVHG7KLV LVVXH PD\ EH RI OLWWOH LPSRUWDQFH
SURYLGHGWKHLQVWUXPHQWXVHGKDVEHHQYDOLGDWHGLQWKH
SHGLDWULFSRSXODWLRQDQGDOOLQYHVWLJDWRUVXVHWKHVDPH
LQVWUXPHQW WKURXJKRXW D WULDO :KHQ WKH LQVWUXPHQWV
QRZ XQGHU GHYHORSPHQW FRPSOHWH YDOLGLW\ WHVWLQJ LW
PD\EHFRPHSRVVLEOHWRUHFRPPHQGRQHWKDWFRPELQHV
PHDVXUHPHQWV RI IXQFWLRQDO DELOLW\ DQG KHDOWKUHODWHG
TXDOLW\RIOLIH
7KH LVVXH RI UHGXQGDQF\ EHWZHHQ WKH QXPEHU
RI MRLQWV ZLWK DFWLYH DUWKULWLV DQG WKH QXPEHU RI MRLQWV
ZLWKOLPLWHGUDQJHRI PRWLRQ ZLOO KDYHWREHUHYLHZHG
FDUHIXOO\ DV WKH GHILQLWLRQV DUH WHVWHG IXUWKHU 2QH
VXJJHVWLRQLV WRXVHWKHQXPEHURIMRLQWVZLWKVZHOOLQJ
UDWKHUWKDQDFWLYHMRLQWVDVLQWKHFRUHVHWIRUDGXOW5$
+RZHYHU WKLV LV QRW OLNHO\ WR VROYH WKH SUREOHP 7KH
QXPEHURIMRLQWV ZLWKVZHOOLQJDOVRFRUUHODWHVZLWKWKH
QXPEHU ZLWK OLPLWHG UDQJH RI PRWLRQ DQG WKH IXUWKHU
SUREOHP RI MRLQWV WKDW DUH VZROOHQ GXH WR ERQ\
HQODUJHPHQWZLWKRXWFXUUHQWO\DFWLYHV\QRYLWLVDULVHV
7KH ODFN RI YDOLG ZLGHO\ DYDLODEOH ODERUDWRU\
PDUNHUVRILQIODPPDWLRQLQFKLOGUHQZLWK-$OHDYHVWKH
FRUHVHWZLWKRQO\WKH(65DVDELRFKHPLFDOPDUNHURI
UHVSRQVH 6RPH FKLOGUHQ HQUROOHG LQ WULDOV RI
VHFRQGOLQH DJHQWV KDYH D QRUPDO (65 WKURXJKRXW WKH
VWXG\WKXVFRPSURPLVLQJWKHXWLOLW\RIWKHGHILQLWLRQRI
LPSURYHPHQW $V WKH VFLHQFH DGYDQFHV LQYHVWLJDWRUV
VKRXOGIHHOIUHHWRUHSODFHWKH(65ZLWKDQRWKHUPRUH
VSHFLILF ODERUDWRU\ PDUNHU 7KLV FDQ EH GRQH LQ WKH
SURVSHFWLYHYDOLGDWLRQSKDVH
:H GLG QRW DWWHPSW WR FUHDWH GLIIHUHQW
GHILQLWLRQV RI LPSURYHPHQW IRU WKH YDULRXV SKHQRW\SHV
RI -$ 5DWKHU ZH DWWHPSWHG WR PDNH WKH GHILQLWLRQ
UREXVWHQRXJKWRFRYHUDOOW\SHVRI-$IRFXVLQJRQWKH
FHQWUDO IHDWXUHV RI DUWKULWLV IXQFWLRQ DQG RYHUDOO
ZHOOEHLQJ 7KH SULPDU\ RXWFRPH LQ D VWXG\ RI
V\VWHPLFDOO\
LOO
DEFINITION OF IMPROVEMENT IN JA
FKLOGUHQPD\EHIHYHUUHGXFWLRQEXWWKHFRUHVHWZLOO
VWLOOEHXVHIXOWRDVVHVV
2WKHU LVVXHV WKDW UHPDLQ LQFOXGH ZKHWKHU LW
ZRXOG EH PRUH GHVLUDEOH WR GHILQH OHYHOV RI
LPSURYHPHQW QRQH PLOG PRGHUDWH PDUNHG E\
XVLQJ DQ DSSURDFK VLPLODU WR WKDW XVHG E\ WKH
(XURSHDQ /HDJXH $JDLQVW 5KHXPDWLVP WKH
SRVVLELOLW\ RI XVLQJ UHGXFHG MRLQW FRXQWV WKH
DFFHSWDQFH RI WKH GHILQLWLRQ RI LPSURYHPHQW DQG
'UXJ$GPLQLVWUDWLRQDQGWKHLPSDFWRIWKHFKDQJLQJ
FODVVLILFDWLRQ FULWHULD IRU WKH LGLRSDWKLF DUWKULWLGHV RI
FKLOGKRRG
,Q VXPPDU\ ZH SURSRVH D GHILQLWLRQ RI -$
LPSURYHPHQW WKDW KDV ! VHQVLWLYLW\ DQG
VSHFLILFLW\ DQG KLJK IDFH YDOLGLW\ 8VH RI D XQLIRUP
GHILQLWLRQ ZLOO KHOS VWDQGDUGL]H WKH FRQGXFW DQG
UHSRUWLQJRIFOLQLFDOWULDOVDQGPD\KHOSSUDFWLWLRQHUV
GHFLGHLIDFKLOGZLWK-$KDVUHVSRQGHGDGHTXDWHO\WR
WKHUDS\
ACKNOWLEDGEMENTS
7KH DXWKRUV ZLVK WR DFNQRZOHGJH WKH IROORZLQJ
LQGL YLGXDOV ZKR ZHUH DWWHQGHHV RI WKH 3DYLD ,WDO\
,QWHUQDWLRQDO &RQVHQVXV &RQIHUHQFH RQ 'HILQLQJ
,PSURYHPHQWLQ-XYHQLOH$UWKULWLVDQGDUHPHPEHUVRIWKH
3HGLDWULF5KHXPDWRORJ\,QWHUQDWLRQDO7ULDOV2UJDQL]DWLRQ
IRUWKHLUGLOLJHQWZRUNGXULQJWKHPHHWLQJ%RHO$QGHUVVRQ
*DUH 0' -RQNRSLQJ 6ZHGHQ =VROW %DORJK 0'
%XGDSHVW+XQJDU\-DPHV 7 &DVVLG\0'&ROXPELD
02-DLPHGH,QRFHQFLR0'0DGULG6SDLQ&LDUDQ0
'XII\ 0' 0RQWUHDO 4XHEHF &DQDGD )ODYLR )DQWLQL
0' 0LODQ ,WDO\ :LHWVH .XLV 0' 8WUHFKW 7KH
1HWKHUODQGV -RVHSK ( /HYLQVRQ 0' &LQFLQQDWL 2+
-RVH $ 0HOR*RPHV 0' /LVERQ 3RUWXJDO +DUWPXW
0LFKHOV0'*DUPLVFK3DUWHQNLUFKHQ, *HUPDQ\5RVV(
3HWW\ 0' 3K' 9DQFRXYHU %ULWLVK &ROXPELD &DQDGD
$QQH0DULH3ULHXU0'3DULV)UDQFH/LVD*5LGHU0'
%HWKHVGD 0' $QQHOL 6DYRODLQHQ 0' +HLQROD
)LQODQG$OH[DQGHU6KDLNRY0'0RVFRZ5XVVLD(DUO
' 6LOYHUPDQ 0' 7RURQWR 2QWDULR &DQDGD )LOLS YDQ
GHQ %RVFK 0' *HQW %HOJLXP 3DWULFLD :RR 0'
/RQGRQ8.7KHDXWKRUVDOVRZLVKWRWKDQNWKHPHPEHUV
RIWKH
20(5$&7 &RPPLWWHH IRU WKHLU DGYLFH
FRQFHUQLQJ WKH PHWKRG RORJ\ XVHG LQ WKLV SURMHFW $
VSHFLDOWKDQNVWR(QULFR6ROFLD0'6FLHQWLILF'LUHFWRURI
WKH ,VWLWXWR GL 5LFRYHUR D &XUD D &DUDWWHUH 6FLHQWLILFR
3ROLFOLQLFR60DWWHR3DYLD,WDO\IRUH[WHQGLQJKLVVXSSRUW
IRUWKH3DYLDPHHWLQJ
REFERENCES
*LDQQLQL (+ %UHZHU (- 3RRU FRUUHODWLRQ EHWZHHQ WKH
HU\WKURF\WH VHGLPHQWDWLRQ UDWH DQG FOLQLFDO DFWLYLW\ LQMXYHQLOH
UKHXPDWRLGDUWKULWLV&OLQ5KHXPDWRO
5XSHUWR1*LDQQLQL(+5HGXQGDQF\RIFRQYHQWLRQDODUWLFXODU
UHVSRQVH YDULDEOHV XVHG LQ MXYHQLOH FKURQLF DUWKULWLV FOLQLFDO
WULDOV$QQ5KHXP'LV
3. Giannini EH, Stillman CM, Brewer EJ: Measuring grip strength
inchildren with the Martin Vigorimeter and adapted sphygmomanometer cuff. Occup Ther J Res 4:234-236, 1984
4. Anderson JJ, Felson DT, Meenan RE, Williams HJ: Which
traditional measures should be used in rheumatoid arthritis clinical trials? Arthritis Rheum 32:1093-1099, 1989
5. Felson DT, Anderson JJ, Meenan RF: Time for changes in the
design, analysis, and reporting of rheumatoid arthritis trials.
Arthritis Rheum 33:140-149, 1990
6. Felson DT: Choosing a core set of disease activity measures for
rheumatoid arthritis clinical trials. J Rheumatol 20:531-534, 199
7. Giannini EH, Lovell DJ, Hepburn B: FDA draft guidelines for the
3 clinical evaluation of antiinflammatory and antirheumatic drugs
in children: executive summary. Arthritis Rheum 38:715-718, 1995
8. Felson DT, Anderson JJ, Boers M, Bombardier C, Chernoff M,
Fried B, Furst D, Goldsmith C, Kieszak S, Lightfoot R, Paulus H,
Tugwell P, Weinblatt M Widmark R, Williams HJ, Wolfe F: The
American College of Rheumatology preliminary core set of disease activity measures for rheumatoid arthritis clinical trials.
Arthritis Rheum 36:729-740, 1993
9. Felson DT, Anderson JJ, Boers M, Bombardier C, Furst D,
Goldsmith C, Katz LM, Lightfoot R Jr, Paulus H, Strand V,
Tugwell P, Weinblatt M, Williams HJ, Wolfe F, Meszak S:
American College of Rheumatology preliminary definition of
improvement in rheumatoid arthritis. Arthritis Rheum 38:727-735,
1995
10. Fried BJ, Boers M, Baker PRA: A method for achieving
consensus on rheumatoid arthritis outcome measures: the
OMERACT conference process. J Rheumatol 20:548-551, 1993
11. Boers M, Tugwell P, Felson DT, van Riel PLCM, Kirwan JR,
Edmonds JP, Smolen JS, Khaltaev N, Muirden KD: World Health
Organization and International League of Associations for Rheumatology core endpoints for symptom modifying antirheumatic
drugs in rheumatoid arthritis clinical trials. J Rheumatol 21 (suppl
41):86-89, 1994
12.Giannini EH, Cassidy JT, Brewer EJ, Shaikov A, Maximov A,
Kuzmina N: Comparative efficacy and safety of advanced drug
therapy in children with juvenile rheumatoid arthritis. Semin
Arthritis Rheum 23:34-46 1993
13.Delbecq AL, van de Ven AH, Gustafson DH: Group Techniques
for Program Planning: A Guide to Nominal Group and Delphi
Processes. Glenview, IL, Scott, Foresman and Co., 1975
14. Brewer EJ Jr, Bass J Baum J, Cassidy JT Fink C, Jacobs J, Hanson
V, Levinson JE, Schaller J, Stillman JS: Current proposed revision
of JRA criteria. Arthritis Rheum 20:195-199, 1977
15. Cassidy JT, Levinson JE, Bass JC, Baum J, Brewer EJ Jr, Fink
CW, Hanson V, Jacobs JC, Masi AT, Schaller JG, Fries JF,
McShane D, Young D: A study of classification criteria for a
diagnosis of juvenile rheumatoid arthritis. Arthritis Rheum 29:
274-281, 1986
16. Ruperto N, Levinson JE, Ravelli A, Shear ES, Tague BL, Murray
K, Martini A, Giannini EH: Long-term health outcomes and
quality of life in American and Italian inception cohorts of patients
with juvenile rheumatoid arthritis. I. Outcome status. J
Rheumatol24:945-951, 1997
17. Ruperto N, Ravelli A, Levinson JE, Shear ES, Murray K, Tague
BL, Martini A, Glass DN, Giannini EH: Long-term health out
comes and quality of life in American and Italian inception
cohorts of patients with juvenile rheumatoid arthritis. II. Early
predictors of outcome. J Rheumatol 24:952-958, 1997
18. Ravelli A, Viola S, Ruperto N, Corsi B, Ballardini G, Martini A:
Correlation between conventional disease activity measures in
juvenile chronic arthritis. Ann Rheum Dis (in press)
19. Howe S, Levinson J, Shear E, Hartner S, McGirr G, Schulte M,
Lovell D: Development of a disability measurement tool for
juvenile rheumatoid arthritis: the Juvenile Arthritis Functional
Assessment Report for children and their parents. Arthritis Rheum
34:873-880, 1991
20. Giannini EH, Brewer EJ, Kuzmina N, Shaikov A, Maximov A,
Vorontsov 1, Fink CW, Newman AJ, Cassidy JT, Zemel LS, for
the
DEFINITION OF IMPROVEMENT IN JA
3HGLDWULF5KHXPDWRORJ\&ROODERUDWLYH6WXG\*URXS0HWKRWUH[
DWHLQUHVLVWDQWMXYHQLOHUKHXPDWRLGDUWKULWLVUHVXOWVRIWKH
86$8665GRXEOHEOLQGSODFHERFRQWUROOHGWULDO1(QJO-
0HG
3DXOXV+((JJHU0-:DUG-5:LOOLDPV+-DQGWKH
&RRSHUDWLYH6\VWHPDWLF6WXGLHVRI5KHXPDWLF'LVHDVHV*URXS
$QDO\VLVRILPSURYHPHQWLQLQGLYLGXDOUKHXPDWRLGDUWKULWLV
SDWLHQWVWUHDWHGZLWKGLVHDVHPRGLI\LQJDQWLUKHXPDWLFGUXJV
EDVHGRQWKHILQGLQJVLQSDWLHQWVWUHDWHGZLWKSODFHER$UWKULWLV
5KHXP
9DQ *HVWHO $0 3UHYRR 0// YDQ
W +RI 0$ YDQ 5LMVZLMN
0+YDQ GH 3XWWH /%$ YDQ 5LHO 3/&0 'HYHORSPHQW DQG
YDOLGDWLRQ RI WKH (XURSHDQ /HDJXH $JDLQVW 5KHXPDWLVP
UHVSRQVH FULWHULD IRU UKHXPDWRLG DUWKULWLV FRPSDULVRQ ZLWK
WKHSUHOLPLQDU\ $PHULFDQ&ROOHJHRI5KHXPDWRORJ\DQGWKH
:RUOG +HDOWK 2UJDQL]DWLRQ,QWHUQDWLRQDO /HDJXH $JDLQVW
5KHXPDWLVPFULWHULD$UWKULWLV5KHXP
)LQN &: 3URSRVDO IRU WKH GHYHORSPHQW RI FODVVLILFDWLRQ
FULWHULD IRU LGLRSDWKLF DUWKULWLGHV RI FKLOGKRRG - 5KHXPDWRO