Detecting Sources of DIF in Translated Verbal Items

'HWHFWLQJ6RXUFHVRI',)LQ7UDQVODWHG9HUEDO,WHPV
$YL$OODORXI
1DWLRQDO,QVWLWXWHIRU7HVWLQJDQG(YDOXDWLRQ1,7(,VUDHO
6WHSKHQ*6LUHFL
8QLYHUVLW\RI0DVVDFKXVHWWV$PKHUVW
3DSHUSUHVHQWHGDWWKHDQQXDOPHHWLQJRIWKH
$PHULFDQ(GXFDWLRQDO5HVHDUFK$VVRFLDWLRQ
6DQ'LHJR&$$SULO
'HWHFWLQJ6RXUFHVRI',)LQ7UDQVODWHG9HUEDO,WHPV
$EVWUDFW
7UDQVODWHGWHVWVDUHEHLQJXVHGLQFUHDVLQJO\LQHGXFDWLRQDOWHVWLQJIRUDVVHVVLQJWKH
NQRZOHGJHDQGVNLOOVRILQGLYLGXDOVZKRVSHDNGLIIHUHQWODQJXDJHV$GLIILFXOWDQG
XQGHUUHVHDUFKHGSUREOHPUHODWHGWRWHVWWUDQVODWLRQLVWKHLGHQWLILFDWLRQRIWUDQVODWHGLWHPV
WKDWGRQRWIXQFWLRQHTXLYDOHQWO\DFURVVODQJXDJHV)XUWKHUPRUHWKHUHLVOLWWOHUHVHDUFK
H[SORULQJZK\WUDQVODWHGLWHPVVRPHWLPHVIXQFWLRQGLIIHUHQWO\DFURVVODQJXDJHV,IWKH
VRXUFHVRIGLIIHUHQWLDOLWHPIXQFWLRQLQJ',)DFURVVODQJXDJHVFRXOGEHSUHGLFWHGWKHVH
VRXUFHVFRXOGEHHOLPLQDWHGDWDQHDUO\VWDJHLQWKHWHVWGHYHORSPHQWSURFHVVDQGWKLV
FRXOGKDYHLPSRUWDQWLPSOLFDWLRQVIRUGHFLVLRQVUHJDUGLQJWHVWGHYHORSPHQWSURFHVV
VFRULQJDQGHTXDWLQJ
7KLVVWXG\IRFXVHVRQWZRTXHVWLRQV,V',)UHODWHGWRLWHPW\SH":KDWDUHWKH
VRXUFHVRI',)"7KHGDWDWKDWZHUHDQDO\]HGFRQVLVWHGRIWKUHHIRUPVRIWKH,VUDHOL
3V\FKRPHWULF(QWUDQFH7HVW3(7LQ+HEUHZVRXUFHDQG5XVVLDQWUDQVODWHG7KH
VRXUFHVRI',)ZHUHDVVHVVHGE\DQDO\]LQJWKH',)GLUHFWLRQZLWKWKHDLGRIWUDQVODWRUVZKR
MXGJHGWKHLWHPV7KHUHVXOWVLQGLFDWHGWKDWRXWRILWHPVIXQFWLRQHG
GLIIHUHQWLDOO\DFURVVODQJXDJHV7KHFRQFOXVLRQVDUHSUHVHQWHGE\LWHPW\SHV$QDORJLHV
KDYHWKHKLJKHVW',)UDWHZLWK5XVVLDQVSHDNLQJH[DPLQHHVSHUIRUPLQJEHWWHURQ
RIWKHDQDORJ\LWHPVZLWK',)6HQWHQFH&RPSOHWLRQVKDYHUHODWLYHO\KLJK',)
ZLWKQRJURXSSHUIRUPLQJEHWWHUWKDQWKHRWKHU5HDGLQJ&RPSUHKHQVLRQKDV
PRGHUDWH',)SUREDEO\UHODWHGWRWKHFRQWHQWRIWKHVSHFLILFSDVVDJH/RJLF
DOPRVWQR',)ZDVIRXQG7KHPDLQVRXUFHVRI',)ZHUHFKDQJHVLQZRUGGLIILFXOW\
FKDQJHVLQLWHPIRUPDWWUDQVODWLRQHUURUVDQGGLIIHUHQFHVLQFXOWXUDOUHOHYDQFH
7UDQVODWHGWHVWVDUHEHLQJXVHGLQFUHDVLQJO\LQHGXFDWLRQDOWHVWLQJIRUDVVHVVLQJ
WKHNQRZOHGJHDQGVNLOOVRILQGLYLGXDOVZKRVSHDNGLIIHUHQWODQJXDJHV$QH[DPSOHRI
WKLVLVWKHUHFHQW7KLUG,QWHUQDWLRQDO0DWKHPDWLFVDQG6FLHQFH6WXG\ZLWK
SDUWLFLSDWLQJFRXQWULHVDQGWHVWVSUHSDUHGLQODQJXDJHV+RZHYHUWUDQVODWLQJDWHVW
IURPDVRXUFHODQJXDJHWRDWDUJHWODQJXDJHGRHVQRWQHFHVVDULO\SURGXFHWZR
SV\FKRPHWULFDOO\HTXLYDOHQWWHVWV$QJRIIDQG&RRNSZURWH,WFDQKDUGO\
EHH[SHFWHGZLWKRXWFDUHIXODQGGHWDLOHGFKHFNVWKDWWKHWUDQVODWHGLWHPVZLOOKDYH
WKHVDPHPHDQLQJDQGUHODWLYHGLIILFXOW\IRUWKHVHFRQGJURXSDVWKH\KDGIRUWKH
RULJLQDOJURXSEHIRUHWUDQVODWLRQ7UDQVODWLRQLVRQHRIWKHILUVWVWDJHVLQWKHORQJ
SURFHVVRIWHVWDGDSWDWLRQDFURVVGLIIHUHQWODQJXDJHV'/7KHILQDOJRDORIWHVW
DGDSWDWLRQLVWRPDLQWDLQFRQVWUXFWHTXLYDOHQFHDQGFRQWHQWUHSUHVHQWDWLRQDFURVVWKH
WZRODQJXDJHYHUVLRQV3URFHGXUHVIRUWUDQVODWLRQDGDSWDWLRQVFDQEHIRXQGLQ
+DPEOHWRQ'LIIHUHQWLDOLWHPIXQFWLRQLQJ',)GHWHFWLRQSOD\VDQLPSRUWDQW
UROHLQWKHWHVWDGDSWDWLRQSURFHVV$QLWHPIXQFWLRQVGLIIHUHQWO\DFURVVJURXSVLI
H[DPLQHHVRIHTXDODELOLW\EXWIURPGLIIHUHQWJURXSVLQWKLVVWXG\VRXUFHDQGWDUJHW
ODQJXDJHVGRQRWKDYHDQHTXDOSUREDELOLW\RIFRUUHFWO\UHVSRQGLQJWRWKDWLWHP
(PSLULFDODQGMXGJPHQWDOPHWKRGVIRUGHWHFWLQJGLIIHUHQWLDOO\IXQFWLRQLQJWHVW
LWHPVZHUHLQLWLDOO\GHYHORSHGWRGHWHFWELDVIRUGLIIHUHQWJURXSVWHVWHGLQWKHVDPH
ODQJXDJH(PSLULFDOELDVVWXGLHVKDYHVRPHWLPHVIRXQGFRQVLVWHQWGLIIHUHQFHVEHWZHHQ
GHPRJUDSKLFJURXSVLHEODFNVYHUVXVZKLWHVDQGPDOHVYHUVXVIHPDOHVDQG
FRQGXFWLQJDQLWHPELDVVWXG\KDVEHFRPHDQHVVHQWLDOSDUWRIWHVWGHYHORSPHQWDQG
WHVWHYDOXDWLRQ6RPHRIWKHVWXGLHVVHHIRUH[DPSOH+DPEOHWRQ-RQHVXVHG
MXGJPHQWDODQGHPSLULFDOSURFHGXUHVVHSDUDWHO\DQGFKHFNHGWKHOHYHORIDJUHHPHQW
EHWZHHQWKHP7KHVHPHWKRGVZHUHDSSOLHGODWHULQWHVWWUDQVODWLRQVLQRUGHUWRGHWHFW
LWHPVWKDWIXQFWLRQGLIIHUHQWO\LQWKHVRXUFHODQJXDJHDQGWKHWDUJHWODQJXDJH
7KHRUHWLFDOO\DQGSUDFWLFDOO\WKH',)PHWKRGVFDQEHXVHGWRGHWHFWDQ\GLIIHUHQFH
EHWZHHQDQ\JURXSVLQWKHLUUHVSRQVHVWRDQ\NLQGRILWHP7KLVVHQ6WHLQEHUJ
:DLQHU
7KHHTXLYDOHQFHRIWKHWHVWVWUXFWXUHLQHDFKODQJXDJHYHUVLRQVKRXOGEH
DVVHVVHGEHIRUHFKRRVLQJDVXLWDEOH',)PHWKRG6LUHFLDQG6ZDPLQDWKDQ
DGGUHVVHGWKLVPDWWHUDQGSUHVHQWHGSURFHGXUHVIRUGLPHQVLRQDOLW\DVVHVVPHQWLQ
GHDOLQJZLWKWHVWDGDSWDWLRQV
$GLIILFXOWDQGXQGHUUHVHDUFKHGSUREOHPUHODWLQJWRWHVWWUDQVODWLRQLVWKH
LGHQWLILFDWLRQRIWUDQVODWHGLWHPVWKDWGRQRWIXQFWLRQHTXLYDOHQWO\DFURVVODQJXDJHV
)XUWKHUPRUHWKHUHLVOLWWOHUHVHDUFKH[SORULQJZK\WUDQVODWHGLWHPVVRPHWLPHV
IXQFWLRQGLIIHUHQWO\DFURVVODQJXDJHV)HZVWXGLHVKDYHGHDOWZLWKGHWHFWLQJVRXUFHVRI
',)DQGRIWKHVHRQO\DIHZLQYROYHGWUDQVODWHGWHVWV,IWKHIDFWRUVDIIHFWLQJWKH',)
RIWUDQVODWHGLWHPVFRXOGEHSUHGLFWHGWKH\FRXOGEHWDNHQLQWRDFFRXQWDWDQHDUO\
VWDJHLQWKHWHVWGHYHORSPHQWSURFHVVUHVXOWLQJLQLPSURYHGGHFLVLRQVUHJDUGLQJWHVW
FRQVWUXFWLQJVFRULQJDQGHTXDWLQJHJH[FOXGLQJIURPWKHHTXDWLQJLWHPVWKDW
IXQFWLRQGLIIHUHQWLDOO\DFURVVODQJXDJHV',)LQWHVWDGDSWDWLRQVLVJHQHUDOO\QRW
DQWLFLSDWHGEHIRUHWHVWDGPLQLVWUDWLRQDQGDVDUHVXOWUHVHDUFKHUVPXVWUHO\RQ
SRVWKRFH[SODQDWLRQVFRQFHUQLQJWKHSUHVXPHGVRXUFHVRI',)
2QO\DIHZVWXGLHVWRGDWHKDYHGHDOWZLWKWKHUHODWLRQVKLSEHWZHHQLWHPFRQWHQW
DQG',)RIWUDQVODWHGLWHPV7KHUHKDVFOHDUO\EHHQVRPHVXFFHVVLQH[SODLQLQJWKH
VRXUFHVRI',)LQWHVWLWHPV+XOLQVXJJHVWHGDPHWKRGIRUDVVHVVLQJWKHVRXUFH
RI',)LQWUDQVODWHGLWHPVXVLQJ,57SDUDPHWHUV$FFRUGLQJWR+XOLQ¶VWKHRU\
ESDUDPHWHULWHPGLIILFXOW\GLIIHUHQFHLVGXHWRWUDQVODWLRQHUURUDQGDSDUDPHWHU
LWHPGLVFULPLQDWLRQGLIIHUHQFHLVGXHWRGLIIHUHQFHLQFXOWXUDOUHOHYDQFH(OOLV
SHUIRUPHGDSDUWLDOWHVWRI+XOLQ¶VWKHRU\UHJDUGLQJWKHDSDUDPHWHUGLIIHUHQFH7KH
VWXG\SRSXODWLRQVSRNHWKHVDPHODQJXDJH*HUPDQEXWFDPHIURPGLIIHUHQWFXOWXUHV
(DVW*HUPDQ\DQG:HVW*HUPDQ\WKHWHVWZDVWKH7ULHU3HUVRQDOLW\,QYHQWRU\
73,(DFKFXOWXUHJURXSFRQVLVWHGRIVRPHSHRSOH(OOLVFRQFOXGHGWKDWWKH
UHVXOWVGLGQRWVXSSRUW+XOLQ¶VWKHRU\DOPRVWDOORIWKHGLIIHUHQFHVIRXQGZHUHLQWKH
ESDUDPHWHUDQGQRWLQWKHDSDUDPHWHU+RZHYHULWLVOLNHO\WKDW(OOLV¶VDPSOHVL]HV
ZHUHQRWODUJHHQRXJKIRUDVWDEOHHVWLPDWLRQRIWKHDSDUDPHWHUDQGFRQVHTXHQWO\VKH
FRXOGQRWDGHTXDWHO\DGGUHVV+XOLQ¶VWKHRU\$QRWKHUFRQFOXVLRQIURPWKLVVWXG\ZDV
WKHQHHGIRUFRQGXFWLQJDFRPSOHPHQWDU\VWXG\XVLQJGLIIHUHQWODQJXDJHVEXW
LQYROYLQJH[DPLQHHVZLWKWKHVDPHFXOWXUDOEDFNJURXQG+RZHYHUVXFKDVWXG\
ZRXOGEHYHU\GLIILFXOWWRFRQGXFWEHFDXVHLQRUGHUWRKDYHWKHVDPHFXOWXUDO
EDFNJURXQGDOOVXEMHFWVZRXOGKDYHWREHFRPSOHWHO\ELOLQJXDOSHRSOHZKRVSHDN
GLIIHUHQWODQJXDJHVFDQQRWKDYHWKHVDPHFXOWXUDOEDFNJURXQG$FFRUGLQJWR(OOLVLW
ZRXOGEHYHU\GLIILFXOWWRPHDVXUHWKHGHJUHHWRZKLFKDSHUVRQLVELOLQJXDO
$QJRIIDQG&RRNIROORZHG$QJRIIDQG0RGXLQDQDWWHPSWWR
HVWDEOLVKVFRUHHTXLYDOHQFHEHWZHHQWKH6$7DQGWKH6SDQLVKODQJXDJHHTXLYDOHQW
WKH3UXHEDGH$SWLWXG$FDGHPLFD3$$7ZR,57PHWKRGVZHUHXVHGIRUDVVHVVLQJ
',)*UHDWHU',)ZDVIRXQGLQWKHDQWRQ\PDQGDQDORJ\LWHPVDQGVPDOOHU',)LQWKH
VHQWHQFHFRPSOHWLRQDQGUHDGLQJFRPSUHKHQVLRQLWHPV7KHLUH[SODQDWLRQZDVLWHPV
ZLWKPRUHFRQWH[WSUREDEO\WHQGWRUHWDLQWKHLUPHDQLQJHYHQLQWKHIDFHRI
WUDQVODWLRQLQWRDQRWKHUODQJXDJHSDJH
,QDVWXG\RIWKHWUDQVODWLRQRIWKH,VUDHOL3V\FKRPHWULF(QWUDQFH7HVW3(7
IURP+HEUHZWR5XVVLDQ*DIQL&DQDDQ<HKRVKDIDWDQG%HOOHU*DIQL
&DQDDQ<HKRVKDIDW%HOOHUHYDOXDWHGWUDQVODWHGLWHPVLQWKHYHUEDO
VHFWLRQRIWKHWHVWIRUWKUHHGLIIHUHQWIRUPV7KHWUDQVODWHGLWHPVLQWKHYHUEDOVHFWLRQ
FRQVLVWHGRIDQDORJLHVVHQWHQFHFRPSOHWLRQVUHDGLQJFRPSUHKHQVLRQDQGORJLFLWHPV
,QRUGHUWRDVVHVV',)WKHUHVHDUFKHUVXVHGDGHOWDSORWWHFKQLTXHSURSRVHGE\$QJRII
7KH\IRXQGWKHJUHDWHVW',)LQWKHDQDORJLHVDQGWKHVPDOOHVW',)LQWKHORJLF
DQGVHQWHQFHFRPSOHWLRQLWHPV7KHVHUHVXOWVDUHVLPLODUWRWKRVHRI$QJRIIDQG&RRN
ZLWKWKHH[FHSWLRQRIWKHORJLFLWHPVZKLFKGRQRWDSSHDURQWKH6$7YHUEDO
VHFWLRQ7KHUHDGLQJFRPSUHKHQVLRQLWHPVVKRZHGUHODWLYHO\KLJKHU',)WKDQLQWKH
$QJRII&RRNVWXG\(OOLVVWXGLHGWKHPHDVXUHPHQWHTXLYDOHQFHRIWUDQVODWHG
$PHULFDQDQG*HUPDQLQWHOOLJHQFHWHVWVXVLQJ,57PHWKRGVIRU',)+HUILQGLQJV
VKRZHGWKDWLQPRVWLWHPVWKH',)UHVXOWHGIURPWUDQVODWLRQHUURUV,QRWKHUFDVHV
VRPHSRVWKRFH[SODQDWLRQVZHUHJLYHQ
,QVXPPDU\DVWXG\RIWKHVRXUFHVRI',)LQWUDQVODWHGYHUEDOLWHPVFRXOG
JUHDWO\HQKDQFHFXUUHQWNQRZOHGJHDQGWKHQHHGIRUDVWXG\RIWKLVW\SHLVJUHDW
3XUSRVH
7KHSXUSRVHRIWKLVVWXG\LVWRDVVHVVVRXUFHVRI',)LQWUDQVODWHGYHUEDOLWHPV
7KHVWXG\FRQVLVWHGRIWZRSDUWV7KHILUVWSDUWRIWKHVWXG\ZDVGHVLJQHGWRLGHQWLI\
WKHW\SHVRIYHUEDOLWHPVZKLFKZHUHPRVWOLNHO\WRGLVSOD\',)ZKHQWUDQVODWHGIURP
RQHODQJXDJHWRDQRWKHU7KHVHFRQGSDUWGHDOWZLWKGHWHFWLQJVRXUFHVRI',)XVLQJ
WZRSURFHGXUHVDQDO\]LQJWKH',)GLUHFWLRQZKLFKJURXSSHUIRUPHGEHWWHURQ
ZKLFKLWHPVDQGLWHPW\SHVDQGDIWHUUHYLHZLQJWKHILQGLQJVIURPWKHSUHYLRXV
SURFHGXUHVWUDQVODWRUVDQDO\]HGWKHW\SHDQGFRQWHQWRIWKH',)LWHPV
7KUHHIHDWXUHVRIWKHVWXG\ZHUHHVSHFLDOO\LPSRUWDQWDVDPSOHVL]HVZHUH
FRQVLGHUDEO\ODUJHUWKDQWKRVHQRUPDOO\DYDLODEOHLQ',)VWXGLHVELWHPVGLVSOD\HGKLJK
',)DFURVVODQJXDJHVLQ',)VWXGLHVWKHGLIIHUHQFHVDUHJHQHUDOO\VPDOOHUDQGF'XHWR
VPDOOJURXSGLIIHUHQFHVLQDELOLW\LGHQWLILFDWLRQRI',)ZDVQRWFRQIRXQGHGZLWKW\SH,HUURUV
0HWKRG
,QVWUXPHQWV
7KHWHVWZKLFKZDVDQDO\]HGZDVWKHYHUEDOVXEWHVWRI3(73V\FKRPHWULF(QWUDQFH
7HVWZKLFKLVDKLJKVWDNHVWHVWXVHGIRUDGPLVVLRQVWRXQLYHUVLWLHVLQ,VUDHOVHH%HOOHU
,WLVDPXOWLSOHFKRLFHWHVWFRQVLVWLQJRIWKUHHVXEWHVWVYHUEDOTXDQWLWDWLYHDQG
(QJOLVKDVDIRUHLJQODQJXDJH,WLVZULWWHQLQ+HEUHZDQGWUDQVODWHGLQWR$UDELF5XVVLDQ
)UHQFK6SDQLVKDQG(QJOLVK7KHVWXG\GHDOWZLWKWKHWUDQVODWLRQIURP+HEUHZWR5XVVLDQ
7KHYHUEDOVXEWHVWFRQVLVWVRILWHPV,WHPW\SHVLQFOXGHV\QRQ\PVDQGDQWRQ\PV
QRWWUDQVODWHGDQDORJLHVVHQWHQFHFRPSOHWLRQVORJLFDQGUHDGLQJFRPSUHKHQVLRQDOPRVW
DOODUHWUDQVODWHG6LQFHWKHUHDUHVRPHYHUEDOLWHPVWKDWFDQQRWEHWUDQVODWHGQHZLWHPVDUH
FRQVWUXFWHGVSHFLILFDOO\IRUWKHWUDQVODWHGIRUP7KUHHWHVWIRUPVWKDWZHUHDGPLQLVWHUHGLQ
ERWK+HEUHZDQG5XVVLDQZHUHXVHGLQWKLVVWXG\7KUHHIRUPVZHUHXVHGLQRUGHUWR
LQFUHDVHWKHQXPEHURILWHPVLQYHVWLJDWHGLQWKHVWXG\DQGWRSURYLGHDEDVLVIRUUHSOLFDWLRQ
7DEOHSUHVHQWVWKHW\SHVRIWKHRIWKHLWHPVLQFOXGHGLQWKHWKUHHIRUPV
FRPPRQLWHPVLQWKHWKUHHIRUPVDQDO\]HGLQWKHVWXG\
7DEOH
7\SH 1XPEHU RI ,WHPV $EEUHYLDWLRQV IRU WKH +HEUHZ5XVVLDQ &RPPRQ ,WHPV
BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
7\SH
1XPEHU RI ,WHPV
$EEUHYLDWLRQ
BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
$QDORJLHV
$1
6HQWHQFH &RPSOHWLRQV
6&
/RJLF
/2
5HDGLQJ FRPSUHKHQVLRQ
5&
7RWDO
BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
([DPLQHHV
6LQFHDSSUR[LPDWHO\RQHPLOOLRQSHRSOHKDYHLPPLJUDWHGWR,VUDHOIURP
WKHIRUPHU6RYLHW8QLRQ$VDUHVXOWWKHQXPEHURI3(7H[DPLQHHVZKRDUHWHVWHGLQ
5XVVLDQKDVJUHDWO\LQFUHDVHG7KHPDLQFULWHULDIRUFKRRVLQJWKHWKUHHVSHFLILFIRUPV
ZHUHWKHLUODUJHVDPSOHVL]HDQGWKHVPDOOHUGLIIHUHQFHVLQYHUEDODELOLW\EHWZHHQWKH
5XVVLDQVSHDNLQJH[DPLQHHVDQGWKH+HEUHZVSHDNLQJH[DPLQHHV7KHYHUEDODELOLW\
GLIIHUHQFHVEHWZHHQWKH5XVVLDQVSHDNLQJH[DPLQHHVDQGWKH+HEUHZVSHDNLQJ
H[DPLQHHVZHUHHVWLPDWHGRQWKHEDVLVRIWKHGLIIHUHQFHVEHWZHHQWKHTXDQWLWDWLYH
VFRUHVRIWKH5XVVLDQVSHDNLQJH[DPLQHHVDQGWKH+HEUHZVSHDNLQJH[DPLQHHV7KH
TXDQWLWDWLYHVFRUHKDVDSUHWW\VWDEOHFRUUHODWLRQDURXQGZLWKWKHYHUEDOVFRUHLQ
ERWKODQJXDJHVVHH7DEOH
'LPHQVLRQDOLW\$VVHVVPHQW
$UHODWHGVWXG\$OODORXIHWDOFRPSDUHGWKHGLPHQVLRQDOVWUXFWXUHRIWKH
FRPPRQLWHPVLQWKH+HEUHZDQGWKHWUDQVODWHG5XVVLDQYHUVLRQRI3(7XVLQJ
FRQILUPDWRU\IDFWRUDQDO\VLV&)$DQGPXOWLGLPHQVLRQDOVFDOLQJ0'6%RWK
PHWKRGVIRXQGWKDWWKHGLPHQVLRQDOVWUXFWXUHRIWKHWHVWDSSHDUVWREHHTXLYDOHQW
DFURVVWKH+HEUHZDQGWKH5XVVLDQYHUVLRQV
',)0HWKRG
7KH0DQWHO+DHQV]HO0+',)GHWHFWLRQPHWKRGZDVXVHGRQWKLVVWXG\
7KH0+SURFHGXUHLQWURGXFHGE\+ROODQGDQG7KD\HULVXVHGWRGHWHUPLQH
ZKHWKHUUHIHUHQFHDQGIRFDOJURXSLWHPSHUIRUPDQFHLVHTXDODWYDULRXVDELOLW\
OHYHOVDORQJWKHDELOLW\FRQWLQXXP7KH0+SURFHGXUHSURYLGHVDQ0+'',)
LQGH[JLYHQLQWKHGHOWDPHWULF7KHPHWKRGGRHVQRWUHTXLUHODUJHVDPSOHVDQGKDV
D FLQGH[IRUWHVWLQJVWDWLVWLFDOVLJQLILFDQFH7KH',&+2',)FRPSXWHUSURJUDP
5RJHUV6ZDPLQDWKDQ+DPEOHWRQSHUIRUPVWKLVDQDO\VLVDQGZDVXVHGLQ
WKLVVWXG\
7KH',)FODVVLILFDWLRQUXOHVDSSOLHGLQWKLVVWXG\ZHUHEDVHGRQWKH',)
FODVVLILFDWLRQUXOHVRIWKH(GXFDWLRQDO7HVWLQJ6HUYLFH'RUDQV+ROODQG7ZR
FDWHJRULHVZHUHGHILQHG/DUJHDQLWHPZDVGHILQHGDVKDYLQJODUJH',)LIWKH
DEVROXWHYDOXHRI0+'',)ZDVDWOHDVWDQGVWDWLVWLFDOO\VLJQLILFDQWJUHDWHUWKDQ
]HURDWWKHOHYHO7KLVFODVVLILFDWLRQLVVLPLODUWR(76FDWHJRU\³&´
0RGHUDWHDQLWHPZDVGHILQHGDVKDYLQJPRGHUDWH',)LIWKHDEVROXWHYDOXH0+'
',)ZDVDWOHDVWDQGOHVVWKDQDQGVWDWLVWLFDOO\VLJQLILFDQWJUHDWHUWKDQ]HURDW
WKHOHYHO7KLVFODVVLILFDWLRQLVVLPLODUWR(76FDWHJRU\³%´
3DUW,GHQWLI\LQJ',)DFURVVODQJXDJHV
$',)GHWHFWLRQGHVLJQZDVXVHGVHSDUDWHO\IRUHDFKRIWKHWKUHHWHVWIRUPV
DQGZKLFKKDGDQGFRPPRQWUDQVODWHGLWHPVUHVSHFWLYHO\,QRUGHUWR
UHILQHWKHPDWFKLQJFULWHULDWKH',)GHWHFWLRQSURFHVVLQYROYHGWZRVWDJHV7KHILUVW
VWDJHXVHGWKHWRWDOUDZVFRUHDVWKHVWUDWLI\LQJYDULDEOH7KHVHFRQGVWDJHXVHGWKH
WRWDOVFRUHRIDOOLWHPVWKDWZHUHQRWIODJJHGDVKDYLQJODUJH',)GXULQJWKHILUVW
VWDJHDVWKHVWUDWLI\LQJYDULDEOH%DVHGRQWKHVHFRQGVWDJHUHVXOWVLWHPVZHUH
FODVVLILHGLQWRRQHRIWKUHHFDWHJRULHVQR',)PRGHUDWH',)DQGODUJH',)
3DUW5HVXOWV
6XPPDU\VWDWLVWLFVIRUWKHWKUHHWHVWIRUPVLQHDFKODQJXDJHDUHSUHVHQWHGLQ
7DEOH
7DEOH
0HDQV6WDQGDUG'HYLDWLRQVIRUWKHWKUHHIRUPVLQHDFKWHVWODQJXDJHIRU9HUEDO &RPPRQ
,WHPV 5DZ 6FRUH DQG 4XDQWLWDWLYH 6FRUH DQG&RUUHODWLRQVEHWZHHQWKH6FRUHV
BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
9HUEDO &RPPRQ ,WHPV D
)RUP
*URXS
1
+HEUHZ
5XVVLDQ
D
0HDQ
4XDQWLWDWLYH E
6'
0HDQ
6'
&RUUHODWLRQ
+HEUHZ
5XVVLDQ
+HEUHZ
5XVVLDQ
7KH UDZ VFRUH LV HTXDO WR QXPEHU ULJKW 7KHUH ZHUH LWHPV LQ )RUP LWHPV
LQ )RUP DQG LWHPV LQ )RUP E
7KHVH VFRUHV KDYH D 0HDQ
DQG 6'
RQ D EDVLV VDPSOH
6DPSOHVL]HVZHUHODUJHDYHUDJLQJRYHUIRUWKH+HEUHZIRUPVDQGFORVH
WRIRUWKH5XVVLDQIRUPV%DVHGRQWKHTXDQWLWDWLYHVFRUHLWFDQEHVHHQWKDWWKH
DELOLW\GLIIHUHQFHVEHWZHHQWKH5XVVLDQH[DPLQHHVDQGWKH+HEUHZH[DPLQHHVZDV
VPDOO7KLVLVDQDGYDQWDJHVLQFHWKHPRUHVLPLODUWKHJURXSVWKHPRUHDFFXUDWHLVWKH
',)GHWHFWLRQ7KHGLIIHUHQFHVEHWZHHQWKHFRUUHODWLRQVZLWKWKHTXDQWLWDWLYHVFRUHDUH
VPDOODQGDUHSUREDEO\GXHWRWKHGLIIHUHQFHVLQYDULDQFH7KHVLPLODULW\RIWKH
YHUEDOTXDQWLWDWLYHFRUUHODWLRQVDFURVVWKH+HEUHZDQG5XVVLDQJURXSVLVFRQVLVWHQW
ZLWKWKHH[SHFWDWLRQVWKDWWKHYHUEDOWHVWLVPHDVXULQJWKHVDPHFRQVWUXFWLQERWK
ODQJXDJHV
$QDO\VLVRI',)
7KH',)DQDO\VLVLGHQWLILHGWKHLWHPVWKDWGLVSOD\HGODUJHDQGPRGHUDWH',)LQ
HDFKIRUP$SSHQGL[$SUHVHQWVWKH0+'',)YDOXHVIRUDOORIWKHLWHPVLQWKH
FRPSDULVRQRIWKH+HEUHZDQG5XVVLDQODQJXDJHWHVWIRUPV,Q)RUPRXWRIWKH
LWHPVGLVSOD\HGODUJH',)DQGGLVSOD\HGPRGHUDWH',)WRWDO',)LWHPV LQ)RUPRXWRIWKHLWHPVGLVSOD\HGODUJH',)DQGGLVSOD\HG
PRGHUDWH',)WRWDO LQ)RUPRXWRIWKHLWHPVGLVSOD\HGODUJH',)
DQGGLVSOD\HGPRGHUDWH',)WRWDO )LJXUHSUHVHQWVGHOWDLWHPGLIILFXOWLHVWUDQVIRUPHGWRWKHGHOWDPHWULFSORWV
IRUDOORIWKHWUDQVODWHGLWHPVLQWKHWKUHHIRUPV7ZRSORWVDUHSUHVHQWHGIRUHDFK
IRUP7KHSORWVRQWKHOHIWUHSUHVHQWDOORIWKHFRPPRQLWHPVWKHSORWVRQWKHULJKW
GLVSOD\RQO\WKRVHLWHPVWKDWZHUHQRWIODJJHGIRUPRGHUDWHRUODUJH',)7KH
FRUUHODWLRQEHWZHHQWKH+HEUHZDQG5XVVLDQGHOWDVUDQJHVIURPWRIRUDOO
FRPPRQLWHPVDQGIURPWRIRUWKHQRQ',)WUDQVODWHGLWHPV
7DEOHSUHVHQWVGDWDUHJDUGLQJWKHQXPEHURILWHPVGLVSOD\LQJ',)RXWRIDOO
WKHFRPPRQLWHPVLQHDFKIRUPE\GHJUHHRI',)/DUJHDEVROXWHYDOXHRI0+'
',)LVDWOHDVWDQGVLJQLILFDQWO\JUHDWHUWKDQ]HURDWWKHOHYHODQG0RGHUDWH
WKHDEVROXWHYDOXHRI0+'',)LVDWOHDVWDQGOHVVWKDQDQGVLJQLILFDQWO\
JUHDWHUWKDQ]HURDWWKHOHYHO
)LJXUH'HOWD3ORWVDQGFRUUHODWLRQVEHWZHHQ5XVVLDQ+HEUHZGHOWDV
No DIF Items
Ru
Ru
All Items
He
)RUP U
He
)RUP U
All Items
Ru
Ru
No DIF Items
U
)RUP All Items
U
No DIF Items
Ru
Ru
He
He
)RUP U
He
He
)RUP ),*85( )RUP 'HOWD 3ORWV IRU WKH WKUHH )RUPV
U
7DEOH
3HUFHQWDJHDRI,WHPV'LVSOD\LQJ',)
LQWKH&RPSDULVRQRIWKH+HEUHZDQG5XVVLDQ7HVW)RUPVE\,WHP7\SH
)RUP ,WHP 7\SH
$QDORJLHV
6HQWHQFH &RPSOHWLRQV
/RJLF
5HDGLQJ &RPSUHKHQVLRQ
7RWDO
/
D 2XW RI WKH VDPH LWHP W\SHV /
0
)RUP /
ODUJH ',) 0
0
)RUP /
0
7RWDO
/ 0 /0
PRGHUDWH
7KHILQGLQJVDUHYHU\FRQVLVWHQW7KHJUHDWHVW',)ZDVIRXQGRQWKHDQDORJLHV
DQGWRDFHUWDLQH[WHQWRQWKHVHQWHQFHFRPSOHWLRQLWHPVDQGDORZHUOHYHORI',)ZDV
IRXQGIRUWKHORJLFDQGWKHUHDGLQJFRPSUHKHQVLRQLWHPV3UHGLFWLRQRIWKH0+'',)
YDOXHVXVLQJWKHFRQWHQWDUHDGHVLJQDWLRQVIRUHDFKLWHPRXWRIDOOWKHLWHPV
UHYHDOHGWKDWWKHUHJUHVVLRQZHLJKWIRUDQDORJLHVLVVLJQLILFDQW564 ) 37KHUHVXOWVDUHVLPLODUWRWKRVHRI$QJRII&RRNDQG*DIQL
&DQDDQ<HKRVKDIDWDQG%HOOHUZKLFKZHUHGHVFULEHGHDUOLHU+RZHYHU
WKH\GLIIHUIURPWKHWZRSUHYLRXVVWXGLHVLQWZRZD\V7KHUHZHUHPRUHVHQWHQFH
FRPSOHWLRQLWHPVWKDWVKRZHG',)LQWKHFXUUHQWVWXG\WKDQLQWKRVHWZRVWXGLHVDQG
WKHUHZHUHUHODWLYHO\OHVVUHDGLQJFRPSUHKHQVLRQLWHPVWKDWVKRZHG',)WKDQLQWKH
*DIQL&DQDDQ<HKRVKDIDWVWXG\7KHH[SODQDWLRQE\$QJRII&RRN
WKDWWUDQVODWHGYHUEDOLWHPVZLWKPRUHFRQWH[WDUHPRUHOLNHO\WRUHWDLQWKHLUPHDQLQJ
PLJKWDOVREHDSSOLFDEOHWRRXUUHVXOWV
3DUW'HWHFWLQJVRXUFHVRI',)
2QFHLWHPVZHUHLGHQWLILHGDVIXQFWLRQLQJGLIIHUHQWLDOO\LQ+HEUHZDQG5XVVLDQ
WZRSURFHGXUHVZHUHXVHGIRUGHWHFWLQJSRVVLEOHVRXUFHVRI',))LUVWWKHGLUHFWLRQRI
',)ZDVGHWHUPLQHG7KLVDQDO\VLVUHFRUGHGWKHJURXS+HEUHZRU5XVVLDQWKDW
SHUIRUPHGEHWWHURQHDFK',)LWHP7KLVDQDO\VLVZDVDOVRFRQGXFWHGVHSDUDWHO\IRU
HDFKLWHPW\SHFRQWHQWDUHDWRDGGUHVVWKHTXHVWLRQRIZKHWKHUWKHWUDQVODWLRQPDGH
WKHLWHPHDVLHURUPRUHGLIILFXOWZKHQWUDQVODWHGLQWR5XVVLDQIRUDQ\RUDOORIWKH
FRQWHQWDUHDV7KHVHFRQGSURFHGXUHIRUGLVFRYHULQJSRWHQWLDOVRXUFHVRI',)XVHGILYH
+HEUHZ5XVVLDQWUDQVODWRUVWRDQDO\]HWKHW\SHDQGFRQWHQWRIWKH',)LWHPV2IWKH
LWHPVJLYHQWRWKHVHWUDQVODWRUVIRUDQDO\VLVZHUHSUHYLRXVO\LGHQWLILHGDV',)
LWHPVWKHRWKHUZHUHQRWIODJJHGIRU',)7KHWUDQVODWRUVZHUHXQDZDUHRIWKH',)
FODVVLILFDWLRQVRIWKHLWHPV$IWHUH[DSODQDWLRQRIWKHWHUP',)HDFKWUDQVODWRU
LQGHSHQGHQWO\FRPSOHWHGDQLWHPUHYLHZTXHVWLRQQDLUHVHH([KLELWIRUHDFKLWHP
7KHWUDQVODWRUVWRRNDERXWWHQKRXUVWRFRPSOHWHWKHVHTXHVWLRQQDLUHV
([KLELW4XHVWLRQQDLUHJLYHQWRWKHWUDQVODWRUV
7UDQVODWRU¶VQDPHBBBBBBBBBBBBBBBBBB'DWHBBBBBBBBBBBBBBBBB
6RPHRIWKHLWHPVSURYLGHGZHUHLGHQWLILHGDVIXQFWLRQLQJGLIIHUHQWO\DFURVV
JURXSV7KLVPHDQVWKDW+HEUHZDQG5XVVLDQH[DPLHHVZLWKWKHVDPHDELOLW\KDGD
GLIIHUHQWOLNHOLKRRGRIDQVZHULQJWKHLWHPFRUUHFWO\0DNHVXUHWKDW\RXXQGHUVWDQG
WKHFRQFHSWRIGLIIHUHQWLDOLWHPIXQFWLRQLQJ',)EHIRUHFRPSOHWLQJWKHVH
TXHVWLRQQDLUHV)RUHDFKLWHPSOHDVHDQVZHUWKHIROORZLQJTXHVWLRQV
7KHFRUUHFWDQVZHUIRUWKHLWHPLVBBBBBBBBBBBBBB
&DQ\RXUDWHWKHLWHPGLVWUDFWRUVWKHPRVWDWWUDFWLYHBBBWKHOHDVWDWWUDFWLYHBB
'R\RXDQWLFLSDWH',)LQWKLVLWHP"BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
!!,IWKHDQVZHUWRTXHVWLRQLVSRVLWLYHSURFHHGWRWKHIROORZLQJTXHVWLRQV
,VWKH',)ODUJHRUPRGHUDWH"BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
)RUZKLFKODQJXDJHJURXSLVWKHLWHPPRUHGLIILFXOW"BBBBBBBBBBBBBBBBBBBBBB
([SODLQBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
:KDWZDVWKHUHDVRQIRUWKH',)"BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
$GGLWLRQDOFRPPHQWVBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
$IWHUDQDO\]LQJWKHWUDQVODWRUV¶DQVZHUVWRWKHTXHVWLRQQDLUHDQHLJKWSHUVRQ
SDQHOZDVIRUPHG7KHSDQHOLQFOXGHGWKHILYHWUDQVODWRUVDQGWKUHH+HEUHZVSHDNLQJ
UHVHDUFKHUV3DQHOPHPEHUVUHYLHZHGHDFKLWHPVHSDUDWHO\XVLQJWKHVWDWLVWLFDOGDWD
DQGWKHTXHVWLRQQDLUHUHVXOWV7KHSXUSRVHRIWKHSDQHOZDVWRDUULYHDWFRQFOXVLRQV
UHJDUGLQJWKHSRVVLEOHVRXUFHVRI',)LQHDFK',)LWHPDQGWRGUDZJHQHUDO
FRQFOXVLRQVDERXWVRXUFHVRI',)LQWUDQVODWHGYHUEDOLWHPV
3DUW5HVXOWV
D',)'LUHFWLRQ
/RRNLQJIRUVRXUFHVRI',)LQYROYHVDQDQDO\VLVRIWKH³',)GLUHFWLRQ´,VWKHUH
DQ\UHODWLRQVKLSEHWZHHQLWHPW\SHDQGJURXSSHUIRUPDQFHRQWKH',)LWHPV"$WRWDO
RIPRGHUDWHDQGODUJH',)LWHPVZHUHIRXQG7KH5XVVLDQVSHDNLQJH[DPLQHHV
SHUIRUPHGEHWWHURQRIWKHVHLWHPVDQGWKH+HEUHZVSHDNLQJH[DPLQHHVSHUIRUPHG
EHWWHURQRIWKHP7KHSURSRUWLRQZDVZKLFKLVFORVHWRWKHH[SHFWHG
UDWLRWKH',)GHWHFWLRQPHWKRGDVVXPHVWKDWWKHWHVWLVJHQHUDOO\QRWELDVHG7DEOH
SUHVHQWVDQLWHPW\SHDQDO\VLVRIWKH',)LWHPVRQZKLFKWKH5XVVLDQVSHDNLQJ
H[DPLQHHVSHUIRUPHGEHWWHUWKDQWKH+HEUHZVSHDNLQJH[DPLQHHV
7DEOH
1XPEHU RI ',) ,WHPV DQG 3HUFHQWDJHV D RI WKH ',) ,WHPV RQ ZKLFK WKH
5XVVLDQ6SHDNLQJ([DPLQHHV3HUIRUPHG%HWWHUWKDQWKH+HEUHZ6SHDNLQJ([DPLQHHV
)RUP
,WHP 7\SH
$QDORJLHV
6HQWHQFH &RPSOHWLRQV
/RJLF
5HDGLQJ &RPSUHKHQVLRQ
7RWDO
7RWDO
3HUFHQWDJHV
',) ,WHPV
D $V SHUFHQWDJHV RI WKH VDPH LWHP W\SH WKDW VKRZHG ',) LQ HDFK IRUP DQG LQ DOO WKUHH IRUPV
WRWDO
7KHILQGLQJVDUHLQWHUHVWLQJ7KH5XVVLDQH[DPLQHHVSHUIRUPHGPXFKEHWWHU
WKDQWKH+HEUHZH[DPLQHHVRQWKHDQDORJ\LWHPVZLWK',)7KHWUDQVODWLRQPLJKW
KDYHPDGHWKHDQDORJLHVPXFKHDVLHULQ5XVVLDQWKDQLQ+HEUHZ$QRWKHUSRVVLEOH
H[SODQDWLRQLVWKDWWKH5XVVLDQODQJXDJHH[DPLQHHVDUHEHWWHUDWDQDORJLHVWKDQWKH
+HEUHZODQJXDJHH[DPLQHHV,QWKHVHQWHQFHFRPSOHWLRQ',)LWHPVQRJURXS
RXWSHUIRUPHGWKHRWKHUJURXS%HFDXVHRIWKHVPDOOQXPEHURI',)LWHPVLQORJLFDQG
LQUHDGLQJFRPSUHKHQVLRQLWZDVQRWSRVVLEOHWRGUDZFRQFOXVLRQVDWWKLVVWDJH
0HDQSYDOXHGLIIHUHQFHV
7DEOHV$%&SUHVHQWVWKHPHDQSYDOXHIRUWKHLWHPVLQERWKODQJXDJHV
E\LWHPW\SHIRUDOOWKHLWHPVIRUWKH',)LWHPVDQGIRUWKHQR',)LWHPV7KH
SYDOXHVRIWKHDQDORJLHVGHPRQVWUDWH DJDLQWKDWWKH5XVVLDQH[DPLQHHVSHUIRUPHG
PXFKEHWWHUWKDQWKH+HEUHZH[DPLQHHVRQWKHDQDORJ\LWHPV
7DEOH$%&0HDQ39DOXHVE\/DQJXDJHDQG,WHP7\SH
$
$OO ,WHPV
$QDORJLHV
6HQWHQFH
/RJLF
&RPSOHWLRQ
$OO ,WHPV
5HDGLQJ
&RPSUHKHQVLRQ
+HEUHZ
5XVVLDQ
$OO ,WHPV
$QDORJLHV
6HQWHQFH
/RJLF
5HDGLQJ
'LIIHUHQFH
%
&RPSOHWLRQ
',) ,WHPV
&RPSUHKHQVLRQ
+HEUHZ
5XVVLDQ
$OO ,WHPV
$QDORJLHV
6HQWHQFH
/RJLF
5HDGLQJ
'LIIHUHQFH
&
&RPSOHWLRQ
1R ',) ,WHPV
&RPSUHKHQVLRQ
+HEUHZ
5XVVLDQ
'LIIHUHQFH
E 3DQHORI7UDQVODWRUV5HVHDUFKHUV
%DVHGRQWKHWUDQVODWRUV¶ZRUNWKHSDQHOIRXQGIRXUPDLQFDXVHVIRU',)LQWKH
',)LWHPV$SSHQGL[ %SUHVHQWVWKHUHDVRQVIRU',)LQWKHVHLWHPV
&KDQJHV LQ 'LIILFXOW\ RI :RUGV RU 6HQWHQFHV 7KHWUDQVODWLRQZDVDFFXUDWHEXW
VRPHZRUGVRUVHQWHQFHVEHFDPHHDVLHURUPRUHGLIILFXOW)RUH[DPSOH$QDQDORJ\LWHP
FRQWDLQHGDYHU\GLIILFXOWZRUGLQWKHVWHPWKDWZDVWUDQVODWHGLQWRDYHU\WULYLDOZRUG
7KHWUDQVODWRUZDVQRWDZDUHRIWKHGLIILFXOW\RIWKHRULJLQDOZRUGRURIWKHLPSRUWDQFHRI
SUHVHUYLQJWKDWGLIILFXOW\
&KDQJHV LQ &RQWHQW 7KLVFDQEHODEHOHGDWUDQVODWLRQSUREOHP7KHPHDQLQJRI
WKHLWHPFKDQJHGLQWKHWUDQVODWLRQWKXVWXUQLQJLWLQWRDGLIIHUHQWLWHP7KLVFRXOGEHGXH
WRDQLQFRUUHFWWUDQVODWLRQWKDWFKDQJHGWKHPHDQLQJRIWKHLWHPRUWKHWUDQVODWLRQRID
ZRUGWKDWKDVDVLQJOHPHDQLQJLQWRDZRUGWKDWKDVPRUHWKDQRQHPHDQLQJ
&KDQJHV LQ )RUPDW 7KHIRUPDWRIWKHLWHPZDVFKDQJHG)RUH[DPSOH$
VHQWHQFHEHFDPHPXFKORQJHU$QRWKHUH[DPSOH,QDWUDQVODWHGVHQWHQFHFRPSOHWLRQ
LWHPZRUGVWKDWRULJLQDOO\DSSHDUHGRQO\LQWKHVWHPQRZDSSHDUHGLQVWHDGLQDOOIRXU
DOWHUQDWLYHUHVSRQVHVWKXVPDNLQJWKHLWHPDZNZDUG,WVKRXOGEHQRWHGWKDWGXHWR
FRQVWUDLQWVRIWKH5XVVLDQODQJXDJHWUDQVODWLQJWKHLWHPLQWKLVZD\FRXOGQRWEHDYRLGHG
'LIIHUHQFHV LQ &XOWXUDO 5HOHYDQFH
7KHLWHPUHPDLQHGH[DFWO\WKHVDPHEXWWKH
WZRJURXSVGLIIHUHGEHFDXVHRIWKHFXOWXUHFRQWHQWRIWKHVSHFLILFLWHP7KLVFRXOGEHWKH
FRQWHQWRIDUHDGLQJFRPSUHKHQVLRQSDVVDJHWKDWZDVPRUHUHOHYDQWWRRQHRIWKHJURXSV
RUWKHFRQWHQWRIDVHQWHQFHFRPSOHWLRQLWHPWKDWZDVPRUHIDPLOLDUWRRQHRIWKHJURXSV
$QDO\VHVRIWKHFDXVHVIRU',)E\LWHPW\SHUHYHDOHGWKDWLQDQDORJ\LWHPVWKH',)
ZDVXVXDOO\GXHWRGLIIHUHQFHVLQZRUGGLIILFXOW\RUWUDQVODWLRQHUURUV,QWKHVHQWHQFH
FRPSOHWLRQLWHPVWKH',)ZDVGXHWRDOOIRXUFDXVHVVWDWHGDERYH,QWKHUHDGLQJ
FRPSUHKHQVLRQLWHPVWKH',)ZDVSUREDEO\GXHWRFXOWXUDOUHOHYDQFH2QVHYHQRIWKH
LWHPVZKLFKFRQVWLWXWHRIWKHLWHPVGLVSOD\LQJ',)LQFOXGLQJDOOORJLFLWHPV
WKHSDQHOFRXOGQRWUHDFKDQ\DJUHHPHQWUHJDUGLQJWKH',)VRXUFH7DEOHVXPPDUL]HV
WKHUHDVRQVIRU',)E\LWHPW\SH
7DEOH
5HDVRQVIRU',)E\,WHP7\SH
5HDVRQV IRU
&KDQJHV ,Q
,WHP 7\SH
$QDORJLHV
6HQWHQFH &RPSOHWLRQV
/RJLF
5HDGLQJ &RPSUHKHQVLRQ
7RWDO
',)
:RUG
'LIILFXOW\
,WHP
&RQWHQW
,WHP
)RUPDW
&XOWXUDO
8QNQRZQ
5HOHYDQFH
5HDVRQ
7RWDO
7KHWUDQVODWRUVSRLQWHGRXWWKDWWKHWZRODQJXDJHVGLIIHULQWKHQXPEHURIGLIILFXOW
ZRUGVZKLFKWKH\FRQWDLQ7KLVIDFWPDNHVWKHWDVNRIWUDQVODWLQJZKLOHSUHVHUYLQJHTXDO
GLIILFXOW\RIWKHZRUGVDOPRVWLPSRVVLEOH7KHYHUEDOVXEWHVWDOVRFRQVLVWVRIDQWRQ\PVDQG
V\QRQ\PVZKLFKDUHFRQVWUXFWHGHVSHFLDOO\IRUWKH5XVVLDQYHUVLRQRIWKHWHVW7KH
WUDQVODWRUVZKRFRQVWUXFWWKHVHLWHPVVD\WKDWWKH5XVVLDQODQJXDJHFRQWDLQOHVVGLIILFXOW
ZRUGVWRFKRRVHIURP,Q+HEUHZWKHUHDUHWZRNLQGVRIGLIILFXOWZRUGV2OGZRUGV
XVXDOO\IURPWKH%LEOH0RGHUQZRUGVZKLFKZHUHLQYHQWHGLQWKHODVWFHQWXU\DQGDUH
UDUHO\XVHGHJ+HEUHZZRUGIRUDQVZHULQJPDFKLQHEDQDQDHWF,Q5XVVLDQWKHVHWZR
VRXUFHVIRUGLIILFXOWZRUGVGRQRWH[LVW/DQJXDJHVDOVRGLIIHULQWKHOHQJWKRIWKHLU
VHQWHQFHV)RUH[DPSOHDOLQHSDVVDJHLQ+HEUHZLVWUDQVODWHGLQWRDOLQHSDVVDJH
LQ5XVVLDQ,WZDVDQWLFLSDWHGWKDWWKHGLIIHUHQWOHQJWKVZRXOGEHWKHUHDVRQIRUDOORUDWOHDVW
PRVWRIWKH',)LQWKHUHDGLQJFRPSUHKHQVLRQLWHPVEXWVXFKDQHIIHFWZDVQRWIRXQG
'LVFXVVLRQ
8QGHUVWDQGLQJ',)VRXUFHVLQWUDQVODWHGYHUEDOLWHPVLVDQDUHDZKLFKZDUUDQWV
IXUWKHUHPSLULFDOUHVHDUFK7KHFXUUHQWVWXG\IRFXVHGRQWZRTXHVWLRQV,V',)UHODWHG
WRLWHPFRQWHQWRULWHPIRUPDW":KDWDUHWKHVRXUFHVRI',)"7KHGDWDZDVWDNHQ
IURPWKUHHIRUPVLQ+HEUHZVRXUFHDQGLQ5XVVLDQWUDQVODWHGRIWKHYHUEDOVXEWHVW
RIWKH,VUDHOL3(7',)ZDVGHWHFWHGXVLQJWKH0DQWHO+DHQV]HOPHWKRG7KHVRXUFHV
RI',)ZHUHDVVHVVHGE\DQDO\]LQJWKH',)GLUHFWLRQDQGE\XVLQJWUDQVODWRUVWR
DQDO\]HWKHW\SHDQGFRQWHQWRIWKH',)LWHPV7KHFRQFOXVLRQVRIWKHVWXG\FDQEH
DUUDQJHGE\LWHPW\SHDQGE\',)VRXUFH
&RQFOXVLRQV E\ LWHP W\SH
$QDORJLHV7KLVZDVWKHPRVWSUREOHPDWLFRIWKHIRXULWHPW\SHV0DQ\RIWKH
DQDORJ\LWHPVGLVSOD\HG',)RIWKHLWHPVGLVSOD\HGODUJH',)
GLVSOD\HGPRGHUDWH',)$QLQWHUHVWLQJILQGLQJLVWKDWLQRIWKH',)LWHPVWKH
5XVVLDQH[DPLQHHVSHUIRUPHGEHWWHUWKDQWKH+HEUHZH[DPLQHHV2QHRIWKHPDLQ
IDFWRUVDIIHFWLQJWKHGLIILFXOW\RIDQDORJLHVLVZRUGGLIILFXOW\VHH%HMDUHWDO
5RFFDV0RVKLQVN\,QWUDQVODWLQJDQDQDORJ\WUDQVODWRUVRIWHQFKRRVHDQ
HDVLHUZRUGZKLFKPDNHVWKHDQDORJ\HDVLHU2IFRXUVHWKH\FRXOGWKHRUHWLFDOO\
FKRRVHZRUGVWKDWDUHPRUHGLIILFXOWEXWWKLVZDVQRWWKHFDVHLQWKLVVWXG\7KHIHZ
LWHPVRQZKLFKWKH+HEUHZVSHDNHUVSHUIRUPHGEHWWHUFRQWDLQHGWUDQVODWLRQHUURUV
$QRWKHUSRVVLELOLW\WKDWVKRXOGEHPHQWLRQHGEXWZKLFKVHHPVWRXVWREHKLJKO\
XQOLNHO\LVWKDWWKH5XVVLDQVSHDNLQJH[DPLQHHVSHUIRUPEHWWHURQDQDORJLHV
6HQWHQFH&RPSOHWLRQV7KLVLWHPW\SHZDVDOVRSUREOHPDWLFEXWOHVVVRWKDQ
DQDORJLHV0DQ\VHQWHQFHFRPSOHWLRQLWHPVGLVSOD\HG',)ODUJH',)
PRGHUDWH',)1HLWKHUJURXSRXWSHUIRUPHGWKHRWKHULQRIWKH',)LWHPV
WKH5XVVLDQVSHDNLQJH[DPLQHHVSHUIRUPHGEHWWHU'XHWRODQJXDJHGLIIHUHQFHVLWLV
GLIILFXOWWRUHWDLQWKHH[DFWIRUPDWRIWKHVHQWHQFHV7KLVZDVWKHPDLQUHDVRQIRU',)
LQWKHVHQWHQFHFRPSOHWLRQLWHPV7KHVHFRQGPDLQUHDVRQIRU',)ZDVZRUG
GLIILFXOW\
/RJLF7KHORJLFLWHPVH[KLELWHGDOPRVWQR',)2QO\RQHLWHPRXWRI
GLVSOD\HGODUJH',)DQGRQO\WZRDGGLWLRQDOLWHPVVKRZHGPRGHUDWH',)7KHVH
UHVXOWVDUHFRQVLVWHQWZLWKWKHFRQFOXVLRQRI$QJRII&RRNWKDWWKHORQJHUWKH
TXHVWLRQWKHOHVV',)WKDWH[LVWV7KH\DUHDOVRFRQVLVWHQWZLWKDQHZLGHD,WLVHDVLHU
WRWUDQVODWHDQGWRSUHVHUYHWKHYDULRXVFKDUDFWHULVWLFVRIYHUEDOORJLFLWHPVEHFDXVH
WKHORJLFFRPSRQHQWRIWKHTXHVWLRQLVOHVVGHSHQGHQWXSRQDWUDQVODWLRQWKDWSURYLGHV
DQH[DFWPHDQLQJ,QWKLVVWXG\WKLVLWHPW\SHZDVWKHEHVWW\SHIRUWHVWDGDSWDWLRQV
5HDGLQJ&RPSUHKHQVLRQ7KHUHDGLQJFRPSUHKHQVLRQLWHPVGLVSOD\HGD
UHODWLYHO\VPDOODPRXQWRI',)7KHUHZHUHGLIIHUHQFHVLQWKH',)RIWKHVL[SDVVDJHV
ZKLFKZHUHDQDO\]HGVHH$SSHQGL[$7KHLWHPVWKDWGLVSOD\HGODUJH',)LQYROYHG
WZRRXWRIVL[SDVVDJHV)URPWKHVHUHVXOWVLWDSSHDUVWKDW',)LVUHODWHGWRWKH
VSHFLILFFRQWHQWRIWKHSDVVDJH)RUH[DPSOHWKUHH',)LWHPVZHUHIRXQGLQDSDVVDJH
ZKRVHFRQWHQWGHDOWZLWKHFRQRPLFVDQGLWFRXOGEHDUJXHGWKDWWKH5XVVLDQVSHDNHUV
ZHUHOHVVIDPLOLDUZLWKWKLVDUHD
',) 6RXUFHV
%DVHGRQWKHUHVXOWVRIWKLVVWXG\DEDVLFIORZFKDUWUHSUHVHQWLQJWKHSURFHVVLQYROYHG
LQLGHQWLI\LQJ',)VRXUFHVLQDQLWHPLVSUHVHQWHGLQ)LJXUH7KLVIORZFKDUWFDQEH
XVHGURXWLQHO\E\WKHWUDQVODWRUV
)LJXUH
)ORZFKDUWIRU',)VRXUFHVIRXQGLQWKLVVWXG\
,WHPLVDGDSWHGIURPVRXUFHWRWDUJHWODQJXDJH
\HV
,VWKHWUDQVODWLRQFRUUHFW" QR
\HV
'RWKHZRUGVKDYHWKHVDPHGLIILFXOW\"
\HV
QR 7KHUHLV
'LGWKHIRUPDWUHPDLQH[DFWO\WKHVDPH"QR
SUREDEO\
',)
\HV
$UHWKHUHGLIIHUHQFHVLQFXOWXUDOUHOHYDQFH" \HV
QR
7KHUHLVSUREDEO\QR',)
2QHOLPLWDWLRQRIWKLVVWXG\LVWKDWWKHUHZHUHRQO\WZRVSHFLILFODQJXDJHVLQYROYHG
+RZHYHUWKHFRQFOXVLRQVGRQRWDSSHDUWREHODQJXDJHVSHFLILF5HSOLFDWLQJWKLV
VWXG\XVLQJRWKHUODQJXDJHVDQGSHUIRUPLQJVLPXOWDQHRXVDQDO\VHVRI³PXOWLSOH
',)´EHWZHHQVHYHUDOODQJXDJHVLVGHVLUDEOH
)XUWKHUUHVHDUFKRQWKHVRXUFHVRI',)LQWUDQVODWHGLWHPVFDQEHGHVLJQHGRQ
WKHEDVLVRIWKHIROORZLQJLGHDV
)RFXVLQJRQWKHLWHPVWKDWGLGQRWGLVSOD\',)IRUH[DPSOHLQWKHDQDORJ\LWHPV
+RZDUHWKHVHLWHPVGLIIHUHQWIURPWKHDQDORJLHVH[KLELWLQJ',)"
*LYHDTXHVWLRQQDLUHWRH[DPLQHHVLQERWKJURXSVDQGDVNLQJWKHPWRH[SODLQWKHLU
DQVZHU$QDO\VLVRIWKHH[SODQDWLRQVRIWKRVHZKRPLVVHG',)LWHPVPD\EULQJ
EHWWHUXQGHUVWDQGRIWKHFDXVHVRIFKDQJHVLQLWHPGLIILFXOW\EHWZHHQWKHJURXSV
&KDQJLQJWKHLWHPVZLWK',)IRUH[DPSOHFKDQJLQJWKHZRUGVGLIILFXOW\DQG
UHWHVWLQJIRU',)HJ&XUOH\6FKPLWW
)LQDOO\IRUWKHSUREOHPDWLFLWHPW\SHVWKHGHVLUDEOHGHJUHHRIGLIILFXOW\PD\EH
DFKLHYHGRQO\E\FRQVWUXFWLQJVRPHLWHPVGLUHFWO\LQWKHWDUJHWODQJXDJHDQGWKDQ
WUDQVODWHWKHPWRWKH³VRXUFH´ODQJXDJH$',)VWXG\RQWKHVHLWHPVZRXOGEHRI
LQWHUHVW
$WUDQVODWHGWHVWFDQEHHTXDWHGWRWKHVRXUFHWHVWHYHQLIVRPHLWHPVLQWKH
WDUJHWODQJXDJHIXQFWLRQGLIIHUHQWO\IURPWKHVRXUFHLWHPV7KLVLVEHFDXVHRQO\QRQ
',)WUDQVODWHGLWHPVVKRXOGEHXVHGIRUHTXDWLQJ+RZHYHUWKHVPDOOHUWKHQXPEHURI
',)LWHPVWKHEHWWHUWKHWUDQVODWHGWHVWLWZLOOEHPRUHVLPLODUWRWKHVRXUFHWHVWDQG
WKHHTXDWLQJZLOOEHPRUHDFFXUDWH
,QVXPPDU\DQGPRVWLPSRUWDQWO\IRUIXWXUHWHVWWUDQVODWLRQZRUN
LGHQWLILFDWLRQRIWKHVRXUFHVRUFDXVHVRI',)LVQRWOLNHO\WREHDFKLHYHGRQO\WKURXJK
VWDWLVWLFDODQDO\VLVRILWHPUHVSRQVHGDWD6WDWLVWLFDODQDO\VHVDUHYHU\KHOSIXOLQ
GHWHFWLQJSUREOHPDWLFLWHPVEXWWKH\GRQRWUHYHDOWKHFDXVHVRIWKHSUREOHPV
:KLOHWKHUHVXOWVDUHRIYDOXHWRLQGLYLGXDOVLQWHUHVWHGLQFURVVQDWLRQDODQG
FURVVFXOWXUDOVWXGLHVWKHILQGLQJVDUHDOVRLPSRUWDQWPHWKRGRORJLFDOO\7KHVWXG\
GHVLJQFRPELQHGZLWKWKHDQDO\VLVPHWKRGVXVHGLQWKLVVWXG\FDQERWKLGHQWLI\',)
LWHPVDQGDOVRLGHQWLI\WKHFDXVHVRI',)IRUPRVWRIWKHLWHPVGHWHFWHGDV',)LQ
RWKHUVWXGLHVLQWKLVDUHDDVZHOO6XFKLGHQWLILFDWLRQVFDQLPSURYHWKHGHYHORSPHQWRI
WUDQVODWHGWHVWVDQGHQKDQFHWKHLUVFRUHYDOLGLW\
5HIHUHQFHV
$OODORXI $ %DVWDUL 6LUHFL 6 * +DPEOHWRQ 5 . 2FWREHU
&RPSDULQJ WKH GLPHQVLRQDOLW\ RI D WHVW DGPLQLVWHUHG LQ WZR ODQJXDJHV 3DSHU
SUHVHQWHG DW WKH PHHWLQJ RI WKH 1RUWKHDVWHUQ (GXFDWLRQDO 5HVHDUFK $VVRFLDWLRQ
(OOHQYLOOH1<
$QJRII : + $ WHFKQLTXH IRU WKH LQYHVWLJDWLRQ RI FXOWXUDO GLIIHUHQFHV
3DSHUSUHVHQWHGDWWKHPHHWLQJRIWKH$PHULFDQ3V\FKRORJLFDO$VVRFLDWLRQ+RQROXOX
(5,&'RFXPHQW5HSURGXFWLRQ6HUYLFH1R('
$QJRII:+&RRN//(TXDWLQJWKHVFRUHVRIWKH3UXHEDGH$SWLWXG
$FDGHPLFD DQG WKH 6FKRODVWLF $SWLWXGH 7HVW &ROOHJH %RDUG 5HSRUW 1R 1HZ
<RUN&ROOHJH(QWUDQFH([DPLQDWLRQ%RDUG
$QJRII : + 0RGX & & (TXDWLQJ WKH VFRUHV RI WKH 3UXHED GH
$SWLWXG $FDGHPLFD DQG WKH 6FKRODVWLF $SWLWXGH 7HVW 5HVHDUFK 5HSRUW 1R 1HZ
<RUN&ROOHJH(QWUDQFH([DPLQDWLRQ%RDUG
%HMDU , , &KDIILQ 5 (PEHUWVRQ 6 &RJQLWLYH DQG SV\FKRPHWULF
DQDO\VHVRIDQDORJLFDOSUREOHPVROYLQJ1HZ<RUN6SULQJHU9HUODJ
%HOOHU 0 3V\FKRPHWULF DQG VRFLDO LVVXHV LQ DGPLVVLRQV WR ,VUDHOL
XQLYHUVLWLHV(GXFDWLRQDO0HDVXUHPHQW,VVXHVDQG3UDFWLFH
%HOOHU 0 7UDQVODWHG YHUVLRQV RI ,VUDHO¶V LQWHUXQLYHUVLW\ 3V\FKRPHWULF
(QWUDQFH7HVW3(7,Q72DNODQG5.+DPEOHWRQ(GV,QWHUQDWLRQDOSHUVSHFWLYH
RIDFDGHPLFDVVHVVPHQW%RVWRQ0$.OXZHU$FDGHPLF3XEOLVKHUV
&XUOH\ : ( 6FKPLWW $ 3 5HYLVLQJ 6$79HUEDO LWHPV WR HOLPLQDWH
GLIIHUHQWLDO LWHP IXQFWLRQLQJ &ROOHJH %RDUG 5HSRUW 1R 1HZ <RUN &ROOHJH
(QWUDQFH([DPLQDWLRQ%RDUG
'RUDQV 1 - +ROODQG 3 : ',) GHWHFWLRQ DQG GHVFULSWLRQ
0DQWHO+DHQV]HO DQG 6WDQGDUGL]DWLRQ ,Q 3: +ROODQG DQG + :DLQHU (GV
'LIIHUHQWLDOLWHPIXQFWLRQLQJ+LOOVGDOH1-(UOEDXP
(OOLV %% 'LIIHUHQWLDO LWHP IXQFWLRQLQJ ,PSOLFDWLRQ IRU WHVW WUDQVODWLRQ
-RXUQDORI$SSOLHG3V\FKRORJ\
(OOLV %% $ SDUWLDO WHVW RI +XOLQ¶V SV\FKRPHWULF WKHRU\ RI PHDVXUHPHQW
HTXLYDOHQFH LQ WUDQVODWHG WHVWV (XURSHDQ -RXUQDO RI 3V\FKRORJLFDO $VVHVVPHQW *DIQL 1 &DQDDQ<HKRVKDIDW = $Q H[DPLQDWLRQ RI GLIIHUHQWLDO LWHP
IXQFWLRQLQJ IRU +HEUHZ DQG 5XVVLDQVSHDNLQJ H[DPLQHHV LQ ,VUDHO 3DSHU SUHVHQWHG DW
WKHFRQIHUHQFHRIWKH,VUDHOL3V\FKRORJLFDO$VVRFLDWLRQ5DPDW*DQ
+DPEOHWRQ 5 . *XLGHOLQHV IRU DGDSWLQJ HGXFDWLRQDO DQG SV\FKRORJLFDO
WHVWV$SURJUHVVUHSRUW(XURSHDQ-RXUQDORI3V\FKRORJLFDO$VVHVVPHQW
+DPEOHWRQ 5 . DQG -RQHV &RPSDULVRQ RI HPSLULFDO DQG MXGJPHQWDO
SURFHGXUHVIRUGHWHFWLQJGLIIHUHQWLDOLWHPIXQFWLRQLQJ(GXFDWLRQDO5HVHDUFK4XDUWHUO\
+ROODQG 3 : 7KD\HU ' 7 'LIIHUHQWLDO LWHP SHUIRUPDQFH DQG WKH
0DQWHO+DHQV]HO SURFHGXUH ,Q + :DLQHU + %UDXQ (GV 7HVW YDOLGLW\ +LOOVGDOH
1-/DZUHQFH(UOEDXP$VVRFLDWHV
+XOLQ & / $ SV\FKRPHWULF WKHRU\ RI HYDOXDWLRQV RI LWHP DQG WHVW
WUDQVODWLRQV )LGHOLW\ DFURVV ODQJXDJHV -RXUQDO RI &URVV&XOWXUDO 3V\FKRORJ\ 5RFFDV 6 0RVKLQVN\ $ )DFWRUV DIIHFWLQJ WKH GLIILFXOW\ RI YHUEDO
DQDORJLHV 1,7( UHSRUW QR 1DWLRQDO ,QVWLWXWH IRU 7HVWLQJ DQG (YDOXDWLRQ
-HUXVDOHP
5RJHUV + - 6ZDPLQDWKDQ + +DPEOHWRQ 5 . ',&+2',) $
)2575$1 SURJUDP IRU ',) DQDO\VLV RI GLFKRWRPRXVO\ VFRUHG LWHP UHVSRQVH GDWD >$
FRPSXWHUSURJUDP@$PKHUVW0$8QLYHUVLW\RI0DVVDFKXVHWWV
6FKHXQHPDQ-'$QH[SHULPHQWDOH[SORUDWRU\VWXG\RIWKHFDXVHVRIELDV
LQWHVWLWHPV-RXUQDORI(GXFDWLRQDO0HDVXUHPHQW
6FKPLWW $ 3 %OHLVWHLQ & $ )DFWRUV DIIHFWLQJ GLIIHUHQWLDO LWHP
IXQFWLRQLQJ RI EODFN H[DPLQHHV RQ 6FKRODVWLF $SWLWXGH 7HVW DQDORJ\ LWHPV 5HVHDUFK
5HSRUW3ULQFHWRQ1-(GXFDWLRQDO7HVWLQJ6HUYLFH
6LUHFL6*3UREOHPVDQGLVVXHVLQOLQNLQJDVVHVVPHQWVDFURVVODQJXDJHV
(GXFDWLRQDO0HDVXUHPHQW,VVXHVDQG3UDFWLFH
6LUHFL 6 * 6ZDPLQDWKDQ + 2FWREHU (YDOXDWLQJ WUDQVODWLRQ
HTXLYDOHQFH 6R ZKDW¶V WKH ELJ GLI" 3DSHU SUHVHQWHG DW WKH DQQXDO PHHWLQJ RI WKH
1RUWKHDVWHUQ(GXFDWLRQDO5HVHDUFK$VVRFLDWLRQ(OOHQYLOOH1<
6ZDPLQDWKDQ+5RJHUV + - 'HWHFWLQJ GLIIHUHQWLDO LWHP IXQFWLRQLQJ
XVLQJORJLVWLFUHJUHVVLRQSURFHGXUHV-RXUQDORI(GXFDWLRQDO0HDVXUHPHQW
7KLVVHQ ' 6WHLQEHUJ / :DLQHU + 'HWHFWLRQ RI GLIIHUHQWLDO LWHP
IXQFWLRQLQJ XVLQJ WKH SDUDPHWHUV RI LWHP UHVSRQVH PRGHOV ,Q 3: +ROODQG +
:DLQHU(GV'LIIHUHQWLDOLWHPIXQFWLRQLQJ+LOOVGDOH1-(UOEDXP
$SSHQGL[$
0+'',)YDOXHVIRUDOORIWKHLWHPVLQWKH+HEUHZ5XVVLDQFRPSDULVRQE\IRUP
3RVLWLYHYDOXHVRI0+'',)PHDQWKDWWKH5XVVLDQH[DPLQHHVSHUIRUPHGEHWWHU$1
$QDORJLHV6& 6HQWHQFH&RPSOHWLRQ/2 /RJLF5& 5HDGLQJ&RPSUHKHQVLRQ
)RUP
,WHP
1R
,WHP 0+
7\SH '',)
$1 6& /2 5& $1 6& /2 5& ,WHP
1R
)RUP
,WHP 0+
7\SH '',)
$1 6& /2 5& $1 6& /2 5& )RUP
,WHP
1R
,WHP 0+
7\SH '',)
$1 6& /2 5& $1 6& /2 5& $SSHQGL[ % 6RXUFHV RI ',) LQ WKH ',) LWHPV
,WHP 7\SH
$QDORJLHV
$1
1R
,WHP
%HWWHU 3HUIRUPDQFH E\
5XVVLDQ 6SHDNHUV
5XVVLDQ 6SHDNHUV
5XVVLDQ 6SHDNHUV
+HEUHZ 6SHDNHUV
5XVVLDQ 6SHDNHUV
5XVVLDQ 6SHDNHUV
5XVVLDQ 6SHDNHUV
5XVVLDQ 6SHDNHUV
+HEUHZ 6SHDNHUV
5XVVLDQ 6SHDNHUV
5XVVLDQ 6SHDNHUV
5XVVLDQ 6SHDNHUV
5XVVLDQ 6SHDNHUV
+HEUHZ 6SHDNHUV
5XVVLDQ 6SHDNHUV
5XVVLDQ 6SHDNHUV
5XVVLDQ 6SHDNHUV
6HQWHQFH
&RPSOHWLRQV
5XVVLDQ 6SHDNHUV
5XVVLDQ 6SHDNHUV
+HEUHZ 6SHDNHUV
5XVVLDQ 6SHDNHUV
5XVVLDQ 6SHDNHUV
+HEUHZ 6SHDNHUV
+HEUHZ 6SHDNHUV
+HEUHZ 6SHDNHUV
5XVVLDQ 6SHDNHUV
+HEUHZ 6SHDNHUV
5XVVLDQ 6SHDNHUV
+HEUHZ 6SHDNHUV
5XVVLDQ 6SHDNHUV
5XVVLDQ 6SHDNHUV
+HEUHZ 6SHDNHUV
&XOWXUDO 'LIIHUHQFHV
:RUG 'LIILFXOW\
6HQWHQFH 'LIILFXOW\
&KDQJHV LQ )RUPDW
7UDQVODWLRQ 3UREOHP
&KDQJHV LQ )RUPDW
&KDQJHV LQ )RUPDW
7UDQVODWLRQ 3UREOHP
8QNQRZQ
&KDQJHV LQ )RUPDW
:RUG 'LIILFXOW\
7UDQVODWLRQ 3UREOHP
:RUG 'LIILFXOW\
8QNQRZQ
&KDQJHV LQ )RUPDW
/2
+HEUHZ 6SHDNHUV
+HEUHZ 6SHDNHUV
+HEUHZ 6SHDNHUV
8QNQRZQ
8QNQRZQ
8QNQRZQ
5HDGLQJ
&RPSUHKHQVLRQ
+HEUHZ 6SHDNHUV
+HEUHZ 6SHDNHUV
+HEUHZ 6SHDNHUV
+HEUHZ 6SHDNHUV
5XVVLDQ 6SHDNHUV
5XVVLDQ 6SHDNHUV
5XVVLDQ 6SHDNHUV
8QNQRZQ
&XOWXUDO 'LIIHUHQFHV
&XOWXUDO 'LIIHUHQFHV
&XOWXUDO 'LIIHUHQFHV
&XOWXUDO 'LIIHUHQFHV
&XOWXUDO 'LIIHUHQFHV
8QNQRZQ
6&
/RJLF
5&
([SODQDWLRQ IRU ',)
:RUG 'LIILFXOW\
:RUG 'LIILFXOW\
:RUG 'LIILFXOW\
7UDQVODWLRQ 3UREOHP
:RUG 'LIILFXOW\
:RUG 'LIILFXOW\
:RUG 'LIILFXOW\
7UDQVODWLRQ 3UREOHP
7UDQVODWLRQ 3UREOHP
:RUG 'LIILFXOW\
7UDQVODWLRQ 3UREOHP
:RUG 'LIILFXOW\
:RUG 'LIILFXOW\
7UDQVODWLRQ 3UREOHP
:RUG 'LIILFXOW\
:RUG 'LIILFXOW\
:RUG 'LIILFXOW\