'LVWULEXWLRQDO5HSUHVHQWDWLRQVRI,GLRPV
)HUQDQGR6D3HUHLUD
&RPSXWHU6FLHQFH
0F*LOO8QLYHUVLW\0RQWUHDO
)LQDO6XEPLVVLRQ$SULO
$WKHVLVVXEPLWWHGWR0F*LOO8QLYHUVLW\LQSDUWLDOIXOILOPHQWRIWKHUHTXLUHPHQWVRI
WKHGHJUHHRI0DVWHURI6FLHQFH
‹)HUQDQGR6D3HUHLUD
Acknowledgements
,WKDVEHHQP\JUHDWSULYLOHJHWRZRUNRQWKLVWKHVLVZLWKP\DGYLVRU3URI'RLQD3UHFXS
6KHKDVSURYLGHGLQQXPHUDEOHVXJJHVWLRQVSUHFLRXVIHHGEDFNDQGPXFKHQFRXUDJHPHQW6KHKDV
DOVR EHHQ VWHDGIDVW LQ PDLQWDLQLQJ DQ XQIODSSDEOH DQG VDQJXLQH SHUVSHFWLYH WKURXJK P\ PDQ\
FUHVWV DQG YDOOH\V RYHU WKHVH ODVW VHYHUDO PRQWKV 7KH ILUVW WKUHH TXDOLWLHV ZHUH RI HQRUPRXV
EHQHILWQRWRQO\WRP\ZRUNEXWWKLVHQWLUHOHDUQLQJH[SHULHQFH7KHODWWHUNHSWPHIRFXVVHGDQG
GLVFLSOLQHG
,DOVRH[WHQGP\VLQFHUHWKDQNVDQGDSSUHFLDWLRQWRP\FRVXSHUYLVRU3URI'HEUD7LWRQH
,QDGGLWLRQWRKHUERXQGOHVVJHQHURVLW\LQVKDULQJKHUH[SHUWLVHDQGLQVLJKWVRQLGLRPVVKHKDV
EHHQHTXDOO\JHQHURXVLQVKDULQJUHVRXUFHVDQGGDWDIURPKHUODE6KHZDVDOZD\VHQFRXUDJLQJ
ZKLFK , DSSUHFLDWHG EXW , DOVR YDOXHG KHU NQDFN IRU DVNLQJ GLIILFXOW TXHVWLRQV DQG KHU PDQ\
KHOSIXOSUHFLVHDQGFRQVWUXFWLYHFRPPHQWVDQGVXJJHVWLRQV
, SUHVHQWHG VRPH LQLWLDO ILQGLQJV WR WKH &RPSXWDWLRQDO /LQJXLVWLF /DE DQG WKH &RJQLWLYH
$UHD 6HPLQDU ERWK DW 0F*LOO 7KH TXHVWLRQV DQG FRPPHQWV ZHUH YHU\ KHOSIXO LQ LGHQWLI\LQJ
ZKDW DVSHFWV QHHGHG PRGLILFDWLRQ FODULILFDWLRQ DQG HPSKDVLV 7KH ILQDO SURGXFW KDV JUHDWO\
EHQHILWWHGDVDUHVXOW
0\ IULHQGV DQG IDPLO\ QHYHU RQFH TXHVWLRQHG P\ FUD]\ GHFLVLRQ WR VWRS ZRUNLQJ DQG JR
EDFN WR VFKRRO DW WKLV SRLQW LQ P\ OLIH $QG WKH\ KDYH QHYHU ZDYHUHG LQ WKHLU VXSSRUW DQG
XQGHUVWDQGLQJ,WKDQNWKHPIRUWKDWIRUWKHLUVDFULILFHVRQP\EHKDOIDQGIRU±TXLWHOLWHUDOO\±
HYHU\WKLQJWKDW,YDOXHLQP\OLIH
Abstract
,GLRPVVXFKDV³VKRRWWKHEUHH]H´XVHGWRPHDQ³FKDW´SRVHDVLJQLILFDQWFKDOOHQJHLQ1DWXUDO
/DQJXDJH3URFHVVLQJ1/32QHGLIILFXOW\LVWKDWE\GHILQLWLRQWKHLGLRPDWLFSKUDVH¶VPHDQLQJ
FDQQRW EH GHULYHGIURP DOLWHUDO UHDGLQJ$QRWKHULV WKDWVRPHSKUDVHVOLNH³KLWDZDOO´FDQ EH
XVHG HLWKHU LGLRPDWLFDOO\ RU OLWHUDOO\ $QG WKLUG WKH VDPH LGLRP FDQ WDNH YDULDQW IRUPV
VRPHWLPHV LW LV LQ WKH SDVVLYH YRLFH ³WKH WDEOHV KDYH WXUQHG´ FRQWDLQV DQ DGGLWLRQDO ZRUG
³ULQJV D IDLQW EHOO´ RU DSSHDUV ZLWK RQH ZRUG VXEVWLWXWHG IRU DQRWKHU ³LQXQGDWH WKH PDUNHW´
UDWKHUWKDQ³IORRG´:HDGGUHVVWKHVHFKDOOHQJHVZLWKDQH[WHQVLRQWRWKHH[LVWLQJDSSURDFKIRU
LQGXFLQJ GLVWULEXWLRQDO UHSUHVHQWDWLRQV XVLQJ WKH SURJUDP ZRUGYHF 7UDLQLQJ UHTXLUHV QR
VXSHUYLVLRQ EH\RQG SURYLGLQJ WKH OLVW RI WDUJHW LGLRPV 7KH UHVXOWLQJ UHSUHVHQWDWLRQV DUH QRW
VSHFLILFWRDVLQJOHWDVN,QGHHGZHWHVWHGWKHLUXWLOLW\DQGUREXVWQHVVE\GHSOR\LQJWKHPLQWDVNV
WKDWUHIOHFWWKUHHFRUH1/3IXQFWLRQVVXPPDUL]DWLRQGLVDPELJXDWLRQDQGSURGXFWLRQ:HIRXQG
WKDWPRGHOVXVLQJWKHVHUHSUHVHQWDWLRQVHTXDOOHGRUPRUHRIWHQRXWSHUIRUPHGSUHYLRXVPRGHOV
7HVWLQJYDULDQWVRILGLRPVVKRZHGWKDWWKHVHUHSUHVHQWDWLRQVDUHUREXVWHYHQZKHQWKHLGLRPVGR
QRWWDNHWKHLUXVXDOIRUPV7KHVHUHSUHVHQWDWLRQVFRXOGEHGHSOR\HGLQVHYHUDOIXWXUHDSSOLFDWLRQV
LQFOXGLQJ 0DFKLQH 7UDQVODWLRQ 6HQWLPHQW $QDO\VLV ,QIRUPDWLRQ 5HWULHYDO DQG 6RFLDO 0HGLD
0RQLWRULQJ
Résumé
/HV H[SUHVVLRQV LPDJpHV WHO TXH ©PDQTXHU OH EDWHDX ª HQ YRXODQW GLUH ©ODLVVHU SDVVHU XQH
RSSRUWXQLWpª SRVHQW XQ GpIL GH WDLOOH GDQV OH GRPDLQH GH WUDLWHPHQW DXWRPDWLTXH GX ODQJDJH
QDWXUHO'¶DERUGSDUODQDWXUHPrPHGHFHVH[SUHVVLRQVOHVHQVGHO¶H[SUHVVLRQGLIIqUHGXVHQV
GHV PRWV GHVTXHOV O¶H[SUHVVLRQ HVW FRPSRVpH 'HX[LqPHPHQW FHUWDLQHV H[SUHVVLRQV GRQW
©IUDSSHUXQPXUªVRQWSDUIRLVXWLOLVpHVDXVHQVOLWWpUDODLQVLTX¶LGLRPDWLTXH(WWURLVLqPHPHQW
ODPrPHH[SUHVVLRQSHXWVHWURXYHUDYHFXQPRWVXSSOpPHQWDLUH©VRQQHUXQHSHWLWHFORFKHªRX
XQPRWjODSODFHG¶XQDXWUH©UDWHUOHEDWHDXªDXOLHXGH©PDQTXHUª/DPpWKRGHTXHQRXV
SURSRVRQV HVW XQH H[WHQVLRQ G¶XQ ORJLFLHO ZRUGYHF GpMj XWLOLVp DILQ GH JpQpUHU GHV
UHSUpVHQWDWLRQVYHFWRULHOOHVGHPRWV,OV¶DJLWG¶XQHPpWKRGHG¶DSSUHQWLVVDJHVDQVVXSHUYLVLRQj
SDUW GHIRXUQLUODOLVWHGHV H[SUHVVLRQVFLEOpHV /HVUHSUpVHQWDWLRQV TXL HQUpVXOWHQW QHVRQWSDV
OLPLWpHV j XQH WkFKH XQLTXH (Q HIIHW QRXV DYRQV WHVWp OHXU XWLOLWp HQ OHV GpSOR\DQW GDQV GHV
WkFKHVTXLUHIOqWHQWWURLVIRQFWLRQVGHEDVHODSDUDSKUDVHODGpVDPELJXwVDWLRQHWODSUpGLFWLRQ
&HV WHVWV GpPRQWUHQW TX¶XQ PRGqOH TXL VH VHUW GH FHV UHSUpVHQWDWLRQV SHXW pJDOHU RX SOXV
VRXYHQWGpSDVVHUOHVPRGqOHVSUpFpGHQWV PrPHORUVTXHO¶H[SUHVVLRQ LPDJpHQ¶HVW SDV VRXV VD
IRUPH KDELWXHOOH &HV UHSUpVHQWDWLRQV SRXUURQW pYHQWXHOOHPHQW VHUYLU DX SURILW GH OD WUDGXFWLRQ
DXWRPDWLVpH O¶DQDO\VH GH VHQWLPHQW OD UHFKHUFKH GH GRFXPHQWV HW OD VXUYHLOODQFH GHV PpGLDV
VRFLDX[
Contents
1 Introduction ..................................................................................................................................... 1
1.1
Outline .................................................................................................................................................... 5
2 Idioms ................................................................................................................................................. 7
2.1
Idioms: A Working Definition ........................................................................................................ 7
2.2
Idioms and Compositionality ......................................................................................................... 9
2.3
Variability in Idioms....................................................................................................................... 11
3 The Computational Linguistics Background ...................................................................... 13
3.1
3.1.1
The Distributional Hypothesis and Distributional Vector Space Models ................... 13
Polysemy ......................................................................................................................................................... 18
3.2
Compositionality.............................................................................................................................. 18
3.3
Interpreting Conventional Metaphors Using Distributional Vector Space Models . 20
3.4
Distinguishing Literal from Figurative Uses .......................................................................... 23
3.5
Other Representations of Idioms............................................................................................... 26
4 Proposed Methods ....................................................................................................................... 29
4.1
Model Selection: Why Distributional Vector Space Models? ........................................... 29
4.2
Training the Word Embeddings ................................................................................................. 32
4.3
Composition Models ....................................................................................................................... 35
4.3.1
Composition of the Idiomatic Phrase .................................................................................................. 35
4.3.2
Composition of the Sentence .................................................................................................................. 35
4.4
4.4.1
4.5
Tasks .................................................................................................................................................... 38
Assessing Variants of Idiomatic Expressions .................................................................................. 39
Scoring Measures............................................................................................................................. 40
5 System Requirements and Processing Time ...................................................................... 42
5.1
Training Time ................................................................................................................................... 42
5.2
Memory Requirements .................................................................................................................. 43
6 The Paraphrasing Tasks ............................................................................................................ 44
6.1
Task Description .............................................................................................................................. 44
6.2
Computational Method .................................................................................................................. 46
6.3
Empirical Results............................................................................................................................. 46
6.3.1
Direct Comparisons of Idioms to Paraphrases................................................................................ 46
6.3.2
The Matching Task...................................................................................................................................... 52
6.3.3
Idiom Variants .............................................................................................................................................. 55
6.4
Summary............................................................................................................................................. 56
7 Classifying Literal vs. Figurative Tokens ............................................................................. 58
7.1
Computational Method .................................................................................................................. 58
7.2
Figurative/Literal Sentence Pairs ............................................................................................. 59
7.2.1
Description of the Dataset ....................................................................................................................... 59
7.2.2
Calculating the Bias .................................................................................................................................... 59
7.2.3
Composition Methods................................................................................................................................ 60
7.2.4
Empirical Results......................................................................................................................................... 61
7.3
Comparison to Previous Work Using the BNC....................................................................... 65
7.3.1
Description of the Dataset ....................................................................................................................... 65
7.3.2
Computational Method ............................................................................................................................. 66
7.3.3
Empirical Results......................................................................................................................................... 66
7.4
Variants ............................................................................................................................................... 69
7.4.1
Syntactic Variants: Passivization .......................................................................................................... 69
7.4.2
Other Variants: Omission and Substitution ...................................................................................... 70
7.5
Summary............................................................................................................................................. 72
8 The Sentence Completion Task ............................................................................................... 73
8.1
Task Description .............................................................................................................................. 73
8.2
Computational Method .................................................................................................................. 74
8.3
Empirical Results............................................................................................................................. 75
8.4
Summary............................................................................................................................................. 81
YL
9 Conclusions and Future Work ................................................................................................. 83
9.1
Summary and Contributions ....................................................................................................... 83
9.2
General Observations .................................................................................................................... 84
9.3
Future Work ...................................................................................................................................... 86
References ............................................................................................................................................. 89
Appendices ............................................................................................................................................ 96
A.
List of idioms used in this work ...................................................................................................... 96
B.
Manually Annotated Idioms ............................................................................................................. 98
C.
Example Sentences from the Figurative/Literal Sentence Pair Dataset .......................... 99
D.
Complete Results for Comparison with Fazly et al. .................................................................. 99
E.
Complete Results for Comparison with Peng et al. ................................................................ 100
F.
Example Sentences from the Sentence Completion Task .................................................... 100
G.
Additional Results for the Sentence Completion Task ......................................................... 101
YLL
1 Introduction
,GLRPVDUHDQLPSRUWDQW DQGSHUYDVLYH SDUWRIRXUHYHU\GD\GLVFRXUVH$OWKRXJKWKH\ DUH
RIWHQ FRQVLGHUHG DVSHFWV RI LQIRUPDO VSHHFK HYHQ VWROLG DFDGHPLF SDSHUV ZLOO XVH SKUDVHV OLNH
³WKH MXU\ LV RXW´ WR PHDQ WKDW VRPHWKLQJ UHPDLQV DQ RSHQ TXHVWLRQ 2QH FRUSXV VWXG\ 0RRQ
XVLQJ WKH 2[IRUG +HFWRU 3LORW &RUSXV IRXQG WKDW DURXQG WKUHH VHQWHQFHV LQ WHQ FRQWDLQ
LGLRPV $QRWKHU IRXQG WKDW LGLRPDWLF YHUE SKUDVHV OLNH ³GURS WKH EDOO´ PHDQLQJ ³IDLO WR PHHW
RQH¶V UHVSRQVLELOLWLHV´ DQG IL[HG FROORFDWLRQV DSSHDU LQ RYHU SHU FHQW RI QHZVSDSHU DUWLFOHV
DQGIHDWXUHV&RZLHFLWHGLQ)HOOEDXP*LYHQWKHLUXELTXLW\DQ\1DWXUDO/DQJXDJH
3URFHVVLQJ 1/3 DSSOLFDWLRQ PXVW DGGUHVV LGLRPV RU ULVN PLVLQWHUSUHWLQJ QHDUO\ RQHWKLUG RI
DQ\ JLYHQ WH[W $QG \HW LQ SUDFWLFH PDQ\ DSSOLFDWLRQV WUHDW DQ LGLRPDWLF SKUDVH DV LI LW ZHUH
PHDQWOLWHUDOO\(YHQWKHPRUHHIIHFWLYHVROXWLRQVDUHVHQVLWLYHWRVOLJKWYDULDWLRQVWKRXJKLWPD\
UHFRJQL]HWKDW³VPHOOLQJDUDW´PHDQVILQGLQJVRPHWKLQJVXVSLFLRXVLWRIWHQZLOOQRWNQRZWKDW
³VPHOOLQJDELJKDLU\UDW´LVMXVWDPRUHHPSKDWLFZD\RIVD\LQJWKHVDPHWKLQJ7RGREHWWHU
WKHVHDSSOLFDWLRQVZRXOGQHHGEHWWHULQWHUQDOUHSUHVHQWDWLRQVRILGLRPVWKDWLVDPRUHHIIHFWLYH
HQFRGLQJRIZKDWIRUPVDQLGLRPPD\WDNHZKDWWKHLGLRPPHDQVDQGKRZLWLVXVHG7KHWRSLF
RI FUHDWLQJ UHSUHVHQWDWLRQV RI ZRUGV KDV EHHQ DQ DFWLYH WRSLF RI UHVHDUFK ZLWKLQ WKH ODVW IHZ
\HDUVEXWVRIDUWKHPHWKRGVFDQQRWEHDSSOLHGYHU\ZHOOWRLGLRPV7KRVHWKDWKDQGOHLGLRPVDW
DOOGRQRWJHQHUDOL]HZHOODFURVVGLIIHUHQWWDVNVDQGODQJXDJHV7KHSUHVHQWZRUNZLOOSURSRVHD
VLPSOHPLQLPDOO\VXSHUYLVHGPHWKRGRIH[WUDFWLQJUHSUHVHQWDWLRQVRILGLRPVIURPDFROOHFWLRQRI
WH[WV,WZLOODOVRGHPRQVWUDWHWKDWVXFKUHSUHVHQWDWLRQVDUHXVHIXOLQDYDULHW\RIUHSUHVHQWDWLYH
FRUH1/3WDVNV
:H VKDOO ILUVW H[SODLQ DQG MXVWLI\ WKH FODLP WKDW FXUUHQW PHWKRGV GR QRW DSSO\ ZHOO WR
LGLRPV ,I ZH ORRN DW VRPH FXUUHQW DSSOLFDWLRQV LW EHFRPHV FOHDU WKDW WKHLU UHSUHVHQWDWLRQV GR
FDSWXUHVRPHDVSHFWVRILGLRPVEXWKDYHQRWDEOHVKRUWFRPLQJVDVZHOO
x 0DFKLQH 7UDQVODWLRQ WKH DXWRPDWHG UHQGHULQJ RI D WH[W LQWR DQRWKHU ODQJXDJH LV D FOHDU
H[DPSOHRIDWDVNLQZKLFKLGLRPVPXVWEHWDNHQLQWRDFFRXQWDVDILJXUDWLYHSKUDVHLQRQH
ODQJXDJHRIWHQGRHVQRWKDYHWKHVDPHPHDQLQJLQDQRWKHU:KHQWKHVHQWHQFH³DIWHUDORQJ
GD\KHKLWWKHVDFN´LHZHQWWREHGLVLQSXWLQWRDQ\RIILYH0DFKLQH7UDQVODWLRQDSSVRU
VLWHVWKH DSSOLFDWLRQ WUDQVODWHV WKH LGLRP DV ³IUDSSp OH VDF´ LQ )UHQFK ³JROSHy HO VDFR´ LQ
6SDQLVK DQG ³WUDI HU GHQ 6DFN´ LQ *HUPDQ $OO WKUHH DUH ZRUGE\ZRUG OLWHUDO WUDQVODWLRQV
WKDW LQ QR ZD\ FRPPXQLFDWH ³JR WR EHG´ LQ WKHLU UHVSHFWLYH ODQJXDJHV $QG ZKLOH *RRJOH
7UDQVODWHDQG35207ERWKWUDQVODWH³VPHOODUDW´LQWRDQDSSURSULDWHFROORTXLDOH[SUHVVLRQ
LQ )UHQFK ³VHQWLU OH URXVVL´ DQG ³VHQWLU TXHOTXH FKRVH GH ORXFKH´ UHVSHFWLYHO\ WKH\ ERWK
UHYHUWWROLWHUDOWUDQVODWLRQVLIWKHLQSXWLV³VPHOODELJUDW´
x 6HQWLPHQW $QDO\VLV D WDVN LQ ZKLFK D JLYHQ WH[W LV FODVVLILHG DFFRUGLQJ WR ZKHWKHU LW
H[SUHVVHVDSRVLWLYHQHXWUDORUQHJDWLYHRSLQLRQPXVWWDNHLQWRDFFRXQW DWWLWXGHVFRQYH\HG
E\LGLRPV$PRYLHUHYLHZWKDWVD\VWKDWWKHILOP³ILUHVRQDOOF\OLQGHUV´LVFOHDUO\SRVLWLYH
ZKLOH D UHVWDXUDQW UHYLHZ VD\LQJ WKDW WKH FKHI ³GURSSHG WKH EDOO´ LV QHJDWLYH <HW WKHVH
VHQWHQFHVDUHPLVFODVVLILHGE\WZRRQOLQHVHQWLPHQWDQDO\VLVWRROVDVEHLQJQHXWUDORUHYHQDV
FRQYH\LQJWKHRSSRVLWHVHQWLPHQW,WLVFOHDUWKDWWKHVHWRROVGLGQRWUHFRJQL]HWKHILJXUDWLYH
VHQVHVRIWKHVHH[SUHVVLRQV
x 6RPHRQH XVLQJ DQ ,QIRUPDWLRQ 5HWULHYDO WRRO OLNH *RRJOH WR ILQG DUWLFOHV DERXW D VXVWDLQHG
SHULRGZKHUHWKHVWRFNPDUNHWSHUIRUPHGSRRUO\PLJKWHQWHU³EHDUPDUNHW´DVDVHDUFKWHUP
7KHVHDUFKHUZRXOGOLNHO\KRSHWKDWKHUVHDUFKZLOODOVRORFDWHDUWLFOHVWKDWXVHH[SUHVVLRQV
OLNH³PDUNHWPHOWGRZQ´³ORQJWHUPFRUUHFWLRQ´³VWRFNPDUNHWFUDVK´RUDQ\RWKHUPHWDSKRU
WKDWILQDQFLDOMRXUQDOLVWVHPSOR\%XWLQWKHLUFXUUHQWVWDWHPRVWVHDUFKWRROVZRXOGRQO\ILQG
DUWLFOHV ZLWK WKHVH RWKHU WHUPV LI WKH\ DOVR KDSSHQ WR KDYH WKH RULJLQDO VHDUFK WHUP ³EHDU
PDUNHW´ VRPHZKHUH LQ WKHLU WH[W .QRZLQJ WKDW WKHVH H[SUHVVLRQV DOO UHIHU WR WKH VDPH
SKHQRPHQRQZRXOGUHVXOWLQDPRUHFRPSOHWHVHDUFKDQGKLJKHUUHFDOO
x 2WKHUPRUHVSHFLDOL]HG1/3WDVNVZRXOGEHQHILWDVZHOO7RJLYHRQHH[DPSOHDQHPHUJLQJ
1/3GRPDLQLVFRQFHUQHGZLWKPRQLWRULQJVRFLDOPHGLDIRUF\EHUEXOO\LQJDQGRWKHUDEXVHV
*RRJOH7UDQVODWH35207IUHHWUDQVODWLRQFRPZRUOGOLQJRFRPDQGFROOLQVGLFWLRQDU\FRPWUDQVODWRU
ZZZVHQWLJHPFRP FODVVLILHV WKH IRUPHU VHQWHQFH DV QHJDWLYH DQG WKH ODWWHU DV SRVLWLYH ZZZWH[W
SURFHVVLQJFRPGHPRVHQWLPHQWFODVVLILHVWKHIRUPHUDVQHJDWLYHDQGWKHODWWHUDVQHXWUDO
7KH&KLHI([HFXWLYH2IILFHRI7ZLWWHUIRUH[DPSOHVWDWHGLQLQWHUQDOPHPRVWKDWKHZDQWVWR
PDNHVXFKGHWHFWLRQDSULRULW\7LNX1HZWRQ$VWKHVHWZHHWVRIWHQLQYROYHVRPH
VRUWRIILJXUDWLYHODQJXDJHXQGHUVWDQGLQJLGLRPVZRXOGEHLQGLVSHQVDEOHWRWKLVWDVN
+RZZRXOGEHWWHUUHSUHVHQWDWLRQVDGGUHVVWKHVHSLWIDOOV"%HIRUHDFRPSXWHUFDQPDNHDQ\
GHFLVLRQRQZKDWDWH[W¶VVXEMHFWLVZKDWHPRWLRQLWH[SUHVVHVRUKRZWRWUDQVODWHLWLQWRDQRWKHU
ODQJXDJHLWPXVWILUVWH[WUDFWWKHPHDQLQJRIWKHWH[WUHQGHULQJLWLQWRDIRUPWKDWWKHFRPSXWHU
FDQ PDQLSXODWH DQG XVH WR PDNH GHFLVLRQV 7KLV HQFRGLQJ LV ZKDW LV PHDQW E\ FUHDWLQJ D
UHSUHVHQWDWLRQ$XVHIXOUHSUHVHQWDWLRQRIDQ\VHPDQWLFXQLWZKHWKHUWKDWEHDZRUGRUDQLGLRP
LVRQHWKDWHQFDSVXODWHVWKHXQLW¶VGHQRWDWLRQWKHFRQWH[WVLQZKLFKLWDSSHDUVWKHFRQQRWDWLRQVLW
KDVDQGZKDWRWKHULQIRUPDWLRQLWLPSOLHV0LOOHU&KDUOHV-XUDIVN\0DUWLQ$
JRRGUHSUHVHQWDWLRQRIDQLGLRPZRXOGQRWRQO\DOORZWKHDSSOLFDWLRQWRKDQGOHWKHLGLRPLQLWV
XVXDOIRUPEXWLQVOLJKWYDULDQWVDVZHOO
$YHU\ VLPSOHUHSUHVHQWDWLRQ LVDORRNXSWDEOH ZKHUHLQLGLRPV DUHPDWFKHGZLWK DOLWHUDO
SDUDSKUDVH RU GHILQLWLRQ HJ 6DOWRQ 5RVV DQG .HOOHKHU :KHQ DQ LGLRP VD\ ³KLW WKH
VDFN´LVIRXQGLQDWH[WZHZRXOGILQGWKHLGLRPLQWKLVWDEOHDQGWH[WXDOO\VXEVWLWXWHLWZLWKLWV
HTXLYDOHQW SKUDVH :H WKHQ SHUIRUP RXU 1/3 WDVN ZKHWKHU WKDW EH 0DFKLQH 7UDQVODWLRQ RU
,QIRUPDWLRQ 5HWULHYDO RQ WKLV OLWHUDO YHUVLRQ 7KH VROXWLRQ LV DSSHDOLQJ EXW KDV LWV GLIILFXOWLHV
2QHLVWKDWZHXVHLGLRPVWRHQFDSVXODWHDIDLUDPRXQWRILQIRUPDWLRQLQWRDVXFFLQFWDQGYLYLG
LPDJH6RPDQ\LGLRPVFDQQRWEHSDUDSKUDVHGLQDVLQJOHZRUGEXWRQO\H[SODLQHGSHUKDSVE\D
VHQWHQFHRUPRUH$QH[DPSOHLV³VKRRWWKHPHVVHQJHU´ZKLFKWKH&DPEULGJH,GLRPV'LFWLRQDU\
GHILQHV DV ³WR EODPH RU SXQLVK WKH SHUVRQ ZKR WHOOV \RX DERXW VRPHWKLQJ EDG« LQVWHDG RI WKH
SHUVRQZKRLVUHVSRQVLEOHIRULW´&DPEULGJH8QLYHUVLW\,IWKHJRDOZDVWRWUDQVODWH
D VHQWHQFH FRQWDLQLQJ WKLV LGLRP DQG WKH VROXWLRQ ZDV WR GURS WKLV OHQJWK\ H[SODQDWLRQ LQ WKH
SODFHRIWKHLGLRPDQGWKHQWUDQVODWHWKHUHVXOWZRXOGEHDYHU\FXPEHUVRPHVHQWHQFHZLWKQRQH
RI WKH RULJLQDO¶V SLWKLQHVV $ VHFRQG GLIILFXOW\ LV WKDW LGLRPV YDU\ DFURVV WLPH DQG JHRJUDSK\
.HHSLQJ WKH OLVW XSWRGDWH DQG DGDSWLQJ LW WR D JLYHQ UHJLRQ ZRXOG UHTXLUH D VXEVWDQWLDO DQG
FRQVWDQWHIIRUW$QGWKHQWKHUHLVWKHDIRUHPHQWLRQHGTXHVWLRQRIYDULDQWV,QGHHGLWPD\EHWKDW
WKHWUDQVODWLRQDSSVDERYHFRXOGQRWWUDQVODWH³VPHOODELJUDW´H[DFWO\EHFDXVHWKH\XVHDPRUH
VRSKLVWLFDWHGYHUVLRQRIWKHWDEOHORRNXSGHVFULEHGDERYH$EHWWHUUHSUHVHQWDWLRQZRXOGEHRQH
WKDW FDSWXUHV PHDQLQJ IURP DFWXDO XVDJH WKDW FDQ EH UHJHQHUDWHG SHULRGLFDOO\ EDVHG RQ DQ
XSGDWHG FROOHFWLRQ RI WH[WV DQG WKDW FDQ UHFRJQL]H D ILJXUDWLYH XVDJH HYHQ ZKHQ WKH LGLRP
DSSHDUVLQYDULDQWIRUPV
)RUWXQDWHO\ D JRRG DOWHUQDWLYH H[LVWV 'LVWULEXWLRQDO UHSUHVHQWDWLRQV DOVR NQRZQ DV ZRUG
HPEHGGLQJV RU YHFWRU VSDFH PRGHOV KDYH EHFRPH ZLGHO\ XVHG LQ WKH 1/3 FRPPXQLW\ 7KH\
UHSUHVHQWDZRUGDVDYHFWRUWKDWLVDQRUGHUHGVHULHVRIQXPEHUV7KHLQGLYLGXDOHOHPHQWVRIWKH
YHFWRUGRQRWFRUUHVSRQGWRDQ\DFWXDOSURSHUWLHVRIWKHZRUG¶VPHDQLQJVXFKDV WKHUHIHUHQW¶V
FRORXU RU VL]H UDWKHU WKH QXPEHUV ZKHQ WDNHQ WRJHWKHU HQFRGH KRZ OLNHO\ WKH ZRUG LV WR EH
IRXQG LQ D YDULHW\ RI FRQWH[WV :RUGV WKDW DUH IRXQG LQ VLPLODU FRQWH[WV OLNH ³F\FORQH´ DQG
³VWRUP´ZLOOKDYHYHU\VLPLODUYHFWRUVZKLOHZRUGVWKDWDUHOHVVIUHTXHQWO\LQWKHVDPHFRQWH[W
³F\FORQH´ DQG ³IDODIHO´ ZLOO QRW 0XFK PRUHGHWDLO LV SURYLGHG LQ 6HFWLRQ 7KLV W\SH RI
UHSUHVHQWDWLRQ KDVEHHQVXFFHVVIXOO\ GHSOR\HG WR UHSUHVHQW ZRUGVSKUDVHVDQGORQJHUVHJPHQWV
RIWH[WLQDEURDGVSHFWUXPRI1/3WDVNVLQFOXGLQJ6HQWLPHQW$QDO\VLV6RFKHUHWDO/H
0LNRORY,QIRUPDWLRQ5HWULHYDO/H0LNRORYDQGPDQ\RWKHUVVHH7XUQH\
3DQWHO IRU D FRPSUHKHQVLYH VXUYH\ RI DSSOLFDWLRQV *HQHUDWLQJ WKHVH UHSUHVHQWDWLRQV LV
GRQHE\VLPSO\SURYLGLQJWKHDSSURSULDWHVRIWZDUHZLWKDFRUSXVRIWH[WVDQGWKHZKROHSURFHVV
FDQ EH FRPSOHWHG ZLWKLQ DQ KRXU 7KDW VDLG FXUUHQW PHWKRGV IRU FUHDWLQJ GLVWULEXWLRQDO
UHSUHVHQWDWLRQVGRQRWSHUIRUPZHOOZKHQLWFRPHVWRLGLRPVDVZHVKDOOGHPRQVWUDWH)RUDOORI
WKHVH UHDVRQV WKLV GLVVHUWDWLRQ ZLOO H[SORUH KRZ ZH FDQ DGDSW WKH H[LVWLQJ PHWKRGV WR WKH
SDUWLFXODUDVSHFWVWKDWPDNHLGLRPVGLIIHUHQWIURPRWKHUNLQGVRIODQJXDJH
,IRXURYHUDOOREMHFWLYHLVWRDUULYHDWXVHIXODQGUREXVWUHSUHVHQWDWLRQVRILGLRPVZHPXVW
GHILQHZKDWLVPHDQWE\³XVHIXO´7KHUHSO\LVWKDWVRPHWKLQJ¶VXVHIXOQHVVLVSURSRUWLRQDOWRKRZ
PDQ\WDVNV ZH FDQSHUIRUPZLWK LWDQGKRZ ZHOO 3DUWRIRXU FULWHULDIRUVXFFHVV LV WKDWWKHVH
UHSUHVHQWDWLRQV EH HIIHFWLYH LQ DV ZLGHDYDULHW\ RIWDVNVDVSRVVLEOH :KLOHZHKDYH GLVFXVVHG
VRPH DSSOLFDWLRQV OLNH 0DFKLQH 7UDQVODWLRQ DQG 6HQWLPHQW $QDO\VLV ZH VKDOO FRQFHQWUDWH KHUH
RQWDVNVWKDWDUHOHVVHODERUDWHKDYHIHZHUSRWHQWLDOFRQIRXQGLQJYDULDEOHVDQGPRVWLPSRUWDQWO\
DUH PRUH HDVLO\ JHQHUDOL]HG ,I WKH UHSUHVHQWDWLRQV DUH VKRZQ WR EH XVHIXO DFURVV WKHVH VPDOOHU
WDVNVIXWXUHZRUNFDQDSSO\WKHPRQODUJHUPRUHFKDOOHQJLQJDQGPRUHVSHFLDOL]HGDSSOLFDWLRQV
*ULIILWKV 6WH\YHUV DQG 7HQHQEDXP LGHQWLILHG WKUHH FRUH FRPSXWDWLRQDO WDVNV RQ
ZKLFK VHPDQWLF UHSUHVHQWDWLRQV VKRXOG EH HYDOXDWHG JLVW H[WUDFWLRQ GLVDPELJXDWLRQ DQG
SUHGLFWLRQ
x *LVWH[WUDFWLRQLV WKHWDVNRISDUDSKUDVLQJ DVHW RIZRUGVZKHWKHUWKDWEHDSKUDVH D
SDUDJUDSKRUDORQJHUWH[W,QRXUJLVWH[WUDFWLRQWDVNVZHVKDOOPHDVXUHKRZZHOODQ
LGLRP¶V UHSUHVHQWDWLRQ FDSWXUHV WKH LGLRP¶V ILJXUDWLYH PHDQLQJ :H GR WKLV E\
FRPSDULQJ WKH UHSUHVHQWDWLRQ WR WKDW RI D VLQJOH ZRUG SDUDSKUDVH HJ E\ FRPSDULQJ
³KLW WKH VDFN´ WR ³VOHHS´ :H DOVR DVN WKH PRGHO WR FKRRVH WKH EHVW VLQJOHZRUG
SDUDSKUDVHRIDQLGLRPRXWRIDOLVWRIFDQGLGDWHV
x 'LVDPELJXDWLRQLQYROYHVFKRRVLQJWKHFRUUHFWVHQVHRIDZRUGRUSKUDVH,QWKHFDVHRI
LGLRPVWKHWDVNLVWRGLVWLQJXLVKOLWHUDOIURPILJXUDWLYHXVHVRISKUDVHVOLNH³KLWDZDOO´
RU³EUHDNWKHLFH´
x 3UHGLFWLRQ LV WKH DELOLW\ WR FKRRVH ZKLFK SKUDVH RU FRQFHSW LV PRVW OLNHO\ WR IROORZ D
VHULHVRIZRUGV)RUWKLVWDVNZHSUHVHQW WKHPRGHOZLWK DVHQWHQFHDQG IRXULGLRPV
DQGWKHQZHDVNWKHPRGHOWRFKRRVHZKLFKLGLRPEHVWFRPSOHWHVWKHVHQWHQFH
,WZRXOGEHLPSRVVLEOHWRDFKLHYHDJRRGVFRUHRQWKHVHWKUHHWDVNVZLWKRXWKDYLQJHQFRGHGWKH
LGLRP¶V ILJXUDWLYH PHDQLQJ 0RUHRYHU ZH WHVW WKH UHSUHVHQWDWLRQV LQ D QXPEHU RI FRQGLWLRQV
LQFOXGLQJ RQHV ZKHUH WKH LGLRP KDV EHHQ PRGLILHG LQ VRPH ZD\ HLWKHU WKH LGLRP KDV EHHQ
DEEUHYLDWHG³VSLOO´LQVWHDGRI³VSLOOWKHEHDQV´WKHZRUGRUGHUKDVEHHQFKDQJHG³WKHWDEOHV
KDYHWXUQHG´RUDZRUGKDVEHHQDOWHUHG³\DQNWKHSOXJ´LQVWHDGRI³SXOO´7KHVHWHVWVWDNHQ
WRJHWKHUZLOOHVWDEOLVKWKHXVHIXOQHVVDQGUREXVWQHVVRIRXUVROXWLRQ
287/,1(
&KDSWHU RIIHUV D YHU\ EULHI RYHUYLHZ RI LGLRPV 7KH PDWHULDO UHOLHV ODUJHO\ RQ WKH
OLQJXLVWLFOLWHUDWXUH7KHDLPLVWRRIIHUDZRUNLQJGHILQLWLRQRILGLRPVDQGH[SRVHVRPHRIZKDW
PDNHVWKHPLQWHUHVWLQJDQGFRPSOH[7KHFRQFHSWRIFRPSRVLWLRQDOLW\LVRQHWRZKLFKZHVKDOO
RIWHQ UHWXUQ KRZ LW UHODWHV WR LGLRPV LV EULHIO\ H[SORUHG LQ 6HFWLRQ 7KH FKDSWHU FRQFOXGHV
ZLWKDVKRUWGLVFXVVLRQRIYDULDWLRQZLWKLQLGLRPV
&KDSWHU UHYLHZV WKH FRPSXWDWLRQDO OLWHUDWXUH WKDW LV JHUPDQH WR WKH SUHVHQW ZRUN 7KH
ILUVWVHFWLRQSURYLGHVDQLQWURGXFWLRQWRGLVWULEXWLRQDOYHFWRUVSDFHPRGHOV5HDGHUVIDPLOLDUZLWK
VXFKPRGHOVDQGWKHLUWUDLQLQJSDUDPHWHUVQHHGQRWGZHOORQWKLVPDWHULDO7KHQH[WVXEVHFWLRQ
GLVFXVVHV KRZ FRPSRVLWLRQDOLW\ KDV EHHQ DSSURDFKHG LQ YHFWRU VSDFH PRGHOV :H WKHQ ORRN DW
SDSHUV WKDW XVH GLVWULEXWLRQDO UHSUHVHQWDWLRQV WR LQWHUSUHW FRQYHQWLRQDO PHWDSKRUV PRGHOV WKDW
GLVWLQJXLVK OLWHUDO IURP ILJXUDWLYH XVHV RI SRWHQWLDOO\ LGLRPDWLF SKUDVHV DQG D IHZ H[DPSOHV RI
KRZLGLRPVDUHUHSUHVHQWHGLQUHFHQWO\LPSOHPHQWHGV\VWHPV
2XUSURSRVHGDSSURDFKLVRXWOLQHGLQH[WHQVLYHGHWDLOLQ&KDSWHU7KLVRYHUYLHZLQFOXGHV
DMXVWLILFDWLRQIRU KDYLQJFKRVHQGLVWULEXWLRQDO UHSUHVHQWDWLRQV DVDVROXWLRQ DQGWKHQGHVFULEHV
WKHFRPSRVLWLRQDOJRULWKPVZHZLOOWU\WKHWUDLQLQJGDWDDQGSUHSURFHVVLQJSURFHGXUHVXVHGWR
LQGXFHWKHYHFWRUUHSUHVHQWDWLRQVDQGDQRYHUYLHZRIWKHWKUHHHYDOXDWLRQVWDVNV
7KLV RYHUYLHZLV IROORZHGE\ &KDSWHUDVKRUWFKDSWHUWKDW JLYHVVRPH VWDWLVWLFV RQWKH
WUDLQLQJWLPHDQGVWRUDJHUHTXLUHPHQWVIRUWKHUHSUHVHQWDWLRQV
7KHQ &KDSWHU SUHVHQWV WKH DOJRULWKPV GDWDVHW DQG HPSLULFDO UHVXOWV IRU WKH WZR JLVW
H[WUDFWLRQWDVNV:HDOVRGHVFULEHWKHVSDUVLW\RIWKHYHFWRUVSDFHWKDWWKHVHGLVWULEXWLRQVFUHDWH
DQLPSRUWDQWHOHPHQWWRFRQVLGHUZKHQLQWHUSUHWLQJWKHVLPLODULW\VFRUHV:HFRQVLGHUWKHJOREDO
VFRUHVRIRXUPHWKRGZKHQFRPSDUHGWRWZREDVHOLQHVDQGWKHQH[SORUHKRZSDUWLFXODULGLRPV
SHUIRUPHG :H DOVR FRQVLGHU KRZ ZHOO WKH UHSUHVHQWDWLRQV JHQHUDOL]H WR YDULDWLRQV RI FHUWDLQ
LGLRPV
&KDSWHU SUHVHQWV WKH DOJRULWKP GDWDVHWV DQG UHVXOWV IRU WKH GLVDPELJXDWLRQ RI ILYH
GLIIHUHQWVHWVRIVHQWHQFHV6HFWLRQSUHVHQWVIRXUDOJRULWKPVIRUFRPELQLQJLQGLYLGXDOZRUG
YHFWRUV WRJHWKHU WR UHSUHVHQW WKH VXUURXQGLQJ FRQWH[W 7KH ILYH GDWDVHWV LQFOXGH WZR SDSHUV
VXUYH\HGLQ 6HFWLRQ IRUFRPSDULVRQ 7KH\ DOVR LQFOXGHWZRVHWV ZKHUHLGLRPV DUHLQ VRPH
YDULDQWIRUP:HGLVFXVVQRWRQO\WKHJOREDOUHVXOWVEXWVSHQGVRPHWLPHORRNLQJDWZKLFKLGLRPV
VHHPWRUHVXOWLQEHWWHUUHSUHVHQWDWLRQVZKLFKVHHPPRUHGLIILFXOWWRUHSUHVHQWDQGZKDWPLJKW
H[SODLQWKHGLIIHUHQFH
7KH IRXU DOJRULWKPV IRU UHSUHVHQWLQJ D VHQWHQFH DUH XVHG DJDLQ IRU WKH SUHGLFWLRQ WDVN
ZKLFKLVWKHVXEMHFWRI&KDSWHU+HUHDJDLQZHSUHVHQWWKHDOJRULWKPVGDWDVHWDQGUHVXOWVZLWK
VRPH GLVFXVVLRQ RI KRZ DQG ZK\ SDUWLFXODU LGLRPV \LHOGHG HIIHFWLYH RU LQHIIHFWLYH
UHSUHVHQWDWLRQV
:HFRQFOXGHLQ&KDSWHUZLWKDVXPPDU\RIWKHUHVXOWVDFURVVWKHWKUHHW\SHVRIWDVNVD
VXPPDU\RIWKHFRQWULEXWLRQVWKDWWKLVZRUNKDVPDGHVRPHJHQHUDOREVHUYDWLRQVDQGGLUHFWLRQV
IRUIXWXUHZRUN
2 Idioms
,',206$:25.,1*'(),1,7,21
7KH TXLWH VXEVWDQWLDO OLWHUDWXUH RQ LGLRPV IUHTXHQWO\ PHQWLRQV WKH GLIILFXOW\ RI GHILQLQJ
SUHFLVHO\ZKDWDQLGLRPLV1XQEHUJHWDOUHIHUWRLGLRPDVD³IX]]\´FDWHJRU\7KHUHDUH
WKUHHFKDUDFWHULVWLFVWKDWDUHFRPPRQO\DVFULEHGWRLGLRPVHJ/DQJORW])HOOEDXP
:XOII WKRXJK HDFK DOVR KDV LWV VKDUH RI GLVVHQWHUV )LUVW LGLRPV DUH PXOWLZRUG
H[SUHVVLRQVZKLFKLVWRVD\DQLGLRPLVFRPSRVHGRIPRUHWKDQRQHZRUG7KXV³DWZLW¶VHQG´
FRXOGEHFRQVLGHUHGDQLGLRPEXW³QLWZLW´FRXOGQRW6HFRQGWKHPHDQLQJRIWKHLGLRPWDNHQDV
D ZKROH LV GLIIHUHQW IURP ZKDW D OLWHUDO UHDGLQJ RI LWV FRPSRQHQWV ZRXOG VXJJHVW ,QGHHG
UHVHDUFKHUV RIWHQ LQVLVW WKDW E\ GHILQLWLRQ WKH LGLRP¶V PHDQLQJ FRXOG QRW EH GHGXFHG IURP WKH
XVXDOPHDQVE\ZKLFKZHFRPELQHZRUGV¶PHDQLQJVWRIRUPPHDQLQJIXOSKUDVHVHJ1XQEHUJ
HWDOPRUHRQWKLVWRSLFLQWKHIROORZLQJVXEVHFWLRQ%\WKLVFULWHULRQ³FRXFKSRWDWR´D
SHUVRQ ZKR GRHV QRWKLQJ EXW ZDWFK WHOHYLVLRQ IRU H[WHQGHG SHULRGV RI WLPH LV FHUWDLQO\
LGLRPDWLFZKLOHWKHQRXQFRPSRXQG³EDNHGSRWDWR´LVQRW7KLUGWKHIRUPRIDQLGLRPLVIDLUO\
IL[HG WKRXJK LW PD\ EH VRPHZKDW YDULDEOH DQG SHUKDSV HYHQ SURGXFWLYH )RU DQ\ LGLRP WKHUH
ZLOO H[LVW VRPH JUDPPDWLFDO RU V\QWDFWLF WUDQVIRUPDWLRQV WKDW LI DSSOLHG WR WKH LGLRP ZRXOG
FDXVHWKHSKUDVHWREHLQWHUSUHWHGOLWHUDOO\&UXVHRIIHUVPDQ\H[DPSOHVRIV\QWDFWLFDQG
JUDPPDWLFDO WUDQVIRUPDWLRQV WKDW PDNH DQ LGLRP ORVH LWV ILJXUDWLYH LQWHUSUHWDWLRQ LQFOXGLQJ
SDVVLYL]DWLRQ³WKHEXFNHWZDVNLFNHGE\KLP´RUDGGLQJDQDGMHFWLYH³VKHSXOOHGKHUEURWKHU¶V
OHIWOHJ´
7KHVH WKUHH FKDUDFWHULVWLFV PLJKW GHVFULEH LGLRPV JHQHUDOO\ EXW WKH\ DUH QRW D OLVW RI
QHFHVVDU\DQGVXIILFLHQWFRQGLWLRQVWKDWZRXOGDOORZXVWRGHFLGHZKLFKH[SUHVVLRQVDUHLGLRPV
DQG ZKLFK DUH QRW ,I WKHVH WKUHH ZHUH WKH RQO\ FULWHULD HLWKHU VRPH QRQFRPSRVLWLRQDO QRXQ
FRPSRXQGV OLNH ³FRPLF ERRN´ DQG QDPHG HQWLWLHV OLNH ³7RURQWR 0DSOH /HDIV´ ZRXOG EH
FRQVLGHUHG LGLRPV RU WKHLU H[SOLFLW H[FOXVLRQ ZRXOG KDYH WR EH DGGHG DV D IRXUWK FRQGLWLRQ
6DUFDVWLF FRPPHQWV ³QLFH JRLQJ´ DOVR KDYH D PHDQLQJ GLIIHUHQW IURP ZKDW D OLWHUDO
LQWHUSUHWDWLRQZRXOGVXJJHVWEXWWKH\DUHQRWLGLRPVDJDLQDFRPSOHWHOLVWRIFRQGLWLRQVZRXOG
QHHGWRH[SOLFLWO\H[FOXGHWKHP
:HSURSRVHDZRUNLQJGHILQLWLRQWKDWZLOOGHOLQHDWHWKHH[SUHVVLRQVIRUZKLFKRXUDSSURDFK
LVLQWHQGHGZHGHILQHDQLGLRPDVDPXOWLZRUGH[SUHVVLRQWKDWWKURXJKFRQYHQWLRQKDVDFTXLUHG
D VHQVH WKDW LV QRW LWV OLWHUDO PHDQLQJ 7KLV GHILQLWLRQ LPSOLHV WKDW WKH H[SUHVVLRQ KDV D OLWHUDO
PHDQLQJ EXW ZH QRWH WKDW WKLV PHDQLQJ FRXOG EH QRQVHQVLFDO 7KXV ³VKRRW WKH EUHH]H´ LV DQ
LGLRP PHDQLQJ ³WR FKDW LGO\´ GHVSLWH WKH XQOLNHOLKRRG RI LWV EHLQJ XVHG OLWHUDOO\ +RZHYHU
FRPSRXQGVOLNH³MHWODJ´DQG³PXWXDOIXQG´KDYHQRPHDQLQJIXOOLWHUDOLQWHUSUHWDWLRQDQGVRDUH
QRWLQFOXGHG/LJKWYHUEFRQVWUXFWLRQVOLNH³WDNHDZDON´DUHDOVRH[FOXGHGIRUWKHVDPHUHDVRQ
6DUFDVP DQG LURQ\ DUH UHFRJQL]HG DV VXFK WKURXJK SUDJPDWLF FRQVLGHUDWLRQV UDWKHU WKDQ E\
FRQYHQWLRQ DQG WKHUHIRUH WKH\ DUH H[FOXGHG DV ZHOO 1RWH WKDW WKH ZRUNLQJ GHILQLWLRQ VD\V
QRWKLQJ DERXW V\QWDFWLF LQIOH[LELOLW\ VXFK LQIOH[LELOLW\ PD\ EH D FKDUDFWHULVWLF EXW IRU RXU
SXUSRVHVLWLVQRWDGHILQLQJRQH
:H UHFRJQL]H WKDW WKH GHILQLWLRQ LV VWLOO LPSHUIHFW IURP D OLQJXLVW¶V SHUVSHFWLYH )RU RQH
WKLQJLWH[FOXGHVH[SUHVVLRQVOLNH³E\DQGODUJH´DQG³WULSWKHOLJKW IDQWDVWLF´ IRUQRWKDYLQJD
OLWHUDO LQWHUSUHWDWLRQ HYHQ WKRXJK VRPH UHVHDUFKHUV HJ )UDVHU FRQVLGHU WKHVH
H[SUHVVLRQVWREHLGLRPV2XUUHSO\LVWKDWH[LVWLQJDSSURDFKHVZRUNSHUIHFWO\ZHOOIRULQYDULDQW
FROORFDWLRQV OLNH ³E\ DQG ODUJH´ VHH 6HFWLRQ WKH VROXWLRQ SURSRVHG KHUH LV LQWHQGHG IRU
H[SUHVVLRQVWKDWFDQEHPLVWDNHQO\WDNHQOLWHUDOO\DQGWKDWFDQYDU\LQIRUP$VIRUFHUWDLQQDPHG
HQWLWLHVWKDWFRXOGDUJXDEO\EHWDNHQOLWHUDOO\PLJKW7KH0LDPL+HUDOGEHLQWHUSUHWHGDVDKHUDOG
OLYLQJLQ0LDPL"LWLVWUXHWKDWWKHGHILQLWLRQKDVQRWH[FOXGHGWKHPEXWWKHDSSURDFKSURSRVHG
KHUHFRXOGYHU\ZHOOZRUNIRUWKHPDVZHOO
)XUWKHUPRUHWKLVZRUNZLOOOLPLWLWVVFRSHWRDSDUWLFXODUVXEFODVVRILGLRPDWLFH[SUHVVLRQV
VRPHWLPHVFDOOHG9HUE1RXQ&RPELQDWLRQV91&&RRN)D]O\DQG6WHYHQVRQRU913
FRQVWUXFWLRQV:XOII([DPSOHVRIVXFKLGLRPVLQFOXGH³VKRRWWKHEUHH]H´³VPHOODUDW´
RU ³WDNH KHDUW´ $SSHQGL[ $ OLVWV WKH LGLRPV XVHG LQ WKLV ZRUN $V WKH QDPHV VXJJHVW WKHVH
FRQVWUXFWLRQVFRQVLVWRIDYHUEDQGLWVGLUHFWREMHFW7KH\DOVRSHUPLWDQGVRPHWLPHVUHTXLUHRQH
RUWZRLQWHUYHQLQJZRUGVDGHWHUPLQHUHJ³D´³DQ\´³WKH´DSRVVHVVLYHSURQRXQRUSHUKDSV
DQDPH³EURNH&DWKHULQH¶VKHDUW´7KHUHDVRQVIRUWKLVUHVWULFWLRQDUHILUVWWKDWWKLVIRUPLVWKH
PRVW FRPPRQ DPRQJ LGLRPV 0RRQ VHFRQG WKDW VR PXFK RI WKH OLWHUDWXUH KDV DOVR
IRFXVVHG RQ VXFK FRQVWUXFWLRQV DOORZLQJ XV WR FRPSDUH RXU UHVXOWV WR WKHLUV DQG ILQDOO\ WKDW
UHVWULFWLQJWKHVFRSHLQWKLVZD\UHPRYHVSRWHQWLDOO\FRQIRXQGLQJYDULDEOHV,WLVRXUH[SHFWDWLRQ
WKDWWKHUHVWULFWLRQFRXOGEHUHOD[HGZLWKRXWJUHDWO\DIIHFWLQJWKHSURSRVHGPHWKRGDOWKRXJKZH
VKDOOQRWWU\WRSURYHKHUHWKDWVXFKLVWKHFDVH
,',206$1'&20326,7,21$/,7<
'HILQLWLRQVRILGLRPVRIWHQLQFOXGHWKHIDFW WKDWWKH\ DUH QRQFRPSRVLWLRQDO HJ )UDVHU
)HOOEDXPD7KLVFKDUDFWHULVWLFLVUHDOO\DGLUHFWUHVXOWRILGLRPV¶QRQOLWHUDOQDWXUH
DV GLVFXVVHG DERYH &RPSRVLWLRQDOLW\ LV D NH\ FRQFHSW LQ LGLRP UHVHDUFK RQH WKDW ZLOO UHFXU
WKURXJKRXWWKLVGLVVHUWDWLRQ³&RPSRVLWLRQDOLW\LVWKHSURSHUW\WKDWDV\VWHPRIUHSUHVHQWDWLRQKDV
ZKHQLLWFRQWDLQVERWKSULPLWLYHV\PEROVDQGV\PEROVWKDWDUHV\QWDFWLFDOO\DQGVHPDQWLFDOO\
FRPSOH[DQGLLWKHODWWHULQKHULWWKHLUV\QWDFWLFVHPDQWLFSURSHUWLHVIURPWKHIRUPHU´)RGRU
/HSRUH ,W LV KRZ ZH FDQ EOHQG VKRUWHU XQLWV RI PHDQLQJ VD\ ZRUGV WR IRUP ORQJHU
RQHV OLNH SKUDVHV RU VHQWHQFHV :H FDQ GHULYH WKH PHDQLQJ RI D UHJXODU VHQWHQFH OLNH ³WKH
FKLFNHQFURVVHGWKHURDG´E\FRPELQLQJWKHPHDQLQJVRILWVLQGLYLGXDOZRUGVDQGWKHLUUHVSHFWLYH
V\QWDFWLFUROHVWKDWLVWKURXJKFRPSRVLWLRQWRDUULYHDWDFRPSOHWHPHQWDOSLFWXUH,QFRQWUDVWLW
ZRXOGEHLPSRVVLEOHWRJXHVVIURPWKHLUFRPSRQHQWZRUGVWKDWWKHLGLRPV³NLFNWKHEXFNHW´DQG
³EX\WKHIDUP´ERWK GHQRWH ³WR GLH´ ,QGHHG&UXVHGHILQHVLGLRPV DVD ³JUDPPDWLFDOO\
FRPSOH[H[SUHVVLRQZKRVHJUDPPDWLFDOFRQVWLWXHQWVDUHQRWVHPDQWLFFRQVWLWXHQWV´ \HWDQRWKHU
ZD\RIVD\LQJWKDWWKH\ DUHQRW PHDQW OLWHUDOO\ +HIXUWKHUVXJJHVWVWKDW WKLVYHU\ IDFW H[SODLQV
ZK\ LGLRPV FDQQRW XQGHUJR WKH VDPH V\QWDFWLF DQG JUDPPDWLFDO WUDQVIRUPDWLRQV ³WKH
VXSHUILFLDOO\DQRPDORXVEHKDYLRXURILGLRPVLVLQIDFWDQDWXUDOFRQVHTXHQFHRIWKHIDFWWKDWWKHLU
FRQVWLWXHQWVDUHLQDUHDOVHQVHPHDQLQJOHVV´
<HW QRW DOO LGLRPV DUH DV RSDTXH DV ³NLFN WKH EXFNHW´ 2WKHU LGLRPV GHQRWLQJ GHDWK OLNH
³SXVKLQJ XS WKH GDLVLHV´ RU ³PHHW RQH¶V PDNHU´ DUH WKH RSSRVLWH WKHLU H[SUHVVLYHQHVV FRPHV
IURPHYRNLQJYHU\VSHFLILFLPDJHVRIZKHUHDERG\RUVRXOZLOOJRDIWHUGHDWK$JODQFHRIWKH
LGLRPV LQ $SSHQGL[ $ ZLOO GHPRQVWUDWH WKDW PRVW RI WKH LGLRPV LQYHVWLJDWHG KHUH DUH DW OHDVW
SDUWLDOO\WUDQVSDUHQW7KHVHQWHQFH³,FDQQRORQJHUWDNHDGYDQWDJHRIWKDWRSSRUWXQLW\´LVIDLUO\
DEVWUDFW DQG DVHSWLF 6D\LQJ ³, PLVVHG WKH ERDW´ WUDQVODWHV WKH VDPH LGHD LQWR D PRUH FRQFUHWH
YLVFHUDO LPDJH ZH LPDJLQH DUULYLQJ DW WKH GRFN DV WKH ERDW SXOOV DZD\ DQG ZH H[SHULHQFH WKH
DQ[LHW\ RI WKDW PRPHQW 7KH H[SUHVVLRQ LV HIIHFWLYH QRW PHUHO\ WKURXJK FRQYHQWLRQ EXW DOVR
EHFDXVHLWVOLWHUDOPHDQLQJLVVWLOODFWLYDWHGFI7LWRQH&RQQLQH
$QRWKHUDVSHFWWKDWVXJJHVWVWKDWLGLRPVDUHQRWFRPSOHWHO\QRQFRPSRVLWLRQDOLVWKDWWKH
LQGLYLGXDO FRPSRQHQWV RI WKH LGLRP FDQ WDNH RQ D VSHFLILF PHDQLQJ ZLWKLQ WKH LGLRP $
IUHTXHQWO\FLWHGH[DPSOHLV³VSLOOWKHEHDQV´ZKHUHWKHYHUE³VSLOO´FRUUHVSRQGVWR³GLYXOJH´DQG
³WKHEHDQV´UHSUHVHQWWKHVHFUHWVEHLQJ³VSLOOHG´,QWKDWVHQVHWKH\DUHGHFRPSRVDEOH1XQEHUJ
HWDOUHIHUWRVXFKLGLRPVDV³LGLRPDWLFDOO\FRPELQLQJH[SUHVVLRQV´7KH\IXUWKHUDUJXHG
WKDW WKHUH DUH WKUHH GLVWLQFW IHDWXUHV RI LGLRPV WKDW DUH RIWHQ FRQIRXQGHG WRJHWKHU WKH LGLRP¶V
FRQYHQWLRQDOLW\ WKH GHJUHH RI GLYHUJHQFH EHWZHHQ WKH ILJXUDWLYH DQG LGLRPDWLF PHDQLQJV LWV
WUDQVSDUHQF\ KRZ HDV\ LW LV WR JXHVV WKH PHDQLQJ RI WKH LGLRP DQG LWV FRPSRVLWLRQDOLW\ WKH
GHJUHHWRZKLFKPHDQLQJFDQEHDWWULEXWHGWRWKHLGLRP¶VFRQVWLWXHQWV
7KLV GLVFXVVLRQ KDV H[SRVHG IRXU FKDUDFWHULVWLFV WKDW DUH XQLTXH WR LGLRPV DQG SRVH
SRWHQWLDOFKDOOHQJHVWRFUHDWLQJDUHSUHVHQWDWLRQIRUWKHP7KHILUVWKDVEHHQVWDWHGVHYHUDOWLPHV
E\QRZEXWWKHRWKHUWKUHHUHTXLUHRXUDWWHQWLRQDVZHOO
%\GHILQLWLRQWKHLGLRP¶VPHDQLQJLVGLIIHUHQWIURPLWVOLWHUDOFRPSRVLWLRQDOPHDQLQJ
1RWRQO\GRHVWKHSKUDVHDVDZKROHKDYHDPHDQLQJHJ³NLFNWKHEXFNHW´PHDQV³WRGLH´
EXWVRPHWLPHVDQDVSHFWRIWKHSKUDVH¶VPHDQLQJPXVWDOVREHDVVRFLDWHGZLWKWKHLQGLYLGXDO
ZRUGVDVZHOOWKHEHDQVDUHVHFUHWVWKHERDWLVDQRSSRUWXQLW\DQGVRRQ
7KHLGLRP¶VFRPSRQHQWVKDYHWKHVHVHQVHVLQWKHVSHFLILFFRQWH[WRIWKHLGLRPDQGIRUWKH
PRVWSDUWRQO\ZLWKLQWKDWFRQWH[WDEHDQWKDWLVQRWVSLOOHGEXWZDVKHGRUHDWHQQRORQJHU
VLJQLILHVDVHFUHW:HPXVWWKHUHIRUHQRWRQO\UHSUHVHQW³EHDQ´DVERWKOHJXPHDQGVHFUHWEXW
DOVRHQVXUHWKDWWKHLGLRPDWLFPHDQLQJLVDFWLYDWHGRQO\ZKHQWKHUHVWRIWKHLGLRPLVSUHVHQW
RUDWOHDVWLPSOLHG
7KHODUJHUFRQWH[WVXUURXQGLQJWKHLGLRPZLOOGHFLGHLIWKHSRWHQWLDOO\LGLRPDWLFPHDQLQJLV
WREHDFWLYDWHGRUQRW³7KHUHIHUHHEOHZWKHZKLVWOH´LVSUREDEO\PHDQWOLWHUDOO\ZKLOH³WKH
HPSOR\HHEOHZWKHZKLVWOHRQKLVERVV´LVPRUHOLNHO\ILJXUDWLYH$JRRGUHSUHVHQWDWLRQRIDQ
LGLRP IDFLOLWDWHV PDNLQJ WKH FRUUHFW FKRLFH ZKHQ JLYHQ WKH FRQWH[W LQ ZKLFK WKH SKUDVH LV
SUHVHQWHG
9$5,$%,/,7<,1,',206
3RLQWRIWKHOLVWDERYHVWDWHGWKDWLGLRPDWLFPHDQLQJVDUHJHQHUDOO\UHVWULFWHGWRWKHLGLRP
LWVHOIDQGQRWRXWVLGHLW:KDWGRZHPHDQE\³RXWVLGH´DQLGLRP"$QLGLRPLVFRQVWLWXWHGQRW
RQO\ E\ WKH ZRUGV ZLWKLQ LW EXW DOVR D SDUWLFXODU V\QWDFWLF VWUXFWXUH $V ZDV SRLQWHG RXW LQ
6HFWLRQ DERYH HYHQ VLPSOH JUDPPDWLFDO WUDQVIRUPDWLRQV FDQ GHDFWLYDWH WKH LGLRPDWLF
LQWHUSUHWDWLRQ,QWKHDEVHQFHRIFRQWH[WZHZRXOGOLNHO\LQWHUSUHW³KLWWKHVDFN´LQWKHLGLRPDWLF
VHQVHRIJRLQJWREHGEXWLIZHKHDURUUHDG³WKHVDFNZDVKLW´ZHDUHPRUHOLNHO\WRWKLQNRIWKH
OLWHUDO PHDQLQJ &RQVLGHU WKH VHQWHQFH ³$IWHU KHU IDWKHU GLHG VKH KDG WR ZHDU WKH SDQWV IRU D
ZKLOH´+HUHDJDLQZHZRXOGOLNHO\IDYRXUWKHILJXUDWLYHLQWHUSUHWDWLRQRIVRPHRQHKDYLQJWREH
WKHOHDGHULQDIDPLO\%XWFRQVLGHU³$IWHUKHUIDWKHUGLHGVKHKDGWRZHDUWKHEODFNSDQWVIRUD
ZKLOH´1RZWKHOLWHUDOLQWHUSUHWDWLRQEHFRPHVPRUHOLNHO\)RUWKHLGLRP³EUHDNDOHJ´PHDQLQJ
³JRRG OXFN´ DOPRVW QR JUDPPDWLFDO WUDQVIRUPDWLRQ UXOH RI DQ\ NLQG FDQ EH DSSOLHG LI WKH
H[SUHVVLRQLVJLYHQLQDQ\PRRGRWKHUWKDQWKHLPSHUDWLYHLWORVHVLWVILJXUDWLYHPHDQLQJ
/DQJORW]LGHQWLILHGWZRFDWHJRULHVRIYDULDWLRQV7KHILUVWLVZKDWKHFDOOHG³IRUPDO´
YDULDWLRQV7KHVHDUHEDVLFPRUSKRORJLFDOV\QWDFWLFDQGJUDPPDWLFDOWUDQVIRUPDWLRQVDVZRXOG
KDSSHQZKHQDSKUDVHLVSXWLQWRDGLIIHUHQWWHQVHRULVUHQGHUHGLQWRWKHSDVVLYHYRLFH$V)UDVHU
SRLQWHG RXW VRPH LGLRPV XQGHUJR FHUWDLQ IRUPDO YDULDWLRQV EXW RWKHUV QHYHU GR 7KH
LGLRP ³WXUQ WKH WDEOHV´ PHDQLQJ FKDQJLQJ D VLWXDWLRQ WR RQH¶V DGYDQWDJH LV DQ H[DPSOH RI
)UDVHU¶V PRVW IOH[LEOH FODVV 6WDQGDUG PRUSKRORJLFDO FKDQJHV DUH VHHQ IRU EDVLF VXEMHFWYHUE
DJUHHPHQW³KHWXUQVWKHWDEOHV´RUFKDQJHRIWHQVHRUDVSHFW³WXUQHGWKHWDEOHV´³LVWXUQLQJWKH
WDEOHV´³ZLOOWXUQWKHWDEOHV´HWF7KHLGLRPFDQEHWUDQVIRUPHGE\SDVVLYL]DWLRQ³WKHWDEOHV
DUH WXUQHG´ RU QRPLQDOL]DWLRQ ³WKH WXUQLQJ RI WKH WDEOHV´ )UDVHU LQVLVWHG WKDW FOHIW
WUDQVIRUPDWLRQVDUHQHYHUSRVVLEOHIRULGLRPVEXW0RRQIRXQGFRXQWHUH[DPSOHVWRWKLV
UXOH 6KH OLVWV D QXPEHU RI H[DPSOHV ZKHUH FRPSOH[ WUDQVIRUPDWLRQV WUDQVIHU WKH YHUE WR D
UHODWLYHFODXVHDQGDQXPEHURIPRGLILHUVDQGVXSSOHPHQWDU\LQIRUPDWLRQDUHLQWHUSRVHG³DZRUN
KDELWWKDWLVKDUGWRNLFN´RU³DEXOOHWRQZKLFKWKH$UWKXU*ROGVRIWKLVZRUOGKDYHVWHDGIDVWO\
IDLOHGWRELWH´
7KH VHFRQG FODVV LQ /DQJORW]¶V FODVVLILFDWLRQ VFKHPH LV VHPDQWLF YDULDWLRQ 2QH H[DPSOH
ZRXOG EH LQVHUWLQJ DQ DGMHFWLYH WR PRGLI\ WKH QRXQ DV LQ ³WXUQ WKH SROLWLFDO WDEOHV´ 7KHUH DUH
DOVR FDVHV ZKHUH WKH LGLRP¶V YHUE RU QRXQ LV VXEVWLWXWHG IRU DQRWKHU 0RRQ IRXQG WKDW
DURXQG RI LGLRPV DSSHDU ZLWK VRPH YDULDWLRQ DQG KDYH PRUH WKDQ RQH YDULDQW WKH
YDULDQWV VKH KDG LQ PLQG DSSHDU WR EH RI WKLV W\SH +HU VWXG\ SURYLGHV VHYHUDO H[DPSOHV RI
IUHTXHQW YHUE YDULDWLRQV HJ ³XSVHWRYHUWXUQ WKH DSSOHFDUW´ DQG QRXQ YDULDWLRQV ³PLVV WKH
ERDWEXV´ $V VKH SRLQWHG RXW UHJLRQDO GLIIHUHQFHV FDQ DFFRXQW IRU VRPH RI WKHVH ³ZHDU WKH
SDQWV´ LV PRUH IUHTXHQWO\ VHHQ LQ $PHULFDQ WH[WV ZKLOH %ULWLVK VRXUFHV XVXDOO\ XVH ³ZHDU WKH
WURXVHUV´7KHQRXQRUYHUEDUHRIWHQFKDQJHGIRUHPSKDWLFRUKXPRURXVHIIHFW2QHFDQ³VKDWWHU
WKHLFH´LQVWHDGRIPHUHO\EUHDNLQJLWRUIDQWKH³HPEHUV´LQVWHDGRIWKHIODPHV
6KH IXUWKHU LGHQWLILHG D SKHQRPHQRQ ZKHUH VHYHUDO LGLRPV IRUP ZKDW ZH PLJKW FDOO D
IDPLO\ RI LGLRPV RU ZKDW VKH FDOOV D ³IUDPH´ ,Q WKHVH FDVHV WKH ILJXUDWLYH PHDQLQJ RI D
SDUWLFXODUZRUGPLJKW EHVKDUHG DFURVVVHYHUDO LGLRPV 7KHYHUE³KLW´LV VKDUHGDPRQJVHYHUDO
LGLRPVLQFOXGLQJ³KLWWKHVDFN´³KLWWKHERRNV´DQG³KLWWKHURDG´,QDOORIWKHVHLQVWDQFHV³KLW´
LVQRWXVHGLQLWVOLWHUDOVHQVHRIVWULNLQJVRPHWKLQJEXWLWGRHVFRQVLVWHQWO\PHDQOHDYLQJRQH¶V
FXUUHQW VWDWH RU SODFH WR JR WR DQRWKHU ZKHWKHU WKDW EH RQH¶V EHG RQH¶V VWXGLHV RU D IRUP RI
WUDYHO,GHDOO\DUHSUHVHQWDWLRQRI³KLW´OHDUQHGIURPWKHVHWKUHHLGLRPVZRXOGH[WHQGHGWRRWKHU
H[SUHVVLRQV WKDW ZHUH QRW GLUHFWO\ OHDUQHG VXFK DV ³KLW WKH VWUHHWV´ ³KLW WKH J\P´ RU ³KLW WKH
EHDFK´6LPLODUO\D³EXQGOH´FDQUHIHUWRDVXPRIPRQH\LQPDQ\H[SUHVVLRQV³GURSDEXQGOH´
³VSHQGDEXQGOH´³VDYHDEXQGOH´DQGVRRQD³EDOO´FDQPHDQ³IXQ´LQPDQ\FRQWH[WVRWKHU
WKDQ³KDYHDEDOO´DQGWKHKHDUWFDQV\PEROL]HURPDQWLFIHHOLQJVRXWVLGHRIWKHH[SUHVVLRQ³EUHDN
RQH¶VKHDUW´7KLVSURFHVVRIJHQHUDOL]DWLRQFDQDOVRPHDQWKDWRQHZRUGFDQVWDQGIRUWKHHQWLUH
LGLRPE\LWVHOI³NLFNLW´FDQPHDQ³NLFNWKHEXFNHW´DQG³6SLOO´FDQEHDFRPPDQGWRWHOORQH¶V
VHFUHWVZLWKRXWQHFHVVDULO\PHQWLRQLQJDQ\EHDQV
7KHIUHTXHQF\ZLWKZKLFKYDULDWLRQVDSSHDUDVDWWHVWHGE\0RRQ¶VVWXG\PDNHVFOHDUZK\
DVROXWLRQWKDWFDQDFFRPPRGDWHYDULDWLRQLV DQLPSRUWDQW JRDO ,WZRXOGDOVREHXVHIXOZKHUH
SRVVLEOH IRU WKH UHSUHVHQWDWLRQV WR EH H[WHQGDEOH WR LGLRPV RQ ZKLFK WKH PRGHO ZDV QRW
QHFHVVDULO\WUDLQHG7KHVHWZRJRDOVPRWLYDWHGWKHFKRLFHRIVROXWLRQWKDWZLOOEHSUHVHQWHGKHUH
7KHHPSKDVLVLQWKLVZRUNZLOOEHRQWKHNLQGVRIYDULDWLRQGHVFULEHGRYHUWKHWKUHHSDUDJUDSKV
DERYH VSHFLILFDOO\ SDVVLYH YRLFH IRUPXODWLRQV OH[LFDO VXEVWLWXWLRQV DQG JHQHUDOL]DWLRQV RI DQ
LGLRPDWLFFRQVWLWXHQWLQWRDQRWKHUFRQWH[W
3 The Computational Linguistics Background
)RUUHDVRQVWKDWZLOOEHPRUHIXOO\IOHVKHGRXWLQ&KDSWHUZHEHOLHYHWKDWGLVWULEXWLRQDO
YHFWRUVSDFHPRGHOVFDQDFKLHYHRXUJRDORIFUHDWLQJXVHIXODQGUREXVWUHSUHVHQWDWLRQVRILGLRPV
ZLWK PLQLPDO VXSHUYLVLRQ 7KLV VHFWLRQ UHYLHZV WKH UHOHYDQW OLWHUDWXUH WKDW ZLOO SURYLGH WKH
QHFHVVDU\EDFNJURXQGIRUWKHSURSRVHGVROXWLRQ7KHUHYLHZEHJLQVZLWKDQH[SODQDWLRQRIZKDW
GLVWULEXWLRQDO YHFWRU VSDFH PRGHOV DUH DQG KRZ WKH\ DUH FUHDWHG WKLV VHFWLRQ LV SDUWLFXODUO\
LQWHQGHG IRU UHDGHUV XQIDPLOLDU ZLWK WKLV DUHD RI UHVHDUFK *LYHQ WKDW VXFK PRGHOV FUHDWH
UHSUHVHQWDWLRQV RI LQGLYLGXDO ZRUGV DQG WKDW LGLRPV DUH E\ GHILQLWLRQ PXOWLZRUG H[SUHVVLRQV D
PHDQV WR FRPELQH ZRUGV WRJHWKHU LV DQ LPSRUWDQW FRQVLGHUDWLRQ )RU WKDW UHDVRQ 6HFWLRQ UHWXUQV WR WKH TXHVWLRQ RI FRPSRVLWLRQDOLW\ WKLV WLPH ZLWK DQ DFFHQW RQ KRZ YHFWRU
UHSUHVHQWDWLRQVRIZRUGVFDQEHDVVHPEOHGWRJHWKHUWRIRUPORQJHUXQLWVRIPHDQLQJVXFKDV D
SKUDVH RU VHQWHQFH :H WKHQ UHYLHZ WKH UHOHYDQW OLWHUDWXUH RQ XVLQJ YHFWRU VSDFH PRGHOV WR
LQWHUSUHW PHWDSKRUV 6HFWLRQ DQG RQ GLVWLQJXLVKLQJ LGLRPV IURP OLWHUDO XVHV RI WKH VDPH
SKUDVH6HFWLRQ:HFRQFOXGHZLWKDIHZUHFHQWH[DPSOHVRIKRZLGLRPVDUHUHSUHVHQWHGLQ
0DFKLQH/HDUQLQJDQG6HQWLPHQW$QDO\VLV6HFWLRQ
7+( ',675,%87,21$/ +<327+(6,6 $1' ',675,%87,21$/ 9(&725
63$&(02'(/6
+RZFDQZHUHSUHVHQWDZRUGLQDIRUPDWWKDWDFRPSXWHUFDQXVH"2QHGLIILFXOW\LVWKDW
ZRUGVDUHQRWPHUHO\SRLQWHUVWKDWVLJQLI\DVLQJOHVWDWLFPHDQLQJ,QVWHDGZRUGVH[HUWDPXWXDO
LQIOXHQFHRQRQHDQRWKHUVXFKWKDWRXUXQGHUVWDQGLQJRIHDFKZRUGLVLQIOXHQFHGE\WKHZRUGVLQ
WKHVXUURXQGLQJFRQWH[W$³IDVWUXQQHU´DQG³DIDVWFDU´DUHERWK³IDVW´EHFDXVHWKH\FDQDWWDLQ
KLJKVSHHG6LPLODUO\D³IDVWW\SLVW´SHUIRUPVWKHDFWRIW\SLQJYHU\TXLFNO\%XWD³IDVWGULYHU´
LVQRW³IDVW´EHFDXVHVKHSHUIRUPVWKHDFWRIGULYLQJTXLFNO\QRULVD³IDVWODQH´LWVHOIIDVW6R
ZKLOHDOORIWKHVHXVHVRI³IDVW´VKDUHDFRPPRQFRUHFRQFHSWKLJKVSHHGWKHLUSUHFLVH
PHDQLQJVDUHLQIOXHQFHGE\WKHQRXQWKDWWKH\PRGLI\([DPSOHWDNHQIURP(YDQV
7KH'LVWULEXWLRQDO+\SRWKHVLVJLYHVDQDOWHUQDWLYHPRUHXVDJHEDVHGDSSURDFKWRZRUG
PHDQLQJ³:KDWSHRSOHNQRZZKHQWKH\VD\WKDWWKH\NQRZDZRUGLVQRWKRZWRUHFLWHLWV
GLFWLRQDU\GHILQLWLRQ±WKH\NQRZKRZWRXVHLWZKHQWRSURGXFHLWDQGKRZWRXQGHUVWDQGLWLQ
HYHU\GD\GLVFRXUVH´0LOOHU&KDUOHV7KLVYLHZSRVLWVWKDWRXULQWHUQDOUHSUHVHQWDWLRQV
RIZRUGVDUHUHSUHVHQWDWLRQVRIWKHFRQWH[WVLQZKLFKWKHZRUGPD\DSSHDULQFOXGLQJV\QWDFWLF
SUDJPDWLFDQGVW\OLVWLFFRQVLGHUDWLRQV,IWKHUHLVDOLQNEHWZHHQWKHZRUGDQGDFRQFHSWWKDWOLQN
LVQRWSDUWRIWKHOLQJXLVWLFUHSUHVHQWDWLRQ³'LVWULEXWLRQDOPRGHOVDUHPRGHOVRIZRUGPHDQLQJ
1RWWKHPHDQLQJVWKDWDUHLQRXUKHDGVDQGQRWWKHPHDQLQJVWKDWDUHRXWWKHUHLQWKHZRUOGEXW
WKHPHDQLQJVWKDWDUHLQWKHWH[W´6DKOJUHQ$ZHDNHUDQGSHUKDSVPRUHSDODWDEOHIRUPRI
WKH'LVWULEXWLRQDO+\SRWKHVLVVHHVDZRUG¶VPHDQLQJDVDODWHQWYDULDEOHWKRXJKZHFDQQRW
REVHUYHWKHPHDQLQJGLUHFWO\ZHFDQGLVFHUQLWE\REVHUYLQJLQZKLFKFRQWH[WVDWRNHQRIWKH
ZRUGLVXVHG
+RZFDQH[DFWO\FDQDZRUG¶VGLVWULEXWLRQDOSURSHUWLHVEHUHODWHGWRLWVPHDQLQJ":HEHJLQ
ZLWKWKHXQFRQWURYHUVLDOREVHUYDWLRQWKDWPDQ\ZRUGVIUHTXHQWO\FRRFFXURQO\ZLWKRWKHUZRUGV
WKDWDUHUHODWHGLQVXEMHFWPDWWHU,IZHVHHWKHZRUGV³WXPRXU´DQG³FDQFHU´LQDSDUDJUDSKZH
ZRXOG QRW EH VXUSULVHG WR VHH ³PDOLJQDQW´ QHDUE\ ,Q FRQWUDVW ZKLOH LW LV QRW LPSRVVLEOH IRU
³FRXFK´³SKRWRV\QWKHVLV´DQG³HLJHQYDOXH´WRDSSHDUZLWKLQDVLQJOHSDUDJUDSKQRRQHZRXOG
DVVLJQDKLJKSUREDELOLW\WRWKDWHYHQW,WIROORZVWKDWWKHSUREDELOLW\RIZRUGV¶FRRFFXUULQJLVD
JRRG LQGLFDWRU RI WKH UHODWHGQHVV RI WKRVH ZRUGV ,Q )LUWK¶V PHPRUDEOH DSKRULVP ³<RX VKDOO
NQRZDZRUGE\WKHFRPSDQ\LWNHHSV´)LUWK
,QWKHFDVHZKHUHWZRZRUGVDUHQRWMXVWUHODWHGEXWDUHV\QRQ\PVZHPLJKWH[SHFWWKHWZR
ZRUGVWREHODUJHO\LQWHUFKDQJHDEOH,QSUREDELOLVWLFWHUPVSHUIHFWLQWHUFKDQJHDELOLW\LPSOLHVWKDW
WKHSUREDELOLWLHVRIWKHZRUGVLQWKHVXUURXQGLQJFRQWH[WVKRXOGDOVREHWKHVDPH7KDWLVLI³FDU´
DQG³DXWRPRELOH´ZHUHFRPSOHWHO\LQWHUFKDQJHDEOHWKRXJKQRWQHFHVVDULO\HTXDOO\IUHTXHQWRXU
HVWLPDWHIRUWKHFRQGLWLRQDOSUREDELOLW\WKDWWKHIROORZLQJZRUGLV³VDOHVPDQ´VKRXOGEHWKHVDPH
IRUERWKZRUGV$QWRQ\PVDOVRDSSHDULQVLPLODUFRQWH[WVDQGDUHKLJKO\UHODWHGVHPDQWLFDOO\DV
WKH\H[SUHVVRSSRVLWHYDOXHVRIWKHVDPHTXDOLW\³\RXQJ´³ROG´³WDOO´³VKRUW´
6RLIZHFDQUHSUHVHQWDZRUG¶VGLVWULEXWLRQLQVXFKDZD\DVWREHDEOHWRFRPSDUHKRZ
VLPLODUDZRUG¶VVXUURXQGLQJFRQWH[WLVWRDQRWKHU¶VZHZRXOGKDYHDZD\RIWHVWLQJKRZVLPLODU
WKH FRQWH[WV DUH DQG WKHUHIRUH KRZ UHODWHG RU HYHQ LQWHUFKDQJHDEOH WKH ZRUGV DUH :H WKXV
DUULYH DW WKLV VXFFLQFW VXPPDU\ RI WKH 'LVWULEXWLRQDO +\SRWKHVLV ³:RUGV WKDW RFFXU LQ VLPLODU
FRQWH[WVWHQGWRKDYHVLPLODUPHDQLQJV«3DLUVRIZRUGVWKDWFRRFFXULQVLPLODUSDWWHUQVWHQGWR
KDYHVLPLODUVHPDQWLFUHODWLRQV´7XUQH\3DQWHO
$ GLVWULEXWLRQDO YHFWRU VSDFH PRGHO LV D FRQFUHWH DSSOLFDWLRQ RI WKH 'LVWULEXWLRQDO
+\SRWKHVLV HQFDSVXODWLQJ WKH ZRUG¶V GLVWULEXWLRQ LQWR D VHULHV RI QXPEHUV DQG XVLQJ WKRVH
QXPEHUVWRUHSUHVHQWWKHZRUGLWVHOI7KDWLVWKHZRUG¶VGLVWULEXWLRQLVHQFRGHGLQDYHFWRUDQG
WKDWYHFWRUVHUYHV DVDNLQG RISUR[\ IRUWKHZRUGLQ DSURFHVVLQJWDVN 7KHWZR ZRUGV FDQEH
FRPSDUHGXVLQJWKHFRVLQHRIWKHDQJOHEHWZHHQWKHLUUHVSHFWLYHYHFWRUV0DWKHPDWLFDOO\JLYHQ
WZR YHFWRUV $ DQG % WKH VLPLODULW\ RI $ DQG % LV WKH FRVLQH RI WKH DQJOH T EHWZHHQ WKHP
FDOFXODWHGDVIROORZV
simA B FRV
A B
A B
$FRVLQHYDOXHRIFRUUHVSRQGVWRDQDQJOHRIqLQGLFDWLQJWKDWWKHYHFWRUVDUHSDUDOOHOWRRQH
DQRWKHU 7KH FORVHU WKDW WZR ZRUG YHFWRUV DUH WR EHLQJ SDUDOOHO WKH PRUH LQGLVWLQJXLVKDEOH WKH
WZRZRUGV¶FRQWH[WVDUHZKLFKE\WKH'LVWULEXWLRQDO+\SRWKHVLVLPSOLHVWKH\DUHKLJKO\UHODWHG
VHPDQWLFDOO\$FRVLQHQHDULQGLFDWHVYHFWRUVWKDWDUHRUWKRJRQDOWRHDFKRWKHULPSO\LQJWKDW
WKH ZRUGV DSSHDU LQ YHU\ GLVVLPLODU FRQWH[WV DQG VR DUH QRW VHPDQWLFDOO\ UHODWHG 6R LQ WKHVH
PRGHOVZRUGVLPLODULW\LVTXDQWLILHGE\DQXPEHUEHWZHHQXQUHODWHGDQGLGHQWLFDO
7KHUH UHPDLQV WKH PHWKRGRORJLFDO TXHVWLRQ RI KRZ WR HQFRGH D ZRUG¶V GLVWULEXWLRQ DV D
YHFWRU7KHOLWHUDWXUHSURYLGHVVHYHUDOZD\V7KHPHWKRGXVHGLQWKLVGLVVHUWDWLRQVWHPVIURPD
RYHUDOODSSURDFKILUVWSURSRVHGE\%HQJLRHWDODQGWKHQUHDOL]HGLQWKH6NLSJUDPPRGHO
0LNRORY HW DO D 7KH 6NLSJUDP PRGHO EHJLQV ZLWK WUDLQLQJ D QHXUDO QHWZRUN VR WKDW
ZKHQ JLYHQ D ZRUG DV LQSXW LW RXWSXWV WKH YHFWRUV RI WKH RWKHU ZRUGV OLNHO\ WR EH ZLWKLQ D
V\PPHWULFZLQGRZDURXQGWKDWZRUG
1RWH WKDW WKHUH LV QRWKLQJ LQHYLWDEOH DERXW WKH FKRLFH RI D YHFWRU WR UHSUHVHQW ZRUG GLVWULEXWLRQV RWKHU PRGHOV
HQFRGHZRUGVGLVWULEXWLRQVDVPDWULFHVRUKLJKHURUGHUWHQVRUV
)LJXUH$UFKLWHFWXUHRIWKH6NLS*UDP0RGHOIURP0LNRORYD
7UDLQLQJWKHPRGHOHQWDLOVILQGLQJWKHDSSURSULDWHSDUDPHWHUVWRDSSO\WRZWWRJHQHUDWHWKH
RXWSXWVZWFWRZWFLQDZD\WKDWPLQLPL]HVWKHPRGHO¶VRYHUDOOHUURUDFURVVDOOZRUGVLQWKH
YRFDEXODU\7KDWLVJLYHQDWUDLQLQJFRUSXVRI7ZRUGVDQGDZLQGRZRIVL]HFWKHQHXUDO
QHWZRUNWUDLQVVXFKWKDWLWPD[LPL]HVWKHDYHUDJHORJSUREDELOLW\
்
ͳ
෍
ܶ
Ž‘‰ሺܲ൫‫ݓ‬௧ା௝ ห‫ݓ‬௧ ൯ሻ
෍
௧ୀଵ ି௖ஸ௝ஸ௖ǡ௝ஷ଴
7KHFRQGLWLRQDOSUREDELOLW\3ZWM_ZWLVFDOFXODWHGDV
ܲ൫‫ݓ‬௧ା௝ ห‫ݓ‬௧ ൯ ൌ
‡š’ሺ‫ݒ‬௪೟శೕ ୘ ‫ݒ‬௪೟ ሻ
୘
σௐ
௡ୀଵ ‡š’ሺ‫ݒ‬௪೙ ‫ݒ‬௪೟ ሻ
ZKHUH‫ݒ‬௪೙ LVWKHYHFWRUUHSUHVHQWDWLRQRIZRUGQ
$UULYLQJ DW DQ H[DFW VROXWLRQ WR WKH RSWLPL]DWLRQ SUREOHP JLYHQ LQ ZRXOG UHTXLUH
FRPSXWLQJWKHFRQGLWLRQDOSUREDELOLW\IRUHYHU\ZRUGSDLULQJRYHUHYHU\ZLQGRZRIWKH FRUSXV
6LQFHDWUDLQLQJFRUSXVZLOOW\SLFDOO\LQYROYHDODUJHQXPEHURIZRUGVWKHWUDLQLQJFRUSXVXVHG
LQ WKLV ZRUN KDV XQLTXH ZRUG W\SHV DQG D KDOIELOOLRQ WRNHQV WKLV WDVN ZRXOG EH
FRPSXWDWLRQDOO\LQWUDFWDEOH7KHVROXWLRQLVLQVWHDGDSSUR[LPDWHGXVLQJVRPHVKRUWFXWV
D ,QVWHDGRIFDOFXODWLQJWKHSUREDELOLWLHVIRUDOOZRUGSDLULQJVZLWKLQWKHV\PPHWULFZLQGRZD
VXEVHWRIWKHZRUGVZW“FDUHFKRVHQDWUDQGRP)RUDZLQGRZRIVL]HFDUDQGRPQXPEHU
EHWZHHQDQGFRIZRUGVZLOOEHVDPSOHG%HFDXVHZRUGVWKDWDUHFORVHUWRWKHWDUJHWDUH
PRUHLQIRUPDWLYHWKHVDPSOLQJLVGRQHVXFKWKDWZRUGVFORVHUWRWKHWDUJHWDUHPRUHOLNHO\WR
EHFKRVHQ
E7UDLQLQJFRQVLVWVRIPDNLQJVPDOODGMXVWPHQWVWRHDFKZRUGYHFWRUVXFKWKDWZRUGVWKDW
DSSHDULQWKHVDPHFRQWH[WKDYHYHFWRUVZLWKKLJKHUGRWSURGXFWV2QHZD\WRPDNHWKLV
SURFHVVJRIDVWHULVWRLQFOXGHQHJDWLYHH[DPSOHVWKDWLVWRQXGJHZRUGYHFWRUVVRWKDWWKH\
KDYHORZHUGRWSURGXFWVLIWKH\GRQRWDSSHDUWRJHWKHULQWKHWUDLQLQJFRUSXV7KLVWHFKQLTXHLV
FDOOHGQHJDWLYHVDPSOLQJ$SDUDPHWHUFRQWUROVKRZPXFKQHJDWLYHVDPSOLQJWRGRIRUHDFK
WUDLQLQJLWHUDWLRQWKHGHIDXOWYDOXHLV
F $QDOWHUQDWLYHDSSUR[LPDWLRQWHFKQLTXHWRQHJDWLYHVDPSOLQJLVKLHUDUFKLFDOVRIWPD[7KLV
DOJRULWKPLQYROYHVFRQVWUXFWLQJDELQDU\WUHHZKRVHOHDYHVDUHHDFKZRUGLQWKHWUDLQLQJ
YRFDEXODU\$ZHLJKWLVDVVLJQHGWRHDFKHGJHIURPDQLQWHUQDOQRGHWRLWVFKLOG7KH
SUREDELOLW\RIDSDUWLFXODUOHDIZRUGLVDSSUR[LPDWHGE\WUDFLQJWKHSDWKIURPWKHWUHH¶VURRW
WRWKHOHDIDQGPXOWLSO\LQJWRJHWKHUWKHZHLJKWVRIHDFKHGJHHQFRXQWHUHGDORQJWKHZD\,I9
LVWKHVL]HRIWKHYRFDEXODU\WKLVWHFKQLTXHPHDQVWKDWWKHUHDUH9ORJ9ZHLJKWVWROHDUQ
UDWKHUWKDQWKH9WKDWZRXOGEHUHTXLUHGWROHDUQWKHFRQGLWLRQDOSUREDELOLWLHVRIDOOZRUG
SDLULQJV7KHWRWDOWUDLQLQJWLPHLVPXFKVKRUWHU7KDWVDLGXVLQJKLHUDUFKLFDOVRIWPD[LV
VORZHUWKDQXVLQJQHJDWLYHVDPSOLQJE\DIDFWRURIIRXULQRXUH[SHULPHQWV
G)UHTXHQWZRUGVFRRFFXUZLWKPDQ\ZRUGVDQGVRXVLQJWKHPIRUWUDLQLQJLVOHVVKHOSIXOWKDQ
XVLQJDZRUGWKDWLVPRUHVSHFLILFWRDJLYHQFRQWH[W6RGXULQJWUDLQLQJIUHTXHQWZRUGVDUH
VXEVDPSOHGLHGURSSHGZLWKDVSHFLILHGSUREDELOLW\IURPWKHWUDLQLQJFRQWH[W,QSUDFWLFH
WKHXVHUVSHFLILHVDWKUHVKROGVXFKWKDWDQ\ZRUGZKRVHSUREDELOLW\LVDERYHWKDWWKUHVKROG
ZLOOEHVXEVDPSOHG:HVKDOOKHUHDIWHUUHIHUWRWKLVWKUHVKROGDVWKH³VDPSOLQJWKUHVKROG´
H 7KHQXPEHURIWUDLQLQJLWHUDWLRQVLVSUHGHWHUPLQHGIURPWKHRXWVHW7KHPRGHOVWRSVWUDLQLQJ
RQFHWKDWQXPEHUKDVEHHQUHDFKHG7KHUHLVWKXVQRJXDUDQWHHRIHYHQORFDORSWLPDOLW\IRU
DQ\VHWRIYHFWRUVJHQHUDWHGE\WKLVPRGHO%\GHIDXOWWKHUHDUHILYHSDVVHVWKURXJKWKH
WUDLQLQJFRUSXVDJDLQWKHXVHUFDQFKDQJHWKLVGHIDXOW
I 7KHYDULDEOHDOSKDLVWKHDPRXQWE\ZKLFKYHFWRUHQWULHVDUHDGMXVWHGDWHDFKLWHUDWLRQWR
QXGJHWRJHWKHUYHFWRUVWKDWVKRXOGEHPRUHVLPLODURUWKURXJKQHJDWLYHVDPSOLQJQXGJH
YHFWRUVDSDUWWKDWVKRXOGEHPRUHGLVVLPLODU7KLVDOSKDLVUHGXFHGRYHUWLPHVRWKDWWKH
DGMXVWPHQWVDUHVPDOOHUDQGVPDOOHUHDFKWLPH7KHDPRXQWE\ZKLFKLWLVUHGXFHGLVVHWVXFK
WKDWDOSKDLVZKHQWKHSURJUDPWHUPLQDWHV7KHGHIDXOWVWDUWLQJYDOXHRIDOSKDLV
6WDUWLQJDWDKLJKHUYDOXHEXWNHHSLQJWKHQXPEHURIWUDLQLQJLWHUDWLRQVFRQVWDQWPHDQVWKDWWKH
QXGJHVZLOOEHVKDUSHUODUJHUDWWKHEHJLQQLQJRIWUDLQLQJEXWZLOOEHEHUHGXFHGPRUH
UDSLGO\DIWHUHDFKDGMXVWPHQW
$OO RI WKHVH SRVVLELOLWLHV DQG WKHLU FRPELQDWLRQV FUHDWH D ODUJH VSDFH RI SRVVLEOH SDUDPHWHUV
([SORULQJWKDWVSDFHLVSDUWRIZKDWWKHSUHVHQWZRUNQHHGVWRGR7KHVHFRQVLGHUDWLRQV ZLOOEH
UHYLVLWHGLQVRPHGHWDLOODWHU
3RO\VHP\
$QRWKHU DVSHFW RI WKH VNLSJUDP PRGHO DV LPSOHPHQWHG DERYH LV WKDW D VLQJOH YHFWRU
UHSUHVHQWDWLRQ LV OHDUQHG IRU HDFK W\SH ZM PHDQLQJ WKDW HDFK RUWKRJUDSKLF W\SH LV WKRXJKW WR
FRUUHVSRQGWRDVHSDUDWHZRUG$VDUHVXOWLIQRWKLQJLVGRQHWRSURYLGHGLIIHUHQWIRUPVIRUHDFK
VHQVH RI D ZRUG OLNH ³EDU´ D VLQJOH YHFWRU ZLOO EH OHDUQHG WKDW ZLOO WU\ WR HQFDSVXODWH WKH
GLVWULEXWLRQVRIDOOLWVWRNHQVZKHWKHUWKDWWRNHQLVUHIHUULQJWRDURGDSLHFHRIVRDSDGULQNLQJ
HVWDEOLVKPHQWRUDQ\RWKHUZRUGWKDWKDSSHQVWRVKDUHWKLVRUWKRJUDSKLFIRUP,QSUDFWLFHJLYHQ
WKHOLPLWHGGLPHQVLRQVRIWKHYHFWRUWKHOLPLWHGWUDLQLQJWLPHDQGDOLPLWHGWUDLQLQJFRUSXVQR
VLQJOHYHFWRUUHSUHVHQWDWLRQLVOLNHO\WRPRGHOWKHGLVWULEXWLRQVRIDOORIWKRVHVHQVHVDGHTXDWHO\
,QVWHDGWKHPRVWIUHTXHQWPHDQLQJVZLOOOLNHO\EHEHWWHUPRGHOOHGWKDQLQIUHTXHQWRQHV
7KLV LVVXH LV SDUWLFXODUO\ LPSRUWDQW IRU LGLRPDWLF H[SUHVVLRQV $V ZH KDYH VHHQ WKH
FRQVWLWXHQWV RI LGLRPDWLF H[SUHVVLRQV DUH QRW JHQHUDOO\ XVHG LQ WKHLU XVXDO VHQVHV EXW RQO\ WDNH
WKHLUVSHFLDOPHDQLQJVLQRQHDQRWKHU¶VSUHVHQFH³VKLUW´FDQPHDQ³PRQH\´EXWRQO\ZKHQXVHG
LQWKHH[SUHVVLRQ³ORVHRQH¶VVKLUW´$JRRGUHSUHVHQWDWLRQRIWKHZRUGPXVWVLPLODUO\UHIOHFWWKH
VSHFLDOPHDQLQJEXWRQO\DFWLYDWHWKDWPHDQLQJZLWKLQWKHDSSURSULDWHFRQWH[W
&20326,7,21$/,7<
,GLRPV DUH E\ GHILQLWLRQ PXOWLZRUG H[SUHVVLRQV DQG VR WKH SKUDVH DV D ZKROH KDV D
PHDQLQJ9HFWRUVSDFHPRGHOVLQGXFHDYHFWRUIRUHDFKLQGLYLGXDOZRUGVRKRZGRZHFRPELQH
WKHVHYHFWRUVWRIRUPDSKUDVH"7KLVTXHVWLRQLVWKDWRIFRPSRVLWLRQDOLW\KRZWRFRPELQHXQLWV
WRJHWKHUWRIRUPODUJHUXQLWV
$VLPSOHVROXWLRQXVHGQRW RQO\LQ YHFWRUVSDFHPRGHOV EXW LQRWKHU1/3 DSSURDFKHVDV
ZHOOLVVRPHWLPHVUHIHUUHGWRDVD³VWDWLFVROXWLRQ´HJ/DPEHUW%DQFKV*KRQHLP
'LDE 3DO 1DVNDU %DQG\RSDGK\D\ 0LNRORY HW DO E 7KH VWDWLF VROXWLRQ
WUHDWV D PXOWLZRUG H[SUHVVLRQ 0:( LQFOXGLQJ IL[HG FROORFDWLRQV OLNH ³E\ DQG ODUJH´
FRPSRXQGVOLNH³PXWXDOIXQG´RU³$IULFDQ$PHULFDQ´RUQDPHGHQWLWLHVOLNH³1HZ<RUN7LPHV´
RU³0RQWUHDO&DQDGLHQV´DVLILWZHUHDVLQJOHZRUG$SUHSURFHVVLQJVWHSMRLQVWKHZRUGVRIWKH
0:( LQWR ZKDW /DPEHUW DQG %DQFKV FDOO D ³VXSHUWRNHQ´ )RU H[DPSOH VSDFHV DUH
UHSODFHG E\ DQ XQGHUVFRUH FKDUDFWHU ³1HZ <RUN 7LPHV´ EHFRPHV ³1HZB<RUNB7LPHV´ ,Q D
GLVWULEXWLRQDO YHFWRU VSDFH PRGHO VHWWLQJ WKH PRGHO WKHQ OHDUQV D UHSUHVHQWDWLRQ IRU WKH 0:(
WKDWLVGLVWLQFWIURPWKRVHLWOHDUQVIRU³QHZ´RU³WLPHV´0LNRORYHWDOEIRXQGWKDWWKLV
VROXWLRQZRUNHGEHVWZLWKGLIIHUHQWOHDUQLQJSDUDPHWHUVWKH\OHQJWKHQHGWKHFRQWH[WZLQGRZWR
IURPLWVGHIDXOWYDOXHRIDQGGHFUHDVHGWKHVXEVDPSOLQJWKUHVKROGIRUIUHTXHQWZRUGVIURP
LWVGHIDXOWRIWR
$V IRU FRPSRVLWLRQ RI VHQWHQFHV SDUDJUDSKV RU ORQJHU XQLWV WKH TXHVWLRQ RI ILQGLQJ D
FRPSRVLWLRQDOPHWKRGLVDQDFWLYHDUHDRIUHVHDUFKLQZKLFKGLYHUVHDSSURDFKHVDUHWDNHQ6RPH
RI WKHVH LQYROYH OHDUQLQJ KLJKHUGLPHQVLRQDO WHQVRUV )RU H[DPSOH &ODUN DUJXHV WKDW
VLQFHDWUDQVLWLYHYHUELQYROYHVDVPDQ\DVWKUHH³DUJXPHQWV´WKDWLVDVXEMHFWDGLUHFWREMHFW
DQG DQ LQGLUHFW REMHFW LW VKRXOG EH UHSUHVHQWHG E\ D WKUHH±GLPHQVLRQDO WHQVRU HDFK GLPHQVLRQ
³RSHUDWLQJ´RQHDFKRILWVDUJXPHQWV6RFKHUHWDODOVRXVHGWKUHHGLPHQVLRQDOWHQVRUV
EXW WKHLU WHQVRUV UHSUHVHQWHG HDFK SDUHQW QRGH RI D ELQDU\ SDUVH WUHH 'HVSLWH WKH WKHRUHWLFDO
DSSHDO RI VXFK PHWKRGV RXU HDUO\ H[SHULPHQWV ZLWK WKHP GLG QRW SURGXFH UHSUHVHQWDWLRQV WKDW
JHQHUDOL]HGEH\RQGRQHWDVNDQGVRZHFRQFHQWUDWHKHUHRQFRQFHSWXDOO\VLPSOHUWHFKQLTXHV
0LWFKHOO DQG /DSDWD SURYLGHG D JRRG UHYLHZ RI VRPH VLPSOH PHWKRGV WKDW WDNH
YHFWRUUHSUHVHQWDWLRQV RIZRUGV DV LQSXWDQGRXWSXW DVLQJOHYHFWRU7KH\ DOVR FRQGXFWHG VRPH
H[SHULPHQWVWRDVVHVVKRZHIIHFWLYHHDFKPHWKRGLV7KH\FRQFOXGHWKDWWKHEHVWUHVXOWVIRUWKHLU
VSHFLILF WDVNV DQG GDWD DUH REWDLQHG E\ HLWKHU QRUPDO YHFWRU DGGLWLRQ RU HOHPHQWZLVH
PXOWLSOLFDWLRQLHDOLJQLQJWZRYHFWRUVDQGPXOWLSO\LQJHDFKHQWU\WRJHWKHUWRIRUPDYHFWRURI
WKH VDPH OHQJWK DV WKH RULJLQDO WZR 7KHVH PHWKRGV KDYH WZR JUHDW DGYDQWDJHV WKH\ DUH
FRPSXWDWLRQDOO\VLPSOHDQGWKHUHVXOWLVDYHFWRURIWKHVDPHGLPHQVLRQVDVWKHRULJLQDOYHFWRUV
2I FRXUVH DGGLWLRQ DQG PXOWLSOLFDWLRQ DUH ERWK FRPPXWDWLYH VR QHLWKHU RI WKHVH WHFKQLTXHV
UHVSHFWVZRUGRUGHU7KDWPHDQVWKDWWKHUHSUHVHQWDWLRQIRU³FRVWHIIHFWLYH´DQG³HIIHFWLYHFRVW´
ZRXOGEHWKHVDPHHYHQWKRXJKWKHVHSKUDVHVKDYHTXLWHGLIIHUHQWPHDQLQJV
3DUWRIRXULQYHVWLJDWLRQVZLOOEHWRDVVHVVFRPSRVLWLRQDOPHWKRGVDQGILQGZKLFKRIWKHVH
UHVXOW LQ WKH PRVW XVHIXO UHSUHVHQWDWLRQ RI DQ LGLRP ZKLFK E\ GHILQLWLRQ LV D PXOWLZRUG
H[SUHVVLRQDQGRIWKHSKUDVH¶VVXUURXQGLQJFRQWH[W:HZLOOSURYLGHDIHZSRWHQWLDOVROXWLRQVLQ
6HFWLRQ
,17(535(7,1*&219(17,21$/0(7$3+25686,1*
',675,%87,21$/9(&72563$&(02'(/6
6RPHFRPSRVLWLRQDOPHWKRGVDUHVSHFLILFWRYHFWRUVSDFHPRGHOVDQGILJXUDWLYHODQJXDJH
ZKLFKDUHFOHDUO\RISDUWLFXODULQWHUHVWWRWKLVZRUN$JRRGH[DPSOHLVWKH3UHGLFDWLRQ$OJRULWKP
.LQWVFK7KHDOJRULWKPSURFHHGVDVIROORZV*LYHQDSUHGLFDWH3DQGDQDUJXPHQW$
)LQGWKHQFORVHVWQHLJKERXUVRI3.LQWVFKUHFRPPHQGHGQ IRUOLWHUDOSUHGLFDWLRQ
DQGEHWZHHQDQGIRUPHWDSKRUV/HWWKLVVHWRIQHLJKERXUVEH6
)LQGDWPRVWNYHFWRUVLQ6WKDWDUHWKHFORVHVWWR$DVORQJDVWKH\DUHZLWKLQDWKUHVKROG
W.LQWVFKUHFRPPHQGHGN /HWWKHVHNYHFWRUVEHV«VN1RWHWKDWWKHUHPD\EH
IHZHUWKDQNYHFWRUVZLWKLQWKHWKUHVKROGWLQGHHGWKHUHPLJKWEHQRQHDWDOO
5HWXUQWKHFHQWURLGRISDDQGV«VN
7RVKRZZK\WKLVDOJRULWKPPLJKWREWDLQDJRRGLQWHUSUHWDWLRQRIDPHWDSKRU.LQWVFKXVHV
³P\ODZ\HULVDVKDUN´DVDQH[DPSOH,QWHUSUHWLQJWKLVPHWDSKRUPHDQVILQGLQJWKHIHDWXUHVWKDW
PLJKWEHVKDUHGE\ERWKDODZ\HUDQGDVKDUN7KHPRVWREYLRXVIHDWXUHVRIDVKDUNLQFOXGHWKDW
LW LV D ILVK DQG VR OLYHV LQ DQG EUHDWKHV ZDWHU %XW WKHVH FKDUDFWHULVWLFV DUH QRW VDOLHQW WR WKLV
PHWDSKRU 7KDW LV ZK\ WKH QHLJKERXUKRRG FDQ EH UHODWLYHO\ VPDOO IRU OLWHUDO LQWHUSUHWDWLRQV EXW
PXVW EH H[SDQGHG IRU PHWDSKRUV 7KH DOJRULWKP SLFNV RXW WKRVH YHFWRUV LQ WKDW QHLJKERXUKRRG
WKDWDUHDOVRWKHFORVHVWWRWKHDUJXPHQW$,QSUDFWLFHYHU\IHZYHFWRUVZLOOVDWLVI\ERWKFULWHULD
7KH FHQWURLG FRPELQHV WKH RULJLQDO SUHGLFDWH DQG DUJXPHQW DV ZHOO DV WKRVH YHFWRUV WKDW DOLJQ
ZLWK ERWK QXGJLQJWKHUHVXOWLQJYHFWRULQWRDSDUWRIWKHVHPDQWLFVSDFHWKDWWKHSUHGLFDWH DQG
DUJXPHQWERWKVKDUH,QHIIHFWWKHDOJRULWKPFKRVHVWKRVHFKDUDFWHULVWLFVWKDWDUHDSSURSULDWHWRD
SDUWLFXODU FRQWH[W D SUHGLFDWH ZKLOH GHHPSKDVL]LQJ WKRVH WKDW DUH QRW ,Q WKH PHWDSKRU ³P\
ODZ\HULV DVKDUN´WKLV DOJRULWKP UHWXUQV DYHFWRUWKDWLV FORVHWR ³YLFLRXV´DQGIDUIURPRWKHU
VKDUNIHDWXUHVOLNH ³ILVK´.LQWVFKWHVWHGVHYHQ PHWDSKRUVDQG IRXQGLQ VL[RIWKRVHFDVHV WKLV
PHWKRGJDYHKLJKHUFRVLQHPHDVXUHVEHWZHHQWKHSUHGLFDWLRQYHFWRUDQGWKHVDOLHQWIHDWXUHVWKDQ
IRUDQLUUHOHYDQWIHDWXUHOLNH³ILVK´LQWKLVH[DPSOH
7KH 3UHGLFDWLRQ $OJRULWKP IRUPHG WKH IRXQGDWLRQ RI DQRWKHU DOJRULWKP SURSRVHG E\
8WVXPL 8WVXPL¶V REMHFWLYH ZDV WR DVVHVV WZR WKHRULHV RI PHWDSKRU NQRZQ DV WKH
&DWHJRUL]DWLRQ 7KHRU\ DQG WKH &RPSDULVRQ 7KHRU\ +H VXJJHVWV WKDW .LQWVFK¶V 3UHGLFDWLRQ
$OJRULWKPLPSOHPHQWVWKH&DWHJRUL]DWLRQ7KHRU\ZKLFKKROGVWKDWLIDOLVWHQHUKHDUVDPHWDSKRU
OLNH ³P\ MRE LV D MDLO´ WKH OLVWHQHU ZLOO WU\ WR ILW ³P\ MRE´ LQWR WKH FDWHJRU\ RI ³MDLO´ 7KH
&RPSDULVRQ7KHRU\SRVLWVLQVWHDGWKDWDOLVWHQHUZLOOVHDUFKWKHFKDUDFWHULVWLFVRIERWKWKHVRXUFH
DQGWKHWDUJHWWRILQGZKDWWKH\KDYHLQFRPPRQ8WVXPLVLPXODWHVWKHODWWHUDSSURDFKZLWKDQ
DOJRULWKP LQVSLUHG E\ WKH 3UHGLFDWLRQ $OJRULWKP EXW ZLWK LPSRUWDQW GLIIHUHQFHV )LUVW WKH
DOJRULWKPVHHNVVHPDQWLFQHLJKERXUKRRGVDURXQGERWKWKHVRXUFHDQGWKHWDUJHWH[SDQGLQJWKH
VL]H RI WKH QHLJKERXUKRRGV XQWLO ERWK QHLJKERXUKRRGV VKDUH N YHFWRUV ZKHUH N LV D VPDOO
LQWHJHU 7KHQ WKH WRSLF ³P\ MRE´ LV DYHUDJHG ZLWK WKRVH FRPPRQ YHFWRUV WR FUHDWH WKH
PHWDSKRUYHFWRU7KHDOJRULWKPILQGVRWKHUYHFWRUVWKDWDUHFORVHWRWKHPHWDSKRUYHFWRU8WVXPL
IRXQGWKDWWKHVHRWKHUYHFWRUVZHUHRIWHQVDOLHQWFKDUDFWHULVWLFVEURXJKWRXWE\WKHPHWDSKRUEXW
WKDW KDG QRW EHHQ IRXQG E\ WKH H[SDQGLQJ QHLJKERXUKRRG VWHS WKDW LV WKH\ ZHUH FORVH WR WKH
PHWDSKRUYHFWRUZLWKRXWEHLQJFORVHWRHLWKHUWKHVRXUFHRUWKHWDUJHW8WVXPLXVHGERWK.LWVFK¶V
3UHGLFDWLRQ $OJRULWKP DQG WKLV FRPSDULVRQ DOJRULWKP RQ -DSDQHVH PHWDSKRUV +H WKHQ
FRPSDUHGWKHIHDWXUHVKLJKOLJKWHGE\WKHWZRPRGHOVWRWKRVHOLVWHGE\KXPDQVXEMHFWV+HGLG
QRW SURYLGH DFFXUDF\ VFRUHV WKDW LQGLFDWH KRZ FORVHO\ KLV PRGHO VLPXODWHG KXPDQV¶
LQWHUSUHWDWLRQV KLV FRQFHUQ ZDV LQVWHDG WR ILQG ZKLFK PRGHO GLG EHWWHU LQ UHODWLYH WHUPV +H
IRXQG WKDW IRU PHWDSKRUV WKH FDWHJRUL]DWLRQ PRGHO JDYH UHVXOWV WKDW ZHUH PRUH VLPLODU WR
WKRVHRIWKHKXPDQVXEMHFWVZKLOHIRUWKHRWKHUWKHFRPSDULVRQPRGHOZDVPRUHVLPLODU,Q
8WVXPL¶V YLHZ WKLV GDWD PLJKW UHIOHFW WKH IDFW WKDW KXPDQV XVH D YDULHW\ RI VWUDWHJLHV ZKHQ
LQWHUSUHWLQJDPHWDSKRU
:HQRZWXUQWRDQRWKHUSDSHUWKDWXVHGGLVWULEXWLRQDOUHSUHVHQWDWLRQVWRLQWHUSUHWILJXUDWLYH
ODQJXDJH RI ZKLFK LGLRPV DUH D VXEFDWHJRU\ 6KXWRYD 9DQ GH &UX\V DQG .RUKRQHQ ORRNHGDWPHWDSKRUVWKDWXQOLNHLGLRPVDUHQRWHQWLUHSKUDVHVWKDWDUHWREHWDNHQILJXUDWLYHO\
,QVWHDG WKH\ DUH FRPSRXQGV ZKHUH RQH ZRUG LV XVHG OLWHUDOO\ DQG WKH RWKHU ILJXUDWLYHO\ OLNH
³EUHDNLQJDFRQWUDFW´RU³WRSSOHWKHJRYHUQPHQW´7KH\XVHGYHFWRUVGHYHORSHGE\9DQGH&UX\V
WKDW HQFRGH ODWHQW IDFWRUV OLQNLQJ D ZRUG LWV VXUURXQGLQJ FRQWH[W DQG LWV GHSHQGHQF\
UHODWLRQV9DQGH&UX\VGLGQRWSURYLGHH[DPSOHVRIWKHGHSHQGHQF\UHODWLRQVEXWSUHVXPDEO\
WKH\ZRXOGLQFOXGHDUHODWLRQOLQNLQJDYHUEWRLWVVXEMHFWDQGDQRWKHUOLQNLQJLWWRLWVREMHFW7KH
UHVXOWLVDYHFWRUTXLWHXQOLNHWKHGLVWULEXWLRQDOUHSUHVHQWDWLRQVZHKDYHVHHQVRIDU5DWKHUWKDQ
WU\WR HQFRGHDZRUG¶VGLVWULEXWLRQDFURVVDOOWH[WV DQGZLWK DOORIWKDWZRUG¶VGLIIHUHQW VHQVHV
WKLV YHFWRU¶V HOHPHQWV GHVFULEH ODWHQW IDFWRUV WKDW GHWHUPLQH WKH LQWHUUHODWLRQVKLSV EHWZHHQ D
ZRUGLWVFRQWH[WDQGLWVGHSHQGHQFLHV
7KH DXWKRUV DWWHPSWHG WR LQWHUSUHW FRQYHQWLRQDO PHWDSKRUV LQ WZR SKDVHV 7KH ILUVW
SKDVHZDVWRWDNHWKHYHFWRURIWKHZRUGEHLQJXVHGPHWDSKRULFDOO\DQGUHZHLJKWLWDFFRUGLQJWR
WKH FRQWH[W LQ ZKLFK LW ILQGV LWVHOI )RU H[DPSOH IRU WKH H[SUHVVLRQ ³UHIOHFW FRQFHUQ´ LW LV
³UHIOHFW´WKDWLV XVHGPHWDSKRULFDOO\VRWKHHOHPHQWV RIWKHYHFWRUIRU³UHIOHFW´DUHUHZHLJKWHG
DFFRUGLQJWRWKHGLVWULEXWLRQDOLQIRUPDWLRQRI³FRQFHUQ´ZKHQ³FRQFHUQ´LVDYHUE¶VDQ\YHUE¶V
GLUHFWREMHFW7KHUHVXOWLVDYHUVLRQRI³UHIOHFW´WKDWKDVEHHQDGDSWHGWRWKLV SDUWLFXODUREMHFW
7KHQ WKH YHFWRUV ZLWKLQ WKH QHLJKERXUKRRG RI WKLV UHZHLJKWHG YHFWRU DUH FKRVHQ DV SRVVLEOH
FDQGLGDWHV WR SDUDSKUDVH³UHIOHFW´7KHFDQGLGDWHV LQ WKLVH[DPSOH ZHUH³DGGUHVV´ ³KLJKOLJKW´
³H[SUHVV´ ³IRFXV´ ³RXWOLQH´ DQG ³FRPPHQW´ $ EDVHOLQH PHWKRG VLPSO\ FKRVH WKH FDQGLGDWH
ZLWKWKHKLJKHVWVLPLODULW\VFRUHZKLFKLQWKLVH[DPSOHZDV³DGGUHVV´
7KHLU VHFRQG PHWKRG LQFRUSRUDWHV WKH VHOHFWLRQDO DVVRFLDWLRQ PHDVXUH ILUVW GHILQHG E\
5HVQLNDQGZKLFK6KXWRYDXVHGZLWKVXFFHVVLQDQRWKHUPRUHNQRZOHGJHEDVHGDSSURDFK
6KXWRYDD6HOHFWLRQDODVVRFLDWLRQTXDQWLILHVKRZH[FOXVLYHO\DYHUELVDVVRFLDWHGZLWKD
SDUWLFXODUFODVVRIQRXQ)RUH[DPSOHWKHFODVVHVOLTXLGDQGDVVHWVPLJKWEHWKHPDLQFODVVHVWKDW
WKH YHUE ³HYDSRUDWH´ ZRXOG WDNH DV D GLUHFW REMHFW 2I WKH WZR OLTXLG ZRXOG EH WKH PRUH
IUHTXHQWVR³HYDSRUDWH´DQGOLTXLGVZRXOGKDYHDKLJKVHOHFWLRQDO DVVRFLDWLRQ 6RUHWXUQLQJWR
WKH ³UHIOHFW FRQFHUQ´ H[DPSOH FKRRVLQJ WKH FDQGLGDWH SDUDSKUDVH ZLWK WKH KLJKHVW VHOHFWLRQDO
DVVRFLDWLRQIRU³FRQFHUQ´DOORZHGWKHPRGHOWRSURSRVH³H[SUHVV´DVWKHFRUUHFWSDUDSKUDVHIRU
³UHIOHFW´ LQ WKLV FRQWH[W 7KH DXWKRUV FRPSDUHG WKHLU WZR PRGHOV¶ FKRLFHV WR WKRVH RI KXPDQ
MXGJHVDQGIRXQGWKDWWKHWZRPHWKRGV\LHOGHGDFFXUDF\UDWHVRIDQGUHVSHFWLYHO\7KH
GLIIHUHQFHEHWZHHQWKHLUEDVHOLQHDQGILQDOUHVXOWLVLQWHUHVWLQJLQWKDWLWVKRZVWKDWFKRRVLQJWKH
QHDUHVW YHFWRU DFFRXQWHG IRU DOPRVW DOO RI WKHLU DSSURDFK¶V VXFFHVV 5HUDQNLQJ E\ VHOHFWLRQDO
DVVRFLDWLRQ RQO\ SURYLGHG WZR DGGLWLRQDO FRUUHFW DQVZHUV IRU WKH RWKHU FDVHV LW PHUHO\
FRQILUPHGWKHFKRLFHWKDWWKHEDVHOLQHPHWKRGKDGDOUHDG\PDGH
6KXWRYDHWDO¶VSDSHULVLPSRUWDQWDVEHLQJRQHRIWKHILUVWPHWKRGVWRLQWHUSUHWILJXUDWLYH
ODQJXDJHZLWKRXWUHIHUULQJWRDNQRZOHGJHEDVHGUHVRXUFH,WLVRISDUWLFXODULQWHUHVWKHUHLQWKDW
LW XVHG D YHFWRU VSDFH PRGHO DQG EHFDXVH LW GHPRQVWUDWHG KRZ SDUWLFXODU DVSHFWV RI D YHFWRU
UHSUHVHQWDWLRQ FDQ EH KLJKOLJKWHG DQG HPSKDVL]HG LQ RUGHU WR DOLJQ D ILJXUDWLYH XVDJH ZLWK D
OLWHUDOHTXLYDOHQW7KHUHVXOWVDOVRJLYHDQLGHDRIZKDWNLQGRIDFFXUDF\UDWHVFDQEHH[SHFWHGLQ
WKLVGRPDLQ
',67,1*8,6+,1*/,7(5$/)520),*85$7,9(86(6
:HUHFDOOIURP6HFWLRQWKDWRQHFKDOOHQJHFRQFHUQLQJLGLRPVLVWKDWRQHFDQQRWVLPSO\
VHDUFK WH[W IRU DQ RFFXUUHQFH RI DQ LGLRP DQG DVVXPH WKDW WKH LQVWDQFH LV XVHG LGLRPDWLFDOO\
3KUDVHV OLNH ³KLW D ZDOO´ DQG ³GURSSHG D ERPE´ DUH DFWXDOO\ PHDQW OLWHUDOO\ PRUH RIWHQ WKDQ
ILJXUDWLYHO\ 2QO\ WKH VXUURXQGLQJ FRQWH[W FDQ WHOO XV ZKDW WKH LQWHQGHG XVH ZDV 6HYHUDO
UHVHDUFKHUV KDYH ORRNHG DW WKLV TXHVWLRQ DV D FODVVLILFDWLRQ WDVN RQFH D WRNHQ RI D NQRZQ
H[SUHVVLRQKDVEHHQIRXQGZLWKLQDJLYHQVHQWHQFHFODVVLI\WKDWWRNHQ DVDOLWHUDO RUILJXUDWLYH
XVH
.DW]DQG*LHVEUHFKWLPSOHPHQWHGDVXSHUYLVHGPHWKRGIRUFODVVLI\LQJILJXUDWLYHYV
OLWHUDO XVHV WKDW OLNH WKH SUHVHQW VWXG\ LQYROYHV YHFWRU UHSUHVHQWDWLRQV RI WKH H[SUHVVLRQ 7KH\
LQGXFHG GLVWLQFW YHFWRUV IRU HDFK WRNHQ RI D WDUJHW H[SUHVVLRQ 7KH YHFWRU ZDV WUDLQHG XVLQJ
/DWHQW 6HPDQWLF $QDO\VLV /DQGXDHU 'XWQDLV RI D ZRUG FRQWH[W VXUURXQGLQJ WKH
H[SUHVVLRQ7KH\WKHQPDQXDOO\JURXSHGWKHVHYHFWRUVLQWRWZRFROOHFWLRQVEDVHGRQZKHWKHUWKH
YHFWRUUHSUHVHQWHGDILJXUDWLYHRUOLWHUDOLQVWDQFH7KHVHYHFWRUVIRUPHGWKHWUDLQLQJVHW7KHQIRU
WHVWLQJ YHFWRUV IRU QHZ WRNHQV ZHUH LQGXFHG XVLQJ WKH VDPH PHWKRG 7KH\ WRRN WKH FRVLQH
VLPLODULW\RIWKHQHZYHFWRUWRWKHWZRFROOHFWLRQV7KHFODVVODEHOZDVGHWHUPLQHGE\VHHLQJWR
ZKLFKFROOHFWLRQWKHYHFWRUPRVWUHVHPEOHG7KLVWHFKQLTXHREWDLQHGDDFFXUDF\VFRUH
7KHLGHDWKDWLGLRPDWLFXVHVZLOOW\SLFDOO\RFFXULQRQHNLQGRIFRQWH[WDQGOLWHUDORQHVLQ
DQRWKHULVDOVRDWWKHFRUHRI)D]O\&RRNDQG6WHYHQVRQ¶VDSSURDFK7KH\WUDLQHGWKUHH
PRGHOVWKHLUWDUJHWPRGHODVXSHUYLVHGPRGHOZKLFKVHUYHGDVDEDVHOLQHDQGDVLPSOHPRGHO
WKDW VHUYHG DV ERWK D EDVHOLQH DQG D PHDQV WR PDNH DQ LQLWLDO GLYLVLRQ RI WKH GDWD 7KH ODWWHU
PRGHOEDVHVLWVGHFLVLRQVROHO\RQZKHWKHUWKHLGLRPDSSHDUVLQLWVXVXDO³FDQRQLFDO´IRUPLI
VRWKHWRNHQLVODEHOOHGDVILJXUDWLYH7KHWDUJHWPRGHOWUDLQVLWVHOIXVLQJWKHODEHOVSURYLGHGE\
WKH ILUVW ,W ILQGV WKH ZRUGV WKDW DUH W\SLFDOO\ DVVRFLDWHG ZLWK ³OLNHO\OLWHUDO´ DQG ³OLNHO\
ILJXUDWLYH´WRNHQV$IWHUWKLVWUDLQLQJWKHPRGHOFODVVLILHVQHZLQVWDQFHVEDVHGRQWKHLUFRQWH[W¶V
-DFFDUGVLPLODULW\WRWKHFRQWH[WVRIWKHWUDLQLQJPRGHOV7KHVXSHUYLVHGPRGHOLVHVVHQWLDOO\WKH
VDPH DV WKH WDUJHW PRGHO H[FHSW WKDW LWV OLWHUDO DQG ILJXUDWLYH WUDLQLQJ VHWV ZHUH QRW DVVHPEOHG
EDVHGRQWKHWRNHQV¶IRUPEXWUDWKHUE\XVLQJWKHDXWKRUV¶PDQXDODQQRWDWLRQRIDVXEVHWRIWKH
%ULWLVK 1DWLRQDO &RUSXV %1& &RQVRUWLXP 7KH\ XVHG WKHLU WKUHH PRGHOV WR FODVVLI\
7KH-DFFDUGVLPLODULW\VFRUHRIVHWV$DQG%LV
ȁ஺‫ת‬஻ȁ
ȁ஺‫׫‬஻ȁ
LGLRPDWLFLHILJXUDWLYHDQGOLWHUDOWRNHQVRIH[SUHVVLRQV7KHLUWDUJHWPRGHO¶VDFFXUDF\ZDV
ZKHQ WKH H[SUHVVLRQ LV PRUH OLNHO\ WR EH XVHG LGLRPDWLFDOO\ EXW ZKHQ WKH
H[SUHVVLRQ LV PRUH OLNHO\ WR EH XVHG OLWHUDOO\ HJ ³EORZ WKH ZKLVWOH´ ³KLW D ZDOO´ 7KHLU
VXSHUYLVHGPRGHODFKLHYHGDQGDFFXUDF\RQWKHVHUHVSHFWLYHJURXSV6HH )LJXUH
DQG$SSHQGL['IRUPRUHGHWDLOHGUHVXOWVIRUVSHFLILFLGLRPV
$GLIIHUHQW VLPLODULW\PHWULFWKH1RUPDOL]HG*RRJOH'LVWDQFH1*'OLHVDW WKHEDVHRI
WZRFODVVLILHUVWKDW6SRUOHGHUDQG/LLPSOHPHQWHGIRUDVLPLODUWDVN7KLVVFRUHFRPSXWHV
KRZ UHODWHG WZR WHUPV DUH E\ VHHLQJ KRZ RIWHQ WKH WZR WHUPV DSSHDU WRJHWKHU RQ ZHE SDJHV
0DWKHPDWLFDOO\1*'[\ ୫ୟ୶ሼ୪୭୥ ௙ሺ௫ሻǡ୪୭୥ ௙ሺ௬ሻሽି୪୭୥௙ሺ௫ǡ௬ሻ
୪୭୥ ெି୫୧୬ሼ୪୭୥ ௙ሺ௫ሻǡ୪୭୥ ௙ሺ௬ሻሽ
ZKHUH0LVWKHWRWDOQXPEHURILQGH[HG
SDJHV7KHLUILUVWFODVVLILHUEXLOGVDOH[LFDOFKDLQIRUDJLYHQFRQWH[W,WHPVDUHLQVHUWHGLQWRWKLV
FKDLQLIWKH\DUHFRKHUHQWZLWKWKHRWKHUWHUPVLQWKHFKDLQWKDWLVLIWKHLU1*'VFRUHLVDERYHD
WKUHVKROGWKDWWKHUHVHDUFKHUVVHW$OLWHUDOXVHRIDQH[SUHVVLRQZLOOOLNHO\ILWZHOOZLWKWKHWHUPV
ZLWKLQWKHFKDLQRIWKHVXUURXQGLQJFRQWH[WLIVRWKHWRNHQLVFODVVLILHGDVOLWHUDO$QLGLRPDWLF
H[SUHVVLRQRQWKHRWKHUKDQGZLOOOLNHO\QRWEHFRKHUHQWLI³IDQWKHIODPHV´LVXVHGLQDFRQWH[W
WKDW PDNHV QR PHQWLRQ RI D ILUH LW LV OLNHO\ XVHG ILJXUDWLYHO\ 7KH\ WHVWHG WKHLU PRGHO RQ DQ
H[WUDFWLRQ RI WKH *LJDZRUG &RUSXV WKDW WKH\ KDG DQQRWDWHG 7KH OH[LFDO FKDLQ PRGHO DFKLHYHG
DFFXUDF\ZLWKSUHFLVLRQDQGUHFDOORQWKHOLWHUDOFODVV6FRUHVIRUWKHILJXUDWLYH
FODVVZHUHQRWSURYLGHG
7KHLUVHFRQGPRGHOIRUPVDQLQWHUFRQQHFWHGJUDSKRIDJLYHQFRQWH[WUDWKHUWKDQDFKDLQ
7KDWLVWKHZRUGVRIWKHFRQWH[WDUHWKHQRGHVRIWKHJUDSKDQGWKHHGJHVFRQQHFWLQJWKHQRGHV
DUHWKH1*'VFRUHRIWKHWZRWHUPV7KHFRKHUHQFHRIWKHJUDSKLVPHDVXUHGZLWKWKHSRWHQWLDOO\
LGLRPDWLFH[SUHVVLRQLQFOXGHG7KHQWKHHGJHVIURPWKHH[SUHVVLRQ¶VZRUGVWRWKHRWKHUZRUGVLQ
WKHFRQWH[WDUHUHPRYHGDQGWKHFRKHUHQFHLVPHDVXUHGDJDLQ,IWKHFRKHUHQFHDIWHUUHPRYDOLV
JUHDWHU WKDQ LW ZDV EHIRUH WKH H[SUHVVLRQ LV FODVVLILHG DV LGLRPDWLF RWKHUZLVH LW LV FRQVLGHUHG
OLWHUDO 7HVWHG RQ WKH VDPH DQQRWDWHG H[WUDFWLRQ IURP WKH *LJDZRUG &RUSXV XVHG DERYH WKLV
PHWKRG REWDLQHG DFFXUDF\ RYHUDOO ZLWK SUHFLVLRQ DQG UHFDOO RQ WKH OLWHUDO
FODVV
3HQJ )HOGPDQ DQG 9\ORPRYD SURYLGH DQRWKHU H[DPSOH RI DQ DSSURDFK WKDW
UHSUHVHQWV WKH FRQWH[W DQG FODVVLILHV DQ H[SUHVVLRQ DV LGLRPDWLF RU OLWHUDO EDVHG RQ KRZ ZHOO LW
FRKHUHV ZLWK WKDW UHSUHVHQWDWLRQ 7KHLU WUDLQLQJ VWHS FRQVLVWV RI FUHDWLQJ D WRSLF VSDFH
UHSUHVHQWDWLRQRIFRQWH[WVXVLQJ/DWHQW'LULFKOHW$OORFDWLRQWKH\FUHDWHGDPDWUL[UHSUHVHQWDWLRQ
RI FRQWH[WV LQ ZKLFK WKH LGLRPDWLF XVH RI WKH H[SUHVVLRQ DSSHDUHG DQG DQRWKHU PDWUL[
UHSUHVHQWDWLRQRIFRQWH[WVIRUWKHOLWHUDOXVHV(DFKPDWUL[LVDJULGWKDWUHODWHVWKHYRFDEXODU\
WHUPVIRXQGLQWKHFROOHFWLRQWRWKHGRFXPHQWVLQZKLFKWKH\ZHUHIRXQG7KHQLQWKHWHVWSKDVH
D TXHU\ PDWUL[ LV JHQHUDWHG IRU HDFK WHVWLQJ LQVWDQFH XVLQJ WKH VDPH PHWKRG $ )LVKHU /LQHDU
'LVFULPLQDQW$QDO\VLVFODVVLILHUFRPSDUHVWKHTXHU\PDWUL[WRWKHOLWHUDODQGILJXUDWLYHUHIHUHQFH
PDWULFHVDQGFKRRVHVWKHFODVVEDVHGRQZKLFKPDWUL[PRVWUHVHPEOHVWKHTXHU\PDWUL[3HQJHW
DO WHVWHG IRXU LGLRPV XVLQJ WKH WHVW VHW WKDW )D]O\ HW DO DQQRWDWHG DV H[SODLQHG DERYH 7KHLU
DFFXUDF\ VFRUHV YDULHG EHWZHHQ DQG GHSHQGLQJ RQ WKH LGLRP 7KH H[DFW DFFXUDF\
SUHFLVLRQDQGUHFDOOVFRUHVDUHJLYHQLQ$SSHQGL[(
3HQJ )HOGPDQDQG-D]PDWL QGODWHUUHWXUQHGWR WKHVDPHTXHVWLRQXVLQJILYHGLIIHUHQW
DSSURDFKHVDOORIZKLFKLQYROYHGLVWULEXWLRQDOUHSUHVHQWDWLRQV7KHPRVWVXFFHVVIXORQHIRUPV
WZRPDWUL[UHSUHVHQWDWLRQVRQHIRUFRQWH[WVRILGLRPDWLFXVHVDQGDQRWKHUIRUOLWHUDORQHV7KHVH
UHSUHVHQWDWLRQV DUH ³VFDWWHU PDWULFHV´ ZKRVH HQWULHV DUH WKH GRW SURGXFWV RI WKH YHFWRU
UHSUHVHQWDWLRQVRIHDFKZRUGZLWKLWVHOIIRUHYHU\ZRUGLQWKHFRQWH[W7RFODVVLI\DWHVWFDVHWKH
VDPHPHWKRGFUHDWHVDPDWUL[UHSUHVHQWDWLRQRIWKHWHVWFDVH¶VFRQWH[WZKLFKLVWKHQFRPSDUHG
ZLWK WKH UHIHUHQFH PDWULFHV XVLQJ WKH )UREHQLXV QRUP 7KH WHVW LV FODVVLILHG EDVHG RQ WKH
UHIHUHQFHPDWUL[WRZKLFKLWPRVWUHVHPEOHV7ZHOYHLGLRPVZHUHWHVWHGXVLQJWKLVPHWKRGDQG
WKHDFFXUDF\VFRUHVYDULHGEHWZHHQDQG:KHQFRPSDUHGWRWKHIRXULGLRPVWHVWHGLQ
WKHSUHYLRXVVWXG\WKHUHVXOWVIRUWKLVPHWKRGDUHORZHU
7KHH[SHULPHQWVVXUYH\HGLQWKLVVXEVHFWLRQKDYHDOORZHGXVWRVHHDUDQJHRIDSSURDFKHV
WKDW KDYH EHHQ WDNHQ WR WKH WDVN LQFOXGLQJ JUDSKLFDO PRGHOV WRSLF PRGHOV DQG GLVWULEXWLRQDO
PRGHOV $FFXUDF\ VFRUHV KDYHJHQHUDOO\EHHQDURXQGZLWK VRPHDURXQG DQGUHFDOO
VFRUHVRIWHQODJJLQJIDUWKHUEHKLQG:HQRWHWKDWWKHUHSUHVHQWDWLRQVXVHGE\WKHVHPRGHOVVHUYH
WKLVGLVDPELJXDWLRQWDVNZLWKQRREYLRXVSRUWDELOLW\WRRWKHU1/3WDVNV:HVKDOOODWHUUHWXUQWR
WKH )D]O\ &RRN DQG 6WHYHQVRQ DQG WKH 3HQJ )HOGPDQ DQG 9\ORPRYD H[SHULPHQWV LQ RUGHU WR
FRPSDUHWKHUHVXOWVZHREWDLQXVLQJRXUUHSUHVHQWDWLRQV
7KHLUSDSHULVQRWGDWHGEXWUHIHUVWRWKHSDSHUDQGVRZDVFOHDUO\ZULWWHQDIWHULW
27+(55(35(6(17$7,2162),',206
,Q FRQWUDVW WR WKH PHWKRGV GLVFXVVHG LQ WKH SUHYLRXV VHFWLRQ LGLRPV DUH PRUH W\SLFDOO\
KDQGOHG XVLQJ VRPH IRUP RI FRUUHVSRQGHQFH WDEOH 7KH QDWXUH RI WKDW WDEOH ZLOO GHSHQG RQ WKH
DSSOLFDWLRQ:HRIIHUKHUHDYHU\EULHIDQGE\QRPHDQVFRPSUHKHQVLYHVXUYH\RIDIHZUHFHQWO\
LPSOHPHQWHGV\VWHPVWKDWLQFOXGHVXFKUHSUHVHQWDWLRQVRILGLRPVWRJLYHDIODYRXUIRUWKHW\SHV
RIVROXWLRQVWKDWDUHFXUUHQWO\LQXVH
7KHJRDORI0DFKLQH7UDQVODWLRQLVWRWDNHDWH[WLQRQHODQJXDJHDVLQSXWDQGRXWSXWDWH[W
ZLWKWKHVDPHPHDQLQJEXWLQDGLIIHUHQWODQJXDJH%/(8LVDZLGHO\XVHGPHWKRGRIDVVHVVLQJ
PDFKLQHJHQHUDWHGWUDQVODWLRQVE\FDOFXODWLQJWKHDPRXQWRIRYHUODSEHWZHHQWKHQJUDPVLQWKH
PDFKLQHSURGXFHG WUDQVODWLRQ DQG D KXPDQ¶V WUDQVODWLRQ RI WKH VDPH WH[W 3DSLQHQL 5RXNRV
:DUG=KX$VIRUDSSURDFKHVWR0DFKLQH7UDQVODWLRQPDSSLQJDZRUGRIWKHVRXUFH
ODQJXDJH WR WKH HTXLYDOHQW ZRUG LQ WKH WDUJHW ODQJXDJH LV D SRRU ZD\ RI JHWWLQJ D JRRG
WUDQVODWLRQ ,W LV D SDUWLFXODUO\ SRRU VWUDWHJ\ IRU LGLRPV XQOHVV WKH WDUJHW ODQJXDJH KDSSHQV WR
KDYH WKH VDPH H[SUHVVLRQ $ FXUUHQWO\ SRSXODU PHWKRG LV SKUDVHEDVHG 6WDWLVWLFDO 0DFKLQH
7UDQVODWLRQZKLFKILQGVPDSSLQJVEHWZHHQVKRUWVHTXHQFHVRIZRUGV7KH³SKUDVHV´LQTXHVWLRQ
DUH VLPSO\ FRQWLJXRXV VHTXHQFHV RI ZRUGV LQ WKH VRXUFH ODQJXDJH WKDW DUH IRXQG WR EH
FRQVLVWHQWO\ WUDQVODWHG LQ WKH VDPH ZD\ LQ WKH WDUJHW ODQJXDJH 7KH\ QHHG QRW EH JUDPPDWLFDO
SKUDVHV RU FRQVWLWXHQWV D PDSSLQJ PLJKW H[LVW EHWZHHQ WKH WZRZRUG SKUDVH ³WKHUH LV´ LQ
(QJOLVK WKH WKUHHZRUG ³LO \ D´ LQ )UHQFK DQG WKH VLQJOH ZRUG ³KD\´ LQ 6SDQLVK IRU H[DPSOH
7KHPDSSLQJVDUHVWRUHGLQDWDEOHZLWKWKHVRXUFHODQJXDJH¶VSKUDVHVHUYLQJDVWKHNH\WKDWLV
WKHYDOXHE\ZKLFKRQHVHDUFKHVWKHWDEOH7KHWDEOH¶VHQWULHVIRUWKDWNH\ZLOO W\SLFDOO\EHWKH
WDUJHWODQJXDJH¶VHTXLYDOHQWVDQGVLQFHPRUHWKDQRQHWUDQVODWLRQPLJKWH[LVWWKHSUREDELOLW\
RIWKDWJLYHQWUDQVODWLRQ
,GLRPV FDQ EH LQFRUSRUDWHG LQWR WKLV VWUXFWXUH LQ GLIIHUHQW ZD\V 2QH LV WR FRPSLOH WKH
PDSSLQJV WDEOH PDQXDOO\ RU IURP D NQRZOHGJHEDVHG UHVRXUFH ,Q WKH IRUPHU FDVH D WHDP RI
ELOLQJXDOH[SHUWVPDSVDQLGLRPWRDQHTXLYDOHQWLGLRPLQWKHWDUJHWODQJXDJHRULIRQHGRHVQRW
H[LVW WR D OLWHUDO SDUDSKUDVH $QDVWDVLRX PDGH XVH RI VXFK D WDEOH WR WUDQVODWH IURP
*HUPDQWR(QJOLVKDQGUHSRUWHG)VFRUHVRIDURXQGIRUFRQWLJXRXVLGLRPVDQGEHWZHHQ
DQG IRU QRQFRQWLJXRXV LGLRPV &KRODNRY DQG .RUGRQL DOVR FRPSLOHG D WDEOH
PDSSLQJ (QJOLVK SKUDVDO YHUEV LQFOXGLQJ EXW QRW UHVWULFWHG WR LGLRPV WR WKHLU HTXLYDOHQWV LQ
%XOJDULDQ7KLV WDEOHLQFOXGHGQRW RQO\WKHPDSSLQJEXW IRXURWKHUOLQJXLVWLFIHDWXUHVWKDWWKH\
REWDLQHGIURP :LNWLRQDU\LQFOXGLQJDVFRUHLQGLFDWLQJWKHGHJUHHWR ZKLFKWKHSKUDVDO YHUELV
FRPSRVLWLRQDO7KH\WUDQVODWHGVHQWHQFHVFRQWDLQLQJDSKUDVDOYHUEDQGREWDLQHGDQDYHUDJH
%/(8VFRUHRIFRPSDUHGWRIRUDV\VWHPZLWKRXWWKHLUWDEOHDQGIRUDV\VWHP
WUDLQHGXVLQJDVWDWLFPHWKRGLHFUHDWLQJDVLQJOHZRUGRXWRIWKHSKUDVHVHH6HFWLRQ
$VPHQWLRQHGLQWKH,QWURGXFWLRQ6DOWRQ5RVVDQG.HOOHKHUXVHGDVHFRQGPHWKRG
7KHILUVWVWHSLVWRWUDQVODWHWKHLGLRPWRDOLWHUDOSDUDSKUDVHZLWKLQWKHVRXUFHODQJXDJHFUHDWLQJ
DQLQWHUPHGLDWHYHUVLRQ7KHVHFRQGVWHSLVWRWUDQVODWHWKDWLQWHUPHGLDWHYHUVLRQLQWRWKHWDUJHW
ODQJXDJH ,I DQ LGLRP ZLWK WKH VDPH PHDQLQJ H[LVWV LQ WKH WDUJHW ODQJXDJH LW FDQ EH XVHG
RWKHUZLVHWKHSDUDSKUDVHLVVLPSO\WUDQVODWHGOLWHUDOO\6DOWRQHWDOXVHGWKLVPHWKRGWRWUDQVODWH
IURP (QJOLVK WR %UD]LOLDQ 3RUWXJXHVH 7KH DYHUDJH %/(8 VFRUH IRU WKH WUDQVODWLRQV LQYROYLQJ
KLJKO\IL[HGLGLRPVZDVFRPSDUHGWRIRUWUDQVODWLRQVPDGHE\WKHVDPHV\VWHP
EXW ZLWKRXW WKHLU HTXLYDOHQFH WDEOHV 7KLV UHVXOW ZDV VWDWLVWLFDOO\ VLJQLILFDQW +RZHYHU ZKHQ
WUDQVODWLQJLGLRPVWKDWVKRZPRUHYDULDELOLW\WKH%/(8VFRUHZDVFRPSDUHGWR
IRUWKHEDVHOLQH 7KRXJKWKHVFRUH ZDVKLJKHUZKHQWKHWDEOHZDVXVHGWKHGLIIHUHQFHZDVQRW
VWDWLVWLFDOO\VLJQLILFDQW)XUWKHUPRUHWKHVDPHV\VWHPDFKLHYHGRQFRUSRUDQRWFRQWDLQLQJ
LGLRPVVXJJHVWLQJWKDWLGLRPVSRVHDFKDOOHQJHHYHQZLWKWKLVYLDEOHVROXWLRQ
:HFRQFOXGHWKLVVHFWLRQE\FRQVLGHULQJDVLPLODUWDEOHXVHGIRU6HQWLPHQW$QDO\VLVUDWKHU
WKDQ0DFKLQH7UDQVODWLRQ7KHWDVNRID6HQWLPHQW$QDO\VLVV\VWHPLVWRFODVVLI\DJLYHQWH[WDV
H[SUHVVLQJ D SRVLWLYH QHXWUDO RU QHJDWLYH RSLQLRQ 6XFK V\VWHPV ZLOO RIWHQ UHO\ RQ D KXPDQ
DQQRWDWRU¶V KDYLQJ FRPSLOHG D WDEOH DVVRFLDWLQJ ZRUGV DQG SKUDVHV ZLWK WKH HPRWLRQV WKH\
FRQYH\7KH6HQWLPHQW$QDO\VLVV\VWHP¶VIXQFWLRQLVWROHDUQWKLVUHODWLRQDQGWRDSSO\ZKDWLW
OHDUQHGWRSUHYLRXVO\XQVHHQWH[WV$UHFHQWH[DPSOHRIKRZWKLVNLQGRIPDSSLQJFDQEHGRQH
IRU LGLRPV LV S6HQWL D V\VWHP FUHDWHG E\ 0XGLQDV =KDQJ DQG /HYHQH 7KLV V\VWHP
LQFOXGHV DSUHSURFHVVLQJV\VWHPZKHUHLGLRPV DUHWH[WXDOO\UHSODFHGE\HLWKHU³B*RRGB2QHB´
RU³B%DGB2QHB´GHSHQGLQJRQWKHKXPDQDVVHVVPHQWRIWKHLGLRP
$OORIWKHVHVROXWLRQVDUHORJLFDODQGFDQEHFUHGLWHGZLWKDGGUHVVLQJWKHLVVXHRILGLRPV¶
QRQOLWHUDOQDWXUH7KH\KDYHWKHLUOLPLWDWLRQVDVZHOO(YHQZLWKLQWKHLUDSSRLQWHGWDVNVWKH\GR
QRWDSSHDUWRGLVWLQJXLVKOLWHUDOXVHVIURPILJXUDWLYHRQHVIRUH[DPSOHDQLVVXHWKDWPLJKWOHDG
WRPLVWUDQVODWLRQVMXVWDVRYHUO\OLWHUDOWUDQVODWLRQVGR7KH\KDYHFOHDUO\EHHQGHVLJQHGZLWKWKHLU
RZQ1/3GRPDLQLQPLQGDQGDUHXQOLNHO\WREHH[WHQGHGWRRWKHUV,QGHHGHYHQZLWKLQDVLQJOH
WDVNWKH\FDQQRWDOZD\VEHJHQHUDOL]HGQHZWUDQVODWLRQWDEOHVDUHQHHGHGIRUHDFKQHZODQJXDJH
DQG QHZ VHQWLPHQW VFRUHV ZLOO EH QHHGHG LI WKH VHQWLPHQW VFDOH FKDQJHV IURP ELQDU\
³IDYRXUDEOHXQIDYRXUDEOH´ MXGJHPHQWV WR PRUH ILQHO\JUDLQHG RQHV 6DOWRQ HW DO¶V WDEOH DYRLGV
WKLVGLIILFXOW\VRPHZKDWEXWLWZRXOGQRWZRUNZHOOIRULGLRPVOLNH³VKRRWWKHPHVVHQJHU´WKDW
UHTXLUHDORQJHUSDUDSKUDVHRUH[SODQDWLRQ1RUFDQWKHVHVROXWLRQVOHDUQQHZLGLRPVRUPHDQLQJV
DV XVDJH FKDQJHV RYHU WLPH RU EHWZHHQ UHJLRQV :KLOH XQVXSHUYLVHG VROXWLRQV H[LVW ERWK IRU
FUHDWLQJ WKH SKUDVH WDEOHV RI SKUDVHEDVHG 6WDWLVWLFDO 0DFKLQH 7UDQVODWLRQ DQG 6HQWLPHQW
$QDO\VLV ZH DUH QRW DZDUH RI DQ\ WKDW DGGUHVV WKH SDUWLFXODU FKDOOHQJHV WKDW LGLRPV SRVH :H
WXUQWKHQWRRXUSURSRVHGDOWHUQDWLYH
4
Proposed Methods
:HQRZFRPHWRGHWDLOLQJWKHPHWKRGE\ZKLFKZHKRSHWRFUHDWHXVHIXOUHSUHVHQWDWLRQVRI
LGLRPV+RZZHZLOODVVHVVWKHXVHIXOQHVVRISDUWLFXODUUHSUHVHQWDWLRQVZLOOEHH[SODLQHGLQPRUH
GHWDLOLQ6HFWLRQ:HZLOOILUVWMXVWLI\ZK\ZHKDYHFKRVHQWRXVHGLVWULEXWLRQDOYHFWRUVSDFH
PRGHOV :H WKHQ HQWHU LQWR WKH VSHFLILFV RI WKH PHWKRG IURP KRZ ZH ZLOO WUDLQ WKH YHFWRU
UHSUHVHQWDWLRQVWRKRZZHDVVHPEOHWKHPWRIRUPDUHSUHVHQWDWLRQRIDQLGLRPRUDVHQWHQFH
:H DVVXPH WKURXJKRXW WKDW ZH KDYH D OLVW RI LGLRPV ZKRVH UHSUHVHQWDWLRQV ZH ZLVK WR
FUHDWH7KHOLVWXVHGIRUWKLVZRUNLVJLYHQLQ$SSHQGL[$,WLVODUJHO\DVXEVHWRIWKHOLVWXVHGE\
/LEEHQDQG7LWRQH&UHDWLQJDQDSSURSULDWHOLVWLVQRWDGLIILFXOWWDVNUHIHUHQFHZRUNVRQ
LGLRPV DERXQG HJ &DPEULGJH 8QLYHUVLW\ DQG WKHUH H[LVWV D VXEVWDQWLDO OLWHUDWXUH RQ
GHWHFWLQJPXOWLZRUGH[SUHVVLRQVHJ0DQQLQJ6FKXW]H3HFLQD6DQJDWDYDQ
&UDQHQEXUJK DQG QRQOLWHUDO ODQJXDJH VHH 6KXWRYD E IRU D UHYLHZ RU 6FKXOGHU +RY\ IRU D UHFHQW H[DPSOH 7KLV ZRUN LV FRQFHUQHG ZLWK WKH TXHVWLRQ RI LQWHUSUHWLQJ
LGLRPVRQZKLFKOHVVUHVHDUFKKDVEHHQGRQH
02'(/ 6(/(&7,21 :+< ',675,%87,21$/ 9(&725 63$&(
02'(/6"
7KHUH DUH DW OHDVW WZR TXHVWLRQV WKDW PLJKW EH DVNHG )LUVW ZK\ VKRXOG ZH H[SHFW D
GLVWULEXWLRQDOPRGHOWREHDJRRGILWIRUWKLVSUREOHP"6HFRQGJLYHQWKDWVXFKPRGHOVDUHOHDUQHG
LQ DQ XQVXSHUYLVHG ZD\ ZK\ ZRXOG ZH XVH DQ XQVXSHUYLVHG PHWKRG LQVWHDG RI XVLQJ RXU
NQRZOHGJHRIZKDWLGLRPVPHDQ"7KHDQVZHUVWRERWKTXHVWLRQVDUHUHODWHG
8SXQWLOQRZDVZHVDZLQ6HFWLRQPDQ\UHSUHVHQWDWLRQVRILGLRPVKDYHEHHQFUHDWHG
XVLQJDVWDWLFDQGRUNQRZOHGJHEDVHGDSSURDFK<HWRYHUWKHODVWWZRGHFDGHVWKHWUHQGLQ1/3
UHVHDUFKKDVEHHQPRYLQJLQH[DFWO\WKHRSSRVLWHGLUHFWLRQ6WDWLVWLFDOPHWKRGVRIIHUDPHDQVWR
PRGHOODQJXDJHEDVHGRQDFWXDOXVDJH2XULQWXLWLRQVRQKRZODQJXDJHZRUNVDVLQGLVSHQVDEOH
DV WKH\ PD\ EH GR QRW DOZD\V JLYH WKH IXOO SLFWXUH DQG WKDW LV SDUWLFXODUO\ WUXH RI ILJXUDWLYH
ODQJXDJH)RUH[DPSOHWKH&DPEULGJH,GLRPV'LFWLRQDU\&DPEULGJH8QLYHUVLW\GHILQHG
³KLWWKHURDG´ DVOHDYLQJ RQH¶V FXUUHQW SODFHIRU DQRWKHU DQG³FKDQJHRQH¶VWXQH´DVFKDQJLQJ
RQH¶VRSLQLRQ%HIRUHZRUNLQJRQWKLVGLVVHUWDWLRQWKHDXWKRUZRXOGKDYHGHILQHGWKHVHLGLRPVLQ
H[DFWO\ WKH VDPH ZD\ <HW DFWXDO XVDJH LQ WKH &RUSXV RI &RQWHPSRUDU\ $PHULFDQ 'DYLHV
VKRZV WKDW ³KLW WKH URDG´ LV PRUH IUHTXHQWO\ XVHG WR GHVFULEH D PXVLFDO JURXS RU D
SROLWLFLDQDERXWWRJRRQWRXUDQGWKDW³FKDQJHRQH¶VWXQH´FDQDOVRPHDQFKDQJLQJRQH¶VSXEOLF
DFFRXQWRIZKDWWUDQVSLUHG7KDWWKHVHPHDQLQJVDOVRH[LVWVHHPVREYLRXVLQUHWURVSHFWEXWQRW
DOOFRPPRQXVHVQHFHVVDULO\FRPHWRPLQGZKHQ RQHIRUPXODWHVDGHILQLWLRQ$QXQVXSHUYLVHG
DSSURDFKGLVFRYHUVWKHDFWXDOXVHVIURPWKHGDWDLWVHOIDQGVRLVOHVVOLNHO\WREHLQIOXHQFHGE\
SUHFRQFHSWLRQV ,QGHHG LW LV WKURXJK WKH FXUUHQW ZRUN WKDW WKHVH WZR H[DPSOHV FDPH WR RXU
DWWHQWLRQ
-XVWDVZRUGVKDYHFRQQRWDWLRQVDVZHOODVGHQRWDWLRQVLGLRPVQRWRQO\KDYHDPHDQLQJEXW
H[SUHVV DQ DWWLWXGH :H KDYH DOUHDG\ JLYHQ WKH H[DPSOH RI WKH HPRWLRQDO LPSDFW RI VD\LQJ
³PLVVLQJWKHERDW´LQVWHDGRIWKHPRUHQHXWUDO³IDLOWRWDNHDGYDQWDJHRIDQRSSRUWXQLW\´:KLOH
³VKRRW WKH EUHH]H´ FHUWDLQO\ PHDQV ³FKDW´ WKH LGLRP PXVW FRQYH\ VRPHWKLQJ WKDW ³FKDW´ GRHV
QRWLILWGLGQRWZHZRXOGVDYHWZRV\OODEOHVE\MXVWVD\LQJ³FKDW´,QWKLVSDUWLFXODUFDVH³VKRRW
WKHEUHH]H´FRPPXQLFDWHVDODFNDGDLVLFDODWWLWXGHDQGDPHDQGHULQJEHWZHHQVXEMHFWVWKDW³FKDW´
E\LWVHOIGRHVQRW7KHVHDWWLWXGHVFDQFHUWDLQO\EHHQFRGHGLQWRDNQRZOHGJHEDVHGUHVRXUFHEXW
EHFDXVH HDFK LGLRP KDV LWV RZQ VHW RI FRQQRWDWLRQV LW LV QRW REYLRXV WKDW DOO RI WKH UHOHYDQW
TXDOLWLHVFRXOGEHLQFOXGHGLQDVLQJOHUHSUHVHQWDWLRQ
,QDGGLWLRQWREHLQJGLIILFXOWWRGHILQHLGLRPVDUHLQDFRQVWDQWVWDWHRIIOX[RYHUWLPHDQG
VSDFH 7KH\ HQWHU LQWR RXU DZDUHQHVV VXEOLPLQDOO\ DW ILUVW RYHU WLPH WKH\ EHFRPH SDUW RI RXU
GDLO\YRFDEXODU\DQGWKHQWKH\FDQIDOORXWRIXVHDVVXGGHQO\RUDVJUDGXDOO\DVWKH\ LQLWLDOO\
FDPH LQWR LW )RU H[DPSOH WKH H[SUHVVLRQ ³WKURZ VRPHRQH XQGHU WKH EXV´ KDV FRPH WR PHDQ
VDFULILFLQJ D IRUPHU DOO\ RIWHQ RSSRUWXQLVWLFDOO\ WR DFKLHYH D EURDGHU JRDO $FFRUGLQJ WR
FROXPQLVW'DYLG6HJDOWKLVH[SUHVVLRQFDPHLQWRXVHLQWKHODWHVEXWZDVLQLWLDOO\
UHVWULFWHG WR WKH VSRUWV GRPDLQ ,W ILUVW EHFDPH SDUW RI SROLWLFDO GLVFRXUVH LQ RQO\
RFFDVLRQDOO\XSWRDQGLQFOXGLQJWKHHOHFWLRQ7KHQIRUUHDVRQVXQNQRZQLWEHFDPH
XVHGVRRIWHQLQDSROLWLFDOFRQWH[WWKDW6HJDOGXEEHGLW³WKHFOLFKpRIWKH>86SUHVLGHQWLDO@
FDPSDLJQ´ +LV FROXPQ H[SUHVVHG D ZLVK WKDW WKH LGLRP EH UHWLUHG EXW VR IDU KLV ZLVK KDV QRW
FRPHWUXH7KHH[SUHVVLRQPLJKWHYHQWXDOO\EHFRPHJHQHUDOL]HGLQWRGRPDLQVRWKHUWKDQVSRUWV
DQGSROLWLFV2ULWPLJKWGLVDSSHDUDVTXLFNO\DVLWFDPH7KLVNLQGRIIOX[FDQEHFDSWXUHGE\D
FRUSXVRIWH[WV EXW LWLVXQOLNHO\WR EHFDSWXUHG LQ DOH[LFDO UHVRXUFHRIWKHNLQG FRQVLGHUHGLQ
6HFWLRQ
$ ILQDO DUJXPHQW LV WKDW LGLRPV FDQ DV ZDV GLVFXVVHG LQ 6HFWLRQ FRPH LQ YDU\LQJ
IRUPV2QFHYDULDQWIRUPVRILGLRPVFRPHLQWRSOD\DUXOHEDVHGRUNQRZOHGJHEDVHGDSSURDFK
EHFRPHVIDUPRUHGLIILFXOWWRLPSOHPHQW:HVDZDQH[DPSOHRIVXFKIDLOXUHLQ&KDSWHUZKHUH
ERWK *RRJOH 7UDQVODWH DQG 35207 FRXOG WUDQVODWH ³VPHOO D UDW´ EXW QHLWKHU VXFFHHGHG ZLWK
³VPHOODELJUDW´7KHGLIILFXOW\LVWKDWKXPDQVPDQLSXODWHPHWDSKRUVXVLQJDQDQDORJLFDOIRUPRI
UHDVRQLQJ QRW DQ DOJRULWKP 8QGHUVWDQGLQJ WKDW ³ULQJV D IDLQW EHOO´ VLJQLILHV D OHVV GLVWLQFW
PHPRU\WKDQ³ULQJVDEHOO´UHTXLUHVDQDQDORJ\WREHPDGHEHWZHHQWKHYROXPHRIWKHEHOOWRWKH
YLYLGQHVVRIWKHUHFROOHFWLRQ7KLVSURFHVVLVRQHWKDWKXPDQVGRXQFRQVFLRXVO\HIIRUWOHVVO\DQG
LQVWDQWDQHRXVO\ %XW LW VHHPV XQOLNHO\ WKDW ZH FDQ VSHFLI\ LQ DGYDQFH ZKDW DUH WKH VDOLHQW
HOHPHQWVWKDWUHTXLUHPDSSLQJ
7KHDERYHDUHDOOUHDVRQVZK\ZHIDYRXUDQXQVXSHUYLVHGDSSURDFKLQJHQHUDOEXWWKH\FDQ
DOVRVHUYHDVDUJXPHQWVIRUDGLVWULEXWLRQDOYHFWRUVSDFHPRGHOLQSDUWLFXODU6XFKUHSUHVHQWDWLRQV
KDYH EHHQ XVHG WR FRPSLOH WKHVDXUL VLQFH WKH V *UHIHQVWHWWH 7KH SURFHVV RI
GLVFRYHULQJ DQ H[SUHVVLRQ¶V VHQVH IURP XVDJH DQG WKHQ XVLQJ WKDW UHSUHVHQWDWLRQ WR IRUP D
VHPDQWLFQHWZRUNRIUHODWHGWHUPVLVH[DFWO\ZKDWLVUHTXLUHGKHUHRQO\QRZWKHWDUJHWGRPDLQLV
LGLRPV UDWKHU WKDQ ZRUGV 8VLQJ ZRUGYHF 0LNRORY HW DO D 0LNRORY HW DO E
*RRJOHQGDVHWRIHPEHGGLQJVFDQEHJHQHUDWHGIURPDFRUSXVZLWKLQDQKRXURQDQRUGLQDU\
GHVNWRSFRPSXWHUSXWWLQJWKLVWHFKQLTXHZLWKLQDQ\RQH¶VUHDFK
%HFDXVH WKH YHFWRU VSDFH PRGHO GRHV QRW HQFRGH VHPDQWLF SURSHUWLHV RI W\SHV EXW UDWKHU
GLVWULEXWLRQVRIWRNHQVLWFDQRYHUFRPHWKHGLIILFXOWLHVVXPPDUL]HGDERYH7KHPRGHOOHDUQVKRZ
DQLGLRPLVXVHGZLWKRXWZRUU\LQJDERXWLWVVSHFLILFV:K\GRZHWDONDERXWDEXVZKHQZHUHIHU
WRVDFULILFLQJDQDOO\",WGRHVQRWPDWWHU:KDWWKHPRGHOZLOOQRWLFHLIWUDLQHGFRUUHFWO\LVWKDW
WKH LGLRPDWLF SKUDVH LV XVHG LQ SROLWLFDO RU VSRUWV FRQWH[WV EXW QRW PLOLWDU\ RU PHGLFDO RQHV ,W
ZLOODOVRQRWLFHWKDWWKHH[SUHVVLRQVKDUHVVRPHRIWKHVDPHFRQWH[WVDVZRUGVOLNH³EHWUD\DO´DQG
³RSSRUWXQLVP´,IWKHPRGHOLVWUDLQHGRQWH[WVIURPWKHDSSURSULDWHWLPHDQGSODFHLWZLOOOHDUQ
WKHH[SUHVVLRQDVLWLVXVHGLQWKDWORFDOH$QGLWFDQKDQGOHYDULDQWVLQZD\VWKDWWKHVWDWLFDQG
NQRZOHGJHEDVHGPHWKRGVFDQQRW
75$,1,1*7+(:25'(0%('',1*6
7RDGGUHVVWKHTXHVWLRQRIOHDUQLQJLGLRPVRIDSDUWLFXODUWLPHDQGSODFHZHLQGXFHGWKH
YHFWRUHPEHGGLQJVE\WUDLQLQJWKHPRQWKH&RUSXVRI&RQWHPSRUDU\$PHULFDQ(QJOLVK&2&$
'DYLHV2QH RIWKLV FRUSXV¶VDGYDQWDJHVLV WKDW LV WKDW LW UHIOHFWV 1RUWK $PHULFDQ
XVDJHIURPWKHODVWWZHQW\ILYH\HDUV$QRWKHULVWKDWLWLQFOXGHVDYDULHW\RIVRXUFHVDFDGHPLF
SDSHUVILFWLRQWUDQVFULSWVRIVSRNHQZRUGGLDORJXHVPDJD]LQHSLHFHVDQGQHZVDUWLFOHV&2&$
DOVRSURYLGHVOHPPDVDQGWKH8&5(/&/$:6SDUWRIVSHHFKWDJIRUHYHU\ZRUGLQWKHFRUSXV
)ROORZLQJDVXJJHVWLRQIURP&ODUNZHPDGHXVHRIWKLVIHDWXUHE\FUHDWLQJDYHUVLRQRI
WKHFRUSXVLQZKLFKHDFKZRUGZDVZULWWHQDVLWVOHPPDZLWKWKHILUVWWZROHWWHUVRIWKHSDUWRI
VSHHFKWDJDWWDFKHGWRLW+HUHLVDQH[DPSOHRIWKHRULJLQDOVHQWHQFHDQGWKHZD\WKDWLWKDVEHHQ
WUDQVIRUPHGIRURXUWUDLQLQJFRUSXV
,
GRXWJURZQLWDQGZDVRYHUMR\HGDWWKHSURVSHFWRIJRLQJWRFROOHJHLQDELJFLW\
L_SS KDYH_YK RXWJURZ_YY LW_SS DQG_FF EH_YE RYHUMR\HG_MM DW_LL WKH_DW SURVSHFW_QQ RI_LR JR_YY WR_LL
FROOHJH_QQLQ_LLD_DWELJ_MMFLW\_QQ
8VLQJWKLVFRQYHQWLRQPHDQVWKDWRQWKHRQHKDQGDVLQJOHRUWKRJUDSKLFIRUPZLOOH[LVWIRUWKH
VDPHZRUGZLWKRXWDQ\ LQIOHFWLRQVIRUSOXUDOVYHUEFRQMXJDWLRQVHWF7KHIRUPV³HDW´³HDWV´
³DWH´ DQG ³HDWHQ´ ZLOO DOZD\V DSSHDU DV ³HDW_YY´ DQG ³PLFH´ DQG ³PRXVH´ ZLOO ERWK EH
³PRXVH_QQ´ 2Q WKH RWKHU KDQG D GLIIHUHQW IRUP ZLOO H[LVW IRU D QRXQ DQG YHUE WKDW KDSSHQ WR
VKDUHWKHVDPHVSHOOLQJOLNH³ERRN´UHQGHUHGDV³ERRN_QQ´DQG³ERRN_YY´UHVSHFWLYHO\6LQFH
WKH VNLSJUDP PRGHO OHDUQV D UHSUHVHQWDWLRQ IRU HDFK WRNHQ RI D SDUWLFXODU RUWKRJUDSKLF IRUP
XVLQJWKHOHPPDDQGSDUWRIVSHHFKKHOSVPDNHWKHUHSUHVHQWDWLRQVPRUHVSHFLILF7KDWVDLGWKH
PRGHOGRHVOHDUQDVLQJOHUHSUHVHQWDWLRQIRUSRO\VHPRXVZRUGVOLNH³EDU´LIWKHVHQVHVDUHRIWKH
VDPHSDUWRIVSHHFK
2QFHWKHWUDLQLQJFRUSXVKDVEHHQFUHDWHGZHUXQWKHZRUGYHFSURJUDP*RRJOHQGLQ
LWVRULJLQDOIRUPQRDOWHUDWLRQVWRWKHRULJLQDOFRGHZHUHUHTXLUHG:KHQUXQQLQJWKHSURJUDP
WKHXVHUPXVWVSHFLI\WKHWUDLQLQJFRUSXVDV ZHOODVVHYHUDO SDUDPHWHUVDVVHHQLQ 6HFWLRQ VXFKDVWKHGLPHQVLRQRIWKHZRUGYHFWRUWKHVL]HRIWKHZLQGRZZKHWKHUKLHUDUFKLFDOVDPSOLQJ
RU QHJDWLYH VDPSOLQJ LV XVHG DQG VR RQ 7KH VSDFH RI SRVVLEOH SDUDPHWHU FRPELQDWLRQV LV WRR
ODUJHIRUDOOFRPELQDWLRQVWREHH[SORUHG:HH[SORUHGDVPDQ\DVSRVVLEOHIDYRXULQJWKRVHSDUWV
RI WKH VSDFH ZKHUH D SDUWLFXODU VHWWLQJ SURGXFHG JRRG UHVXOWV )RU H[DPSOH LI ZLQGRZV RI SURGXFHGJRRGUHVXOWVWKDWSDUDPHWHUZRXOGEHKHOGFRQVWDQWDQGRWKHUSDUDPHWHUFRPELQDWLRQV
ZHUHYDULHG
7KH PRGHO GHVFULEHG WKXV IDU LV XVHG DV RXU EDVHOLQH :H FUHDWHG WZR RWKHU WUDLQLQJ
PHWKRGVWKDWLGHQWLI\SRWHQWLDOO\ILJXUDWLYHLQVWDQFHVDQGUHVXOWLQGLVWLQFWLYHZRUGHPEHGGLQJV
EHLQJFUHDWHGIRU RXULGLRPV¶FRPSRQHQWV7KHRQO\GLIIHUHQFHEHWZHHQWKHVHPHWKRGVDQGWKH
EDVHOLQH LV LQ WKH SUHSURFHVVLQJ RI WKH WUDLQLQJ FRUSXV $JDLQ ZH PDGH QR DOWHUDWLRQ WR WKH
ZRUGYHFFRGHRUKRZWKHYHFWRUUHSUHVHQWDWLRQVDUHJHQHUDWHG
7KH NH\ LGHD EHKLQG RXU PHWKRGV LV WR WDJ ZRUG WRNHQV WKDW DUH OLNHO\ EHLQJ XVHG
ILJXUDWLYHO\VRWKDWWKH\DUHGLVWLQFWIURPWKHVDPHZRUGXVHGOLWHUDOO\'RLQJVRIRUFHVWKHPRGHO
WROHDUQDQGRXWSXWGLVWLQFWYHFWRUVIRUHDFKZRUGW\SH7KDWLVE\FKDQJLQJDOOWRNHQVRI³KLW_YY
WKH_DW ERRN_QQ´ WR ³KLW_YI ERRN_QI´ ZKLOH OHDYLQJ SKUDVHV OLNH ³UHDG_YY ERRN_QQ´ DQG ³ERRN_YY
DQ_DWDSSRLQWPHQW_QQ´DVWKH\DUHWKHVNLSJUDPPRGHORXWSXWVGLVWLQFWYHFWRUVIRU³ERRN_YY´DQG
³ERRN_QQ´ DV EHIRUH DQG D WKLUG YHFWRU IRU ³ERRN_QI´ 7KLV ODWWHU YHFWRU HQFRGHV WKH
GLVWULEXWLRQDO LQIRUPDWLRQ RI ³ERRN_QI´ EXW LJQRUHV FRQWH[WV FRQWDLQLQJ ³ERRN_QQ´ %\ WKH
'LVWULEXWLRQDO +\SRWKHVLV WKLV QHZ UHSUHVHQWDWLRQ ZLOO FDSWXUH WKH ILJXUDWLYH PHDQLQJ WKDW
³ERRN´WDNHVLQWKHSKUDVH³KLWWKHERRNV´ZKLFKLQWKHEDVHOLQHPHWKRGZDVVXEVXPHGLQWKH
YHFWRU UHSUHVHQWDWLRQ RI ³ERRN_QQ´ +HUH DUH WKH WZR ZD\V ZH XVHG WR SUHSURFHVV WKH WUDLQLQJ
FRUSXV
7KH ILUVW VWUDWHJ\ LV WR ILQG DOO XVHV RI D SDUWLFXODU ZRUG SDWWHUQ OLNH ³KLW D ZDOO´ RU ³ORVW
P\\RXUKLVKHU« PDUEOHV´ DQG DGG WKH ³ILJXUDWLYH´ WDJ WR HDFK ZRUG 7KLV WH[WXDO
VXEVWLWXWLRQ PHDQV WKDW DOO WRNHQV RI WKLV SKUDVH DUH WDJJHG LGHQWLFDOO\ ZKHWKHU WKH\ ZHUH
XVHG LQGHHG XVHG ILJXUDWLYHO\ RU QRW 7KH UHVXOW LV WKDW WKH PRGHO ZLOO OHDUQ GLVWLQFW
UHSUHVHQWDWLRQVIRU³KLW´DQG³ZDOO´ZKHQWKRVHWZRZRUGVDSSHDULQWKHVSHFLILFSKUDVH³KLWD
ZDOO´ WKDW ZLOO EH LQGHSHQGHQW RI ³KLW´ XVHG ZLWK RWKHU QRXQV RU ³ZDOO´ ZLWK RWKHU YHUEV
+RZHYHU WKH UHSUHVHQWDWLRQV VR OHDUQHG DUH QRW VSHFLILF WR LGLRPDWLF LQVWDQFHV RI ³KLW´ RU
³ZDOO´LWZLOOLQFOXGHWUDIILFDFFLGHQWVDVZHOODVREVWDFOHV7KHUHDUHDIHZDGGLWLRQDOGHWDLOV
WKDWVKRXOGEHPDGHFOHDUDERXWWKLVPHWKRG
D $ VHFRQG SDVV ZDV PDGH WR WDJ ERWK WKH YHUE DQG WKH QRXQ IRU H[SUHVVLRQV WKDW DUH
FRPPRQO\XVHGLQWKHSDVVLYHYRLFH)RUH[DPSOH³[KDVWXUQHGWKHWDEOHV´LVHTXLYDOHQW
WR ³WKH WDEOHV KDYH WXUQHG´ )RU VXFK H[SUHVVLRQV WKH QRXQ DQG YHUE DUH DOVR WDJJHG LI
WKH\DSSHDULQVXFKDFRQVWUXFWLRQ
E 7RNHQV DUH QRW JLYHQ WKH VSHFLDO WDJ LI WKH\ DSSHDU LQ PRUH HODERUDWH YDULDQWV RI ³KLW D
ZDOO´VLPSO\EHFDXVHWKRVHWRNHQVGRQRWFRQIRUPWRWKHWH[WXDOSDWWHUQEHLQJVHDUFKHG
)RUH[DPSOHSKUDVHVOLNH³KLWDEULFNZDOO´³KLWDQLPSHQHWUDEOHZDOO´RU³VWUXFNDZDOO´
DUH OHIW LQ WKHLU RULJLQDO IRUPV HYHQ WKRXJK WKHVH SKUDVHV PLJKW EH XVHG LGLRPDWLFDOO\
7KDW PHDQV WKH UHSUHVHQWDWLRQ IRU WKH OLWHUDO XVHV ZLOO EH WUDLQHG ZLWK VRPH ILJXUDWLYH
XVHV
F 7KLVPHWKRGGHOLEHUDWHO\FKRRVHVWRFUHDWHDVLQJOHUHSUHVHQWDWLRQIRUDOOXVHVRIDZRUGLI
LWDSSHDUVLQVHYHUDOLGLRPV)RUH[DPSOH³KLW´LVWKHYHUELQHLJKWLGLRPVRQRXUOLVW%XW
WKHUH ZLOO EH RQH W\SH ³KLW_YI´ WKDW ZLOO EH OHDUQHG IRU DQG VKDUHG E\ DOO RI WKHVH
H[SUHVVLRQV2XUH[SHFWDWLRQZDVWKDWWKHVHQVHVRIWKHLGLRPDWLFXVHVRIWKHZRUGZRXOG
DOOKDYHVRPHWKLQJLQFRPPRQ$VGLVFXVVHGLQ6HFWLRQWKH³KLW´LQH[SUHVVLRQV³KLW
WKH VDFN´ ³KLW WKH URDG´ ³KLW WKH ERRNV´ DOO FRQYH\ WKH VHQVH RI OHDYLQJ RQH¶V FXUUHQW
VWDWH IRU DQRWKHU ,I VR FRQVROLGDWLQJ WKHVH WRNHQV ZRXOG LPSURYH WKH XVHIXOQHVV RI WKH
UHSUHVHQWDWLRQ ,W PD\ EH KRZHYHU WKDW VRPH ZRUGV DUH XVHG LQ GLIIHUHQW LGLRPV ZLWK
YHU\ GLIIHUHQW PHDQLQJV HJ ³VKRRW WKH EUHH]H´ DQG ³VKRRW WKH PHVVHQJHU´ DQG VR
IRUFLQJWKHPRGHOWROHDUQRQHUHSUHVHQWDWLRQIRU³VKRRW´ZRXOGQRWEHDVDGYDQWDJHRXVLQ
WKDWFDVH3DUWRIWKHH[SORUDWLRQRIRXUUHVXOWVZLOOH[DPLQHWKLVLVVXH
$Q DOWHUQDWLYH VWUDWHJ\ LV IRU RQH RU VHYHUDO KXPDQ DQQRWDWRUV WR JR WKURXJK WKH FRUSXV
PDQXDOO\VHDUFKLQJIRUDOORFFXUUHQFHVRI³KLW´QHDU³ZDOO´7KHDQQRWDWRUZLOOGHFLGHLIWKH
SKUDVH LV D ILJXUDWLYH XVH DQG LI VR ZLOO DGG WKH VSHFLDO WDJ WR WKDW LQVWDQFH 7KDW LV D
VHQWHQFHDERXWDSURMHFWKLWWLQJDZDOOZRXOGEHWDJJHGZKLOHRQHDERXWDFDUKLWWLQJDZDOO
ZRXOGQRW2XUH[SHFWDWLRQZDVWKDWWKHPRGHO¶VUHSUHVHQWDWLRQVZRXOGEHPRUHVSHFLILFWR
LGLRPDWLF XVHV DQG WKHUHIRUH PRUH XVHIXO :H WKXV DSSOLHG WKLV SURFHGXUH WR WKH FRUSXV DV
PRGLILHG DERYH ZLWK WKH DXWKRU UHDQQRWDWLQJ RI WKH H[SUHVVLRQV ZKRVH OLWHUDO DQG
ILJXUDWLYH PHDQLQJV VHHPHG WKH PRVW OLNHO\ WR EH FRQIXVHG 7KDW LV WKH OLWHUDO LQVWDQFHV RI
WKRVHH[SUHVVLRQVZHUHFKDQJHGEDFNWRWKHLUXVXDOWDJV7KHVHH[SUHVVLRQVDUHOLVWHGLQ
$SSHQGL[%7KHRWKHULGLRPVZHUHOHIWDVWKH\ZHUH
2QFH WKH WUDLQLQJ FRUSXV LV WDJJHG LQ WKHVH ZD\V WKH VDPH ZRUGYHF SURJUDP ZDV UXQ LQ WKH
XVXDO ZD\7KHEDWFKHV RIYHFWRUVRXWSXWE\ HDFKUXQRIZRUGYHFZHUHODEHOOHGWREH DEOHWR
GLVWLQJXLVKZKLFKWUDLQLQJFRUSXVZDVXVHGDQGZLWKZKLFKWUDLQLQJSDUDPHWHUV
&20326,7,2102'(/6
&RPSRVLWLRQDOLW\LVDQHVVHQWLDOSDUWRILQWKHZRUNGHVFULEHGKHUH)LUVWWKHUHLVWKHPDWWHU
RIFRPELQLQJWKHUHSUHVHQWDWLRQV RIWKHLGLRPV¶LQGLYLGXDOZRUGVWR IRUP WKHLGLRPDWLFSKUDVH
7KLVTXHVWLRQLVRQHRIFRPSRVLWLRQDOLW\DWDORFDOVFDOH0RUHLPSRUWDQWO\WZRRIWKHWKUHHWDVNV
LQYROYH DVVHVVLQJ WKH JRRGQHVV RI ILW RI D SKUDVH IRU D SDUWLFXODU VHQWHQFH 7KLV MXGJHPHQW LV
PDGHE\FRPSDULQJWKHLGLRPWRWKHUHVWRIWKHVHQWHQFH+RZHYHUWKDWOHDYHVRSHQWKHTXHVWLRQ
RIKRZWRFRPSDUHDSKUDVHWRDORQJHUJURXSRIZRUGV7KLVDVSHFWLVFRPSRVLWLRQDOLW\DWDPRUH
JOREDOVFDOH
&RPSRVLWLRQRIWKH,GLRPDWLF3KUDVH
2QFHZHKDYHWKHUHSUHVHQWDWLRQVRIWKHLQGLYLGXDOFRPSRQHQWVRIWKHLGLRPOLNH³KLW_YI´
DQG ³ZDOO_QI´ ZH PXVW FRPELQH WKHP WR IRUP WKH FRPSOHWH LGLRP :H DWWHPSWHG WKH WKUHH
PHWKRGV GHVFULEHG LQ 6HFWLRQ DQG VLPSOH YHFWRU DGGLWLRQ HOHPHQWZLVH PXOWLSOLFDWLRQ
DQG WKH 3UHGLFDWLRQ $OJRULWKP )RU WKH ODWWHU ZH WULHG D ODUJH QXPEHU RI GLIIHUHQW SDUDPHWHU
FRPELQDWLRQV IRU Q WKH VL]H RI WKH SUHGLFDWH¶V QHLJKERXUKRRG N WKH PD[LPXP QXPEHU RI
YHFWRUVIURPWKDWYHFWRUWRXVHDQGWWKHWKUHVKROGWKDWGHFLGHVZKHWKHUDYHFWRULVFKRVHQ:H
UHSRUW KHUH WKH RQHV WKDW .LQWVFK UHFRPPHQGHG LQ KLV SDSHU DQG WKH RQHV WKDW ZH IRXQG PRVW
HIIHFWLYH
&RPSRVLWLRQRIWKH6HQWHQFH
7KH WDVN KHUH LV WR SXW DOO RI WKH ZRUGV LQ WKH VHQWHQFH WRJHWKHU LQWR D IRUP WKDW FDQ EH
FRPSDUHGWRWKHLGLRPIRUJRRGQHVVRIILW:HUHFDOOIURP6HFWLRQWKDWWKHFRVLQHVLPLODULW\
VFRUH LQGLFDWHV KRZ VLPLODU WZR ZRUGV DUH E\ FRPSDULQJ WKHLU YHFWRU UHSUHVHQWDWLRQV 7KH
TXHVWLRQ LV WKHQ KRZ WR DSSO\ WKLV VFRUH WR D JURXS RI ZRUGV :H H[SHULPHQWHG ZLWK IRXU
FRPSRVLWLRQ PHWKRGV H[SODLQHG LQ GHWDLO EHORZ HDFKRI ZKLFK UHSUHVHQW DVHQWHQFHDV HLWKHUD
VLQJOHYHFWRURUDVHWRIYHFWRUV(DFKWDNHVDGLIIHUHQWWKRXJKDOZD\VYHU\VLPSOHDSSURDFKWR
WKLVTXHVWLRQ0RUHIRUPDOO\ZHFDVWWKHSUREOHPDVIROORZV*LYHQ
x 6DVHTXHQFHRI1ZRUGVZZ«ZSZS«Z1
x YHFWRUUHSUHVHQWDWLRQVIRUDOORIWKHZRUGV^Z«Z1`
x DPXOWLZRUGH[SUHVVLRQHUHSUHVHQWHGE\WKHYHFWRUH
x WKHSRVLWLRQSLQ6ZKHUHWKHSRWHQWLDOO\LGLRPDWLFH[SUHVVLRQLVVLWXDWHG
ZHFUHDWHDYHFWRURUPDWUL[UHSUHVHQWDWLRQRIWKHVHQWHQFH6ZKLFKZHVKDOOGHVLJQDWHDV6
:LWK WKLV QRWDWLRQ LQ PLQG ZH FDQ WXUQ WR WKH TXHVWLRQ RI KRZ ZH VKDOO FUHDWH WKH
UHSUHVHQWDWLRQ6WRZKLFKVHYHUDOLGLRPYHFWRUVHLFDQEHFRPSDUHGLQRUGHUWRGHWHUPLQHZKLFK
H LV WKH PRVW VLPLODU LH EHVW VXLWHG IRU WKH VHQWHQFH 6 ,W LV LPSRUWDQW WR QRWH WKDW DOO RI WKH
PHWKRGVXVHRQO\WKHQRXQVDGMHFWLYHVDQGYHUEVRIWKHVHQWHQFHV)XQFWLRQZRUGV³WKH´³DQG´
³RI´HWFDUHRIOLWWOHYDOXHZKHQGHFLGLQJKRZZHOODSKUDVHILWVWKHVHQWHQFHDQGLQGHHGWKHLU
XELTXLW\ DFURVV DOO FRQWH[WV PHDQV WKDW DOPRVW DQ\ YHFWRU ZLOO DOLJQ ZHOO ZLWK WKHP $GYHUEV
SURYHG LQ HDUO\ WHVWV WR EH XQUHOLDEOH VRPHWLPHV IRROLQJ WKH PRGHO WR DFFHSW D GHFR\ WKDW LW
ZRXOGKDYHRWKHUZLVHUHMHFWHG7KLVGLIILFXOW\LVSDUWLFXODUO\WUXHRIEDJRIZRUGPRGHOVZKLFK
GRQRWWDNHLQWRDFFRXQWWRZKLFKZRUGWKHDGYHUELVDWWDFKHG
&KRRVH D NH\ ZRUG 7\SLFDOO\ RQH RU WZR HOHPHQWV LQ D VHQWHQFH ZLOO EH WKH FXH WKDW
LQGLFDWHV ZKHWKHU D ILJXUDWLYH RU OLWHUDO VHQVH ZDV LQWHQGHG RU ZKLFK LGLRP LV WKH PRVW
DSSURSULDWH IRU WKDW VHQWHQFH 7KH RWKHU HOHPHQWV PHQWLRQHG LQ WKH VHQWHQFH DUH DW ZRUVW
GLVWUDFWLRQVIURPWKHGHFLVLRQ7KHPRVWHIIHFWLYHZD\WRILQGWKHEHVWVLQJOHFXHLVWRILQGWKDW
ZRUG IRU ZKLFK WKHUH LV WKH ODUJHVW GLIIHUHQFH EHWZHHQ SRVVLEOH FKRLFHV $V DQ H[DPSOH
FRQVLGHUWKHWDVNRIFKRRVLQJZKHWKHU³VSLOOWKHEHDQV´ZDVXVHGOLWHUDOO\RUILJXUDWLYHO\7KH
ILJXUDWLYH VHQVH VKRXOG PDWFK ZHOO ZLWK ZRUGV UHIHUULQJ WR VHFUHWV RU WDONLQJ DQG WKRVH DUH
H[DFWO\WKHZRUGVWKDWZRXOGQRWFRKHUHZLWKUHIHUHQFHVWROLWHUDOEHDQV
0DWKHPDWLFDOO\ WKLV FDOFXODWLRQ LV GRQH E\ FUHDWLQJ D UHSUHVHQWDWLRQ RI WKH OLWHUDO
LQWHUSUHWDWLRQ ZKLFK ZH GHVLJQDWH DV HOLW DQG D UHSUHVHQWDWLRQ RI WKH ILJXUDWLYH PHDQLQJ
GHVLJQDWHGDVHILJ:HWKHQFRPSXWHWKHVLPLODULW\VFRUHEHWZHHQWKRVHWZRLQWHUSUHWDWLRQVDQG
HDFKQRXQDGMHFWLYHDQGYHUELQWKHVHQWHQFHDQGVHOHFWWKHVLQJOHZRUGZPD[IRUZKLFKWKH
GLIIHUHQFHEHWZHHQWKHFDQGLGDWHVLVWKHJUHDWHVW
ZPD[ ƒ”‰ƒš௪೔ ห•‹ሺ‫ ܑܟ‬ǡ ‫ ܜܑܔ܍‬ሻȂ •‹൫‫ ܑܟ‬ǡ ‫ ܏ܑ܎܍‬൯ห
7KLVZRUGDFWVDVDSUR[\IRUWKHVHQWHQFHDVDZKROH7KDWLV6 ZPD[
6XP XS WKH YHFWRU UHSUHVHQWDWLRQV RI HDFK ZRUG LQ WKH VHQWHQFH $V ZDV GLVFXVVHG LQ
6HFWLRQ WKH HDVLHVW FRPSRVLWLRQ PHWKRG LV VLPSO\ WR XVH YHFWRU DGGLWLRQ ZKLFK KDV
SURYHQHIIHFWLYHLQSDVWZRUNDQGKDVVRPHWKHRUHWLFDOVXSSRUW0LNRORYHWDOE7KLV
SURFHVVHVVHQWLDOO\PDNHVDVLQJOHYHFWRUWKDWLQFRUSRUDWHVHOHPHQWVIURP HYHU\ZRUGZLWKLQ
WKH VHQWHQFH WKXV WUDYHUVLQJ WKH VHQWHQFH¶V HQWLUH VHPDQWLF VSDFH ,Q SUDFWLFH WKH LGLRP¶V
YHFWRU ZLOO DOLJQ ZLWK VRPH RI WKHVH HOHPHQWV LQFUHDVLQJ WKH RYHUDOO VLPLODULW\ VFRUH ZKLOH
WKRVH HOHPHQWV WKDW GR QRW DOLJQ ZLOO QRW FRQWULEXWH WR WKH VFRUH 7KH DGYDQWDJH RI WKH
DSSURDFK LV WKDW QR KHXULVWLFV QHHG EH HPSOR\HG WR GHFLGH ZKLFK SRUWLRQ RI WKH VHQWHQFH WR
LQFOXGH6RIRUWKLVPRGHO6LVWKHYHFWRUVXPRIWKHUHSUHVHQWDWLRQVRIWKHQRXQVYHUEVDQG
DGMHFWLYHVRIWKHVHQWHQFH6OHDYLQJRXWWKHZRUGVIURPWKHPXOWLZRUGH[SUHVVLRQ6LQFHWKH
LGLRPV XVHG KHUH KDYH WZR ZRUGV D YHUE DQG D QRXQ WKLV PHDQV H[FOXGLQJ WKH ZRUGV DW
SRVLWLRQVSDQGS
ே
6 σ௣ିଵ
௜ୀଵ ‫ ܑܟ‬൅ σ௜ୀ௣ାଶ ‫ ܑܟ‬
0RGHOWKHVHQWHQFHDVDJURXSRIYHFWRUVRQHIRUHDFKZRUGWRNHQLQWKHVHQWHQFH7KLV
DSSURDFKLVURXJKO\HTXLYDOHQWWRWKHRQHDERYH+RZHYHULQVWHDGRIFUHDWLQJDVLQJOHYHFWRU
WRUHSUHVHQWWKHHQWLUHVHQWHQFHZHPDNH6DJURXSRIYHFWRUVRQHYHFWRUIRUHDFKQRXQ
DGMHFWLYHRUYHUELQWKHVHQWHQFH1RHIIRUWLVPDGHWRUHPRYHGXSOLFDWHZRUGV
6 ‫ ܑܟ‬ሻIRUL WRSDQGSWR1
8VHDGHSHQGHQF\SDUVHRIWKHVHQWHQFHWRVHOHFWUHOHYDQWFXHV7KHDERYHWKUHHPHWKRGV
DUHDOOEDJRIZRUGVPRGHOVDQGDVVXFKWKH\PDNHQRXVHRIWKHVHQWHQFH¶VLQWHUQDOVWUXFWXUH
$Q\ RI WKHP FDQ EH IRROHG E\ HOHPHQWV LQ WKH VHQWHQFH WKDW KDYH QR EHDULQJ RQ WKH
LQWHUSUHWDWLRQ RI WKH H[SUHVVLRQ LQ IDFW WKH ORQJHU WKH VHQWHQFH WKH PRUH OLNHO\ VXFK HUURUV
EHFRPH2QHZD\WR DYRLGWKHVHGLVWUDFWLRQVZRXOGEHWRUHVWULFWDWWHQWLRQWRWKHZRUGVWKDW
DUHQHDUHUWKHH[SUHVVLRQEXWDEHWWHUPHWKRGZRXOGEHWRILQGWKRVHHOHPHQWVWKDWKDYHVRPH
GHSHQGHQF\ UHODWLRQ WR WKH H[SUHVVLRQ LQ TXHVWLRQ $ GHSHQGHQF\ SDUVH EUHDNV WKH VHQWHQFH
LQWRLWVFRQVWLWXHQWVDQGGHVFULEHVWKHUHODWLRQVKLSEHWZHHQWKHPDQRXQSKUDVHPLJKWEHWKH
VXEMHFWRIDVXERUGLQDWHFODXVHZKLFKLWVHOILVWKHGLUHFWREMHFWRIDYHUESKUDVHDQGVRRQ
+HUH ZH DUH LQWHUHVWHG LQ WKRVH FRQVWLWXHQWV WKDW DUH HLWKHU GHSHQGHQW RQ RXU PXOWLZRUG
H[SUHVVLRQRURQZKLFKWKHH[SUHVVLRQLVGHSHQGHQW7KHQRXQVDGMHFWLYHVDQGYHUEVRIZKLFK
WKHVHFRQVWLWXHQWVDUHFRPSRVHGWKHQEHFRPHRXUEDJRIZRUGVZKLOHWKHRWKHUZRUGVLQWKH
VHQWHQFHDUHLJQRUHG:HWKHQVXPWKHVLPLODULW\VFRUHVRIWKHWZRLQWHUSUHWDWLRQVRULGLRPVWR
HDFKRIWKHZRUGVLQWKLVVXEVHW:HFDQWKLQNRIWKLVPHWKRGDVDPRUHVHOHFWLYHYHUVLRQRI
0HWKRGDERYH
7KH GHSHQGHQF\ SDUVH ZDV GRQH XVLQJ WKH 6WDQIRUG 3DUVHU &KHQ 0DQQLQJ )RU HDFK FDQGLGDWH FM ZH REWDLQ WKH GHSHQGHQF\ SDUVH RI WKH VHQWHQFH 6 WKDW LQFOXGHV WKH
PXOWLZRUG H[SUHVVLRQ RI LQWHUHVW ZS WR ZSP 8VLQJ WKDW SDUVH ZH FKRRVH D VXEVHW RI 6
FRPSULVHG RI WKH QRXQV YHUEV DQG DGMHFWLYHV WKDW DUH HLWKHU GHSHQGHQW RQ WKH WDUJHW SKUDVH
EXW H[FOXGH WKH ZRUGV RI WKH SKUDVH LWVHOI /HW WKLV VXEVHW RI 6 EH ; :H WKHQ DVVHPEOH WKH
YHFWRUUHSUHVHQWDWLRQVRIHDFKHOHPHQWRI;
6 ‫ ܑܟ‬ሻIRUZL;
7$6.6
$VH[SODLQHGLQWKH,QWURGXFWLRQZHKDYHFKRVHQDVHULHVRIWDVNVWKDWFRYHUDOOWKUHHFRUH
1/3 WDVNV *ULIILWKV 6WH\YHUV DQG 7HQHQEDXP JLVW H[WUDFWLRQ GLVDPELJXDWLRQ DQG
SUHGLFWLRQ1RQHRIWKHVHWDVNVLVSRVVLEOHXQOHVVWKHPRGHOSHUIRUPLQJWKHPKDVFDSWXUHGVRPH
DVSHFWRIWKHPHDQLQJRIWKHWH[W6XFFHVVLQDOOWKUHHWDVNVLVDQLQGLFDWLRQWKDWWKHUHSUHVHQWDWLRQ
VKRXOGJHQHUDOL]HDFURVVRWKHU1/3WDVNV)RUWKDWUHDVRQZHKDYHFKRVHQWKUHHFODVVHVRIWHVWV
RQHFRUUHVSRQGLQJWRHDFKRIWKH³FRUHWDVNV´7KHWKUHHFODVVHVDUHDOVRLQVSLUHGE\RWKHU1/3
UHVHDUFKDQGLQGHHGODQJXDJHWHVWVDGPLQLVWHUHGLQPDQ\VFKRROV
:HKDYHWZR³JLVWH[WUDFWLRQ´WDVNV7KHILUVWLVWRWDNHWKHFRVLQHVLPLODULW\VFRUHEHWZHHQ
WKH UHSUHVHQWDWLRQ RI DQ LGLRP DQG WKDW RI D RQHZRUG OLWHUDO SDUDSKUDVH $V H[SODLQHG LQ
6HFWLRQ WKH WUDLQLQJ REMHFWLYH PDNHV YHFWRUV RI VHPDQWLFDOO\ UHODWHG ZRUGV REWDLQ D
KLJKHU FRVLQH VLPLODULW\ VFRUH DQG XQUHODWHG ZRUGV D ORZHU VFRUH 7KH RWKHU WDVN LQYROYHV
XVLQJWKHUHSUHVHQWDWLRQWRVHOHFWIRUHDFKLGLRPWKHEHVWRQHZRUGSDUDSKUDVHRXWRIDOLVWRI
SRVVLEOH SDUDSKUDVHV :H ZLOO FRPSDUH WKH UHVXOWV REWDLQHG E\ RXU GLVWULEXWLRQDO
UHSUHVHQWDWLRQV WR WKRVH RI GLVWULEXWLRQDO UHSUHVHQWDWLRQV WUDLQHG XVLQJ WKH VWDQGDUG PHWKRG
LHZLWKRXWVSHFLDOWDJJLQJ
7KHGLVDPELJXDWLRQWDVNLVDFODVVLILFDWLRQWDVNZKHUHDSKUDVHLVFODVVLILHGDVEHLQJOLWHUDORU
ILJXUDWLYHJLYHQWKHVXUURXQGLQJVHQWHQFH7KLVWDVNLVFRYHUHGLQWKHOLWHUDWXUHUHYLHZHGLQ
6HFWLRQ:HVKDOOFRPSDUHVRPHRIWKH VFRUHVREWDLQHGLQ WZR RIWKRVHSDSHUVWR WKRVH
REWDLQHGE\PRGHOVHPSOR\LQJRXUUHSUHVHQWDWLRQV
7KHSUHGLFWLRQWDVNWDNHVWKHIRUPRIDPXOWLSOHFKRLFHWHVW+HUHWKHPRGHOLVSUHVHQWHGZLWK
DVHQWHQFHDQGIRXULGLRPVDQGPXVWFKRRVHZKLFKLGLRPLVWKHEHVWILWIRUWKHVHQWHQFH7KH
WDVNLVVLPLODUWRWKHVHQWHQFHFRPSOHWLRQWDVNVHWE\=ZHLJDQG%XUJHVDQGRQZKLFK
0LNRORY DQG FROOHDJXHV WHVWHG WKHLU ZRUG HPEHGGLQJV 0LNRORY HW DO D +HUH DJDLQ
ZHSLWRXUGLVWULEXWLRQDOUHSUHVHQWDWLRQVDJDLQVWWKRVHWUDLQHGXVLQJWKHVWDQGDUGPHWKRGDV
ZHOODVD1DwYH%D\HVFODVVLILHU
&RQVLGHUDEOHGHWDLORIWKHVHWDVNVDQGWKHDOJRULWKPVEHKLQGWKHPZLOOEHSURYLGHGLQHDFKRIWKH
IROORZLQJVHFWLRQV(DFKDOJRULWKPGHVFULSWLRQZLOOEHIROORZHGE\WKHHPSLULFDOUHVXOWVDFKLHYHG
RQWKDWWDVN7KHVHUHVXOWVLQFOXGHDFFXUDF\DQGZKHUHSHUWLQHQWSUHFLVLRQUHFDOODQG)VFRUHV
5HDGHUVXQIDPLOLDUZLWKWKHODWWHUWKUHHVFRUHVDUHUHIHUUHGWR6HFWLRQ
$VVHVVLQJ9DULDQWVRI,GLRPDWLF([SUHVVLRQV
:KLOHWKHIRFXVRIWKLVZRUNLVRQLGLRPVLQWKHLU³FDQRQLFDO´IRUPZHZDQWDVROXWLRQWKDW
FDQ DOVR KDQGOH YDULDWLRQV RI GLIIHUHQW NLQGV 6HFWLRQ GLVFXVVHG WKH NLQGV RI YDULDWLRQV RQ
ZKLFK ZH ZLOO IRFXV :H QRZ VSHFLI\ IXUWKHU KRZ ZH ZRXOG ZDQW YHFWRU UHSUHVHQWDWLRQV RI
ZRUGVWRSHUIRUPRQWKLVDVSHFW
$VLPSOHYDULDWLRQLVUHQGHULQJWKHLGLRPLQWRWKHSDVVLYHYRLFH³7KHWDEOHVDUHWXUQHG´LVWKH
UHDOO\ WKH VDPH LGLRP DV ³WXUQ WKH WDEOHV´ :H ZRXOG OLNH WKH VDPH YHFWRU UHSUHVHQWDWLRQV
OHDUQHGIRUWKHIRUPHUIRUPXODWLRQWRZRUNIRUWKHODWWHUDVZHOO
:HZRXOGDOVRZDQWWKHUHSUHVHQWDWLRQVWREHHIIHFWLYHLQDSKUDVHZKHUHRQHRIWKHLGLRP¶V
ZRUGV KDV EHHQ FKDQJHG IRU DQRWKHU ZLWKRXW FKDQJLQJ WKH PHDQLQJ :KHWKHU D WH[W VD\V
³ZHDUWKHSDQWV´RU³ZHDUWKHWURXVHUV´ZLOOODUJHO\GHSHQGRQZKHWKHULWZDVZULWWHQLQWKH
8QLWHG .LQJGRP RU LQ 1RUWK $PHULFD ,GHDOO\ WKH PRGHO ZRXOG QRW KDYH WR EH WUDLQHG RQ
ERWK
$Q LGLRP¶V YHUE FDQ DSSHDU E\ LWVHOI ZKLOH PDLQWDLQLQJ LWV LGLRPDWLF VHQVH WR VD\ WKDW
VRPHRQH³NLFNHG´LVDVKRUWIRUPRIVD\LQJWKDWWKHSHUVRQ³NLFNHGWKHEXFNHW´,QDVLPLODU
YHLQ D FRQVWLWXHQW RI DQ LGLRP FDQ KDYH WKH VDPH LGLRPDWLF PHDQLQJ LQ PDQ\ FRQWH[WV
³+HDUW´ LV D IDYRXULWH PHWDSKRU IRU URPDQWLF HPRWLRQV ,I ZH FUHDWH D UHSUHVHQWDWLRQ IRU
³EUHDN VRPHRQH¶V KHDUW´ ZH ZRXOG DJDLQ OLNH WKH OHDUQHG UHSUHVHQWDWLRQ RI ³KHDUW´ WR EH
H[WHQGDEOHWRRWKHUFRQWH[WVLQZKLFK³KHDUW´KDVWKHVDPHILJXUDWLYHPHDQLQJ
$Q LGLRPDWLF FRQVWLWXHQW FDQ VRPHWLPHV EH FRPELQHG ZLWK D QHZ ZRUG WR IRUP D QHZ
PHWDSKRUWKH³IODPHV´LQ³LJQLWHWKHIODPHV´FDQKDYHWKHVDPHPHWDSKRULFDOPHDQLQJDVLQ
³IDQ WKH IODPHV´ ,Q RWKHU FDVHV DQ LGLRP LV IRXQGHG RQ WKH VDPH FRQFHSWXDO PHWDSKRU DV
DQRWKHUDVLQWKH³VSLOO´LQ³VSLOORQH¶VJXWV´:HZRXOGOLNHWKHPRGHOWREHDEOHWRH[WHQG
PHWDSKRULFDOVHQVHVRIZRUGVWRQHZFRQWH[WVDJDLQZLWKQRRUPLQLPDOUHWUDLQLQJ
6&25,1*0($685(6
2XUXVXDOSHUIRUPDQFHPHWULFIRUWKHWKUHHWDVNVZLOOEHWKHPRGHO¶VDFFXUDF\WKDWLVWKH
QXPEHURIFRUUHFWDQVZHUVRXWRIWKHQXPEHURIDQVZHUVJLYHQ)RUVRPHWDVNVZHDOVRUHSRUW
SUHFLVLRQ UHFDOO DQG ) VFRUHV 7KHVH VFRUHV PHDVXUH KRZ ZHOO D FODVVLILHU KDV UHFRJQL]HG
LQVWDQFHVRIDJLYHQFODVV7KHVHPHDVXUHVFDQEHPRUHWHOOLQJZKHQDWDUJHWFODVVLVOHVVIUHTXHQW
WKDQWKHGHIDXOWFODVVDVLQGHHGLVWKHFDVHIRUILJXUDWLYHXVHVRIVRPHSKUDVHV,QWKHIRUPXODH
EHORZZHXVHWKHIROORZLQJQRWDWLRQ
x
73WUXHSRVLWLYHVLVWKHQXPEHURIWRNHQVFRUUHFWO\ODEHOOHGDVEHORQJLQJWRWKHWDUJHWFODVV
HJODEHOOHGDVILJXUDWLYHDQGDUHLQGHHGILJXUDWLYH
x )3IDOVHSRVLWLYHVLVWKHQXPEHURIWRNHQVZKLFKZHUHODEHOOHGDVEHORQJLQJWRWKHWDUJHW
FODVVEXWZKLFKGRQRWEHORQJHJODEHOOHGDVILJXUDWLYHEXWDUHOLWHUDO
x )1IDOVHQHJDWLYHVLVWKHQXPEHURIWRNHQVODEHOOHGDVQRWEHORQJLQJWRWKHWDUJHWFODVVEXW
ZKLFKDUHLQGHHGPHPEHUVHJODEHOOHGDVOLWHUDOEXWDUHILJXUDWLYH
7KH SUHFLVLRQ VFRUH VRPHWLPHV FDOOHG WKH VSHFLILFLW\ PHDVXUHV WKH SURSRUWLRQ RI WRNHQV
WKDWWKHFODVVLILHUKDVODEHOOHGDVEHORQJLQJWRWKHWDUJHWFODVVDQGWKDWGRLQGHHGEHORQJWRWKDW
FODVV,QDFULPLQDOMXVWLFHFRQWH[WZHPLJKWZDQWWRNQRZZKDWSURSRUWLRQRISHRSOHFRQYLFWHG
்௉
RIDFULPHDUHDFWXDOO\JXLOW\7KHVFRUHLVFDOFXODWHGXVLQJWKHIRUPXOD‫ ݊݋݅ݏ݅ܿ݁ݎ݌‬ൌ ்௉ାி௉
7KH UHFDOO VFRUH VRPHWLPHV FDOOHG WKH VHQVLWLYLW\ PHDVXUHV WKH SURSRUWLRQ RI DOO WRNHQV
EHORQJLQJ WR WKH WDUJHW FODVV WKDW WKH FODVVLILHU IRXQG )RU H[DPSOH ZKDW SURSRUWLRQ RI DOO
்௉
FULPLQDOVKDYHEHHQFRQYLFWHG",WLVFDOFXODWHGDV‫ ݈݈ܽܿ݁ݎ‬ൌ ்௉ାிே
2QH ZD\ WR LQFUHDVH SUHFLVLRQ LV WR DSSO\ WKH SRVLWLYH ODEHO PRUH FRQVHUYDWLYHO\ IRU
H[DPSOH E\ UHTXLULQJ RYHUZKHOPLQJ HYLGHQFH %XW VXFK D SROLF\ ZRXOG GHFUHDVH UHFDOO
&RQYHUVHO\RQHFDQDSSO\WKHODEHOZLWKOHVVHYLGHQFHLQFUHDVLQJUHFDOOEXWUHGXFLQJSUHFLVLRQ
7KH ) VFRUH DWWHPSWV WR HYDOXDWH WKH EDODQFH EHWZHHQ SUHFLVLRQ DQG UHFDOO ,W LV LQ IDFW WKH
KDUPRQLFPHDQRIWKHWZR,WLVFDOFXODWHGDV‫ ͳܨ‬ൌ ʹ
ሺ௣௥௘௖௜௦௜௢௡ሻሺ௥௘௖௔௟௟ሻ
௣௥௘௖௜௦௜௢௡ା௥௘௖௔௟௟
5 System Requirements and Processing Time
7KURXJKRXW WKH SUHVHQWDWLRQ WR IROORZ ZH VKDOO UHIHU WR ³EDWFKHV´ RI YHFWRUV 7KLV WHUP
GHVLJQDWHV RQH VHW RI YHFWRUV WUDLQHG DW WKH VDPH WLPH ZLWK D JLYHQ VHW RI WUDLQLQJ SDUDPHWHUV
(DFK EDWFK FRQWDLQV RQH YHFWRU UHSUHVHQWDWLRQ RI HDFK ZRUG LQ WKH FRUSXV¶V YRFDEXODU\ (YHQ
ZKHQ WUDLQHG XVLQJ H[DFWO\ WKH VDPH WUDLQLQJ FRUSXV DQG SDUDPHWHUV WZR EDWFKHV FDQ SHUIRUP
TXLWH GLIIHUHQWO\ IURP RQH DQRWKHU VLQFH HDFK EDWFK LV JHQHUDWHG E\ D QHXUDO QHWZRUN
DSSURDFKLQJDORFDOPLQLPXPGXULQJLWVWUDLQLQJ)RUWKDWUHDVRQZHJHQHUDWHGDVPDQ\DVVL[
EDWFKHVZLWKWKHVDPHSDUDPHWHUVDVZHOODVH[SORULQJDVPXFKRIWKHSDUDPHWHUVSDFHDV ZDV
IHDVLEOH ,Q DOO ZH JHQHUDWHG EDWFKHV XVLQJ WKH EDVHOLQH WUDLQLQJ FRUSXV XVLQJ WKH
DXWRPDWLFDOO\DQQRWDWHGFRUSXV6HFWLRQ0HWKRGDQGXVLQJWKHFRUSXVRQZKLFK
LGLRPVZHUHPDQXDOO\DQQRWDWHG0HWKRG
75$,1,1*7,0(
:HWUDLQHGDOOEDWFKHVRQDVLQJOHFRUSXVRIDERXWPLOOLRQZRUGVDQGLJQRUHGDQ\ZRUG
WKDWDSSHDUHGIHZHUWKDQWLPHV7KDWJDYHDYRFDEXODU\RIURXJKO\XQLTXHZRUGV:H
XVHGVNLSJUDPWKURXJKRXW7KHWUDLQLQJZDVGRQHRQDFRPSXWHUZLWKWKUHH,QWHO;HRQ(
FKLSVUXQQLQJDW*+]8VLQJWKHIROORZLQJSDUDPHWHUVDEDWFKZDVJHQHUDWHGDIWHUDERXW
PLQXWHV
x WKUHDGV
x GLPHQVLRQV
x ZLQGRZVL]H x QHJDWLYHVDPSOLQJ x QRKLHUDUFKLFDOVRIWPD[
x WUDLQLQJLWHUDWLRQV
7KH WLPH QHHGHG WR WUDLQ D VLQJOH EDWFK RI YHFWRUV YDULHG EHWZHHQ DURXQG PLQXWHV WR
RYHU WZHOYH KRXUV GHSHQGLQJ RQ WKH WUDLQLQJ SDUDPHWHUV :H REVHUYH WKH IROORZLQJ UXOHV RI
WKXPEIRUGLIIHUHQWSDUDPHWHUV
x /RZHULQJWKHVDPSOLQJWKUHVKROGWRHRUHKDVUHODWLYHO\OLWWOHHIIHFWRQWUDLQLQJ
WLPH/RZHULQJLWWRHVWDUWVWRPDNHDPRUHQRWLFHDEOHGLIIHUHQFHUHGXFLQJWKHWRWDO
WUDLQLQJWLPHE\DERXWWKUHHTXDUWHUV
x 'RXEOLQJWKHQXPEHURILWHUDWLRQVIURPWRGRXEOHVWKHWUDLQLQJWLPHDVPLJKWEH
H[SHFWHG
x ,QFUHDVLQJWKHZLQGRZVL]HKDVDURXJKO\SURSRUWLRQDOHIIHFWRQWKHWUDLQLQJVL]H)RU
H[DPSOHLQFUHDVLQJLWIURPWRQHDUO\GRXEOHVWKHWUDLQLQJWLPH
x 5HGXFLQJWKHQXPEHURIGLPHQVLRQVIURPWRFXWVWUDLQLQJWLPHURXJKO\LQKDOI
,QFUHDVLQJIURPWRLQFUHDVHVWUDLQLQJWLPHVE\DERXWWZRWKLUGV
x 8VLQJKLHUDUFKLFDOVRIWPD[VORZVGRZQWUDLQLQJE\DIDFWRURIDERXW
0(025<5(48,5(0(176
)RU D YRFDEXODU\ RI WKLV VL]H URXJKO\ ZRUGV D FRPSOHWH EDWFK RI GLPHQVLRQDO
YHFWRUVUHTXLUHGDERXW0%RIGLVNVSDFHLIVWRUHGDVWH[WDQGDERXW0%ZKHQORDGHG
LQWROLYHPHPRU\5$0+DOYLQJWKHQXPEHURIGLPHQVLRQVRIWKHYHFWRUVURXJKO\KDOYHVWKH
GLVN DQG PHPRU\ UHTXLUHPHQWV GRXEOLQJ WKH GLPHQVLRQDOLW\ GRXEOHV ERWK /RDGLQJ D EDWFK RI
YHFWRUVLQWRPHPRU\WDNHVURXJKO\VHFRQGV
6 The Paraphrasing Tasks
2QFHZHKDYHDSSOLHGWKHPHWKRGVRXWOLQHGLQ&KDSWHUDQGWKHUHSUHVHQWDWLRQVKDYHEHHQ
JHQHUDWHGZHILUVWPHDVXUHWKHLUXVHIXOQHVVE\DVVHVVLQJZKHWKHURUQRWWKH\KDYHFDSWXUHGWKH
JLVWRIWKHLGLRP7KLVDVVHVVPHQWLVGRQHE\FRPSDULQJWKHLGLRPV¶UHSUHVHQWDWLRQVWRWKRVHRI
RWKHU H[SUHVVLRQV ZLWK VLPLODU RU UHODWHG PHDQLQJV :H FDUULHG RXW WZR NLQGV RI FRPSDULVRQV
GHWDLOV DUH SURYLGHG LQ 6HFWLRQ EHORZ 6LQFH SDUW RI RXU REMHFWLYH ZDV WR FUHDWH
UHSUHVHQWDWLRQV WKDW FRXOG DOVR EH XVHG IRU LGLRPV LQ DQ DEEUHYLDWHG RU PRGLILHG IRUP ZH
SHUIRUPWKHVDPHWZRWDVNVRQDVHFRQGGDWDVHWPDGHVSHFLILFDOO\IRUVXFKYDULDQWV
:H DVVHVV QRW RQO\ WKH UHSUHVHQWDWLRQV WKHPVHOYHV EXW DOVR WKH HIILFDF\ RI WKH WUDLQLQJ
SDUDPHWHUV XVHGWR JHQHUDWHWKHP7KHSDUDPHWHUVOLVWHGDQG H[SODLQHGLQ 6HFWLRQ LQFOXGH
WKH GLPHQVLRQDOLW\ RI WKH YHFWRUV WKH ZLQGRZ VL]H WKH VDPSOLQJ WKUHVKROGV WKH RSWLPL]DWLRQ
PHWKRG DQG WKH QXPEHU RI WUDLQLQJ LWHUDWLRQV :H KDYH H[SORUHG D QXPEHU RI FRPELQDWLRQV RI
WKHVH SDUDPHWHUV LQ RXU LQYHVWLJDWLRQV 0XFK RI ZKDW IROORZV ZLOO GRFXPHQW WKH HIIHFW RI WKH
DERYHSDUDPHWHUV
7$6.'(6&5,37,21
$VZHKDYHVHHQ6HFWLRQ(TXDWLRQWKHFRVLQHVLPLODULW\VFRUHDOORZVXVWRFRPSDUH
WZR YHFWRU UHSUHVHQWDWLRQV RI ZRUGV 6R LI ZH ZDQW WR WHVW D UHSUHVHQWDWLRQ¶V TXDOLW\ WKH PRVW
GLUHFW WHVW ZRXOG EH WR FRPSDUH WKH LGLRP¶V UHSUHVHQWDWLRQ WKDW WKDW RI D OLWHUDO HTXLYDOHQW 2I
FRXUVHWKHPHWKRGDVVXPHVWKDWDVLQJOHZRUGSDUDSKUDVHH[LVWV7KHLGLRP³VKRRWWKHEUHH]H´
IRUH[DPSOHLVVXPPDUL]HGZHOOE\WKHYHUE³FKDW´$VKDVEHHQDUJXHGQRWDOOLGLRPVDUHHDVLO\
HQFDSVXODWHGLQDVLQJOHZRUG
:H REWDLQHG VRPH SDUDSKUDVHV IURP D SV\FKRORJLFDO H[SHULPHQW *LOEHUW 7LWRQH LQ
SURJUHVVWKHIDFWWKDWSHRSOHKDYHKDGWRUHODWHWKHVHSDUDSKUDVHVWRWKHLGLRPVLQDWDVNZRXOG
KRSHIXOO\ JLYH WKH SDUDSKUDVHV DQ H[WHUQDO VRXUFH RI YDOLGDWLRQ +RZHYHU RQO\ D GR]HQ VXFK
SDUDSKUDVHV ZHUH DYDLODEOH DQG RXU OLVW RI LGLRPV LV FRQVLGHUDEO\ PRUH H[WHQVLYH 2WKHU
SDUDSKUDVHV ZHUH WDNHQ E\ FKRRVLQJ D NH\ZRUG IURP D GHILQLWLRQ LQ WKH &DPEULGJH ,GLRPV
'LFWLRQDU\ &DPEULGJH 8QLYHUVLW\ ,GLRPV WKDW DUH OHVV HDVLO\ VXPPDUL]HG LQ D VLQJOH
ZRUGOLNH³VKRRWWKHPHVVHQJHU´FDQDWOHDVWEHFRPSDUHGWRVRPHFRQFHSWWKDWLVFHQWUDOWRWKH
LGLRPVD\³EODPH´RU³VFDSHJRDW´
2QFHWKHSDUDSKUDVHVZHUHHVWDEOLVKHGLWZDVDVLPSOHPDWWHUWRILQGWKHFRVLQHVLPLODULW\
VFRUHV7KHUHVXOWVZHUHDKHOSIXOJXLGHEXWQRWDOZD\VHDV\WRLQWHUSUHW:KDWLVWKHWKUHVKROG
IRUVXFFHVVRUIDLOXUH",VDVLPLODULW\VFRUHRI JRRGRUEDG",IWKHVFRUHZDVOHVVWKDQZKDW
PLJKWKDYHEHHQKRSHGGRHVWKHGLIILFXOW\OLHZLWKWKHUHSUHVHQWDWLRQRUZLWKDSDUDSKUDVHWKDW
IDLOVWRFDSWXUHWKHHVVHQFHRIWKHLGLRP"$QGHYHQLIZHREWDLQDKLJKVLPLODULW\VFRUHEHWZHHQ
WKHLGLRPDQGLWVSURSRVHGSDUDSKUDVHKRZFDQZHEHVXUHWKDWWKHLGLRP¶VUHSUHVHQWDWLRQLVQRW
DOVRVLPLODUWRDQXQUHODWHGZRUG"
:HDGGUHVVHGWKHVHGLIILFXOWLHVE\VXSSOHPHQWLQJWKLVILUVWWDVNZLWKDQRWKHULQZKLFKWKH
PRGHOLVJLYHQDOLVWRISRVVLEOHSDUDSKUDVHVDQGKDVWRFKRRVHWKHRQHWKDWLVFORVHVWLQPHDQLQJ
'HILQLQJWKHWDVNLQWKLVZD\PHDQVWKDWZHQHHGQRWILQGSHUIHFWSDUDSKUDVHVIRUDQLGLRPEXW
RQO\ZRUGVWKDWDUHVXIILFLHQWO\UHODWHGWRLWIRUDPRGHOWRIDYRXULWRYHURWKHUFKRLFHV:HDOVR
GHWHFWKRZVSHFLILFDQLGLRP¶VUHSUHVHQWDWLRQLVLQWKDWZHHYDOXDWHQRWRQO\ZKHWKHUWKHPRGHO
ZLOOSLFNDUHODWHGZRUGEXWDOVRZKHWKHULWFDQEHGLVWUDFWHGE\DQXQUHODWHGZRUG7KLVWHVWLV
OLNHWKH³PDWFKXS´WHVWVRIWHQXVHGLQ VFKRROV ZKHUHLQ DVWXGHQWLV SUHVHQWHGZLWK DVHULHV RI
WHFKQLFDOZRUGVRQRQHVLGHDQGWKHLUGHILQLWLRQVLQUDQGRPRUGHURQWKHRWKHUDQGWKH VWXGHQW
PXVWPDWFKWKHDSSURSULDWHFXHVWRJHWKHU
:H FUHDWHG IRXU VHWV RI WZHOYH LGLRPV DQG WZHOYH ZRUGV &UHDWLQJ IRXU VHSDUDWH VHWV
DOORZHGLGLRPVZLWKSRWHQWLDOO\VLPLODURUUHODWHGPHDQLQJVHJ³KLWWKHURRI´³PDGHDVFHQH´
DQG ³ORVW KLV FRRO´ WR EH VHJUHJDWHG LQWR VHSDUDWH WHVWV DQG VR QRW FDXVH SRWHQWLDOO\ QRLV\
FRQIXVLRQ :LWKLQ HDFK VHW WKH SDUDSKUDVHV DUH VFUDPEOHG UDQGRPO\ 7KH ILQDO VFRUH LV WKH
QXPEHURIFRUUHFWSDLULQJVWKDWWKHPRGHOPDNHVRXWRIWKHSDLUV
$VLPLODUEXWVPDOOHUWHVWVHWZDVXVHGWRVHHLIWKHPRGHOFDQSHUIRUPWKHPDWFKLQJWDVNRQ
YDULDQWVRILGLRPV:HGHYLVHGDVHWRIYDULDWLRQV)LYHRIWKHVHLQYROYHVXEVWLWXWLQJRQHZRUG
IRU DQRWKHU HJ ³VSLOO JXWV´ LQVWHDG RI ³EHDQV´ 7KH RWKHU ILYH DUH PHDQW WR WHVW WKDW D VLQJOH
ZRUG FDQ EH XVHG RQ LWV RZQ DQG PDLQWDLQ LWV LGLRPDWLF VHQVH )RU H[DPSOH ZH ZDQW WKH
ILJXUDWLYH ³EDOO´ WR PHDQ ³IXQ´ HYHQ LI ³KDYH D´ GRHV QRW SUHFHGH LW ,Q HYHU\ RWKHU ZD\ WKH
SURFHGXUHXVHGIRUWKLVGDWDVHWLVOLNHWKHRQHIRUWKHIRXUVHWVGHVFULEHGDERYH
&20387$7,21$/0(7+2'
,Q WKH ILUVW WDVN ZH VLPSO\ WDNH WKH LGLRP¶V UHSUHVHQWDWLRQ DQG WKDW RI LWV SURSRVHG
SDUDSKUDVH DQG RXWSXW WKH FRVLQH VLPLODULW\ RI WKH WZR )RU WKH PDWFKLQJ WDVN WKH PRGHO
FRPSXWHVWKHGLVWDQFHEHWZHHQHDFKLGLRP DQGHDFKSDUDSKUDVHWKHGLVWDQFHEHWZHHQ$DQG%
EHLQJVLP$%7KLVGLVWDQFHLVWKH³FRVW´RISDLULQJDQLGLRPZLWKLWVSDUDSKUDVH,WWKHQXVHV
WKH .XKQ0XQNUHV DOJRULWKP 0XQNUHV LPSOHPHQWHG E\ &ODSSHU WR FUHDWH D
ELSDUWLWHPDWFKLQJWKDWPLQLPL]HVWKHRYHUDOOFRVWWKDWLVWKHWRWDOGLVWDQFHVDFURVVDOOSDLUVRI
LGLRPVDQGSDUDSKUDVHV
(03,5,&$/5(68/76
'LUHFW&RPSDULVRQVRI,GLRPVWR3DUDSKUDVHV
7KH FRVLQH VLPLODULW\ VFRUH LV D QXPEHU EHWZHHQ QR VLPLODULW\ DQG KLJKO\
VLPLODUV\QRQ\PRXV7KLVUDQJHPD\JLYHWKHLPSUHVVLRQWKDWDVFRUHRILVH[DFWO\KDOIZD\
EHWZHHQQRUHODWLRQDQGFRPSOHWHLQWHUFKDQJHDELOLW\SHUKDSVRIWZRZRUGVEHDULQJDWDQJHQWLDO
UHODWLRQ LQ YHU\ VSHFLILF FRQWH[WV ,Q SUDFWLFH YHFWRUV DUH QRW VSUHDG RXW XQLIRUPO\ DFURVV WKH
YLUWXDOVSDFH)LJXUHVKRZVWKHYHU\VNHZHGGLVWULEXWLRQRIVLPLODULW\VFRUHV:HREWDLQHGWKLV
GLVWULEXWLRQ XVLQJ D EDWFK RI YHFWRUV RI OHQJWK :H WRRN HDFK RQH RI RXU LGLRPV DQG
FDOFXODWLQJLWVVLPLODULW\WRHYHU\RWKHUYHFWRULQWKHFRPSOHWHZRUGYRFDEXODU\\LHOGLQJ
RYHU PLOOLRQ VFRUHV )XOO\ RI WKHVH VFRUHV DUH EHORZ 7KH SURSRUWLRQ RI YHFWRUV
DFKLHYLQJ KLJKHU VFRUHV IDOOV RII H[SRQHQWLDOO\ DV WKH VFRUH LQFUHDVHV 2QO\ RI WKH HQWLUH
VSDFHDFKLHYHGDVLPLODULW\VFRUHRIRUDERYHDQGWKHSURSRUWLRQFRQWLQXHVWRGLPLQLVKIURP
WKHUH)LJXUHEJLYHVDGHWDLORIWKHGLVWULEXWLRQRIVFRUHVEHWZHHQDQGWRLOOXVWUDWHKRZ
TXLFNO\WKHSURSRUWLRQVGHFUHDVH7KLVGLVWULEXWLRQPHDQVWKDWDVFRUHRIRUPRUHLVDFWXDOO\D
ILUVWSHUFHQWLOHVFRUHDQGHYHQVPDOOGLIIHUHQFHVLQVFRUHVDERYHFDQUHSUHVHQWDVXEVWDQWLDO
GLIIHUHQFHLQSUR[LPLW\
percentage of all vectors
percentage of all vectors
25%
20%
15%
10%
5%
0%
0
0.2
0.4
0.6
cosine similarity
0.8
1
0.35%
0.30%
0.25%
0.20%
0.15%
0.10%
0.05%
0.00%
0.50
0.60
0.70
cosine similarity
)LJXUH'LVWULEXWLRQRIYHFWRUVE\FRVLQHVLPLODULW\VFRUH7KHJUDSKRQWKHOHIWKDQGVKRZV
WKHGLVWULEXWLRQRIVFRUHVIRUDOOYHFWRUVLQDJLYHQEDWFKRIHQWU\YHFWRUVZKHQFRPSDUHGWR
WKHLGLRPVXVHGLQWKLVWDVN7KHGLVWULEXWLRQLVKLJKO\VNHZHG%\IDUWKHODUJHVWPDVVLV
EHWZHHQDQGDIWHUZKLFKVFRUHVIDOORIISUHFLSLWRXVO\7KHULJKWKDQGJUDSKJLYHVDGHWDLO
RIWKRVHYHFWRUVWKDWDFKLHYHGVFRUHVEHWZHHQDQG(YHQZLWKLQWKLVQDUURZUDQJHWKH
VFRUHVGHFOLQHH[SRQHQWLDOO\7KLVGLVWULEXWLRQVKRZVWKDWLIWKHVLPLODULW\VFRUHRIWZRYHFWRUVLV
RUKLJKHUWKDWVFRUHLVZLWKLQWKHWRSRIDOOVLPLODULW\VFRUHVIRUYHFWRUVDFURVVWKHHQWLUH
VSDFH6FRUHVDERYHDUHLQWKHWRSSHUFHQWLOH
:LWKWKLVLQPLQGZHSUHVHQWWKHVLPLODULW\VFRUHVIRUWKHLGLRPVDQGWKHLUSDUDSKUDVHV
7KHUHDUH WZREDVHOLQHV7KHILUVW LV DVHW RISXEOLFO\ DYDLODEOHYHFWRUVWUDLQHGRQDELOOLRQ
ZRUGFRUSXV*RRJOHQG(DFKYHFWRUKDVHQWULHV7KHVHFRQGEDVHOLQHLVRQHWUDLQHGRQ
WKHVDPHFRUSXVDVRXUWDUJHWPHWKRGH[FHSWWKDWWKHFRUSXV¶VZRUGVDUHOHPPDWL]HGDQGWDJJHG
E\WKHLUSDUWRIVSHHFKRQO\7RPDNHDIDLUFRPSDULVRQZHXVHGYHFWRUVRIHQWULHVIRUWKLV
EDVHOLQHEDWFKDVZHOODQGVKRZWKHEDWFKZKRVHDYHUDJHFRVLQHVLPLODULW\VFRUHLVWKHKLJKHVW
:HFRPSDUHWKHVHWZREDVHOLQHVWRWZREDWFKHVXVLQJRXUWDUJHWPHWKRGV7KHILUVWZDVWUDLQHG
RQ D FRUSXV ZKHUH SRWHQWLDO LGLRPV ZHUH WDJJHG DXWRPDWLFDOO\ DQG LQ ZKLFK WKHUH DUH VHSDUDWH
YHFWRUVIRUWKHFRPSRQHQWZRUGVRIWKHLGLRP7KHVHFRQGZDVWUDLQHGRQDFRUSXVZKHUHRI
WKHLGLRPVZHUHWDJJHGPDQXDOO\
7DEOHEHORZVKRZVWKHFRVLQHVLPLODULW\VFRUHVIRUWKHLGLRPYHFWRUDQGWKHSDUDSKUDVH
)LJXUHSURYLGHVDER[SORWRIWKHVDPHUHVXOWV
7KH WUDLQLQJ RI WKHVH YHFWRUV XVHG WKH IROORZLQJ SDUDPHWHUV ZLQGRZ VL]H VDPSOLQJ WKUHVKROG IRU IUHTXHQW
ZRUGV HLWHUDWLRQVDQGQHJDWLYHVDPSOLQJ +HUHWKHSDUDPHWHUVZHUHZLQGRZVL]H VDPSOLQJWKUHVKROG HLWHUDWLRQVDQGQHJDWLYHVDPSOLQJ Baseline
Google
Baseline
POS tagged
Method 1:
Auto-tagged
endure
0.2436
0.2711
0.3666
Method 2:
Some
manual tags
0.3386
bite the dust
defeat
0.0846
0.1548
0.2388
0.2349
bite one’s tongue
silence (n.)
0.1311
0.3079
0.3159
0.3678
blow a fuse
angry
0.1508
0.2507
0.3880
0.4743
blow the whistle
alert (v.)
0.1723
0.3214
0.3988
0.3568
break someone’s heart
hurt (v.)
0.2699
0.4542
0.4982
0.4962
break the ice
initiate (v.)
0.1101
0.0779
0.1454
0.1768
bury the hatchet
reconciliation
0.2109
0.2061
0.3433
0.3769
buy the farm
die
0.1879
0.3101
0.4482
0.4669
call the shots
command (v.)
0.1703
0.2565
0.3205
0.3347
cover one’s tracks
conceal
0.3033
0.2550
0.4201
0.4070
drop a bundle
expensive
0.1358
0.2050
0.2295
0.2233
drop a line
mail (v.)
0.1366
0.2033
0.1363
0.1501
face the music
punishment
0.1897
0.1641
0.2642
0.3222
fan the flames
provoke
0.1943
0.2362
0.5026
0.5288
flood the market
glut (v.)
0.3658
0.5673
0.5163
0.5443
get the ax
layoff (v.)
0.2588
0.2468
0.4062
0.4513
have a ball
enjoy
0.2336
0.3355
0.5083
0.4336
hit the books
study (v.)
0.1038
0.3024
0.2296
0.2868
hit the hay
sleep (v.)
0.1737
0.3979
0.4799
0.4516
hit the road
depart
0.1011
0.2007
0.3126
0.3062
hit the roof
rage (v.)
0.1199
0.2082
0.2394
0.2262
hit a wall
stall (v.)
0.1210
0.2217
0.3594
0.4052
kick the bucket
die
0.1603
0.3003
0.4587
0.3662
kick the habit
quit
0.2772
0.4825
0.5934
0.5396
know the score
understand
0.5217
0.4619
0.3658
0.3090
lose one’s marbles
crazy
0.1709
0.2549
0.6091
0.5815
lose one’s nerve
cower
0.2186
0.2412
0.4774
0.4111
lose one’s shirt
bankrupt (v.)
0.2311
0.2660
0.3385
0.2984
make a bundle
profit (n.)
0.0632
0.1821
0.3627
0.3798
make a scene
tantrum
0.0337
0.1840
0.3893
0.3981
miss the boat
fail
0.2787
0.3306
0.3467
0.3152
pack one’s bags
leave (v.)
0.1857
0.1918
0.3007
0.3788
pass the buck
renege
0.2242
0.1904
0.3105
0.3299
play ball
cooperate
0.1500
0.1433
0.2160
0.2814
pull someone’s leg
fool (v.)
0.1933
0.2619
0.3302
0.3171
Idiom
Paraphrase
bear one’s cross
Baseline
Google
Baseline
POS tagged
Method 1:
Auto-tagged
cancel
0.3415
0.2391
0.3469
Method 2:
Some
manual tags
0.3558
ring a bell
familiar
0.0841
0.3147
0.4195
0.4166
shoot the breeze
chat (v.)
0.2053
0.3275
0.5054
0.5210
shoot the messenger
blame (v.)
0.1915
0.2749
0.4143
0.4232
smell a rat
suspect (v.)
0.2082
0.3059
0.4471
0.4762
spill the beans
divulge
0.0730
0.1308
0.4904
0.4537
straddle the fence
vacillate
0.2499
0.1938
0.3432
0.3729
take heart
encourage
0.2443
0.2106
0.2715
0.2988
tie the knot
marry
0.3965
0.2544
0.6375
0.6501
trim one’s sails
economize
0.3241
0.2736
0.2335
0.2916
turn the tables
outwit
0.1442
0.1383
0.4239
0.4488
wear the pants
parent (v.)
0.0317
0.1155
0.3389
0.4267
AVERAGES:
0.1952
0.2588
0.3758
0.3834
MEDIANS:
0.1888
0.2525
0.3642
0.3778
1
4
19
21
Idiom
Paraphrase
pull the plug
# in first percentile:
7DEOH6LPLODULW\VFRUHVIRULGLRPVDQGWKHLUSDUDSKUDVHVXVLQJWZREDVHOLQHDQGWKH
WDUJHWWUDLQLQJPHWKRGV6FRUHVYDU\EHWZHHQDQGZLWKLPSO\LQJSHUIHFWV\QRQ\P\
6FRUHVDERYHDUHLQWKHWRSSHUFHQWLOHWKHQXPEHURILGLRPVRYHUWKDWWKUHVKROGLVDOVRJLYHQ
6FRUHVIRUWKHWZRWDUJHWPHWKRGVDUHJHQHUDOO\PXFKEHWWHUWKDQWKHEDVHOLQH(QWULHVDUHLQ
LWDOLFVIRUWKHLGLRPVWKDWZHUHWDJJHGPDQXDOO\LQ0HWKRG
)LJXUH&RPSDULVRQRIFRVLQHVLPLODULW\VFRUHVIRUIRXUEDWFKHVRIYHFWRUV9HFWRUVWUDLQHG
RQWKH*RRJOHDQGEDVHOLQHFRUSRUDDFKLHYHVLJQLILFDQWO\ORZHUDYHUDJHVWKDQGRWKRVHWUDLQHG
XVLQJWKHWDUJHWPHWKRGV7KHGLIIHUHQFHEHWZHHQWUDLQLQJRQDQDXWRPDWLFDOO\WDJJHGYHUVXVD
SDUWLDOO\PDQXDOO\WDJJHGFRUSXVLVOHVVDSSDUHQW
:HILUVWQRWHWKDWWKHSXEOLFO\DYDLODEOH*RRJOHYHFWRUVKDYHDORZHUDYHUDJHVFRUHWKDQGR
WKRVH WUDLQHG XVLQJ WKH EDVHOLQH PHWKRG DQG RQO\ RQH VFRUH LV LQ WKH WRS SHUFHQWLOH 7KH
GLIIHUHQFHEHWZHHQWKHWZRPHWKRGVLVVWDWLVWLFDOO\VLJQLILFDQW) S 7KLVEHWWHU
SHUIRUPDQFHLQGLFDWHVWKDWXVLQJWKHILYHKXQGUHGPLOOLRQZRUG&2&$FRUSXVIRUWUDLQLQJLVQRW
D GLVDGYDQWDJH IRU WKLV WDVN HYHQ WKRXJK WKH FRUSXV LV WZHQW\ WLPHV VPDOOHU WKDQ WKH RQH RQ
ZKLFK WKH *RRJOH YHFWRUV ZHUH WUDLQHG 7KH VHFRQG DQG PRUH LPSRUWDQW UHVXOW LV WKDW WKH
SURSRVHG PHWKRGV DFKLHYHG VLJQLILFDQWO\ EHWWHU DYHUDJH VFRUHV WKDQ GLG WKH EDVHOLQH ) S 2YHUDOOWKHPHGLDQVFRUHVLQGLFDWHWKDWPRVWYHFWRUVDUHZLWKLQWKHWRSRIWKHLU
SDUDSKUDVHV¶ UHVSHFWLYH QHLJKERXUV ,Q VRPH FDVHV OLNH ³VSLOO WKH EHDQV´ DQG ³WXUQ WKH WDEOHV´
WKHGLIIHUHQFHEHWZHHQWKHWDUJHWPHWKRG¶VUHSUHVHQWDWLRQDQGWKDWRIWKHEDVHOLQHLVVXEVWDQWLDO
7KDWVDLGLIVOLJKWO\RYHUKDOIRIWKHYHFWRUVDUHZLWKLQWKHWRSWKHQWKHUHPDLQLQJKDOI
DUH QRW 7KHUH DUH FDVHV OLNH ³EUHDN VRPHRQH¶V KHDUW´ ZKHUH WKH GLIIHUHQFHV EHWZHHQ WKH
PHWKRGVDUHTXLWHVPDOODQGRWKHUVHJ³IORRGWKHPDUNHW´DQG³NQRZWKHVFRUH´ZKHUHWKH
EDVHOLQH PHWKRG DFWXDOO\ SHUIRUPV EHWWHU WKDQ WKH SURSRVHG RQHV 1HLWKHU PHWKRG DFKLHYHV
SDUWLFXODUO\LPSUHVVLYHUHVXOWVIRUVRPHLGLRPVOLNH³KLWWKHURRI´RU³WDNHKHDUW´7KHUHFDQEH
VHYHUDOUHDVRQVIRUWKHVHORZVFRUHV7KHREYLRXVSRVVLELOLW\LVWKDWWKHUHSUHVHQWDWLRQPLJKWQRW
EHDJRRGRQHHLWKHUEHFDXVHWKHSURSRVHGPHWKRGVDUHLPSHUIHFWRUEHFDXVHWKHWUDLQLQJFRUSXV
GRHV QRW FRQWDLQ H[DPSOHV WKDW EULQJ RXW WKH LGLRP¶V PHDQLQJ $QRWKHU SRVVLELOLW\ DV VWDWHG
SUHYLRXVO\LVWKDWQRWDOORIWKHSDUDSKUDVHVDUHSDUWLFXODUO\DSWZHVKDOOUHYLVLWWKLVSRVVLELOLW\LQ
WKHQH[WVXEVHFWLRQ
$WKLUGH[SODQDWLRQLVWKDWWKHFRPSRVLWLRQPHWKRGPLJKWEHDWIDXOW,QWKHDERYHFDVHVZH
XVHG VLPSOH YHFWRU DGGLWLRQ /HW XV QRZ FRQVLGHU WKUHH RWKHU PHWKRGV HOHPHQWZLVH
PXOWLSOLFDWLRQZKLFK0LWFKHOODQG/DSDWDIRXQGWREHWKHPRVWHIIHFWLYHPHWKRGIRUVRPH
YHFWRUV WKH 3UHGLFDWLRQ $OJRULWKP XVLQJ WKH SDUDPHWHUV VXJJHVWHG E\ .LQWVFK DQG WKH
3UHGLFDWLRQ $OJRULWKP XVLQJ Q DQG N WKH SDUDPHWHUV WKDW ZH IRXQG PRVW HIIHFWLYH LQ
LQLWLDO WHVWV :H VKRZ HDFK EDWFK¶V DYHUDJH FRVLQH VFRUHV IRU D KDOIGR]HQ UDQGRPO\ FKRVHQ
EDWFKHVRIGLPHQVLRQDOYHFWRUV
%DWFK
9HF
9HF[
0.0603
3UHGLFDWLRQ
Q N 0.3545
3UHGLFDWLRQ
Q N 0.3599
1
0.3597
2
0.3540
0.0568
0.3439
0.3548
3
0.3567
0.0576
0.3570
0.3571
%DWFK
9HF
9HF[
0.0571
3UHGLFDWLRQ
Q N 0.3004
3UHGLFDWLRQ
Q N 0.2955
4
0.2731
5
0.2776
0.0477
0.3120
0.3005
6
0.3570
0.0508
0.3540
0.3584
7DEOH&RPSDULVRQRIFRPSRVLWLRQPHWKRGVIRUVL[UDQGRPO\VHOHFWHGEDWFKHVRI
GLPHQVLRQDOYHFWRUV³9HF´PHDQVVLPSOHYHFWRUDGGLWLRQDQG³YHF[´LVHOHPHQWZLVH
PXOWLSOLFDWLRQ9HFWRUPXOWLSOLFDWLRQFOHDUO\JLYHVYHU\ORZFRVLQHVLPLODULW\+RZHYHULQVRPH
FDVHVWKH3UHGLFDWLRQ$OJRULWKPDSSHDUVWRLPSURYHWKHDYHUDJHVFRUH
&OHDUO\ HOHPHQWZLVH PXOWLSOLFDWLRQ GRHV QRW SURGXFH XVDEOH UHVXOWV ,Q FRQWUDVW WKH DYHUDJH
FRVLQHVLPLODULWLHVIRUWKHYHFWRUDGGLWLRQDQGWKUHHSUHGLFDWLRQPHWKRGVDUHDERXWWKHVDPH,IZH
WDNHUDQGRPO\FKRVHQEDWFKHVRIGLPHQVLRQDOYHFWRUVWKHDYHUDJHVLPLODULW\VFRUHVDUHDV
IROORZV
0HWKRG
$YHUDJHFRVLQH
9HFWRUDGGLWLRQ
3UHGLFDWLRQQ N 3UHGLFDWLRQQ N 7DEOH$YHUDJHVFRUHVRIGLIIHUHQWSKUDVDOFRPSRVLWLRQPHWKRGVDFURVVUDQGRPO\
FKRVHQEDWFKHVRIGLPHQVLRQDOYHFWRUV7KH3UHGLFDWLRQ$OJRULWKPVHHPVWRJLYHEHWWHU
DYHUDJHVFRUHVEXWWKHGLIIHUHQFHLVQRWVWDWLVWLFDOO\VLJQLILFDQW
7KH DYHUDJH VFRUH ZKHQ XVLQJ 3UHGLFDWLRQ LV KLJKHU 7KDW VDLG WKH PHWKRGV PD\ JLYH YHU\
GLIIHUHQW UHVXOWV RQ SDUWLFXODU H[DPSOHV 3UHGLFDWLRQ GRHV PXFK EHWWHU RQ ³KLW WKH URRI´ DQG
³IORRG WKH PDUNHW´ ZKLOH VLPSOH DGGLWLRQ GRHV PXFK EHWWHU RQ ³WLH WKH NQRW´ 2YHUDOO WKH
$129$RIDYHUDJHVFRUHVLQGLFDWHVWKDWWKHWKUHHPHWKRGVDUHVWDWLVWLFDOO\HTXLYDOHQW) S 0HDQZKLOH SURFHVVLQJ WLPHV IRU WKH DOJRULWKPV DUH YDVWO\ GLIIHUHQW :KHQ YHFWRU
DGGLWLRQLVXVHGDOOFRPSDULVRQVDUHGRQHLQ VHFRQGV3UHGLFDWLRQDGGVVHFRQGVRI
RYHUKHDG DW WKH EHJLQQLQJ DQG WKHQ UHTXLUHV VHFRQGV WR SURFHVV WKH FRPSDULVRQV 7KLV
PHDQV WKDW 3UHGLFDWLRQ $OJRULWKP VORZV WKH SURFHVV GRZQ E\ D IDFWRU RI RYHU IRU QR
VWDWLVWLFDOO\VLJQLILFDQWEHQHILW6RIRUWKLVWDVNDWOHDVWWKHYHFWRUDGGLWLRQPHWKRGRIFRPSRVLQJ
YHFWRUVZLOOFRQWLQXHWREHXVHG
/HWXVQRZWXUQWRWKHTXHVWLRQRIWKHWZRDQQRWDWLRQPHWKRGVWKDWLVZKHWKHULWLVKHOSIXO
WR WDJ WKH WUDLQLQJ FRUSXV PDQXDOO\ RU ZKHWKHU DXWRPDWHG WDJJLQJ ZLOO VXIILFH $V PLJKW EH
H[SHFWHGWKHYHFWRUVWUDLQHGRQWKHPDQXDOO\DQQRWDWHGFRUSXVKDYHDKLJKHUDYHUDJHVLPLODULW\
VFRUHWKDQWKHRQHVWUDLQHGRQWKHFRUSXVZKHUHDOOLQVWDQFHVDUHWDJJHGLQWKHVDPHZD\,IZH
ORRN DW LQGLYLGXDOLGLRPV ³ZHDUWKHSDQWV´KDV EHQHILWWHGWKHPRVWDIDFW ZKLFKLV QRGRXEWD
UHIOHFWLRQ RI WKH SURSRUWLRQ RI OLWHUDO WR ILJXUDWLYH WRNHQV WR %\ WUDLQLQJ RQ D FRUSXV
ZKHUH WKH OLWHUDO XVHV ZHUH QRW WKH PLVODEHOOHG DV ILJXUDWLYH WKH UHSUHVHQWDWLRQV IRU ILJXUDWLYH
³ZHDU´ DQG ³SDQWV´ ZRXOG EHWWHU UHIOHFW FRQWH[WV LQ ZKLFK ³ZHDU WKH SDQWV´ ZDV XVHG
LGLRPDWLFDOO\ 6WLOOWKHGLIIHUHQFH DFURVVWKH H[SUHVVLRQVWKDWZHUHDQQRWDWHG LV TXLWHVPDOO
DURXQGRQDYHUDJH6HYHUDOLGLRPVLQFOXGLQJ³EX\WKHIDUP´³ORVHRQH¶VVKLUW´DQG³WLHWKH
NQRW´ KDG SHUIHFWO\ JRRG FRVLQH VLPLODULW\ VFRUHV HYHQ ZLWKRXW WKH PDQXDO DQQRWDWLRQV
VXJJHVWLQJWKDWWKHLQVWDQFHVZKHUHWKHVHSKUDVHVZHUHXVHGOLWHUDOO\GLGQRWSUHYHQWWKHPRGHO
IURP PDNLQJ D XVHIXO UHSUHVHQWDWLRQ RI WKH ILJXUDWLYH VHQVH ,Q IDFW LI ZH FRPSDUH WKH FRVLQH
VFRUHVDFURVVVHYHUDOEDWFKHVRIWKHGLPHQVLRQDOLW\WKHGLIIHUHQFHEHWZHHQWKHWZRWUDLQLQJ
PHWKRGV LV QRW VWDWLVWLFDOO\ VLJQLILFDQW ) S :H VKDOO FRQVLGHU ODWHU WKH
SHUIRUPDQFHGLIIHUHQFHVIRUWKHVHWZRPHWKRGVLQWKHRWKHUWDVNV
7KH0DWFKLQJ7DVN
7KHSUHYLRXVWDVNFRQVLGHUVDEDWFKRIUHSUHVHQWDWLRQVWREHJRRGLIWKHDYHUDJHVLPLODULW\
EHWZHHQ WKH LGLRPV¶ UHSUHVHQWDWLRQV DQG WKHLU SURSRVHG SDUDSKUDVHV DUH KLJK 7KLV HYDOXDWLRQ
PHWKRGLVIODZHGLQDIHZUHVSHFWV2QHLVWKDWEDWFKHVZKRVHYHFWRUVDUHDOOVLPLODUWRRQHRWKHU
ZLOOREWDLQDKLJKDYHUDJH9HFWRUVRIORZHUGLPHQVLRQVWHQGWRKDYHKLJKHUVLPLODULW\VFRUHVEXW
WKLV KLJKHU DYHUDJH GRHV QRW QHFHVVDULO\ PHDQ WKDW ORZHUGLPHQVLRQDO YHFWRUV DUH PRUH XVHIXO
0RUHRYHUPHDVXULQJWKHDYHUDJHVLPLODULW\VFRUHGRHVQRWHVWDEOLVKZKHWKHUWKHLGLRP¶VYHFWRUV
DUHFORVHUWRWKHLUSDUDSKUDVHVWKDQWRRWKHUZRUGV$QGDVKDVEHHQVDLGZHFDQQRWDOZD\VEH
FHUWDLQ WKDW D SDUDSKUDVH LV DV DSW DV LW PLJKW EH 7KH PDWFKLQJ WDVN DGGUHVVHV WKHVH LVVXHV E\
DVNLQJDVOLJKWO\GLIIHUHQWTXHVWLRQLQVWHDGRITXDQWLI\LQJDVSHFLILFVHWRIVLPLODULW\VFRUHVWKLV
WDVN DVNV ZKHWKHU WKH UHSUHVHQWDWLRQV DUH VSHFLILF HQRXJK IRU WKH PRGHO WR IDYRXU D UHOHYDQW
FRQFHSWRYHUDQXPEHURIOHVVUHOHYDQWRQHV%DWFKHVDUHFRQVLGHUHGEHWWHUWRWKHH[WHQWWKDWWKH\
DOORZWKHPRGHOWRVHOHFWWKHEHVWSDUDSKUDVHRXWRIDVHWRIFDQGLGDWHV7KLVFKDQJHDOVRUHPRYHV
WKHEXUGHQRIILQGLQJSHUIHFWRQHZRUGSDUDSKUDVHVIRULGLRPVIRUZKLFKWKHUH RIWHQDUHQRQH
:KDW LV LPSRUWDQW LV WKDW WKH PRGHO FKRVHV D FRQFHSW WKDW LV UHODWHG WR WKH LGLRP HYHQ LI WKDW
FRQFHSWGRHVQRWFDSWXUHDOORIWKHLGLRP¶VPHDQLQJ
$JDLQ ZH FRQWUDVW WKH EDVHOLQH PHWKRG ZLWK QR SUHSURFHVVLQJ DJDLQVW WKH WZR WDUJHW
PHWKRGV)LJXUHFRQWUDVWVWKHUDQJHVRIVFRUHVDFKLHYHGE\WKHGLIIHUHQWEDWFKHVLQGXFHGE\WKH
WZR PHWKRGV 7KH RWKHU ILJXUHV EHORZ VKRZ WKH HIIHFWV RI WKUHH WUDLQLQJ SDUDPHWHUV RQ
SHUIRUPDQFHRQO\WKHSDUDPHWHUVZLWKDVWDWLVWLFDOO\VLJQLILFDQWLPSDFWDUHVKRZQ
)LJXUH &RPSDULVRQ RI WKUHH WUDLQLQJ
PHWKRGV 1RWH WKDW WKH EHVW DFFXUDF\ VFRUH
DFKLHYHG E\ WKH EDVHOLQH PHWKRG LV
OHVVWKDQWKHORZHVWVFRUHREWDLQHGE\HLWKHU
RIWKHRWKHUV
)LJXUH(IIHFWRIWKHRSWLPL]DWLRQ
PHWKRGRQSHUIRUPDQFH&OHDUO\WKH
DYHUDJHVFRUHIRUQHJDWLYHVDPSOLQJLV
KLJKHUWKDQWKDWRIXVLQJKLHUDUFKLFDO
VRIWPD[
)LJXUH 6FRUHV E\ GLPHQVLRQDOLW\ IRU WKH
DXWRWDJJLQJ PHWKRG¶V YHFWRUV *RRG VFRUHV
ZHUH DFKLHYHG E\ DOO GLPHQVLRQV RI RU
DERYH+LJKHUGLPHQVLRQVVHHPWROHDGWRPRUH
H[WUHPHVFRUHVERWKKLJKDQGORZ
)LJXUH(IIHFWRIVDPSOLQJWKUHVKROGVRQ
SHUIRUPDQFH$YDOXHRIHGRHVEHVWDOWKRXJK
ERWKKLJKDYHUDJHDQGORZVFRUHVVHHPWREH
GLVWULEXWHGDFURVVDEURDGUDQJHRIYDOXHV
9HFWRUVWUDLQHGXVLQJWKHEDVHOLQHPHWKRGDFKLHYHEHWZHHQDQGRQWKLVWDVNZLWK
WKH DYHUDJH VFRUH EHLQJ 7KH SXEOLFO\ DYDLODEOH *RRJOH 1HZV YHFWRUV REWDLQ D VFRUH RI
7KH DXWRPDWHG WDJJLQJ PHWKRG DFKLHYHV IDU EHWWHU VFRUHV LWV ORZHVW VFRUH LV KLJKHU WKDQ WKH EDVHOLQH PHWKRG¶V EHVW VFRUH 9HFWRUV XVLQJ WKH DXWRPDWHG WDJJLQJ PHWKRG
UHVXOWHG LQ DYHUDJH VFRUHV RI ZLWK RQH EDWFK DWWDLQLQJ DFFXUDF\ 7KH PDQXDO
WDJJLQJPHWKRGJHWVVOLJKWO\ORZHUVFRUHVFRPSDUHGWRWKHDXWRPDWHGPHWKRGZLWKDQDYHUDJHRI
7KH GLIIHUHQFH LV VWDWLVWLFDOO\ VLJQLILFDQW ) S 7KLV ODWWHU UHVXOW LV
VXUSULVLQJPDQXDOO\DQQRWDWLQJWKHWUDLQLQJFRUSXVKDVJRQHIURPEHLQJQRWSDUWLFXODUO\KHOSIXO
LQWKHFRVLQHVLPLODULW\FRPSDULVRQWRDFWXDOO\KXUWLQJSHUIRUPDQFHLQWKLVWDVN:HZLOOFRQVLGHU
VRPHUHDVRQVDERXWZK\WKDWPLJKWEHLQWKH&RQFOXVLRQDIWHUZHKDYHVHHQZKDWHIIHFWPDQXDO
WDJJLQJKDVRQWKHGLVDPELJXDWLRQDQGSUHGLFWLRQWDVNV
:H WXUQ QRZ WR WKH EHVW WUDLQLQJ SDUDPHWHUV WR XVH IRU WKLV WDVN +HUH LV DQ DQDO\VLV RI
YDULDQFH$129$RYHU VHYHQSDUDPHWHUV7KH DQDO\VLVVKRZQLQ7DEOH LQGLFDWHVWKDW IRXU
SDUDPHWHUV DUH VLJQLILFDQW 'LPHQVLRQDOLW\ LV RQO\ VLJQLILFDQW LQ WKDW YHFWRUV RI GLPHQVLRQV
REWDLQ ORZHU VFRUHV LI ZH ORRN RQO\ DW GLPHQVLRQDOLWLHV RI RU DERYH WKLV SDUDPHWHU LV QR
ORQJHUVLJQLILFDQW
)DFWRU
2SWLPXPYDOXHV
)VFRUH
S
'LPHQVLRQDOLW\
RUPRUH
:LQGRZVL]H
2SWLPL]DWLRQPHWKRG
QHJDWLYHVDPSOLQJ
6DPSOLQJWKUHVKROG
H
1XPEHURILWHUDWLRQV
1HJDWLYHVDPSOHVL]H
$OSKD
7DEOH6LJQLILFDQFHDQGRSWLPDOYDOXHRIWUDLQLQJSDUDPHWHUVIRUWKHPDWFKLQJWDVN7KH
IRXULWHPVLQEROGLWDOLFDUHVLJQLILFDQWLHS'LPHQVLRQDOLW\LVRQO\VLJQLILFDQWZKHQ
YHFWRUVRIHQWULHVDUHLQFOXGHG,IWKHVHYHFWRUVDUHH[FOXGHGGLPHQVLRQDOLW\ORVHVVLJQLILFDQFH
) S 6RIDUZHKDYHFRQVLGHUHGJOREDOVWDWLVWLFVIRUWKHPDWFKLQJWDVNDFURVVWKHHQWLUHVHWRI
LGLRPV ,WLVDOVRLQVWUXFWLYHWRVHHKRZWKHUHVXOWVYDULHGIRUVSHFLILFLGLRPV1RWVXUSULVLQJO\
WKHPRGHOFRQVLVWHQWO\JRWFRUUHFWPDWFKHVIRUWKRVHLGLRPVWKDWREWDLQHGKLJKFRVLQHVLPLODULW\
VFRUHVUHJDUGOHVVRIWKHYHFWRUXVHGIRUH[DPSOH³EORZWKHZKLVWOH´³IORRGWKHPDUNHW´³WULP
RQH¶V VDLOV´ ³ORVH RQH¶V VKLUW´ ,Q FRQWUDVW DOPRVW HYHU\ PRGHO PDGH DQ LQFRUUHFW FKRLFH IRU
³SXOO VRPHRQH¶V OHJ´ (YHQ WKH EDWFKHV WKDW OHG WR WKH KLJKHVW VFRUHV KDG GLIILFXOW\ ZLWK WKH
LGLRPV³ORVHRQH¶VQHUYH´³NLFNWKHEXFNHW´³GURSDOLQH´³EUHDNWKHLFH´DQG³EORZDIXVH´
7KHFOHDUGLIILFXOW\WKDW WKHVHWKUHHLGLRPV SRVH FRXOG KDYHVHYHUDOH[SODQDWLRQVRQHVWKDWDUH
QRWPXWXDOO\H[FOXVLYH2QHSRVVLELOLW\DOUHDG\GLVFXVVHGLVWKDWWKHJLYHQSDUDSKUDVHPD\QRW
EH VXIILFLHQWO\ DSW $QRWKHU LV WKDW WKH WUDLQLQJ FRUSXV PD\ QRW KDYH H[DPSOHV WKDW PDNH WKH
LGLRPDWLFPHDQLQJVXIILFLHQWO\FOHDU$WKLUGLV WKDWVLQFH³EORZDIXVH´³GURSDOLQH´³EUHDN
WKH LFH´ DQG ³SXOO VRPHRQH¶V OHJ´ DUH RIWHQ XVHG LQ D OLWHUDO VHQVH WKH IDFW WKDW DOO RI WKH
RFFXUUHQFHVZHUHWDJJHGLQWKHVDPHZD\PD\KDYHPDGHLWGLIILFXOWIRUWKHPRGHOWROHDUQWKH
LGLRPDWLF VHQVH ,I WKH ODWWHU H[SODQDWLRQ ZHUH FRUUHFW ZH ZRXOG H[SHFW WKDW WUDLQLQJ RQ WKH
PDQXDOO\ DQQRWDWHG FRUSXV ZRXOG VROYH WKH SUREOHP +RZHYHU ZKHQ ZH ORRN DW WKH PRGHO¶V
SHUIRUPDQFH ZKHQ XVLQJ WKH EHVWVFRULQJ EDWFKHV RI WKDW VHFRQG PHWKRGWKH\ DOVR KDG WURXEOH
ZLWK ³EORZD IXVH´ ³GURSDOLQH´ DQG³EUHDNWKHLFH´ ³SXOOVRPHRQH¶VOHJ´ZDVQRW PDQXDOO\
DQQRWDWHGVRZHZRXOGH[SHFWQRFKDQJHKHUHDQGLQGHHGWKHUHZDVQRQH7KHODWWHUH[SODQDWLRQ
DOVRIDLOVWRDFFRXQWIRUZK\³ORVHRQH¶VQHUYH´LVDOVRVRGLIILFXOWRUZK\WKHVHSDUWLFXODULGLRPV
SRVHG D SUREOHP ZKHQ RWKHU OLWHUDOO\ SODXVLEOH H[SUHVVLRQV OLNH ³ZHDU WKH SDQWV´ RU ³PLVV WKH
ERDW´GLGQRW:HVKDOOFRPHEDFNWRWKLVTXHVWLRQZKHQZHFRQVLGHURWKHUWKHRWKHUWDVNV
:KHQFRQVLGHULQJWKHHUURUVVHHQLQWKLVWDVNLWLVLPSRUWDQWWRNHHSLQPLQGWKDWQRWDOORI
WKHPDUHGXHWRDEDGUHSUHVHQWDWLRQ$VPLJKWEHH[SHFWHGRQHLQFRUUHFWFKRLFHKDVDGRPLQR
HIIHFWWKDWOHDGVWRRWKHUPLVPDWFKHVVLQFHWKHFRUUHFWDQVZHUIRUDJLYHQLGLRPPLJKWDOUHDG\EH
WDNHQWKHPRGHOKDVQRFKRLFHEXWWRPDWFKWKHLGLRPZLWKDQLQFRUUHFWSDUDSKUDVHZKLFKWDNHV
WKHFRUUHFWDQVZHUDZD\IURPDQRWKHULGLRPDQGVRRQ7KHDOJRULWKPVHHNVWRORZHUWKHRYHUDOO
FRVW RI PLVPDWFKLQJ LH WKH ORZHVW WRWDO VXP RI FRVLQH GLVWDQFHV EXW QRW WKH WRWDO QXPEHU RI
PLVPDWFKHV 7KLV H[SODLQV ZK\ VRPH LGLRPV WKDW REWDLQHG JRRG FRVLQH VLPLODULW\ VFRUHV ZKHQ
FRPSDUHGWRWKHLUSDUDSKUDVHVOLNH³NLFNWKHEXFNHW´DUHRIWHQPLVPDWFKHGLQWKLVWDVN
,GLRP9DULDQWV
:HFRQFOXGHWKLVVHFWLRQRIUHVXOWVE\ORRNLQJDWWKHPRGHO¶VSHUIRUPDQFHRQDVPDOOJURXS
RIYDULDQWV:HFRQWUDVWSHUIRUPDQFHRIEDVHOLQHYHFWRUEDWFKHVFKRVHQDWUDQGRPZLWKWKDW
RI WKH VDPH QXPEHU RI EDWFKHV WUDLQHG XVLQJ RXU PHWKRG $V VHHQ LQ )LJXUH IRXU RI WKH WHQ
WDUJHWYHFWRUEDWFKHVJRWSHUIHFWVFRUHV(YHQWKHORZHVWSHUIRUPLQJEDWFKRIWDUJHWYHFWRUVGLG
EHWWHUWKDQWKHEHVWSHUIRUPLQJEDWFKRIEDVHOLQHYHFWRUV
)LJXUH6FDWWHUSORWRIDFFXUDF\VFRUHVIRUEDVHOLQHDQGWDUJHWYHFWRUVIRULGLRPYDULDQWV
5HVXOWVIRUUDQGRPO\FKRVHQEDWFKHVRIEDVHOLQHDQGWDUJHWYHFWRUEDWFKHVDUHVKRZQ%DVHOLQH
YHFWRUVFRUHVDUHUHGFLUFOHVWDUJHWYHFWRUVFRUHVDUHDVWHULVNV1RWHWKDWWKHEHVWVFRUHDFKLHYHG
E\DEDVHOLQHEDWFKLVORZHUWKDQWKHORZHVWVFRUHIRUDWDUJHWPHWKRGEDWFK)RXURXW
RIWHQWDUJHWYHFWRUEDWFKHVDFKLHYHGVFRUHVRI
7KHVH UHVXOWV DUH DQ HQFRXUDJLQJ VLJQ WKDW WKH SURSRVHG WUDLQLQJ PHWKRG LQGXFHV UREXVW
UHSUHVHQWDWLRQV WKDW FDQ FDSWXUH WKH PHDQLQJ RI DW OHDVW VRPH LGLRPDWLFSKUDVHV WKDW KDYH EHHQ
DEEUHYLDWHGRUDOWHUHGLQVRPHZD\,QGHHGWKLVDVSHFWLVRQHRIWKHUHDVRQVZK\WKHSURSRVHG
PHWKRG OHDUQV D VHSDUDWH UHSUHVHQWDWLRQ IRU HDFK ZRUG LQ WKH H[SUHVVLRQ DQG QRW IRU WKH
H[SUHVVLRQDVDXQLWDVGRHVWKH³VWDWLF´VROXWLRQRXWOLQHGLQ6HFWLRQ
6800$5<
,QWKLVVHFWLRQZHZHUHLQWHUHVWHGLQWKHYHFWRUUHSUHVHQWDWLRQV¶DELOLW\WRFDSWXUHWKHJLVW
RILGLRPDWLFH[SUHVVLRQV:HGLGVRE\FRPSDULQJWKHLGLRP¶VUHSUHVHQWDWLRQWRWKDWRIDVLQJOH
ZRUG WKDW HLWKHU SDUDSKUDVHV WKH LGLRP RU HQFDSVXODWHV D FRUH DVSHFW RI WKH LGLRP 7KHVH
FRPSDULVRQVVKRZWKDWPRVWRIWKHLGLRPYHFWRUVREWDLQVLPLODULW\VFRUHVWKDWDUHZLWKLQWKHWRS
ZKHQFRPSDUHGWRVLPLODULW\VFRUHVIRUDOOWKHYHFWRUVLQWKHVSDFH ,QFRQWUDVWRQO\RQHLQ
VL[YHFWRUVWUDLQHGLQWKHXVXDOZD\DUHZLWKLQWKLVUDQJH:HSXVKHGWKLVLQYHVWLJDWLRQIXUWKHUE\
VHHLQJ LI WKH PRGHO FRXOG FKRRVH WKH FORVHVW ZRUG IURP D OLVW RI SRWHQWLDO SDUDSKUDVHV +HUH
YHFWRUVWUDLQHGXVLQJRXUPHWKRGJRWVFRUHVDVKLJKDVFRPSDUHGWRWKHPD[LPXPVFRUHRI
IRU YHFWRUV WUDLQHG LQ WKH XVXDO PHWKRG )LQDOO\ ZH ZDQWHG WR VHH LI WKH UHSUHVHQWDWLRQV
ZRXOG SHUIRUPYHU\ ZHOO ZKHQXVHGLQ DWUXQFDWHGRUDOWHUHGLGLRPDWLFH[SUHVVLRQV7KHWDUJHW
PHWKRG¶V YHFWRUV GLG YHU\ ZHOO RQ WKLV WDVN VRPHWLPHV DFKLHYLQJ D SHUIHFW VFRUH ZKLOH WKH
EDVHOLQHYHFWRUVDFKLHYHGDWEHVWDVFRUHRI$OOLQDOOPRVWRIWKHUHSUHVHQWDWLRQVWUDLQHG
XVLQJ RXU SURSRVHG PHWKRGV DSSHDU ZHOOVXLWHG WR D JLVW H[WUDFWLRQ WDVN :H QRZ FRQVLGHU WKH
VHFRQGFRUH1/3IXQFWLRQGLVDPELJXDWLRQ
7 Classifying Literal vs. Figurative Tokens
$Q\ 1/3 DSSOLFDWLRQ WKDW GHDOV ZLWK WKH VHPDQWLFV RI LGLRPV PXVW GHFLGH LI D SDUWLFXODU
LQVWDQFHRIDQH[SUHVVLRQLVEHLQJXVHGOLWHUDOO\RUILJXUDWLYHO\7KLVUHTXLUHPHQWLVSDUWLFXODUO\
LPSRUWDQW IRU SKUDVHV OLNH ³KLW D ZDOO´ IRU ZKLFK ERWK OLWHUDO DQG LGLRPDWLF XVHV DUH TXLWH
FRPPRQ7KLVLVWKHGLVDPELJXDWLRQWDVN1RWRQO\LVWKHWDVNLPSRUWDQWLQSUDFWLFDODSSOLFDWLRQV
WKHH[LVWLQJOLWHUDWXUHRQWKLVWDVNUHYLHZHGLQ6HFWLRQSURYLGHVDJRRGEDVLVIRUFRPSDULQJ
RXU UHSUHVHQWDWLRQV WR RWKHUV :H XVHG WKUHH GDWDVHWV HDFK KLJKOLJKWLQJ D GLIIHUHQW VHW RI
FRQGLWLRQV RI LQWHUHVW 7KH ILUVW GDWDVHW LV FRPSRVHG RI VHQWHQFHV ZKRVH FRQWH[W PDNHV FOHDU
ZKLFKXVHLVFRUUHFW7KHVHWHVWVSHUPLWXVWRHYDOXDWHWKHSHUIRUPDQFHZLWKDPLQLPDOQXPEHURI
FRQIRXQGV7KHVHFRQGGDWDVHWLQFOXGHVWKHVDPHVHQWHQFHVIURPWKH%ULWLVK1DWLRQDO&RUSXVWKDW
ZHUH XVHG E\ )D]O\ &RRN DQG 6WHYHQVRQ DQG 3HQJ )HOGPDQ DQG 9\ORPRYD +HUH WKH REMHFWLYH LV WR DVVHVV WKH UHSUHVHQWDWLRQ¶V SHUIRUPDQFH RQ VHQWHQFHV XQGHU OHVV
FRQWUROOHG DQG PRUH UHDOLVWLF FRQGLWLRQV DQG WR FRPSDUH WKRVH UHVXOWV DJDLQVW WKRVH RI WKH
SUHYLRXV H[SHULPHQWV )LQDOO\ WZR RWKHU GDWDVHWV DOORZ XV WR DVVHVV KRZ ZHOO WKHVH
UHSUHVHQWDWLRQV FDQ EH XVHG LQ YDULDQWV RI LGLRPDWLF H[SUHVVLRQV ERWK V\QWDFWLF YDULDWLRQV DQG
LQVWDQFHV ZKHUH D ZRUG RI WKH LGLRP KDV EHHQ VXEVWLWXWHG RU UHPRYHG %HIRUH SUHVHQWLQJ WKH
UHVXOWVZHVXPPDUL]HWKHPHWKRGVWKDWDUHXVHGWRFKRRVHWKHLQWHUSUHWDWLRQ
&20387$7,21$/0(7+2'
7KH WDVN LV WR ILQG WKH PRVW OLNHO\ FODVV F IURP WKH VHW ^) /` ³ILJXUDWLYH´ DQG ³OLWHUDO´
UHVSHFWLYHO\ 7KH PRGHO ILUVW FUHDWHV D UHSUHVHQWDWLRQ 6 RI WKH VHQWHQFH DV D ZKROH PLQXV WKH
LGLRPDWLF H[SUHVVLRQ $OO IRXU PRGHOV GHVFULEHG LQ 6HFWLRQ ZHUH WULHG 7KH WDUJHW
H[SUHVVLRQ LV UHSUHVHQWHG E\ WZR YHFWRUV HILJ IRU WKH LGLRPDWLF PHDQLQJ DQG HOLW IRU WKH OLWHUDO
PHDQLQJ7KHREMHFWLYHLVWRFKRRVHWKHYHFWRUHWKDWLVWKHEHVWILWIRU6ZKLFKZHTXDQWLI\E\D
VLPLODULW\VFRUH,QSUDFWLFHZHDOVRUHTXLUHDELDVWHUPEWRHQVXUHDQRSWLPDOEDODQFHEHWZHHQ
SUHFLVLRQ DQG UHFDOO LH EHWZHHQ ODEHOOLQJ D XVDJH DV LGLRPDWLF RQO\ LI LW DFWXDOO\ LV LGLRPDWLF
SUHFLVLRQ YHUVXV ILQGLQJ DOO LGLRPDWLF XVDJHV UHFDOO VHH 6HFWLRQ IRU GHWDLOV 6R WKH
FODVVLILFDWLRQGHFLVLRQLVPDGHDVIROORZV
ܿƸ ൌ ൜
‫‹•ˆ‹ܨ‬൫‫ ܏ܑ܎܍‬ǡ ‫܁‬൯ ൅ ܾ ൐ •‹ሺ‫ ܜܑܔ܍‬ǡ ‫܁‬ሻ
‫‡•‹™”‡Š–‘ܮ‬
7KHELDVWHUPEZDVGHWHUPLQHGIURPWKHGDWD+RZLWZDVFDOFXODWHGLVVSHFLILHGLQWKHGHWDLOVRI
HDFKVXEWDVN
),*85$7,9(/,7(5$/6(17(1&(3$,56
'HVFULSWLRQRIWKH'DWDVHW
:H XVHG SDLUV RI VHQWHQFHV WKDW ZHUH FRQFHLYHG E\ WKH 7LWRQH /DQJXDJH DQG
0XOWLOLQJXDO /DERUDWRU\ IRU RQJRLQJ SV\FKRORJLFDO H[SHULPHQWV (DFK SDLU LQYROYHV WKH VDPH
SKUDVHEHLQJXVHGLQDOLWHUDODQGWKHQDILJXUDWLYHZD\)RUH[DPSOH
x -DQHGURSSHGDEXQGOHZKHQVKHWULHGWRFDUU\WRRPXFKODXQGU\XSWKHVWDLUVOLWHUDO
x -DQHGURSSHGDEXQGOHZKHQVKHVWD\HGLQDILYHVWDUKRWHOIRUKHUYDFDWLRQILJXUDWLYH
7KUHHPRUHH[DPSOHVDUHSURYLGHGLQ$SSHQGL[&7KHRULJLQDOVHQWHQFHSDLUVZHUHPDWFKHGIRU
V\QWDFWLFFRPSOH[LW\ZRUGOHQJWKDQGIUHTXHQF\FRQWUROOLQJIRUWKRVHSRWHQWLDOFRQIRXQGV7KDW
VDLG VRPHRIWKH VHQWHQFHVUHOLHGRQ DQLPSOLFDWLRQWKDW DKXPDQZLOOLPPHGLDWHO\XQGHUVWDQG
EXWWKDWDVLPSOHODQJXDJHPRGHOFRXOGQRWVRWKHVHVHQWHQFHVZHUHVRPHZKDWVLPSOLILHG
7KH PRGHO LV SUHVHQWHG ZLWK HDFK VHQWHQFH RQH DW D WLPH DQG LW PXVW GHFLGH LI WKH
H[SUHVVLRQLVXVHGLQLWVILJXUDWLYHRULWVOLWHUDOVHQVH1RWHWKDWHYHQLIWKHVHQWHQFHVDUHLQSDLUV
WKHPRGHOFKRLFHVDUHLQGHSHQGHQWRIRQHDQRWKHUWKHIDFWWKDWWKHPRGHOGHFLGHVWKDWWKHILUVW
VHQWHQFHLVXVHGOLWHUDOO\LQQRZD\LQIRUPVLWWKDWWKHVHFRQGVHQWHQFHPXVWEHILJXUDWLYH
&DOFXODWLQJWKH%LDV
)RU WKLV GDWDVHW WKH ELDV ZDV GHWHUPLQHG LQ HDUO\ WHVWLQJ E\ WDNLQJ WKH PHGLDQ RI WKH
PDUJLQ IRU PLVFODVVLILHG VHQWHQFHV WKDW LV WKH GLIIHUHQFH EHWZHHQ WKH ILJXUDWLYH DQG OLWHUDO
VFRUHV7KHVDPHELDVWHUPZDVXVHGIRUDOOH[SUHVVLRQV$PRUHQXDQFHGDSSURDFKZKHUHWKH
ELDVLVGHWHUPLQHGE\LGLRPZLOOEHSRVVLEOHZKHQWUDLQLQJGDWDLVDYDLODEOHDVLWZLOOEHLQWKH
QH[WVXEWDVN
&RPSRVLWLRQ0HWKRGV
:HUHWXUQWRWKHTXHVWLRQRIZKLFKFRPSRVLWLRQPHWKRGSURGXFHVWKHEHVWUHVXOWV$JDLQ
ZHFRPSDUHYHFWRUDGGLWLRQDQGWKHWZRVHWVRISDUDPHWHUVIRUWKH3UHGLFDWLRQ$OJRULWKP*LYHQ
LWV IDLOXUH LQ WKH SUHYLRXV WDVN ZH KHUHDIWHU DEDQGRQ HOHPHQWZLVH PXOWLSOLFDWLRQ 7KH UHVXOWV
EHORZ ZHUH REWDLQHG XVLQJ ILYH GLIIHUHQW EDWFKHV RI ZRUG HPEHGGLQJV YDU\LQJ WKH YHFWRU VL]H
DQGYHFWRUWUDLQLQJSDUDPHWHUVWRHQVXUHREWDLQLQJDJRRGFURVVVHFWLRQRIUHVXOWV
0HWKRG
9HFWRUDGGLWLRQ
3UHGLFDWLRQQ N 3UHGLFDWLRQQ N VHW
VHW
VHW
VHW
VHW
GZ
VPSOH
GZ
VPSOH
GZ
VPSOH
GZ
VPSOH
GZ
VPSOH
7DEOH(IIHFWRISKUDVHOHYHOFRPSRVLWLRQPHWKRGRQDFFXUDF\RIFODVVLILFDWLRQ
8QOLNH WKH SDUDSKUDVH WDVN LQ WKH SUHYLRXV VHFWLRQ WKHUH LV FOHDUO\ D VLJQLILFDQW GLIIHUHQFH
EHWZHHQ WKH DOJRULWKPV¶ SHUIRUPDQFHV KHUH 7KH 3UHGLFDWLRQ $OJRULWKP VXEVWDQWLDOO\
XQGHUSHUIRUPV VLPSOH YHFWRU DGGLWLRQ 7KH GLIIHUHQFH LV VWDWLVWLFDOO\ VLJQLILFDQW ) S 0HDQZKLOHWKH3UHGLFDWLRQ$OJRULWKPLVRUGHUVRIPDJQLWXGHVORZHUSURFHVVLQJWKH
VHQWHQFHV WDNHV EHWZHHQ DQG VHFRQGV ZLWK WKH 3UHGLFDWLRQ $OJRULWKP ZKLOH YHFWRU
DGGLWLRQSURFHVVHVWKHVDPHQXPEHULQEHWZHHQDQGVHFRQGV
1H[W ZH FRQVLGHU E\ ZKLFK RI WKH ILYH PHWKRGV GLVFXVVHG LQ 6HFWLRQ ZH FDQ
UHSUHVHQW WKH VHQWHQFH :H XVHG WKH VDPH ILYH EDWFKHV RI ZRUG HPEHGGLQJV DV DERYH WZR
DGGLWLRQDO EDWFKHV KDG WR EH DGGHG WR DFKLHYH VLJQLILFDQFH 7DEOH EHORZ VKRZV WKH DFFXUDF\
VFRUHVIRUWKHVHYHQEDWFKHVDVZHOODVWKHDYHUDJHWLPHUHTXLUHGWRSURFHVVWKHVHQWHQFHV
0HWKRG
.H\ZRUG
$GGLWLRQ
6XPRIVFRUHV
'HSHQGHQF\SDUVH
VHW
VHW
VHW
VHW
VHW
VHW
VHW
WLPHV
7DEOH(IIHFWRIVHQWHQFHOHYHOFRPSRVLWLRQPHWKRGRQDFFXUDF\RIFODVVLILFDWLRQDQG
DYHUDJHSURFHVVLQJWLPH7KHORQJHUWLPHIRUWKHGHSHQGHQF\SDUVHLVEHFDXVHWKHSDUVLQJVWHS
LWVHOIUHTXLUHVVHFRQGVIRUDOOVHQWHQFHV,QSUDFWLFHWKLVVWHSLVGRQHRQO\RQFHDQG
WKHQUHXVHGIRUDOOEDWFKHVWKHSURFHVVLQJWLPHLWVHOILVVHFRQGVIRUDOOVHQWHQFHV
$OWKRXJK HDFK RI WKH WKUHH RWKHU UHSUHVHQWDWLRQV RFFDVLRQDOO\ REWDLQHG EHWWHU UHVXOWV WKDQ WKH
DGGLWLRQ PHWKRG WKH ODWWHU LV WKH RQH WKDW DFKLHYHV WKH PRVW FRQVLVWHQW UHVXOWV DQG RQ DYHUDJH
SHUIRUPV ZLWK EHWWHU DFFXUDF\ 7KLV KLJKHU SHUIRUPDQFH LV VWDWLVWLFDOO\ VLJQLILFDQW ) S ,WLVDOVRWKHIDVWHVWPHWKRGWDNLQJRQO\VHFRQGVIRUWKHVHQWHQFHV
(PSLULFDO5HVXOWV
+DYLQJFKRVHQRXUWZRFRPSRVLWLRQ DOJRULWKPVZHSURFHHGWR WHVWLQJDVHULHVRIYHFWRUV
EDWFKHV ZLWK YDULRXV SDUDPHWHUV )RU WKLV WDVN ZH FDQQRW FRPSDUH SHUIRUPDQFH WR WKDW RI WKH
EDVHOLQH YHFWRUV XVHG IRU WKH PDWFKLQJ WDVN VLQFH WKRVH EDWFKHV PDNH QR GLVWLQFWLRQ EHWZHHQ
OLWHUDO DQG ILJXUDWLYH VHQVHV :H VKDOO EH DEOH WR FRPSDUH SHUIRUPDQFHV ZLWK RWKHU
UHSUHVHQWDWLRQVLQWKHQH[WVHFWLRQ)RUQRZZHVKRZWKHRYHUDOOILQGLQJVRQRXUVHQWHQFHSDLUV
DQG WKHQ H[DPLQH KRZ WUDLQLQJ SDUDPHWHUV DIIHFW DFFXUDF\ 7DEOH SUHVHQWV SHUIRUPDQFH
VWDWLVWLFVIRUYHFWRUVWUDLQHGXVLQJWKHWDUJHWPHWKRG,QDGGLWLRQWRRYHUDOODFFXUDF\LWDOVRJLYHV
WKHSUHFLVLRQUHFDOODQG)IRUWKHILJXUDWLYHH[DPSOHV2YHUDOOZHVHHDYHUDJHVFRUHVRIDURXQG
IRUWKUHHRIWKHIRXUPHDVXUHV
3UHFLVLRQ
5HFDOO
)
$FFXUDF\
/RZHVW
$YHUDJH
+LJKHVW
7DEOH3UHFLVLRQUHFDOODQGDFFXUDF\RQWKHILJXUDWLYHOLWHUDOFODVVLILFDWLRQWDVN7KH
ILJXUDWLYHFODVVLVVHHQDVWKH³SRVLWLYH´FODVVIRUWKHSUHFLVLRQUHFDOODQG)VFRUHV
7KHRSWLPDOWUDLQLQJSDUDPHWHUVWKDWFRQWULEXWHWRWKHHIIHFWLYHQHVVRIWKHYHFWRUVIRUWKLV
WDVNDUHLQVRPHFDVHVGLIIHUHQWIURPZKDWZRUNHGEHVWIRUWKHPDWFKLQJWDVNDQGLQVRPHFDVHV
WKH\DUHTXLWHFRXQWHULQWXLWLYH:HZRXOGQRUPDOO\H[SHFWYHFWRUVZLWKPRUHGLPHQVLRQVWREH
PRUHSUHFLVH\HWZHVHHKHUHWKDWSHUIRUPDQFHLVEHWWHUDVWKHQXPEHURIGLPHQVLRQVLVUHGXFHG
DOWKRXJK SHUIRUPDQFH VWDUWV WR VXIIHU EHORZ /DUJHU ZLQGRZV VKRXOG PDNH WKH PRGHO WDNH
LQWRDFFRXQWDEURDGHUFRQWH[WEXWRQWKLVWDVNWKHGHIDXOWZLQGRZVL]HRIZRUGVRQHLWKHUVLGH
GRHVPXFKEHWWHUWKDQZLQGRZVRIRUPRUH2QWKHRWKHUKDQGVRPHSDUDPHWHUVWKDWZHPLJKW
KDYH H[SHFWHG WR EH KHOSIXO DUH QRW 7KH QXPEHU RI WUDLQLQJ LWHUDWLRQV KDV QR VWDWLVWLFDOO\
VLJQLILFDQWHIIHFWSRVLWLYHRUQHJDWLYHRQSHUIRUPDQFH$QGILQDOO\XVLQJKLHUDUFKLFDOVRIWPD[
LQVWHDGRIQHJDWLYHVDPSOLQJKXUWVSHUIRUPDQFH7DEOHVXPPDUL]HVWKHILQGLQJV
)LJXUH(IIHFWVRIWUDLQLQJSDUDPHWHUVRQSHUIRUPDQFHLQWKHFODVVLILFDWLRQRIILJXUDWLYHYV
OLWHUDOXVHV7KHILUVWILJXUHVKRZVWKHHIIHFWRIWKHWUDLQLQJPHWKRGWKHRWKHUVVKRZWKHHIIHFWVRI
WKHUHPDLQLQJSDUDPHWHUVIRUWKHPDQXDOO\WDJJHGPHWKRGRQO\2YHUDOOZHVHHWKDWUHODWLYHO\ORZ
GLPHQVLRQDOLW\VPDOOZLQGRZVDQGIDLUO\ORZVDPSOLQJWKUHVKROGVRUDQG
QHJDWLYHVDPSOLQJSURGXFHWKHEHVWUHVXOWV
3DUDPHWHU
)VFRUH
S
2SWLPDO6HWWLQJ
'LPHQVLRQV
:LQGRZVL]H
QHJDWLYHVDPSOLQJ
+LHUDUFKLFDO6RIWPD[YV
1HJDWLYH6DPSOLQJ
6DPSOLQJWKUHVKROG
HRUH
$OSKD
LWHUDWLRQV
QHJDWLYHVDPSOHV
7DEOH1:D\$129$RISDUDPHWHUVIRUWKHPDQXDOWDJJLQJPHWKRGLQWKH
ILJXUDWLYHOLWHUDOFODVVLILFDWLRQWDVN,WHPVLQLWDOLFVDUHVWDWLVWLFDOO\VLJQLILFDQWS
)URP WKHVH SDUDPHWHUV LW ZRXOG DSSHDU WKDW WKH ILJXUDWLYHOLWHUDO GHFLVLRQ LV EHVW PDGH E\ D
YHFWRUWKDWHQFDSVXODWHVWKHPRVWVDOLHQWDVSHFWVRIDZRUG¶VORFDOFRQWH[WDQGLJQRUHVWKHUHVW$
ODUJHUZLQGRZPRUHGLPHQVLRQVDQGPRUHWUDLQLQJLWHUDWLRQVRQO\DGGGLVWUDFWLRQVIRUWKHWDVNDW
KDQG7KHIDFWWKDWKLHUDUFKLFDOVRIWPD[KXUWVSHUIRUPDQFHLQGLFDWHVWKDWLWVDOWHUQDWLYHQHJDWLYH
VDPSOLQJ KHOSV LH WKH PRGHO OHDUQV EHWWHU E\ KDYLQJ QHJDWLYH H[DPSOHV DV ZHOO DV SRVLWLYH
RQHV
7KH WZR EHVW SHUIRUPLQJ EDWFKHV RI YHFWRUV UHVXOWHG LQ WKH PRGHO¶V PDNLQJ VL[
PLVFODVVLILFDWLRQVRXWRIVHQWHQFHVDQGWKHQH[WEHVWEDWFKUHVXOWHGLQVHYHQHUURUV$OOWKUHH
RI WKHVH EDWFKHV ZHUH WUDLQHG RQ WKH PDQXDOO\ DQQRWDWHG FRUSXV 2I WKH EDWFKHV WUDLQHG RQ WKH
DXWRPDWLFDOO\WDJJHGFRUSXVWKHEHVWWZRUHVXOWHGLQHLJKWPLVWDNHV7KHHUURUVDFURVVWKHVHEHVW
SHUIRUPHUVDQGLQGHHG DFURVVPRVWEDWFKHV DUHTXLWHVLPLODU7KHUHLV QRFDVHZKHUHERWK WKH
OLWHUDODQGWKHILJXUDWLYHVHQWHQFHVRIWKHVDPHH[SUHVVLRQZHUHPLVFODVVLILHGDOORIWKHPLVWDNHV
ZHUH RQ RQH RI WKH VHQWHQFHV EXW QRW WKH RWKHU 0RVW RI WKH PLVWDNHV ZHUH FODVVLI\LQJ D OLWHUDO
VHQWHQFHDVILJXUDWLYHIDOVHSRVLWLYHWKHRWKHUVZHUHIDOVHQHJDWLYHV7KHLGLRPVWKDWZHUHPRVW
IUHTXHQWO\PLVFODVVLILHGDUHJLYHQEHORZ0RVWRIWKHVHHUURUVFRXOGEHDWWULEXWHGWRDOHVVWKDQ
RSWLPDOYDOXHEHLQJXVHGIRUWKHELDVWHUP7KDWVDLGLWHPVLQEROGZHUHPLVFODVVLILHGEHFDXVHRI
DVXEVWDQWLDO PDUJLQ EHWZHHQWKH ILJXUDWLYH DQG OLWHUDO VFRUHV IRUZKLFK QR ZRUNDEOHYDOXHIRU
WKHELDVWHUPZRXOGKDYHKDGDQLPSDFW,WHPVLQLWDOLFVZHUHPLVFODVVLILHGZKHQXVLQJEDWFKHV
WUDLQHGRQWKHDXWRPDWLFDOO\WDJJHGFRUSXVEXWZHUHFRUUHFWO\FODVVLILHGE\WKRVHWUDLQHGRQWKH
PDQXDOO\WDJJHGFRUSXV
)DOVHSRVLWLYHVOLWHUDOFODVVLILHGDVILJXUDWLYH )DOVHQHJDWLYHVILJXUDWLYHFODVVLILHGDVOLWHUDO
EHDUWKHFURVV
PLVVWKHPDUN
ELWHRQH¶VWRQJXH
WULPRQH¶VVDLOV
KLWWKHFHLOLQJ
EORZDIXVH
SDFNRQH¶VEDJ
WXUQWKHWDEOHV
GURSDERPE
7DEOH)DOVHSRVLWLYHVDQGQHJDWLYHVIRUWKHYHFWRUVJLYLQJWKHEHVWSHUIRUPDQFH
7XUQLQJQRZWRWKHLGLRPVZKLFKZHUHFRUUHFWO\FODVVLILHGDVLJQRIDJRRGUHSUHVHQWDWLRQ
IRU WKLV WDVN LV WKDW WKHPDUJLQ EHWZHHQ WKH ILJXUDWLYH DQG OLWHUDO VFRUHV LV ODUJH ERWK ZKHQ WKH
H[SUHVVLRQLVXVHGOLWHUDOO\DQGZKHQLWLVXVHGILJXUDWLYHO\%\WKDWFULWHULRQWKHVHLGLRPVKDYH
WKHEHVWFRQWUDVWEHWZHHQWKHLUOLWHUDODQGILJXUDWLYHXVHV
x VSLOOWKHEHDQV
x ZHDUWKHSDQWV
x
x
x
x
x
x
x
x
FRYHURQH
VWUDFNV
FOHDUWKHGHFN
OLFNRQH¶VZRXQGV
FOHDUWKHDLU
FKDQJHRQH¶VWXQH
VFUDWFKRQH¶VKHDG
GURSDEXQGOH
PDNHDEXQGOH
+HUHDUHVRPHREVHUYDWLRQVRQVRPHVSHFLILFFDVHV
x /RRNLQJ DW WKH DERYH OLVW RI HUURUV LW EHFRPHV FOHDUHU ZK\ WKH PDQXDO DQQRWDWLRQV GR QRW
KDYHDVVLJQLILFDQWDQHIIHFWRQSHUIRUPDQFHDVPLJKWKDYHDQWLFLSDWHG2IWKHH[SUHVVLRQV
WKDW ZHUH PDQXDOO\ DQQRWDWHG QLQH DUH XVHG LQ WKLV GDWDVHW 2I WKRVH QLQH WKH PRGHO
FODVVLILHVVL[FRUUHFWO\DQGPLVFODVVLILHVWZR³PLVVWKHPDUN´DQG³WULPRQH¶VVDLOV´ZKHWKHU
LWXVHVYHFWRUVWUDLQHGRQWKHDXWRPDWLFDOO\RUWKHPDQXDOO\WDJJHGFRUSXV7KDWOHDYHVWZR
LGLRPV³EORZDIXVH´DQG³GURSDERPE´IRUZKLFKWUDLQLQJRQDPDQXDOO\DQQRWDWHGFRUSXV
PDNHV D GLIIHUHQFH ERWK DUH PLVFODVVLILHG ZKHQ XVLQJ WKH DXWRPDWLFDOO\ WDJJHG FRUSXV EXW
FRUUHFWO\ FODVVLILHG ZKHQ XVLQJ WKH PDQXDO DQQRWDWLRQV ,Q RWKHU ZRUGV PDQXDO DQQRWDWLRQ
KHOSHGLPSURYHDFFXUDF\E\WZRFODVVLILFDWLRQV7KHLPSRUWDQWSRLQWKHUHLVWKDWLWLVSRVVLEOH
WRREWDLQJRRGUHSUHVHQWDWLRQVZLWKRXWPDQXDODQQRWDWLRQ
x 7KHH[SUHVVLRQV³WULPRQH¶VVDLOV´³EORZDIXVH´³EHDURQH¶VFURVV´DQG³KLWWKHFHLOLQJ´DOO
RI ZKLFK DSSHDU RQ WKH DERYH OLVW RI PLVFODVVLILFDWLRQV RFFXU UHODWLYHO\ IHZ WLPHV LQ WKH
WUDLQLQJ FRUSXV DQG WLPHV UHVSHFWLYHO\ ³'URS D ERPE´ DSSHDUV PRUH
IUHTXHQWO\²WLPHVLQDOO²EXWLVXVXDOO\XVHGOLWHUDOO\6RDSRVVLEOHH[SODQDWLRQRIWKH
HUURUV LV WKDW WKHUH PLJKW QRW EH HQRXJK WUDLQLQJ GDWD IRU WKH YHFWRUV WR FUHDWH D JRRG
UHSUHVHQWDWLRQRIWKHLGLRPDWLFPHDQLQJ
x 2QH RI WKH H[SUHVVLRQV WKDW SURYHG GLIILFXOW LQ WKH PDWFKLQJ WDVN ³EUHDN WKH LFH´ GLG QRW
SRVHDSUREOHP LQ WKLV WDVN 7KHRWKHUWZRH[SUHVVLRQVWKDWSURYHGGLIILFXOWLQ WKHSUHYLRXV
WDVN³SXOORQH¶VOHJ´DQG³ORVHRQH¶VQHUYH´ZHUHQRWSDUWRIWKLVRQH
x $Q HOHPHQW WR FRQVLGHU LV ZKHWKHU LW SRVHV D SUREOHP ZKHQ D VLQJOH UHSUHVHQWDWLRQ LV
JHQHUDWHG IRU ZRUGV WKDW DUH FRPPRQ WR GLIIHUHQW LGLRPV 7KH HYLGHQFH WKRXJK QRW
FRPSOHWHO\FDWHJRULFDOVXJJHVWVWKDWLWGRHVQRW%RWK³FOHDUWKHDLU´DQG³FOHDUWKHGHFN´DUH
DPRQJRXUOLVWRIWRSSHUIRUPHUVDVDUH³PDNHDEXQGOH´DQG³GURSDEXQGOH´:HVDZDERYH
WKDW³ELWHWKHEXOOHW´DQG³ELWHRQH¶VWRQJXH´ERWKUHVXOWHGLQHUURUVPLJKWWKRVHKDYHEHHQ
EHFDXVH RQH UHSUHVHQWDWLRQ IRU ³ELWH´ LV PDGH WR FDSWXUH YHU\ GLIIHUHQW VHQVHV" 7ZR RWKHU
SKUDVHVWKDWVKDUHDZRUGZLWKDQRWKHULGLRPZHUHFRQVLVWHQWO\PLVFODVVLILHG³KLWWKHFHLOLQJ´
DQG³PLVVWKHPDUN´6WLOOVHYHUDORWKHUSKUDVHVZLWKWKDWXVH³KLW´³WKHERRNV´³DZDOO´
³WKH URRI´ DQG ³PLVV´ ³WKH ERDW´ ZHUH FRUUHFWO\ FODVVLILHG 7KHUH DUH DOVR VL[ SKUDVHV
VKDULQJ D YHUE ZLWK DQRWKHU SKUDVH WKDW ZHUH FRQVLVWHQWO\ FODVVLILHG FRUUHFWO\ ³ORVH RQH¶V
VKLUW´ DQG ³ORVH WKH WKUHDG´ ³JHW WKH PHVVDJH´ DQG ³JHW WKH VDFN´ :H VKDOO UHWXUQ WR WKLV
TXHVWLRQZKHQORRNLQJDWWKHUHVXOWVIRUWKHVHQWHQFHFRPSOHWLRQWDVN
&203$5,6217235(9,286:25.86,1*7+(%1&
'HVFULSWLRQRIWKH'DWDVHW
&RRN )D]O\ DQG 6WHYHQVRQ DQQRWDWHG WKH %ULWLVK 1DWLRQDO &RUSXV %1& IRU
LGLRPDWLFDQGILJXUDWLYHRFFXUUHQFHVRIPDQ\9HUE1RXQSKUDVHV7KHVHDQQRWDWLRQVSURYLGHDQ
H[FHOOHQWWHVWIRURXUHPEHGGLQJVLQWKDWWKHVHQWHQFHVLQWKH%1&ZHUHQRWXVHGIRURXUWUDLQLQJ
DQG EHFDXVH WKH DQQRWDWLRQV SURYLGH D ³JURXQG WUXWK´ DJDLQVW ZKLFK WKH HPEHGGLQJV FDQ EH
WHVWHG)XUWKHUPRUHWZRSDSHUVGLVFXVVHGLQ6HFWLRQ)D]O\&RRN6WHYHQVRQ3HQJ
)HOGPDQ 9\ORPRYD XVHG WKHVH DQQRWDWLRQV IRU D WDVN H[DFWO\ OLNH WKH RQH EHLQJ
GHVFULEHGKHUH7KHVHSDSHUVSURYLGHXVZLWKUHVXOWVDJDLQVWZKLFKRXUUHVXOWVFDQEHFRPSDUHG
3HQJ)HOGPDQDQG-D]PDWLQGPLJKWDOVREHDJRRGSRLQWRIFRPSDULVRQLQWKDWOLNHXVWKH\
XVHGZRUGYHFUHSUHVHQWDWLRQVRIZRUGVDQGWKHFRVLQHVLPLODULW\WRVHQWHQFHVLQRUGHUWRFODVVLI\
DQH[SUHVVLRQDVOLWHUDO RUILJXUDWLYH+RZHYHUWKHLUGDWDVHW LV QRWSXEOLFO\ DYDLODEOHDQGWKH\
DFKLHYHGPRUHFRPSHWLWLYHUHVXOWVLQWKHLUIRUPHUVWXG\VRZHZLOOXVHWKDWRQHKHUH
:H UHSOLFDWHG DV FORVHO\ DV SRVVLEOH WKH SURFHGXUH RXWOLQHG LQ 3HQJ HW DO :H IXUWKHU
LQFOXGHQRWRQO\WKHLUIRXULGLRPVEXWDOVRDQRWKHUILYHXVHGE\)D]O\HWDO
7KHGDWDVHWZDVFUHDWHGDVIROORZV
:HVHOHFWHGWKRVHLGLRPVWKDWDSSHDUHGLQRQHRIWKHWZRDERYHVWXGLHVDQGIRUZKLFKWKHUH
DUHDWOHDVWOLWHUDODQGILJXUDWLYHXVHVUHFRUGHGLQWKH%1&DQQRWDWLRQV
:HH[WUDFWHGWKHVHQWHQFHVIURPWKH%1&WKDWZHUHDQQRWDWHGDVHLWKHULGLRPDWLFRUOLWHUDO
:HPDGHXVHRIWKHOHPPDWL]DWLRQDQGSDUWRIVSHHFKWDJVRIWKH%1&LWVHOI
:HUDQGRPO\DVVLJQHGHDFKH[DPSOHWRHLWKHUDWHVWLQJRUDWUDLQLQJVHW7RHQVXUHWKDWWKH
UHVXOWVDUHFRPSDUDEOHWR3HQJHWDO¶VZHPDGHRXUWHVWVHWVWKHVDPHVL]HDVWKRVHWKDWWKH\
XVHGLQWKHLUH[SHULPHQWV)RUWKRVHH[SUHVVLRQVXVHGRQO\E\)D]O\HWDOZHXVHGDWOHDVW
DQGDWPRVWWUDLQLQJH[DPSOHVGHSHQGLQJRQZKHWKHUWKHUHZHUHHQRXJKH[DPSOHVLQWKH
DQQRWDWLRQV 7KH WUDLQLQJ H[DPSOHV ZHUH XVHG WR ILQG WKH SDUDPHWHUV IRU WKH FODVVLILFDWLRQ
VHHQH[WVXEVHFWLRQ
&RPSXWDWLRQDO0HWKRG
+HUHDUHWKHGHWDLOVRIWKHDOJRULWKP
7KHPRGHOILUVWVFDQVWKHVHQWHQFHIRUWKHLGLRPLQTXHVWLRQ,ILWGRHVQRWILQGLWLWFODVVLILHV
WKH VHQWHQFH DV EHLQJ OLWHUDO $Q H[DPSOH RI WKLV VLWXDWLRQ LV WKH VHQWHQFH ³LW ZRXOG WDNH D
KDUGKHDUWQRWWRJHWLQYROYHG´6LQFHWKHLGLRPLV³WDNHKHDUW´WKHLGLRPGRHVQRWDFWXDOO\
DSSHDULQWKHVHQWHQFH2QHFRXOGULJKWO\REMHFWWKDWWKH³KDUGKHDUW´LQWKLVVHQWHQFHLVQR
PRUH OLWHUDO WKDQ LV ³WDNH KHDUW´ EXW WKH VHQWHQFH LV DQQRWDWHG DV ³OLWHUDO´ LQ &RRN HW DO¶V
GDWDVHW,QDQ\FDVHWKHVHFDVHVDUHFODVVLILHGZLWKRXWXVLQJWKHYHFWRUUHSUHVHQWDWLRQV
,IWKHLGLRPLVIRXQGWKHPRGHOFRQVWUXFWVDUHSUHVHQWDWLRQIRUWKHVHQWHQFHE\VXPPLQJXS
WKH YHFWRU UHSUHVHQWDWLRQV RI WKH QRXQV DGMHFWLYHV DQG YHUEV DQG WKH ZRUGV LPPHGLDWHO\
EHIRUHDQGDIWHUWKHLGLRPDVOLJKWPRGLILFDWLRQWR6HQWHQFH0RGHOLQ6HFWLRQ7KH
YHUEDQGQRXQRIWKHLGLRPDUHQRWLQFOXGHGLQWKLVUHSUHVHQWDWLRQ:HKHUHDIWHUUHIHUWRWKLV
YHFWRUUHSUHVHQWDWLRQDV6
7KHOLWHUDODQGILJXUDWLYHVFRUHVIRUWKLVVHQWHQFHDUHREWDLQHGE\WDNLQJWKHFRVLQHVLPLODULW\
RI6DQGWKHYHFWRUUHSUHVHQWDWLRQVRIWKHLGLRPDWLFDQGOLWHUDOSKUDVHV
)RUWKHWUDLQLQJH[DPSOHVWKHGLIIHUHQFHEHWZHHQWKHWZRVFRUHVDUHIHGLQWRDOLQHDU690
FODVVLILHULPSOHPHQWHGXVLQJWKHVFLNLWOHDUQOLEUDU\3HGUHJRVDHWDO7KHSXUSRVHRI
WKLVFODVVLILHULVVROHO\WRILQGWKHELDVWHUPE
)RUWKHWHVWLQJH[DPSOHVVWHSVDWRFDUHIROORZHGDQGWKHQWKHOLQHDUFODVVLILHUDERYHXVHV
WKHGLIIHUHQFHEHWZHHQWKHWZRVLPLODULW\VFRUHVWRSURGXFHLWVSUHGLFWLRQ
$JDLQIROORZLQJ3HQJHWDO¶VSURFHGXUHZHDYHUDJHGWKHUHVXOWVRYHUUXQV7KHPRGHO¶V
SUHFLVLRQ UHFDOO ) VFRUH DQG DFFXUDF\ DUH EDVHG RQ WKH PRGHO¶V DELOLW\ WR GLIIHUHQWLDWH
LGLRPDWLFIURPILJXUDWLYHXVHVLQWKHWHVWVHWVHQWHQFHV
(PSLULFDO5HVXOWV
:HEHJLQ ZLWK )D]O\ HW DO ,QIDFW WKH\SUHVHQW UHVXOWVIURP WKHLUWKUHH GLIIHUHQW PRGHOV
LQFOXGLQJDEDVHOLQHZKLFKW\SLFDOO\RXWSHUIRUPVWKHRWKHUWZRPRGHOV:KLFKHYHURQHGLGEHVW
RQDSDUWLFXODULGLRPLVWKHRQHUHSRUWHGKHUHPHDQLQJWKDWWKH)D]O\HWDOUHVXOWVEHORZFRPH
IURPDPL[RIPHWKRGV2XUUHVXOWVDUHDYHUDJHGRYHUUXQVDQGXVLQJWKHEDWFKRIYHFWRUVWKDW
SHUIRUPHGEHVWIRUWKHVHQWHQFHSDLUGDWDVHWDERYH)D]O\HWDOUHSRUWWKHSUHFLVLRQDQGUHFDOOIRU
WKHILJXUDWLYHDQGOLWHUDOFODVVZHFRPELQHWKHVHLQWRDQ)VFRUHKHUH$EUHDNGRZQLVSURYLGHG
LQ$SSHQGL['
)LJXUHFRPSDULVRQRIUHVXOWVIURP)D]O\&RRNDQG6WHYHQVRQWRWKRVHREWDLQHG
E\RXUPRGHOXVLQJWKHVDPHGDWDDYHUDJHGRYHUUXQV(DFKLGLRPKDVWZRSDLUVRIVFRUHV
WKHILUVWSDLUEHLQJWKHILJXUDWLYH))D]O\HWDODQGWKHQRXUVWKHVHFRQGEHLQJWKHOLWHUDO)
DJDLQ)D]O\HWDODQGWKHQRXUV,QHDFKFDVHWKHUHOHYDQWFRPSDULVRQLVEHWZHHQWKHEOXHEDU
)D]O\HWDODQGWKHUHGRQHWRLWVLPPHGLDWHULJKWRXUV1RWHWKDWIRU³KLWWKHURRI´³KLWWKH
URDG´DQG³JHWWKHVDFN´WKH)D]O\HWDOPRGHOREWDLQHGVFRUHVRIDQGVRWKHEOXHEDULVQRW
YLVLEOH7KLVFRPSDULVRQVKRZVWKDWIRURIWKHSDLUVRXUPRGHOREWDLQHGDKLJKHU)VFRUH
2YHUDOORXUPRGHOREWDLQHGDQ)VFRUHRIYHUVXVIRU)D]O\HWDO¶VWKUHHPRGHOV
,QPRVWFDVHVZKHUHWKHWDUJHWPRGHOSHUIRUPVOHVVZHOOWKHGLIIHUHQFHVDUHTXLWHVPDOO1RWHDV
ZHOO WKDW WKH WKUHH FRPSHWLQJ PRGHOV JHW ) VFRUHV RI ]HUR IRU WKH OLWHUDU\ XVDJHV RI ³KLW WKH
URRI´³KLWWKHURDG´DQG³JHWWKHVDFN´7KHVHVFRUHVPHDQWKDWWKHFODVVLILHUVDUHVLPSO\QDPLQJ
DOOLQVWDQFHVDVILJXUDWLYHLQRWKHUZRUGVWKHPRGHOVDUHQRWUHDOO\FODVVLI\LQJDWDOO2XUPRGHO
DYRLGVVXFKH[WUHPHVDOWKRXJKRI FRXUVHLWV) VFRUHVIRUWKHOLWHUDU\XVDJHVRI ³KLWWKHURDG´
DQG³KLWWKHURRI´DUHTXLWHORZ
7KH FRPSDULVRQ LV DOVR LQVWUXFWLYH LQ GHPRQVWUDWLQJ WKDW VRPH H[SUHVVLRQV DUH PRUH
LQKHUHQWO\GLIILFXOWWKDQRWKHUV%RWKPRGHOVKDYHVXEVWDQWLDOO\ORZHUVFRUHVIRUILJXUDWLYHXVHV
RI ³SXOO VRPHRQH¶V OHJ´ WKDQ IRU WKH RWKHU H[SUHVVLRQV 6R WKH ORZ VFRUH IRU WKLV H[SUHVVLRQ
HQFRXQWHUHGLQRWKHUWDVNVVHHPVQRWWREHDQDUWHIDFWRIWKHDSSURDFKWDNHQKHUHEXWUDWKHURIWKH
H[SUHVVLRQLWVHOI
³*HW WKH VDFN´ LV DQ H[SUHVVLRQ PRUH FRPPRQO\ XVHG LQ *UHDW %ULWDLQ WKDQ LQ 1RUWK
$PHULFDDVHYLGHQFHGE\WKHIDFWWKDWLWDSSHDUVLQWKH%1&DQQRWDWLRQVEXWRQO\VHYHQWLPHV
LQWKH&2&$WUDLQLQJFRUSXV*LYHQWKDWWKHUHZHUHIHZRFFXUUHQFHVRQZKLFKWRWUDLQWKHIDFW
WKDWWKHYHFWRUVDFKLHYHGVXFKDJRRGVFRUHLQWKLVWDVNLVTXLWHUHPDUNDEOH:HVKDOOVHHWKDWWKLV
SHUIRUPDQFHLVQRWDVJRRGLQVHQWHQFHFRPSOHWLRQWDVN
:H QRZ WXUQ WR WKH UHVXOWV REWDLQHG E\ 3HQJ HW DO 7KH\ UHSRUW SUHFLVLRQ UHFDOO DQG
DFFXUDF\IRUWZRPRGHOV$JDLQZHUHSRUWZKLFKHYHUUHVXOWZDVEHVWDQGFRPELQHWKHSUHFLVLRQ
DQG UHFDOO LQWR DQ ) VFRUH )RU WKH PRVW SDUW WKH UHVXOWV DUH YHU\ FORVH 2QO\ RQ ³EORZ WKH
ZKLVWOH´LVWKHUHDVXEVWDQWLDOGLIIHUHQFHEHWZHHQWKH RXUVDQGWKHLUV7KHQXPHULFDOUHVXOWVDUH
JLYHQLQ$SSHQGL[(
)LJXUH&RPSDULVRQRIUHVXOWVREWDLQHGE\3HQJHWDOWRWKRVHREWDLQHGE\WKH
FXUUHQWPRGHO7KHUHOHYDQWFRPSDULVRQLVEHWZHHQHDFKEOXHEDUWKHVFRUHUHSRUWHGE\3HQJHW
DODQGWKHUHGRQHWRLWVLPPHGLDWHULJKWRXUPRGHO7KHJUDSKVKRZVWKDWWKHWZRPRGHOV
SHUIRUPHGDWDOPRVWWKHVDPHOHYHOVZLWKWKHH[FHSWLRQRI³EORZWKHZKLVWOH´ZKHUHWKHWDUJHW
PRGHOKDGDVXEVWDQWLDOO\EHWWHU)VFRUH
7KHOHVVRQWRWDNHLVQRWWKDWWKHUHSUHVHQWDWLRQVSURSRVHGKHUHDUHDOOWKDWPXFKEHWWHUWKDQ
WKHVHRWKHUFODVVLILHUVZKLFKDOUHDG\SHUIRUPDWTXLWHKLJKOHYHOV,WLVLQVWHDGWKDWJHQHUDOL]HG
QRQWDVNVSHFLILF UHSUHVHQWDWLRQV FDQGRDVZHOO DVVSHFLDOL]HGPRGHOV7KHUHLV OLWWOHUHDVRQ WR
EHOLHYH WKDW WKH FRPSHWLQJ PRGHOV FRXOG EH GHSOR\HG IRU D VHQWHQFH FRPSOHWLRQ RU SDUDSKUDVH
WDVN WKH OHDUQLQJ WKH\ GR LV XVHIXO IRU WKLV WDVN DQG WKLV RQH DORQH 2XU PHWKRG¶V PRUH
JHQHUDOL]DEOHUHSUHVHQWDWLRQVGRQRWFRPHDWWKHFRVWRIORZHUHIIHFWLYHQHVV
9$5,$176
6\QWDFWLF9DULDQWV3DVVLYL]DWLRQ
:HWHVWHGLGLRPDWLFYDULDQWVXVLQJWZRVPDOOGDWDVHWV7KHILUVWGDWDVHWLQFOXGHVERWKOLWHUDO
DQG ILJXUDWLYH H[DPSOHV RI SDVVLYH FRQVWUXFWLRQV 2I WKH LGLRPV ZH XVHG LQ WKLV ZRUN WKH WZR
WKDW VHHP WKH PRVW DPHQDEOH WR SDVVLYL]DWLRQ DUH ³WXUQ WKH WDEOHV´ DQG ³EUHDN WKH LFH´ ZLWK
GLIIHUHQWHIIHFWV:KLOH³WKHWDEOHVDUHWXUQHG´LVXVXDOO\XVHGLQDILJXUDWLYHVHQVH³WKHLFHZDV
EURNHQ´ LV JHQHUDOO\DOLWHUDO XVH:HH[WUDFWHGVHQWHQFHVIURP &2&$'DYLHV
DQGWKH:LNLSHGLD&RUSXV'DYLHVFRQWDLQLQJWKHVHLGLRPVLQWKHSDVVLYHYRLFHOLWHUDO
DQGLGLRPDWLFXVHVRI³WDEOHVWXUQHG´DQGOLWHUDODQGLGLRPDWLFXVHVRI³LFHEURNHQ´ +HUH
DUHIRXUH[DPSOHV
x $OOWKHWDEOHVZHUHWXUQHGLQWRZRUNVRIDUWVZLWKZLQGRZVGRRUVSLFWXUHVDQGSDLQWOLWHUDO
x <RXSUHVXPHWRZRUU\DERXW\RXUPRWKHU
VVDIHW\\HW\RXDUHLQGLJQDQWZKHQWKHWDEOHVDUH
WXUQHGILJXUDWLYH
x $WWKDWWLPHGXULQJWKHODWHZLQWHULFHZDVEURNHQDQGFDUULHGRQKRUVHDQGFDUWWRWKHLFH
FHOODUVDQGLFHKRXVHVOLWHUDO
x ,WVHHPVRQFHWKHLFHZDVEURNHQDQGHJRVFDOPHGJHWWLQJGRZQWRWKHHLJKWNH\WDVNVZLWK
KHOSIURPRXWVLGHUVGLGQRWVHHPVREDGDIWHUDOOILJXUDWLYH
$JDLQWKHPRGHO¶VWDVNZDVWRFODVVLI\WKHVHQWHQFHDVDILJXUDWLYHRUOLWHUDOXVDJH7DEOH
EHORZVKRZVWKHUHVXOWV
([SUHVVLRQ
WDEOHVWXUQHG
LFHEURNHQ
/LWHUDO)LJXUDWLYH &RUUHFW 2XWRI
$FFXUDF\
/LWHUDO
)LJXUDWLYH
/LWHUDO
)LJXUDWLYH
$YHUDJH
7DEOH$FFXUDF\VFRUHIRUSDVVLYHYRLFHGDWDVHW
7KHRYHUDOODFFXUDF\VFRUHLVFRPSDUDEOHWRWKHRWKHUH[SHULPHQWV UHSRUWHGLQWKHSUHYLRXVWZR
VXEVHFWLRQV6RIRUWKHVHWZRH[SUHVVLRQVDWOHDVWWKHUHSUHVHQWDWLRQVDSSHDUWREHDVHIIHFWLYHIRU
WKHSDVVLYHDVWKH\DUHIRUWKHDFWLYHYRLFH
2WKHU9DULDQWV2PLVVLRQDQG6XEVWLWXWLRQ
$NH\ FRPSRQHQWRI)D]O\ HW DO¶VPHWKRGZDV WR RQO\ FODVVLI\DXVDJH DVLGLRPDWLFLILW
DSSHDUHG LQ LWV XVXDO ³FDQRQLFDO´ IRUP <HW DV ZDV GLVFXVVHG LQ 6HFWLRQ DQG LQGHHG
DFNQRZOHGJHGE\)D]O\DQGFROOHDJXHVLQWKHLUSDSHULGLRPVFDQYDU\2QHH[DPSOHLVDZRUG
VXEVWLWXWLRQ VXFK DV ³\DQN WKH SOXJ´ DV D PRUH HPSKDWLF YHUVLRQ RI ³SXOO WKH SOXJ´ 6RPH
LQGLYLGXDO ZRUGV OLNH ³EXQGOH´ LQ ³PDNH D EXQGOH´ RU ³GURS D EXQGOH´ UHWDLQ WKHLU LGLRPDWLF
VHQVH LQ RWKHU FRQVWUXFWLRQV DV ZHOO VXFK DV ³VSHQG D EXQGOH´ RU ³ORVH D EXQGOH´ ,GHDOO\ RXU
UHSUHVHQWDWLRQVRIVXFKZRUGVVKRXOGJHQHUDOL]HWRRWKHUH[SUHVVLRQVRQZKLFKWKHYHFWRUVZHUH
QRWVSHFLILFDOO\WUDLQHG
7RWHVW WKHGHJUHHWR ZKLFKDSSURSULDWH JHQHUDOL]DWLRQLV SRVVLEOHD VHFRQGGDWDVHW RI
VHQWHQFHV ZDV H[WUDFWHG IURP &2&$ 'DYLHV DQG WKH :LNLSHGLD &RUSXV 'DYLHV
,QRIWKRVHVHQWHQFHVDOORIWKHZRUGVDUHXVHGLQWKHLUXVXDOOLWHUDOVHQVH,QWKHRWKHU
D ZRUG LV XVHG LGLRPDWLFDOO\ EXW QRW ZLWKLQ WKH LGLRPDWLF H[SUHVVLRQ RQ ZKLFK WKH PRGHO
WUDLQHG )RU H[DPSOH WKH LGLRP ³GURSSHG WKH EDOO´ LV RQH RI WKH LGLRPV IRU ZKLFK WKH PRGHO
FUHDWHG GLVWLQFW LGLRPDWLF YHUVLRQV RI ³GURS´ DQG ³EDOO´ 2QH VHQWHQFH LQ WKH GDWDVHW LV ³7KH
FDQGLGDWHKDVIXPEOHGWKHEDOOHYHU\WLPHKHKDVWDONHGDERXWIRUHLJQSROLF\´7KHZRUG³EDOO´
KHUHFOHDUO\KDVWKHVDPHLGLRPDWLFPHDQLQJHYHQWKRXJKLWLVXVHGZLWKWKHYHUE³IXPEOH´IRU
ZKLFKWKHPRGHOKDVQRLGLRPDWLFUHSUHVHQWDWLRQ+HUHDUHIRXURWKHUH[DPSOHVIURPWKLVGDWDVHW
HPSKDVLVDGGHGKHUHVROHO\IRUFODULW\
x 0V,YLQVPD\SUHIHUWRDWWDFNWKHPHVVHQJHUZKHQVKHGRHVQ
WDJUHHZLWKWKHPHVVDJHILJXUDWLYH
x $KDSURXGER\ODXJKHGWKHPHVVHQJHUDVWKH\WXUQHGLQWRWKHVQRZILOOHGSDUNWKLVLVDJUHDW
GD\LVLWQRWIRUWKHH"´OLWHUDO
x %XWVRPHHDJHUKRPHRZQHUVZKRVSRQVRUWKHFRQWHVWVVWDQGWRORVHDEXQGOHILJXUDWLYH
x 0DUJDUHWKXUULHGWRKHUFDUDQGUHWXUQHGFDUU\LQJDEXQGOHZUDSSHGLQDFRWWRQEHGVKHHWOLWHUDO
:HXVHGWKLVGDWDVHWIRUWZRWDVNV7KHILUVWLVDQDORJRXVWRWKHWDVNVGHVFULEHGDERYHWKH
PRGHO LV DVNHG LI WKH ILJXUDWLYH RU OLWHUDO UHSUHVHQWDWLRQ LV WKH EHVW ILW IRU WKH JLYHQ VHQWHQFH
8VLQJWKHEDWFKRIYHFWRUVWKDWSHUIRUPHGWKHEHVWWKURXJKRXWWKLVVHFWLRQDQGQRELDVWHUPWKH
PRGHO DFKLHYHG RYHUDOO DFFXUDF\ +HUH DUH WKH GHWDLOV EURNHQ GRZQ E\ ILJXUDWLYH DQG
OLWHUDOFDVHV
)LJXUDWLYH
/LWHUDO
3UHFLVLRQ
5HFDOO
)
7DEOHSUHFLVLRQUHFDOODQG)VFRUHVIRUFODVVLI\LQJVLQJOHZRUGVXVHGILJXUDWLYHO\
7KH VHFRQG WDVN LV SRWHQWLDOO\ PRUH GLIILFXOW \HW PRUH UHDOLVWLF WKH PRGHO LV JLYHQ WKH
VHQWHQFH EXW LV QRW WROG ZKLFK LI DQ\ ZRUG PLJKW EH XVHG ILJXUDWLYHO\ 7KH PRGHO¶V WDVN LV WR
DVVHVVHDFKZRUGWKDWKDVWZRVHQVHVDQGGHFLGHZKLFKVHQVHPDNHVWKHEHVWILWLQWRWKHRYHUDOO
FRQWH[W )RU H[DPSOH LQ WKH VHQWHQFH ³KH ZDV DIUDLG RI EHLQJ FDOOHG D UDW´ ³UDW´ LV XVHG
ILJXUDWLYHO\ZKLOH³FDOO´ZKLFKFDQEHSDUWRIDQLGLRP³FDOOWKHVKRWV´LVXVHGOLWHUDOO\6RLQ
WKLVVHQWHQFHWKHPRGHOPXVWPDNHWZRGHFLVLRQV$QH[FHSWLRQLVPDGHIRUDX[LOLDU\YHUEVOLNH
³KDYH´ZKLFKDUHFODVVLILHGDVOLWHUDOE\GHIDXOW7KHDOJRULWKPRSHUDWHVLQDJUHHG\ZD\RQFHD
GHFLVLRQKDVPDGHDGHFLVLRQIRUDSDUWLFXODUZRUGHDUO\LQWKHVHQWHQFHLWLVQRWUHYLVLWHGEDVHG
RQRWKHUGHFLVLRQVPDGHODWHULQWKHVHQWHQFH
$JDLQXVLQJWKHEDWFKRIYHFWRUVWKDWREWDLQHGWKHEHVWUHVXOWVRQWKHSUHYLRXVWDVNVDQGQR
ELDVWHUPWKLVPRGHODFKLHYHGDQRYHUDOODFFXUDF\VFRUHRIZKLFKEUHDNVGRZQDVVKRZQ
LQ7DEOH
)LJXUDWLYH
/LWHUDO
3UHFLVLRQ
5HFDOO
)
7DEOH3UHFLVLRQUHFDOODQG)VFRUHVZKHQWKHPRGHOPXVWPDNHDGHFLVLRQIRUHDFKZRUG
:KDW DSSHDUHG WR EH D PRUH GLIILFXOW WDVN KDV DFWXDOO\ UHVXOWHG LQ EHWWHU VFRUHV RQ WKH OLWHUDO
LQVWDQFHV ,Q WKH PHDQWLPH WKH SUHFLVLRQ RI WKH ILJXUDWLYH FDVHV KDV GLPLQLVKHG VRPHZKDW
2YHUDOOWKRXJKWKHVFRUHVDUHTXLWHFRPSDUDEOHWRZKDWWKH\ZHUHIRUWKHRWKHUWDVNVGLVFXVVHG
LQWKLVFKDSWHU7KLVOHYHORISHUIRUPDQFHOHQGVVXSSRUWWRDFRQFOXVLRQWKDWWKHUHSUHVHQWDWLRQV
FDQJHQHUDOL]HWKHILJXUDWLYHPHDQLQJVRILQGLYLGXDOZRUGVWRRWKHUFRQWH[WVZKHQDSSURSULDWH
DQGWKDWWKHUHSUHVHQWDWLRQVFDQEHXVHGHYHQIRUFDVHVRIVOLJKWYDULDWLRQVZLWKLQDQLGLRP
:H UHLWHUDWH D SRLQW PDGH ZKHQ GLVFXVVLQJ WKH UHVXOWV IRU WKH LGLRP YDULDQWV LQ WKH
0DWFKLQJ7DVN6HFWLRQ7KHGHFLVLRQWROHDUQLQGLYLGXDOUHSUHVHQWDWLRQVIRUHDFKZRUGLQDQ
LGLRPKDVSHUPLWWHGWKHJHQHUDOL]DWLRQVHHQKHUH7KHFRPPRQO\XVHG³VWDWLF´VROXWLRQZKHUHE\
WKHZRUGVRIDQLGLRPDUHPHUJHGWRIRUPD³VXSHUWRNHQ´6HFWLRQZRXOGQRWKDYHZRUNHG
RQWKLVGDWDVHW7KDWVROXWLRQGRHVQRW DOORZWKHFRPSRQHQWZRUGVRIDQLGLRPWREHVHSDUDWHG
IURPWKHZKROHVRWKHUHLVQRUHSUHVHQWDWLRQIRU³EDOO´RU³EXQGOH´E\WKHPVHOYHV7KLVDVSHFWLV
DFOHDUDGYDQWDJHRIRXUSURSRVHGVROXWLRQ
6800$5<
2QHRIWKHPRVWIXQGDPHQWDO FKDUDFWHULVWLFVRILGLRPVLVWKDWWKH\PHDQVRPHWKLQJRWKHU
WKDQ ZKDW D OLWHUDO UHDGLQJ ZRXOG VXJJHVW 6R D UHSUHVHQWDWLRQ RI DQ LGLRP PXVW FDSWXUH WKDW
GLVWLQFWILJXUDWLYHPHDQLQJ 7KHREMHFWLYHRIWKLVVHFWLRQ KRZZHOO WKHVH UHSUHVHQWDWLRQV GRVR
IRULWLV RQO\E\KDYLQJ FOHDU DQGGLVWLQFW UHSUHVHQWDWLRQVRIWKHOLWHUDO DQGILJXUDWLYHPHDQLQJV
WKDWDPRGHOFDQDVVHVVZKLFKPHDQLQJLVWKHEHVWILWIRUDVHQWHQFH:KDWZHKDYH VHHQLQWKH
DERYHVHFWLRQVLVWKDWPRGHOVXVLQJRXUSURSRVHGUHSUHVHQWDWLRQVKDYHJHQHUDOO\SHUIRUPHGTXLWH
ZHOO7KURXJKRXWWKHYDULRXVGDWDVHWVDQGVHWXSVSUHVHQWHGLQWKLVVHFWLRQDFFXUDF\VFRUHVKDYH
JHQHUDOO\EHHQDERYHDQGPRUHRIWHQQHDUHU7KHVHVFRUHVKDYHJHQHUDOO\PDWFKHGRU
H[FHHGHG WKRVH RI RWKHU PRGHOV WKDW ZHUH VSHFLILFDOO\ HQJLQHHUHG IRU WKLV WDVN 2XU
UHSUHVHQWDWLRQVKDYHWKHDGYDQWDJHRIEHLQJXVDEOHE\RWKHUWDVNVDQGRIKDQGOLQJYDULDQWV$OORI
WKLV HYLGHQFH LQGLFDWHV WKDW WKH UHSUHVHQWDWLRQV DUH ZHOOVXLWHG WR WKH VHFRQG FRUH 1/3 WDVN RI
GLVDPELJXDWLRQ:HZLOOQRZVHHKRZZHOOWKH\VHUYHWKHWKLUGWDVNSUHGLFWLRQ
8 The Sentence Completion Task
7$6.'(6&5,37,21
$KXPDQVSHDNHUZLOOJHQHUDOO\GHPRQVWUDWHNQRZOHGJHRIDZRUGRUH[SUHVVLRQE\XVLQJLW
FRUUHFWO\ LQ D VHQWHQFH %\ DQDORJ\ D PRGHO WKDW FDQ VHOHFW WKH DSSURSULDWH LGLRP IRU D JLYHQ
FRQWH[WFDQEHVDLGWRKDYHFDSWXUHGVRPHWKLQJHVVHQWLDODERXWWKDWLGLRP7KLVWDVNLVNQRZQLQ
WKH1/3OLWHUDWXUHDVWKHVHQWHQFHFRPSOHWLRQWDVNHJ=ZHLJ%XUJHV7KHWHVWLVDOVR
VLPLODU WR PDQ\ PXOWLSOHFKRLFH WHVWV JLYHQ WR (6/ VWXGHQWV WR DVVHVV WKHLU XQGHUVWDQGLQJ RI
LGLRPV$QGLWFRUUHVSRQGVWRWKH³SUHGLFWLRQ´WDVNLQ*ULIILWKV6WH\YHUDQG7HQHQEDXP¶VWULQLW\
RIFRUH1/3IXQFWLRQVLQWKDWLWLQYROYHVDPRGHO¶VSUHGLFWLQJDQLGLRPEDVHGRQWKHUHVW
RIWKHVHQWHQFH
)RU WKLV WDVN HDFK SURPSW LV D VHQWHQFH FKRVHQ IURP WKH 2QH %LOOLRQ :RUG %HQFKPDUN
&RUSXV&KHOEDHWDOLQZKLFKRQHRIWKHWDUJHWLGLRPV$SSHQGL[$ZDVXVHG8VLQJD
UHDOVHQWHQFHUHPRYHVWKHULVNRILQWURGXFLQJXQFRQVFLRXVELDVLQWRWKHZD\WKDWWKHLGLRPVDUH
XVHG RU WKH VHQWHQFHV DUH IRUPXODWHG XVLQJ VHQWHQFHV IURP D VHSDUDWH FRUSXV HQVXUHV WKDW WKH
PRGHOZLOOQRWKDYHEHHQH[SRVHGWRWKHVHQWHQFHGXULQJWUDLQLQJ7KHPRGHOLVSUHVHQWHGZLWK
WKH VHQWHQFH ZLWK WKH LGLRP H[WUDFWHG DQG LV JLYHQ WKH SODFH LQ WKH VHQWHQFH ZKHUH WKH LGLRP
ZRXOGEHORQJDQDORJRXVWRWKHEODQNWKDWZRXOGEHSURYLGHGLQDKXPDQWHVW7KHPRGHOLVWKHQ
JLYHQIRXUGLIIHUHQWLGLRPVWKHRQHWKDWZDVLQWKHRULJLQDOVHQWHQFHDQGWKUHHGHFR\VWKDWDUHLQ
RXUMXGJHPHQWDSRRUILWIRUWKHVHQWHQFH+HUHLVDQH[DPSOH
,KDGWROHDUQWRBBBBBBVRDVQRWWRVHHPLPSHUWLQHQW
x SDYHWKHZD\
x ELWHP\WRQJXH
x PLVVWKHERDW
x WXUQWKHWDEOHV
7KHPRGHO¶VWDVNLVWRFKRRVHWKHLGLRPWKDWEHVWILWVWKHVHQWHQFHKHUH³ELWHP\WRQJXH´7KUHH
PRUHH[DPSOHVDUHSURYLGHGLQ$SSHQGL[)
7KHUH DUH WZR VHQWHQFHV IRU HDFK RI VL[W\ LGLRPV PDNLQJ WHVW VHQWHQFHV LQ DOO 7KH
GHFR\DQVZHUVZHUHGLVWULEXWHGDVUDQGRPO\DQGDVHYHQO\DVSRVVLEOH(DFKGHFR\LVRIIHUHGDW
OHDVWILYHEXWQRPRUHWKDQVHYHQWLPHVWKURXJKRXWWKHWHVWVHW,WZDVQRWSRVVLEOHWRPDNHWKH
GLVWULEXWLRQFRPSOHWHO\UDQGRPRUH[DFWO\XQLIRUPJLYHQWKHFRQVWUDLQWWKDWDGHFR\VKRXOGQRW
EHDSODXVLEOHILWIRUWKHJLYHQVHQWHQFH
)RUWKLVWDVNZHGRQRWLQFOXGHDWHVWVHWIRUYDULDWLRQV7KHQDWXUHRIWKHWDVNLVWKDWWKH
PRGHO PXVW FKRRVH EHWZHHQ SRWHQWLDOO\ DWWUDFWLYH FDQGLGDWHV +RZHYHU WKHUH LV QR ZD\ WR
FRQWUROKRZ³DWWUDFWLYH´DYDULDWLRQZLOOEHFRPSDUHGWRDQRWKHU³7XUQWKHSROLWLFDOWDEOHV´DQG
³ELWHWKHSROLWLFDOEXOOHW´ERWKVHHPOLNHDFRKHUHQWDQGSODXVLEOHLGLRPYDULDWLRQVEXW³GURSD
SROLWLFDOEXQGOH´LVOHVVVR7KHSRWHQWLDOFRQIRXQGVWKDWVXFKGLIIHUHQFHVZRXOGPDNHWKHUHVXOWV
LQFRQFOXVLYH
&20387$7,21$/0(7+2'
$JDLQ WKH VHQWHQFH PLQXV WKH H[SUHVVLRQ LV UHSUHVHQWHG E\ 6 ZKLFK LV HLWKHU D VLQJOH
YHFWRU RU D FROOHFWLRQ RI YHFWRUV ,Q HLWKHU FDVH WKH K\SRWKHVLV DGYDQFHG KHUH LV WKDW WKH EHVW
FKRLFHWRFRPSOHWHWKHVHQWHQFHrLVWKHRQHZKRVHYHFWRUUHSUHVHQWDWLRQEHVWDOLJQVZLWK67KH
PRGHOFKRRVHVWKHEHVWLGLRPE\FRQGXFWLQJDSDLUZLVHFRPSDULVRQIRUHYHU\SDLUHGFRPELQDWLRQ
RIFDQGLGDWHLGLRPV7KDWLVIRUHDFKSDLULQJRISRVVLEOHDQVZHUVHDDQGHE
r DUJPD[HVLP6HM
)RU D FKRLFH EHWZHHQ LGLRPV WKHUH DUH ൫ସଶ൯ ൌ SDLUZLVH FRPSDULVRQV WR EH PDGH 2QH
FRPSDULVRQUHVXOWVLQRQHYRWH2QFHDOOSDLUZLVHFRPSDULVRQVDUHPDGHWKHPRGHOFKRRVHVWKH
LGLRPWKDWUHFHLYHGWKHPRVWYRWHV
7KHGHWDLOVRIKRZWKHVFRUHLVFDOFXODWHGGLIIHUVOLJKWO\DFFRUGLQJWRWKHPRGHOFKRVHQWR
FUHDWH6
:KHQ6LVDYHFWRUPHWKRGVDQGWKHVFRUHLVVLPSO\WKHFRVLQHVLPLODULW\VFRUHEHWZHHQ
WKHH[SUHVVLRQ¶VYHFWRUDQG6)RUWKHPHWKRGZKHUHDVLQJOHNH\ZRUGLVFKRVHQWKHUHZLOO
EHRQHNH\FKRVHQIRUHDFKSDLUZLVHFRPELQDWLRQ,QSUDFWLFHIRUPRVWRUDOORIWKH
FRPSDULVRQVWKHVDPHNH\ZRUGZLOORIWHQEHFKRVHQ
r DUJPD[HVLP6HM
:KHQ6LVDFROOHFWLRQRIYHFWRUVPHWKRGVDQGWKHVFRUHLVWKHVXPRIWKHFRVLQH
VLPLODULWLHVRIWKHH[SUHVVLRQDQGDOORIWKHYHFWRUVLQ6
r DUJPD[Hσே
௜ୀଵ •‹ሺ‫ ܑܟ‬ǡ ‫ ܒ܍‬ሻIRUZL6
(03,5,&$/5(68/76
$WWKLVSRLQWZHKRSHWKDWWKHTXHVWLRQRIKRZWRFRPSRVHWKHLGLRPDWLFSKUDVHKDVEHHQ
VHWWOHGJLYHQWKHYHU\FOHDUGLIIHUHQFHVVHHQLQWKHODVWWDVN7KHUHVXOWVRQUHSUHVHQWDWLRQVRIWKH
VHQWHQFH ZHUH OHVV GHFLVLYH DQG FRXOG EH WDVNGHSHQGHQW DQG VR ZH UHH[DPLQH WKLV TXHVWLRQ
QRZ:HDJDLQSUHVHQWDVHULHVRIUHVXOWVIRUWKHIRXUZD\VRIUHSUHVHQWLQJWKHVHQWHQFHWHVWHGRQ
WKH VDPH ILYH EDWFKHV DV ZHUH XVHG LQ 6HFWLRQ EXW WKLV WLPH RQ WKH VHQWHQFH FRPSOHWLRQ
WDVN
0HWKRG
.H\ZRUG
$GGLWLRQ
6XPRIVFRUHV
'HSHQGHQF\SDUVH
VHW
VHW
VHW
VHW
VHW
WLPHV
7DEOH&RPSDULVRQRIDFFXUDF\VFRUHVDQGWRWDOSURFHVVLQJWLPHREWDLQHGIRUIRXU
FRPSRVLWLRQPHWKRGVLQWKHVHQWHQFHFRPSOHWLRQWDVN(DFKPHWKRGZDVXVHGZLWKWKHVDPH
ILYHEDWFKHVRQWKHIXOOLWHPWHVWVHW8VLQJWKHGHSHQGHQF\SDUVHFOHDUO\UHVXOWVLQWKH
KLJKHVWDYHUDJHDFFXUDF\DQGPD[LPXPVFRUHVIRUWKHVHILYHEDWFKHV,WLV
KRZHYHUE\IDUWKHVORZHVWRIWKHIRXUPHWKRGV
7KH GHSHQGHQF\ SDUVH LV FOHDUO\ WKH OHDGHU KHUH 7KH GLIIHUHQFH LV VWDWLVWLFDOO\ VLJQLILFDQW
) S 7KH PHWKRG DOVR WDNHV ORQJHU WKDQ WKH RWKHUV EHFDXVH WKH SDUVLQJ VWHS
LWVHOI WDNHV VHFRQGV IRU WKH VHQWHQFHV 7KH SDUVHU DOVR WDNHV DURXQG 0% RI
DGGLWLRQDO5$0,QSUDFWLFHKRZHYHUWKHSDUVLQJVWHSZDVUXQRQFHDQGWKHQLWVUHVXOWVZHUHUH
XVHG*LYHQWKHFOHDUDGYDQWDJHRIWKHGHSHQGHQF\SDUVHDOJRULWKPWKHUHPDLQGHURIWKHUHVXOWV
UHSRUWHGLQWKLVVHFWLRQZLOOXVHWKLVPHWKRG
:HEHJLQE\FRQWUDVWLQJWKHSHUIRUPDQFHRIWKHEDVHOLQHWRWKHWZRPHWKRGVRIWDJJLQJWKH
WUDLQLQJFRUSXV7KHUDQJHRIVFRUHVLVGHSLFWHGLQ)LJXUH$VZDVWKHFDVHLQWKHRWKHUWDVNV
WKH FRQWUDVW EHWZHHQ WKH EDVHOLQH DQG WKH WZR WDJJLQJ PHWKRGV LV TXLWH VWDUN DQG KLJKO\
VLJQLILFDQW)VFRUH S8VLQJWKHSXEOLFO\DYDLODEOH*RRJOH1HZVYHFWRUVJRW
FRUUHFWDQVZHUVDQGDFFXUDF\WKHORZHVWVFRUHRIDOOPHWKRGV7UDLQLQJYHFWRUEDWFKHV
RQDOHPPDWL]HGFRUSXVZLWKRQO\SDUWRIVSHHFKWDJJLQJJRWDWPRVWFRUUHFWDQVZHUVDVFRUH
RI 6R HYHQ WKH EHVW VFRUHV RI WKH EDVHOLQH PHWKRG DUH ORZHU WKDQ WKH ORZHVW VFRUHV
REWDLQHGE\DEDWFKWUDLQHGXVLQJHLWKHURIWKHSURSRVHGPHWKRGVZKLFKVWLOOPDQDJHGWRJHW
FRUUHFWDQVZHUV
)LJXUH&RPSDULVRQRIWKHSHUIRUPDQFHRQWKHVHQWHQFHFRPSOHWLRQWDVNE\IRXUPRGHOV
XVLQJUHVSHFWLYHO\WKH*RRJOH1HZVYHFWRUVRUYHFWRUVWUDLQHGUHVSHFWLYHO\XVLQJWKH
EDVHOLQHWKHDXWRPDWLFDOO\WDJJHGDQGWKHPDQXDOO\DQQRWDWHGFRUSXV7KHWZRSURSRVHG
PHWKRGVFOHDUO\RXWSHUIRUPWKHEDVHOLQHE\DVXEVWDQWLDOPDUJLQWKHKLJKHVWVFRUHREWDLQHGE\D
EDWFKWUDLQHGXVLQJWKHEDVHOLQHPHWKRGLVOHVVWKDQWKHORZHVWVFRUHREWDLQHGE\WKHWZR
SURSRVHGPHWKRGV
/HWXVQRZORRNDWWKHWDUJHWPHWKRG¶VEHVWVFRUHV2IEDWFKHVJHQHUDWHGE\WKHDXWR
WDJJLQJPHWKRGWKHEHVWSHUIRUPHUDFKLHYHGDFFXUDF\WKDWLVFRUUHFWDQVZHUVIRU
TXHVWLRQV7KHEHVWSHUIRUPHUDPRQJEDWFKHVWUDLQHGXVLQJWKHPDQXDOO\DQQRWDWHGFRUSXVJRWD
VLPLODUVFRUHRIFRUUHFWDQVZHUV7KLVSHUIRUPDQFHOHYHOLVORZHUWKDQZKDWZHKDYHVHHQVR
IDU2QWKHRWKHUKDQGVHQWHQFHFRPSOHWLRQLVDGLIILFXOWWDVNPRGHOVLQWKHOLWHUDWXUHW\SLFDOO\
KDYHDFFXUDF\UDWHVRIDURXQGRQWKHLUGDWDVHWVHJ=ZHLJ%XUJHV
$OWKRXJK WKH UHVXOWV REWDLQHG E\ XVLQJ WKH DXWRWDJJLQJ DQG PDQXDO WDJJLQJ PHWKRGV
UHVHPEOH RQH DQRWKHU WKH\ DUH GLIIHUHQW HQRXJK WR EH DEOH WR VD\ WKDW DXWRPDWLF WDJJLQJ GRHV
EHWWHU) S,QRWKHUZRUGVWKHUHLVQRGLVFHUQDEOHSD\EDFNIRUWKHH[WUDKXPDQ
HIIRUWUHTXLUHGWRDQQRWDWHWKHFRUSXV7KLVUHVXOWLVTXLWHFRXQWHULQWXLWLYHDQGZLOOEHH[SORUHG
IXUWKHUEHORZDQGLQWKH&RQFOXVLRQ
2WKHUWKDQWUDLQLQJPHWKRGRQO\IRXUWUDLQLQJSDUDPHWHUVKDYHDVLJQLILFDQWHIIHFWRQWKH
SHUIRUPDQFHRIEDWFKHVWUDLQHGRQWKHDXWRPDWLFDOO\WDJJHGFRUSXVGLPHQVLRQDOLW\ZLQGRZVL]H
WKHVDPSOLQJWKUHVKROG DQGWKHQXPEHURILWHUDWLRQVVHH 7DEOH2QFHDJDLQ UHODWLYHO\ORZ
GLPHQVLRQDOLW\DFWXDOO\LPSURYHVSHUIRUPDQFHEXWRQO\XSWRDSRLQWGLPHQVLRQVLVQRWTXLWH
HQRXJK6PDOOZLQGRZVDQGORZVDPSOLQJWKUHVKROGVSURYHDGYDQWDJHRXVKHUHDVWKH\GLGIRUWKH
OLWHUDOILJXUDWLYHFODVVLILFDWLRQWDVNOLNHO\IRUWKHVDPHUHDVRQV
)LJXUH(IIHFWVRIWUDLQLQJSDUDPHWHUVRQSHUIRUPDQFHLQWKHVHQWHQFHFRPSOHWLRQWDVNRI
EDWFKHVWUDLQHGRQDQDXWRPDWLFDOO\WDJJHGFRUSXV$JDLQZHVHHWKDWUHODWLYHO\ORZ
GLPHQVLRQDOLW\VPDOOZLQGRZVDQGIDLUO\ORZVDPSOLQJWKUHVKROGVSURGXFHWKHEHVW
UHVXOWV,QFUHDVLQJWKHQXPEHURIWUDLQLQJLWHUDWLRQVFOHDUO\LPSURYHVSHUIRUPDQFH
3DUDPHWHU
)VFRUH
S
2SWLPDO6HWWLQJ
'LPHQVLRQV
:LQGRZVL]H
3DUDPHWHU
)VFRUH
S
2SWLPDO6HWWLQJ
6DPSOLQJWKUHVKROG
H
LWHUDWLRQV
2SWLPL]DWLRQPHWKRG
$OSKD
QHJDWLYHVDPSOHV
7DEOH1ZD\$129$RISDUDPHWHUVIRUWKHDXWRPDWLFWDJJLQJPHWKRGLQWKHVHQWHQFH
FRPSOHWLRQWDVN,WHPVLQLWDOLFVDUHVWDWLVWLFDOO\VLJQLILFDQWS
7KHRSWLPDOSDUDPHWHUVGLIIHUHQRXJKIURPWKRVHRIWKHRWKHUWDVNVWRVXJJHVWWKDWVRPHYHFWRU
EDWFKHVPLJKWGRUHDVRQDEO\ZHOORQERWKWDVNVEXWWKDWLISHUIRUPDQFHZHUHFULWLFDOLWZRXOGEH
EHVWWRWUDLQYHFWRUVDFFRUGLQJWRDVSHFLILFWDVN
+DYLQJORRNHGDWWKHJHQHUDOIDFWRUVWKDWDIIHFWSHUIRUPDQFHOHWXVQRZORRNDWWKHRXW
FRPHVIRUWKHLQGLYLGXDOLGLRPV:HEHJLQE\ORRNLQJDWZKLFKLGLRPVZHUHFRQVLVWHQWO\FKRVHQ
ZKHQDSSURSULDWHFRPSDUHGWRWKRVHFKRVHQZKHQLQDSSURSULDWHRUQRWFKRVHQZKHQWKH\VKRXOG
KDYHEHHQ)RUH[DPSOHDOORIWKHWUDLQHGPRGHOVFKRVH³KLWWKHERRNV´HDFKDQGHYHU\WLPH
LWZDVDSSURSULDWHDQGRQO\RQHFKRVHLWIRUDVHQWHQFHZKHUHLWZDVQRWWKHFRUUHFWDQVZHU:H
FDQVD\WKDWPRGHOVXVLQJWKLVUHSUHVHQWDWLRQKDYHKLJKUHFDOOZLWKUHVSHFWWRWKLVLGLRPLH
FKRRVHWKHLGLRPZKHQWKHFRQWH[WLVDSSURSULDWHDQGDUHKLJKO\SUHFLVHGRQRWFKRRVHWKH
LGLRPIRULQDSSURSULDWHFRQWH[WV³5LQJDEHOO´KDVWKHPRVWSUHFLVHUHSUHVHQWDWLRQLQWKDWLWLV
QHYHUFKRVHQZKHQQRWDSSURSULDWHEXWORZHUUHFDOOKDYLQJEHHQFKRVHQIRULWVFRUUHFWFRQWH[W
WLPHVRXWRI,QFRQWUDVW³FKDQJHJHDUV´ZDVFRUUHFWO\FKRVHQWLPHVDJDLQRXWRI
EXWLWZDVDOVRFKRVHQWLPHVZKHQLWZDVQRWWKHFRUUHFWDQVZHU,QJHQHUDOWKH
UHSUHVHQWDWLRQVWHQGHGWRUHVXOWLQEHWWHUUHFDOOWKDQSUHFLVLRQEXWPDQ\PDQDJHGWRJHWJRRG
VFRUHVIRUERWKUHFDOODQGSUHFLVLRQ
,WPD\EHPRUHLQVWUXFWLYHWRORRNDWWKHFKRLFHVRIWKHEHVWSHUIRUPLQJPRGHOVVLQFHWKH
FKRLFHVRIDSRRUO\SHUIRUPLQJPRGHOPLJKWKDYHPRUHWRGRZLWKZHDNQHVVHVLQWKHPRGHO¶V
WUDLQLQJWKDQZLWKWKHLGLRPLWVHOI7DEOHJLYHVWKHSUHFLVLRQDQGUHFDOOVFRUHVIRUWKH
LGLRPVXVHGLQWKLVWDVN$OVRLQFOXGHGDUHWKHQXPEHURIWUXHSRVLWLYHVLHFRUUHFWFKRLFHVDQG
IDOVHSRVLWLYHVLHWLPHVWKDWWKHLGLRPZDVFKRVHQZKHQLWVKRXOGQRWKDYHEHHQ,GLRPV
ZKRVH)VFRUHVDUHLQWKHWRSTXLQWLOHDUHLQEROGW\SHLGLRPVLQWKHERWWRPTXLQWLOHDUHLQ
LWDOLFV
Idiom
bear cross
bite bullet
bite dust
bite tongue
blow fuse
blow whistle
break heart
break ice
bury hatchet
buy farm
call shot
change gear
change tune
chew fat
clear air
cover track
crack smile
crack whip
drop ball
drop bomb
drop bundle
drop line
face music
fan flame
flood market
force hand
get ax
get sack
have ball
hit book
hit hay
hit road
hit sack
hit wall
hold fort
kick bucket
kick habit
know score
lose marble
lose shirt
TP
5
5
8
10
10
5
6
7
5
0
7
9
7
10
5
5
10
2
3
0
9
3
10
10
10
10
8
7
8
10
10
10
10
5
4
8
10
9
6
10
FP
8
4
6
6
6
3
1
0
6
0
0
0
4
2
0
2
0
0
0
0
6
0
4
0
5
0
5
13
6
0
5
2
10
9
2
9
4
1
2
2
Precision
0.385
0.556
0.571
0.625
0.625
0.625
0.857
1.000
0.455
0.000
1.000
1.000
0.636
0.833
1.000
0.714
1.000
1.000
1.000
0.000
0.600
1.000
0.714
1.000
0.667
1.000
0.615
0.350
0.571
1.000
0.667
0.833
0.500
0.357
0.667
0.471
0.714
0.900
0.750
0.833
Recall
0.500
0.500
0.800
1.000
1.000
0.500
0.600
0.700
0.500
0.000
0.700
0.900
0.700
1.000
0.500
0.500
1.000
0.200
0.300
0.000
0.900
0.300
1.000
1.000
1.000
1.000
0.800
0.700
0.800
1.000
1.000
1.000
1.000
0.500
0.400
0.800
1.000
0.900
0.600
1.000
F1
0.435
0.526
0.667
0.769
0.769
0.556
0.706
0.824
0.476
0.000
0.824
0.947
0.667
0.909
0.667
0.588
1.000
0.333
0.462
0.000
0.720
0.462
0.833
1.000
0.800
1.000
0.696
0.467
0.667
1.000
0.800
0.909
0.667
0.417
0.500
0.593
0.833
0.900
0.667
0.909
Idiom
Precision
Recall
F1
TP
FP
make scene
10
4
0.714
1.000
0.833
miss boat
8
0
1.000
0.800
0.889
miss mark
0
0
0.000
0.000
0.000
pass buck
10
2
0.833
1.000
0.909
pave way
8
0
1.000
0.800
0.889
play ball
5
5
0.500
0.500
0.500
pull leg
8
1
0.889
0.800
0.842
pull plug
10
4
0.714
1.000
0.833
ring bell
7
0
1.000
0.700
0.824
shoot breeze
8
0
1.000
0.800
0.889
shoot messenger
10
0
1.000
1.000
1.000
show card
9
5
0.643
0.900
0.750
smell rat
5
0
1.000
0.500
0.667
spill bean
5
4
0.556
0.500
0.526
straddle fence
5
2
0.714
0.500
0.588
take heart
9
0
1.000
0.900
0.947
tie knot
6
1
0.857
0.600
0.706
trim sail
8
2
0.800
0.800
0.800
turn table
8
1
0.889
0.800
0.842
wear pant
10
1
0.909
1.000
0.952
AVERAGES
7.25
2.75
0.735
0.725
0.702
7DEOH7UXHSRVLWLYHVIDOVHSRVLWLYHVSUHFLVLRQUHFDOODQG)VFRUHVIRULQGLYLGXDO
LGLRPVLQWKHVHQWHQFHFRPSOHWLRQWDVN7KHVFRUHVDUHEDVHGRQWKHFKRLFHVPDGHE\WKHILYH
EHVWSHUIRUPLQJPRGHOV6FRUHVRFFXS\WKHIXOOUDQJHIURPWR,WHPVLQEROGD
UHLQWKHWRSTXLQWLOHRIWKH)VFRUHLWHPVLQLWDOLFVDUHLQWKHERWWRPTXLQWLOH
1RW RQO\ ZHUH WKHUH QR FRUUHFW DQVZHUV IRU ³GURS D ERPE´ RU ³EX\ WKH IDUP´ WKHVH
H[SUHVVLRQVZHUHW\SLFDOO\WKHPRGHOV¶WKLUGRUIRXUWKFKRLFHVDPRQJWKHIRXUSRVVLEOHDQVZHUV
1RUZDVHLWKHUH[SUHVVLRQFKRVHQZKHQLWZDVRIIHUHGDVDGHFR\³&UDFNWKHZKLS´DQG³PLVV
WKHPDUN´ZHUHDOVRVHOGRPFKRVHQZKHQRIIHUHGDVGHFR\V,WZRXOGDSSHDUWKDWWKHLUYHFWRUVGR
QRWILWDQ\FRQWH[W,QFRQWUDVW³JHWWKHVDFN´ZDVFKRVHQWLPHVE\WKHILYHPRGHOVWRJHWKHU
IRU VHQWHQFHV IRU ZKLFK LW ZDV QRW DSSURSULDWH 2WKHU LGLRPV KDYH D ODUJH QXPEHU RI IDOVH
SRVLWLYHV ³KLW WKH VDFN´ DQG ³KLW D ZDOO´ DQG ³NLFN WKH EXFNHW´ HDFK 7KH QXPEHU RI
IDOVHSRVLWLYHVLVDJDLQVXJJHVWLYHRIDSUREOHPLQWUDLQLQJSRVVLEO\WRRIHZWUDLQLQJH[DPSOHV
2I WKH VHYHQ SUREOHPDWLF LGLRPV LGHQWLILHG LQ WKH SUHFHGLQJ SDUDJUDSK IRXU ³GURS D
ERPE´ ³EX\ WKH IDUP´ ³PLVV WKH PDUN´ DQG ³KLW D ZDOO´ ZHUH DPRQJ WKH LGLRPV WKDW ZHUH
DQQRWDWHG PDQXDOO\ :H PLJKW H[SHFW WKDW VXFK DQQRWDWLRQV ZRXOG KHOS JLYHQ WKDW WKHVH
H[SUHVVLRQVDUHIUHTXHQWO\XVHGOLWHUDOO\,QVRPHFDVHVLWGLG7KHILYHEHVWSHUIRUPLQJEDWFKHV
WUDLQHGWKLVZD\JRWDQ)VFRUHRIRQ³EX\WKHIDUP´DVRSSRVHGWRWKHVFRUHRIUHSRUWHG
DERYH +RZHYHU ³KLW WKH ZDOO´ VWLOO JHWV IDOVH SRVLWLYHV ZKLOH ³PLVV WKH PDUN´ DQG ³GURS D
ERPE´ VWLOO REWDLQ DQ ) VFRUH RI /RRNLQJ EH\RQG WKHVH IRXU SUREOHP FDVHV ZH JHW DQ
LPSURYHPHQWLQDIHZVFRUHV³ULQJDEHOO´DFWXDOO\JHWVDSHUIHFW)VFRUHRIZKLOH³GURSD
OLQH´JRHVIURPWR%XWRYHUDOOPRUHVFRUHVGHFOLQHWKDQLPSURYHPHDQLQJWKDWWKH
QHW FKDQJH LV QHJDWLYH )RU WKH FRPSOHWH UHVXOWV RI WKHVH ILYH EDWFKHV DQG WKH FRPSDULVRQ WR
WKRVHUHSRUWHGLQ7DEOHVHH$SSHQGL[*6RZKLOHPDQXDODQQRWDWLRQLVDSDUWLDOVROXWLRQLQ
FHUWDLQ FDVHVLWLV QRW D FRPSUHKHQVLYHRQH1RULV LWREYLRXVZKDW GLIIHUHQWLDWHVWKHFDVHV IRU
ZKLFKLWLVKHOSIXOIURPWKRVHIRUZKLFKLWLVQRW
:HFORVHWKLVDQDO\VLVE\UHWXUQLQJWRWKHTXHVWLRQRIVKDULQJDVLQJOHZRUGUHSUHVHQWDWLRQ
DFURVV VHYHUDO LGLRPV 7KH UHVXOWV VKRZQ DERYH LQGLFDWH WKDW IRU WKH PRVW SDUW WKLV VWUDWHJ\ LV
HLWKHU EHQHILFLDO RU DW OHDVW QRW KDUPIXO LGLRPV VWDUWLQJ ZLWK ³KLW´ KDYH PDQ\ RI WKH KLJKHVW
VFRUHVZLWKWKHQRWDEOHH[FHSWLRQRI³KLWWKHZDOO´7KHODWWHU¶VSRRUVKRZLQJPD\PRUHKDYHWR
GRZLWKLWVIUHTXHQWOLWHUDOXVH)RUPDQ\RWKHULGLRPIDPLOLHVWKHUHVXOWVDUHTXLWHPL[HG³GURS´
EHLQJRQHH[DPSOH2QHUHDVRQWKDW³JHWWKHVDFN´PLJKWKDYHVRPDQ\IDOVHSRVLWLYHVFRXOGEH
WKDW LW DSSHDUV LQIUHTXHQWO\ LQ WKH WUDLQLQJ FRUSXV DQG IXUWKHUPRUH VKDUHV D ZRUG ZLWK ³KLW WKH
VDFN´ *LYHQ WKDW WKH WZR ³VDFNV´ DUH YHU\ GLIIHUHQW DQG WKH ZRUG ³JHW´ VR XQLQIRUPDWLYH WKLV
LGLRPPD\EHWKHH[DPSOHZKHUHWKHVKDULQJZDVKDUPIXO2QHZD\WRWHVWWKLVHIIHFWZRXOGEH
WR LQFOXGH PRUH VHQWHQFHV IURP D %ULWLVK FRUSXV ZKHUH ³JHW WKH VDFN´ ZLOO RFFXU PRUH
IUHTXHQWO\LQWUDLQLQJ2QHFRXOGWKHQVHHLIWKHUHSUHVHQWDWLRQIRU³VDFN´ZRXOGILWERWK³KLWWKH
VDFN´DQG³JHWWKHVDFN´ZLWKRXWVRPDQ\IDOVHSRVLWLYHV
6800$5<
7KLVILQDOWDVNPHDVXUHGWKHXVHIXOQHVVRIWKHVHUHSUHVHQWDWLRQVLQDSUHGLFWLRQWDVNWKDWLV
DWDVNZKHUHWKHPRGHOKDGWRSUHGLFWZKDWLGLRPZDVPRUHOLNHO\WREHXVHGLQDVHQWHQFH7KH
SUHYLRXVWZRWDVNVZHQWLQWKHRSSRVLWHGLUHFWLRQWKH\DVNHGWKHPRGHOWRJRIURPDQLGLRPWRLWV
PHDQLQJZKLOHWKLVRQHDVNHGLWWRJRIURPDPHDQLQJRIDFRQWH[WWRDQLGLRP*RLQJLQWKLV
GLUHFWLRQLVPRUHOLNHSURGXFLQJODQJXDJHWKDQXQGHUVWDQGLQJLW-XVWDVDKXPDQOHDUQLQJDQHZ
ODQJXDJHILQGVSURGXFWLRQPRUHGLIILFXOWWKDQFRPSUHKHQVLRQVRGRHVWKHPRGHO
7KDWVDLGWKHVFRUHVRQWKHEHVWSHUIRUPLQJEDWFKHVDUHFRQVLVWHQWO\LQWKHORZVHYHQWLHV
DFFXUDF\ SUHFLVLRQ DQG UHFDOO 7KHVH VFRUHV LQGLFDWH WKDW DOO LQ DOO WKH
UHSUHVHQWDWLRQV KDYH SHUIRUPHG UHDVRQDEO\ ZHOO $ PRUH LQGHSWK DQDO\VLV KDV SHUPLWWHG XV WR
VHH ZKLFK LGLRPV KDYH DIIRUGHG EHWWHU VFRUHV WKDQ RWKHUV 7KH ORZHU VFRUHV PD\ LQGLFDWH DQ
RSSRUWXQLW\WRLPSURYHWKHWUDLQLQJFRUSXVWKHWUDLQLQJPHWKRGLWVHOIRUWKHWHVWVHWRQZKLFKWKH
UHSUHVHQWDWLRQVDUHWHVWHG$OORIWKHVHSRVVLELOLWLHVFRXOGEHH[SORUHGLQIXWXUHZRUNDVZLOOEH
HODERUDWHGLQRXUFRQFOXGLQJFKDSWHU
9
Conclusions and Future Work
6800$5<$1'&2175,%87,216
7KLVZRUN¶VPDLQREMHFWLYHZDVWRILQGDPHWKRGE\ZKLFKXVHIXOUHSUHVHQWDWLRQVRILGLRPV
FRXOGEHFUHDWHG:HZDQWHGWRGRVRLQDZD\WKDWGLVFRYHUHGLGLRPV¶PHDQLQJIURPWKHLUDFWXDO
XVDJH ZDV IOH[LEOH DQG UHSHDWDEOH DQG JHQHUDOL]HG ZHOO WR YDULDWLRQV DQG UHODWHG LGLRPV 7KH
SDWKFKRVHQWRDFFRPSOLVKWKLVJHQHUDOJRDOZDVWRFUHDWHGLVWULEXWLRQDOYHFWRUUHSUHVHQWDWLRQVE\
DGDSWLQJ DQ H[LVWLQJ PHWKRG ² ZRUGYHF¶V VNLSJUDP PHWKRG ² ZLWK DV IHZ FKDQJHV DV
SRVVLEOH
5HVXOWVSURYLGHGKHUHKDYHVKRZQWKDWWKHXVXDOPHWKRGRIUXQQLQJWKHZRUGYHFSURJUDP
RQ D FRUSXV HYHQ D YHU\ ODUJH RQH ZLOO QRW E\ LWVHOI SURGXFH VDWLVIDFWRU\ UHSUHVHQWDWLRQV RI
LGLRPDWLFH[SUHVVLRQV7KHSURSRVHGPHWKRGLQYROYHVVRPHSUHSURFHVVLQJRIDWUDLQLQJFRUSXV
DQGFUHDWHVVHSDUDWHUHSUHVHQWDWLRQVIRUHDFKZRUGZLWKLQWKHLGLRP7KHH[LVWLQJZRUGYHFFRGH
FDQ WKHQ EH UXQ ZLWK QR PRGLILFDWLRQV WR JHQHUDWH WKH ZRUG HPEHGGLQJV :H KDYH IXUWKHU
SURYLGHGDWKRURXJKH[DPLQDWLRQRIWKHK\SHUSDUDPHWHUVWKDWPDNHWKHUHVXOWLQJYHFWRUVPRUH
HIIHFWLYHIRUJLYHQWDVNV$VLWWXUQVRXWUHODWLYHO\ORZGLPHQVLRQDOLW\VPDOOZLQGRZVDQGORZ
VDPSOLQJ WKUHVKROGV KDYH SURYHQ WR ZRUN EHVW IRU GLVDPELJXDWLRQ DQG SUHGLFWLRQ WDVNV ZKLOH
GLPHQVLRQVRIRUPRUHDQGZLQGRZVL]HRIZRUNVEHVWIRUJLVWH[WUDFWLRQ:HKDYHDOVR
VKRZQWKDWVLPSOHYHFWRUDGGLWLRQLVEHVWIRUFRPELQLQJWKHZRUGYHFWRUVWRIRUPWKHLGLRPDWLF
SKUDVH
7KHYHFWRUVFDQEHJHQHUDWHGZLWKLQDQKRXUDQGWDNHOHVVWKDQKDOIDJLJDE\WHRIPHPRU\
2QFHWKHUHSUHVHQWDWLRQVDUHDYDLODEOHFRPSXWLQJZLWKWKHPLVYHU\IDVWGDWDVHWVRIDURXQG
WHVW FDVHV ZHUH SURFHVVHG ZLWKLQ VHFRQGV 0RUH LPSRUWDQWO\ D VXEVWDQWLDO QXPEHU RI
H[SHULPHQWDOUHVXOWVDFURVVWKUHHFRUH1/3WDVNV KDYHVKRZQWKDWWKHVHUHSUHVHQWDWLRQVFDSWXUH
WKH PHDQLQJ RI WKH LGLRP LQ D XVHIXO ZD\ $ PRGHO XVLQJ WKHVH YHFWRUV FDQ DFKLHYH RYHU DFFXUDF\ LQ D SDUDSKUDVH PDWFKLQJ WDVN RQ D SUHGLFWLRQ WDVN DQG QHDU RQ VHYHUDO
GLVDPELJXDWLRQWDVNV,QFRQWUDVWDPRGHOXVLQJUHSUHVHQWDWLRQVWUDLQHGZLWKWKHGHIDXOWPHWKRG
JHWV D VFRUH RQ WKH PDWFKLQJ WDVN DQG RQ WKH SUHGLFWLRQ WDVN 2XU VFRUHV RQ WKH
GLVDPELJXDWLRQ WDVN PDWFK WKRVH RI WKH EHWWHU PRGHOV LQ WKH OLWHUDWXUH KDYLQJ WKH DGGLWLRQDO
DGYDQWDJHRIEHLQJJHQHUDOL]DEOHWRRWKHUWDVNVDVZHOO
2QH RI WKH SURSRVHG PHWKRG¶V JUHDW VWUHQJWKV LV WKDW LW GHDOV ZLWK VOLJKW YDULDWLRQV RI
LGLRPV 7KHVH LQFOXGH LQVWDQFHV ZKHUH D ZRUG LQ WKH LGLRP KDV EHHQ VXEVWLWXWHG IRU DQRWKHU
³IXPEOHWKHEDOO´LQVWHDGRI³GURS´RUDSDUWRIWKHLGLRP DSSHDUVRQLWVRZQEXW VWLOOKDV LWV
ILJXUDWLYH VHQVH DV ZKHQ ³6SLOO´ KDV WKH VDPH PHDQLQJ DV ³VSLOO WKH EHDQV´ %HFDXVH WKH
FRQVWLWXHQWVRIWKHLGLRPHDFKKDYHWKHLURZQUHSUHVHQWDWLRQ DQGDUHSUHVHQWDWLRQHQFDSVXODWHV
WKH FRQWH[WV LQ ZKLFK WKH ZRUG LV XVHG D VHQWHQFH OLNH ³KH IXPEOHG WKH EDOO´ FDQ VWLOO EH
XQGHUVWRRG LQ LWV ILJXUDWLYH VHQVH 7KH IDFW WKDW WKH VLQJOH ZRUG ³EDOO´ KDV YHU\ GLIIHUHQW
PHDQLQJV LQ ³GURS WKH EDOO´ DQG ³KDYH D EDOO´ GRHV QRW SUHYHQW LWV UHSUHVHQWDWLRQ IURP EHLQJ
XVHIXOLQERWKH[SUHVVLRQV
$ORQJ WKH ZD\ VRPH VHFRQGDU\ REMHFWLYHV KDYH EHHQ PHW DV ZHOO 2QH LV ZKDW ZH KDYH
WHUPHGWKH³GHSHQGHQF\SDUVH´UHSUHVHQWDWLRQRIDVHQWHQFHZKLFKLVDVLPSOHPHDQVWRLGHQWLI\
WKHHVVHQWLDOZRUGVLQDVHQWHQFHDQGFRPELQHWKHLUYHFWRUUHSUHVHQWDWLRQVWRIRUPWKHVHQWHQFH¶V
UHSUHVHQWDWLRQ7KLVPHWKRGXVHVWKHVWUXFWXUHRIWKHVHQWHQFHEXWUHPDLQVVLPSOHDQGLWGRHVQRW
UHTXLUH WKH FRPSOH[ PRGHOOLQJ RU KLJKGLPHQVLRQDO WHQVRUV WKDW VRPH RWKHU FRPSRVLWLRQDO
DOJRULWKPVGR,QGHHGVRPHRIWKHUHVXOWVKHUHKDYHRIIHUHGDGGLWLRQDOVXSSRUWWRRWKHUVWXGLHV
HJ0LWFKHOO/DSDWDWKDWIRXQGWKDWYHFWRUDGGLWLRQLVRQHRIWKHPRUHHIIHFWLYHPHDQV
RIFRPSRVLWLRQIRUDYHFWRUVSDFHPRGHO
$VVHVVLQJWKHLGLRP¶VUHSUHVHQWDWLRQUHTXLUHGDVHULHVRIGDWDVHWV$Q\IXWXUHZRUNLQWKLV
DUHDFDQPDNHXVHRIWKHVHDVVHVVPHQWWRROVUHJDUGOHVVRIZKHWKHURUQRWWKRVHUHSUHVHQWDWLRQV
EHDUDQ\UHVHPEODQFHWRRXUV7KHWHVWVHWVFDQEHH[SDQGHGIDLUO\HDVLO\WRLQFOXGHPRUHWHVWVRU
RWKHULGLRPDWLFH[SUHVVLRQV
*(1(5$/2%6(59$7,216
:KDW DVSHFWV RI LGLRPV GR WKHVH UHSUHVHQWDWLRQV DFWXDOO\ FDSWXUH" :KDW FDQ UHDOO\ EH
GHULYHG IURP KDYLQJ D FRPSXWHU SURJUDP LWHUDWLQJ FRPSOHWHO\ XQVXSHUYLVHG WKURXJK D ODUJH
FROOHFWLRQ RI WH[W" 'LVWULEXWLRQDO UHSUHVHQWDWLRQV PRGHO D ZRUG¶V PHDQLQJ E\ HQFRGLQJ WKH
GLVWULEXWLRQV RI WKDW ZRUG¶V WRNHQV LQ LWV FRQWH[WV 7KH UHYLHZ RI SUHYLRXV ZRUN VKRZHG WKDW
FODVVLI\LQJWRNHQVRILGLRPVDVILJXUDWLYHRUOLWHUDOUHOLHGRQFODVVLI\LQJWKHVXUURXQGLQJFRQWH[WV
UHDVRQLQJ WKDW WKH FRQWH[WV RI OLWHUDO XVHV ZLOO FORVHO\ UHVHPEOH RQH DQRWKHU %RWK WKH
FODVVLILFDWLRQDQGVHQWHQFHFRPSOHWLRQWDVNVDOVR FRPSDUHGDQLGLRP¶VUHSUHVHQWDWLRQ WR RQHRI
LWVFRQWH[W%XWLIFRQWH[WZDVWKHVROHIDFWRUUHTXLULQJDWWHQWLRQLWLVDSX]]OHZK\WKHEDVHOLQH
PHWKRGZRXOGQRWKDYHDFKLHYHGEHWWHUUHVXOWVWKDQLWGLG)RUH[DPSOHWKHVHQWHQFHFRPSOHWLRQ
WDVNLQYROYHGGHFLGLQJZKHWKHU³IORRGWKHPDUNHW´RU³WLHWKHNQRW´ZDVWKHEHVWZD\WRFRPSOHWH
DVHQWHQFH,IDOOWKDWZDVQHHGHGWRPDNHWKLVFKRLFHZDVWRNQRZWKDWILQGLQJ³FKHDSLPSRUWHG
JRRGV´VRPHZKHUHLQWKHVHQWHQFHPDNHVWKHILUVWH[SUHVVLRQOLNHO\ZKLOHILQGLQJ³ILDQFpH´LVD
SURPSWWRFKRRVHWKHODWWHUWKHQWKHEDVHOLQHPHWKRGKDGDOORIWKHLQIRUPDWLRQQHHGHGWRPDNH
WKDWGHWHUPLQDWLRQ
7KHIDFWWKDWWKHPHWKRGDGYRFDWHGKHUHDFKLHYHGQHDUO\WZLFHWKHDFFXUDF\RIWKHEDVHOLQH
WHOOV XV ILUVW WKDW WKH PHUH SUHVHQFH RI ZRUGV RU SKUDVHV LQ WKH VXUURXQGLQJ FRQWH[W LV QRW
HQRXJK DQG VHFRQG WKDW WKHVH GLVWULEXWLRQDO UHSUHVHQWDWLRQV FDSWXUH PRUH WKDQ VXFK EDVLF
FRQWH[WXDOLQIRUPDWLRQ2IFRXUVHWKH\DUHGHULYHGIURPFRQWH[WXDOLQIRUPDWLRQ%XWRXUPHWKRG
IRUFHGWKHUHSUHVHQWDWLRQWRHQFRGHWKHZRUGV¶PXWXDOLQIOXHQFHRQHDFKRWKHUDSKHQRPHQRQZH
DOOXGHGWRZKHQFRQVLGHULQJOH[LFDOPHDQLQJLQ6HFWLRQ³)DQ´ZKHQQHDU³IODPHV´GRHVQRW
KDYHWKHVDPHPHDQLQJDVLWGRHVZKHQQHDU³EDVHEDOO´RU³FHLOLQJ´&RQVWUDLQLQJWKHPRGHOVXFK
WKDWLW FRQVLGHUV WKHVH GHSHQGHQFLHVDVWKHSURSRVHGPHWKRGGRHVDOORZVWKHUHSUHVHQWDWLRQ WR
FDSWXUH VRPH DVSHFWV RI PHDQLQJ WKDW JR EH\RQG MXVWWKH FRQWH[WV LQ ZKLFK WKH ZRUG LV IRXQG
7KLVDELOLW\ZHFODLPLVZKDWH[SODLQVWKHEHWWHUVFRUHVWKDWWKHVHUHSUHVHQWDWLRQVDFKLHYHGRQ
WKHJLYHQWDVNV
7KH SUHYLRXVO\ SXEOLVKHG PRGHOV IURP 6HFWLRQ ZKLFK FODVVLILHG WRNHQV DV OLWHUDO RU
ILJXUDWLYHGLGQRWKLQJZKDWVRHYHUWRUHSUHVHQWWKHPHDQLQJRIWKHLGLRPSUHVXPDEO\EHFDXVHLW
ZDV IHOW WKDW VXFK PRGHOOLQJ ZDV XQQHFHVVDU\ :KLOH WKDW DSSURDFK PLJKW ZRUN IRU WKDW
FODVVLILFDWLRQ WDVN ZH ZRXOG DUJXH WKDW DSSOLFDWLRQV VXFK DV 0DFKLQH 7UDQVODWLRQ 6HQWLPHQW
$QDO\VLV RU ,QIRUPDWLRQ 5HWULHYDO ZRXOG KDYH WR WUHDW WKH TXHVWLRQ RI ILJXUDWLYH PHDQLQJ DV D
FHQWUDO FRQFHUQ )RU WKLV UHDVRQ ZH SUHGLFW WKDW UHSUHVHQWDWLRQV VXFK DV WKH RQHV FUHDWHG KHUH
FRXOGLPSURYHXSRQUHVXOWVREWDLQHGXVLQJWKHFRUUHVSRQGHQFHWDEOHVSUHVHQWO\LQXVH
7KHELJJHVWVXUSULVHLQWKLVZRUNZDVWKDWPDQXDOO\DQQRWDWLQJWKHWUDLQLQJFRUSXVDFWXDOO\
GHJUDGHV SHUIRUPDQFH RQ VRPH WDVNV DQG GRHV OLWWOH WR KHOS RQ WKH RWKHUV 7KLV RXWFRPH ZDV
H[DFWO\WKHRSSRVLWHRIZKDWZDVH[SHFWHGRULQWHQGHG ,QPRVWFDVHVWKHUHSUHVHQWDWLRQVZHUH
DEOHWR GRDJRRGMRE RIPRGHOOLQJWKHLGLRPDWLFVHQVHZLWKRXWWKHDQQRWDWLRQV ,W ZRXOGVHHP
WKDWWKHUHSUHVHQWDWLRQVJDLQUREXVWQHVVIURPKDYLQJWRGHDOZLWKDFHUWDLQGHJUHHRIQRLVH
7KHUHDUHRWKHUUHDVRQVWRVXSSRVHWKDWVRPHLPSUHFLVLRQFDQKDYHEHQHILFLDOHIIHFWV:H
IRXQG WKDW XVLQJ WKH VDPH UHSUHVHQWDWLRQ IRU ZRUGV WKDW DUH FRPPRQ WR VHYHUDO LGLRPV KDG D
PL[HGHIIHFWEXWIRUWKHPRVWSDUWZDVEHQHILFLDO$QRWKHUILQGLQJWKDWZHQWDJDLQ H[SHFWDWLRQ
ZDVWKDWORZGLPHQVLRQDOLW\DQGDUHODWLYHO\VPDOOZLQGRZVL]HHQKDQFHSHUIRUPDQFH$JDLQLW
FRXOGEHDTXHVWLRQRIDOORZLQJWKHPRGHOWRFRQFHQWUDWHRQZKDWLVHVVHQWLDOZLWKQXDQFHRQO\
FRQWULEXWLQJ FRQIXVLRQ ,Q D VLPLODU YHLQ LW LV FOHDU WKDW KDYLQJ UHODWLYHO\ IHZ H[DPSOHV LQ WKH
WUDLQLQJFRUSXVFDQEHDQREVWDFOHWREXLOGLQJDJRRGUHSUHVHQWDWLRQEXWLWQHHGQRWEHSURYLGHG
WKDWWKHH[DPSOHVDUHLQIRUPDWLYH
)8785(:25.
7KLVZRUNKDVIRFXVVHGH[FOXVLYHO\RQYHUEQRXQFRQVWUXFWLRQV7KHUHVKRXOGEHQRUHDVRQ
ZK\WKHVDPHDSSURDFKFRXOGQRWEHXVHGIRUPRUHHODERUDWHYHUESKUDVHV³NLFNWKHFDQGRZQ
WKH URDG´ QRXQQRXQ FRQVWUXFWLRQV ³SLH LQ WKH VN\´ RU RWKHU FRQVWUXFWLRQV DOWRJHWKHU 7KH
VDPHWHFKQLTXHVIRUWUDLQLQJDQGWKHQFRPELQLQJWKHUHSUHVHQWDWLRQVZRXOGDSSO\,WZRXOGDOVR
EH LQWHUHVWLQJ WR VHH KRZ ZHOO WKLV DSSURDFK VFDOHV WR ODUJHU VHWV RI LGLRPV SDUWLFXODUO\ LI WKH
LGLRPVVKDUHPRUHDQGPRUHZRUGVXVHGLQGLIIHUHQWVHQVHV
7KHUHLVDJUHDWGHDOPRUHWKDWFRXOGEHGRQHZLWKLGLRPDWLFYDULDWLRQVWKDQZKDWZDVGRQH
KHUH,QWHUSUHWLQJKRZWKHPHWDSKRUZLWKLQWKHYDULDQWDIIHFWVWKHLGLRPDWLFPHDQLQJDVLQ³ULQJ
DIDLQWEHOO´RU³KLWDEULFNZDOO´LVDGLIILFXOWSUREOHPODUJHO\EHFDXVHFRVLQHVLPLODULW\VFRUHV
EHWZHHQ GLVWULEXWHG UHSUHVHQWDWLRQV FDQ WHVW VHPDQWLF UHODWHGQHVV EXW QRW VSHFLILF UHODWLRQV OLNH
³OHVVYLYLG´RU³PRUHGLIILFXOW´$QGKRZZRXOGDPRGHODFFRXQWIRUWKHYHU\GLIIHUHQWPHDQLQJV
RI³KLWWKHFHLOLQJ´DQG³KLWDJODVVFHLOLQJ´"
:KLOH WKH SUHVHQW ZRUN FRXOG QRW SURYH WKDW PDQXDOO\ DQQRWDWLQJ WKH WUDLQLQJ FRUSXV LV
EHQHILFLDOLWFDQQRWUXOHRXWWKHSRVVLELOLW\WKDWLWPLJKWEH2QHRIWKHGLIILFXOWLHVHQFRXQWHUHG
KHUHZDVWKDWRIVFDOHDQQRWDWLRQVRIPRUHH[DPSOHVZHUHOLNHO\QHHGHG0RUHLPSRUWDQWO\WKH
IDFWWKDWHDFKWDVNDOORZHGWZRWHVWVSHULGLRPUHGXFHGDQ\SRWHQWLDOEHQHILWWKDWPLJKWKDYHEHHQ
VHHQ 0RUH H[WHQVLYH GDWD VHWV IRU H[SUHVVLRQV OLNH ³KLW D ZDOO´ ZLWK D ODUJHU QXPEHU RI ERWK
ILJXUDWLYHDQGOLWHUDOXVHVZRXOGKHOSLOOXPLQDWHWKLVTXHVWLRQ
7KH VHQWHQFH FRPSRVLWLRQ WHFKQLTXH LQYROYLQJ WKH GHSHQGHQF\ SDUVH FRXOG EH HQKDQFHG
DQG UHILQHG $W WKH PRPHQW DOO GHSHQGHQF\ UHODWLRQV DUH WUHDWHG WKH VDPH ZD\ 0RUH
H[SHULPHQWDWLRQ FRXOG DVFHUWDLQ ZKLFK UHODWLRQV DUH WKH PRVW LQIRUPDWLYH /HVV LQIRUPDWLYH
UHODWLRQV FRXOG HLWKHU EH LJQRUHG DOWRJHWKHU RU ZHLJKWHG GLIIHUHQWO\ ZLWK WKH ZHLJKWV EHLQJ
GHWHUPLQHGWKURXJKPDFKLQHOHDUQLQJ
7KHVHQWHQFHFRPSOHWLRQGDWDVHWKDVEHHQXVHGRQWKHYHFWRUEDWFKHVEXWQRWRQKXPDQV,W
ZRXOG EH LQWHUHVWLQJ WR VHH KRZ ZHOO QDWLYH (QJOLVK VSHDNHUV ILOO LQ WKH EODQNV IRU WKHVH
VHQWHQFHV,WZRXOGDOVREHEHQHILFLDOWRWHVWRWKHUVHQWHQFHFRPSOHWLRQV\VWHPVRQWKLVGDWDDQG
FRPSDUHWKHLUVFRUHVWRWKRVHREWDLQHGZLWKRXUUHSUHVHQWDWLRQV
,QRXUYLHZWKHPRVWLPSRUWDQWIXWXUHZRUNZRXOGEHWRUHWXUQWRWKHH[DPSOHVJLYHQLQWKH
,QWURGXFWLRQ DQG XVH WKHVH UHSUHVHQWDWLRQV LQ ,QIRUPDWLRQ 5HWULHYDO 6HQWLPHQW $QDO\VLV RU
0DFKLQH7UDQVODWLRQ)RU6HQWLPHQW$QDO\VLVRQHZRXOGQHHGDVHWRILGLRPVWKDWFRQYH\DFOHDU
VHQWLPHQWDQGZKRVHVHQWLPHQWFDQQRWEHSUHGLFWHGIURPWKHLGLRPV¶LQGLYLGXDOZRUGV7HVWLQJ
WKHVHUHSUHVHQWDWLRQVZRXOGIXUWKHUUHTXLUHDGDWDVHWZLWKUHYLHZVXVLQJWKRVHLGLRPV 7UDLQLQJ
YHFWRUVIRU0DFKLQH7UDQVODWLRQ FRXOG EHGRQHE\ XVLQJDELOLQJXDO WUDLQLQJ FRUSXVSUHIHUDEO\
RQHWKDWKDV EHHQOHPPDWL]HGDQGWDJJHGE\SDUW RIVSHHFK ,QGXFLQJWKHYHFWRUUHSUHVHQWDWLRQ
ZRXOGIROORZWKHSURFHGXUHVRXWOLQHGWKHUH)RUERWKDSSOLFDWLRQVWKHQHZFKDOOHQJHZRXOGEH
GLVFRYHULQJWKHPDSSLQJEHWZHHQDQLGLRPDQGWKHVHQWLPHQWLWFRQYH\VRULWVHTXLYDOHQWLQWKH
RWKHUODQJXDJH,WLVDQRSHQHPSLULFDOTXHVWLRQDVWRZKHWKHUVXFKDPDSSLQJFDQEHGLVFRYHUHG
IURPWKHGDWDDORQHRUZKHWKHUVRPHIRUPRIKXPDQLQWHUYHQWLRQZRXOGEHUHTXLUHG
7KH VWXG\ RI KRZ WR XVH D ZRUG¶V GLVWULEXWLRQ WR PRGHO LWV VHPDQWLF SURSHUWLHV LV RIWHQ
UHIHUUHG WR DV ³GLVWULEXWLRQDO VHPDQWLFV´ 7KLV ZRUN LV DQ H[DPSOH RI ZKDW PLJKW EH WHUPHG
³GLVWULEXWLRQDO SUDJPDWLFV´ :KHUHDV VHPDQWLFV LV FRQFHUQHG ZLWK ZKDW LV DFWXDOO\ VDLG
SUDJPDWLFVLVFRQFHUQHGZLWKZKDWLVPHDQWWKDWLVKRZWKDWVHQWHQFHRUWH[WLVXVHGWRFRQYH\
PHDQLQJ LQ D SDUWLFXODU FRQWH[W 'HFLGLQJ LI D JLYHQ LQVWDQFH RI D YHUEQRXQ SKUDVH LV PHDQW
OLWHUDOO\ RU ILJXUDWLYHO\ LV D JRRG H[DPSOH RI SUDJPDWLFV 0LJKW WKH PHWKRGV RXWOLQHG LQ WKLV
GLVVHUWDWLRQ EH XVHG IRU RWKHU FRPSXWDWLRQDO SUDJPDWLFV DSSOLFDWLRQV EH\RQG LGLRPV" )RU
H[DPSOH LW PD\ EH SRVVLEOH WR GHWHFW VDUFDVP E\ GRLQJ ZKDW ZH GLG KHUH IRU LGLRPV FUHDWH D
UHSUHVHQWDWLRQ RI D FRQYHQWLRQDO VDUFDVWLF SKUDVH ³1LFH JRLQJ´ DQG FRPSDUH LWV ILW ZLWK D
VHQWHQFHWRWKDWRIWKHVDPHSKUDVHXVHGOLWHUDOO\$QRWKHUH[DPSOHLVVSHHFKDFWLQWHUSUHWDWLRQD
WDVNFRQFHUQHGZLWKFODVVLI\LQJDQXWWHUDQFHDVDVWDWHPHQWDSURPLVHDFRPPDQGDTXHVWLRQ
DQGVRRQ7KLVWDVNLVFRPSOLFDWHGLQSUDFWLFHEHFDXVHDTXHVWLRQOLNH³$UH\RXIUHHWRPRUURZ"´
PLJKWDFWXDOO\EHDQLQYLWDWLRQRUHYHQDFRPPDQG*LYHQWKHFRQYHQWLRQDOL]HGQDWXUHRIWKHVH
LQGLUHFW VSHHFK DFWV ZH FRXOG HQYLVLRQ FUHDWLQJ D YHFWRU UHSUHVHQWDWLRQ IRU HDFK LQWHUSUHWDWLRQ
DQGWKHQDVVHVVLQJZKLFKYHUVLRQILWVEHWWHULQ DJLYHQFRQWH[W7KHVHDUHMXVWWZRH[DPSOHVRI
KRZ WKH DSSURDFK GHVFULEHG KHUH FDQ SRWHQWLDOO\ EH JHQHUDOL]HG WR GHSOR\ GLVWULEXWLRQDO
UHSUHVHQWDWLRQVLQQHZZD\VWKDWLQFOXGHQRWRQO\LGLRPVEXWPDQ\IRUPVRIQRQOLWHUDOODQJXDJH
10 References
$QDVWDVLRX',GLRPWUHDWPHQWH[SHULPHQWVLQ0DFKLQH7UDQVODWLRQ3K'WKHVLV
%LUG6/RSHU(.OHLQ(1DWXUDO/DQJXDJH3URFHVVLQJZLWK3\WKRQ%RVWRQ0$
2¶5HLOO\0HGLD,QF
%HQJLR < 'XFKDUPH 5 9LQFHQW 3 -DXYLQ & $ QHXUDO SUREDELOLVWLF ODQJXDJH
PRGHO-RXUQDORI0DFKLQH/HDUQLQJ5HVHDUFK
%LFNHO 6 +DLGHU 3 6FKHIIHU 7 /HDUQLQJ WR FRPSOHWH VHQWHQFHV ,Q - *DPD 5
&DPDFKR 3 %UD]GLO $ -RUJH / 7RUJR (GV 0DFKLQH /HDUQLQJ (&0/ SS
%1&&RQVRUWLXP7KH%ULWLVK1DWLRQDO&RUSXVYHUVLRQ%1&;0/(GLWLRQ
&KHOED&0LNRORY76FKXVWHU0*H4%UDQWV7.RHKQ35RELQVRQ72QH
ELOOLRQZRUGEHQFKPDUNIRUPHDVXULQJSURJUHVVLQVWDWLVWLFDOODQJXDJHPRGHOLQJ5HWULHYHG
IURP KWWSZZZVWDWPWRUJOPEHQFKPDUNELOOLRQZRUGODQJXDJHPRGHOLQJEHQFKPDUN
URXWSXWWDUJ]
&KHQ'0DQQLQJ&'$IDVWDQGDFFXUDWHGHSHQGHQF\SDUVHUXVLQJQHXUDOQHWZRUNV
3URFHHGLQJVRI(01/3
&KRODNRY . .RUGRQL 9 %HWWHU 6WDWLVWLFDO 0DFKLQH 7UDQVODWLRQ WKURXJK OLQJXLVWLF
WUHDWPHQWRISKUDVDOYHUEV,Q3URFHHGLQJVRIWKH&RQIHUHQFHRQ(PSLULFDO0HWKRGV
LQ1DWXUDO/DQJXDJH3URFHVVLQJ(01/3±
&ODSSHU%0XQNUHV2EWDLQHGIURPKWWSVS\SLS\WKRQRUJS\SLPXQNUHV
&ODUN69HFWRUVSDFHPRGHOVRIOH[LFDOPHDQLQJ ,Q6/DSSLQ&)R[(GV7KH
+DQGERRN
RI
FRQWHPSRUDU\
VHPDQWLF
WKHRU\
QG
HG
5HWULHYHG
IURP
KWWSZZZFOFDPDFXN(VFSXEVVHPBKDQGERRNSGI
&RRN 3 )D]O\ $ 6WHYHQVRQ 6 7KH 91&WRNHQV GDWDVHW ,Q 3URFHHGLQJV RI WKH
/5(&:RUNVKRS7RZDUGVD6KDUHG7DVNIRU0XOWLZRUG([SUHVVLRQV0:(±
&UXVH$0HDQLQJLQODQJXDJH$QLQWURGXFWLRQWR6HPDQWLFVDQG3UDJPDWLFVUGHG
1HZ<RUN2[IRUG8QLYHUVLW\3UHVV
'DYLHV 0 7KH &RUSXV RI &RQWHPSRUDU\ $PHULFDQ (QJOLVK PLOOLRQ ZRUGV
'DYLHV07KH:LNLSHGLD&RUSXVPLOOLRQDUWLFOHVELOOLRQZRUGV$GDSWHGIURP
:LNLSHGLD
(YDQV9+RZZRUGVPHDQ1HZ<RUN2[IRUG8QLYHUVLW\3UHVV
)D]O\ $ &RRN 3 6WHYHQVRQ 6 8QVXSHUYLVHG W\SH DQG WRNHQ LGHQWLILFDWLRQ RI
LGLRPDWLFH[SUHVVLRQV&RPSXWDWLRQDO/LQJXLVWLFV±
)HOOEDXP & D 7RZDUGV D UHSUHVHQWDWLRQ RI LGLRPV RQ :RUG1HW ,Q 3URFHHGLQJV RI WKH
:RUNVKRSRQWKH8VHRI:RUG1HWLQ1DWXUDO/DQJXDJH3URFHVVLQJ6\VWHPV &ROLQJ$&/
)HOOEDXP&E:RUG1HW$Q(OHFWURQLF/H[LFDO'DWDEDVH&DPEULGJH0$0,73UHVV
)HOOEDXP&(G,GLRPVDQGFROORFDWLRQV1HZ<RUN&RQWLQXXP
)LUWK -5 $ V\QRSVLV RI OLQJXLVWLF WKHRU\ ± ,Q 3DOPHU )5 (G 6HOHFWHG
3DSHUVRI-5)LUWK%ORRPLQJWRQ,QGLDQD8QLYHUVLW\3UHVV
)RGRU-/HSRUH(7KHFRPSRVLWLRQDOLW\SDSHUV7RURQWR2[IRUG8QLYHUVLW\3UHVV
*ROGEHUJ < /HY\ 2 :RUGYHF H[SODLQHG 'HULYLQJ 0LNRORY HW DO¶V QHJDWLYH
VDPSOLQJZRUGHPEHGGLQJPRGHODU;LYSUHSULQWDU;LY
*RRJOHQG:RUGYHF5HWULHYHGIURPKWWSVFRGHJRRJOHFRPSZRUGYHF
*RRJOHQG*RRJOH1HZV9HFWRUV5HWULHYHGIURP
KWWSVGULYHJRRJOHFRPILOHG%;N&ZS,.'<1O1877O66S4P0HGLW"XVS VKDULQJ
*UHIHQVWHWWH * ([SORUDWLRQV LQ DXWRPDWLF WKHVDXUXV GLVFRYHU\ 1RUZHOO 0$ .OXZHU
$FDGHPLF3XEOLVKHUV
*ULIILWKV 7/ 6WH\YHUV 0 7HQHQEDXP -% 7RSLFV LQ VHPDQWLF UHSUHVHQWDWLRQ
3V\FKRORJLFDO5HYLHZ
-XUDIDVN\' 0DUWLQ-+ 6SHHFKDQGODQJXDJHSURFHVVLQJ QGHG 8SSHU6DGGOH
5LYHU1-3HDUVRQ3UHQWLFH+DOO
.DW] * *LHVEUHFKW ( $XWRPDWLF LGHQWLILFDWLRQ RI QRQFRPSRVLWLRQDO PXOWLZRUG
H[SUHVVLRQVXVLQJ/DWHQW6HPDQWLF$QDO\VLV,Q3URFHHGLQJVRIWKH:RUNVKRSRQ0XOWLZRUG
([SUHVVLRQV,GHQWLI\LQJDQG([SORLWLQJ8QGHUO\LQJ3URSHUWLHV±
.LQWVFK:0HWDSKRUFRPSUHKHQVLRQ$FRPSXWDWLRQDOWKHRU\3V\FKRQRPLF%XOOHWLQ
5HYLHZ
.LQWVFK:+RZWKHPLQGFRPSXWHVWKHPHDQLQJRIPHWDSKRU$VLPXODWLRQEDVHGRQ
/6$ ,Q 5: *LEEV(G7KH&DPEULGJHKDQGERRNRI PHWDSKRUDQGWKRXJKWSS
1HZ<RUN&DPEULGJH8QLYHUVLW\3UHVV
/DNRII*-RKQVRQ00HWDSKRUVZHOLYHE\&KLFDJR8QLYHUVLW\RI&KLFDJR3UHVV
/DPEHUW 3 %DQFKV 5 'DWD LQIHUUHG PXOWLZRUG H[SUHVVLRQV IRU 6WDWLVWLFDO 0DFKLQH
7UDQVODWLRQ ,Q 3URFHHGLQJV RI 0DFKLQH 7UDQVODWLRQ 6XPPLW ; 6HS 3KXNHW
7KDLODQG
/DQGDXHU 7. 'XWQDLV 67 $ VROXWLRQ WR 3ODWR¶V SUREOHP 7KH ODWHQW VHPDQWLF
DQDO\VLV WKHRU\ RI DFTXLVLWLRQ LQGXFWLRQ DQG UHSUHVHQWDWLRQ RI NQRZOHGJH 3V\FKRORJLFDO
5HYLHZ
/DQJORW] $ ,GLRPDWLF FUHDWLYLW\ $ FRJQLWLYHOLQJXLVWLF PRGHO RI LGLRPUHSUHVHQWDWLRQ
DQGLGLRPYDULDWLRQLQ(QJOLVK3KLODGHOSKLD-RKQ%HQMDPLQV
/H 4 0LNRORY 7 'LVWULEXWHG UHSUHVHQWDWLRQV RI VHQWHQFHV DQG GRFXPHQWV ,Q
3URFHHGLQJVRIWKHVW,QWHUQDWLRQDO&RQIHUHQFHRQ0DFKLQH/HDUQLQJ
/LEEHQ057LWRQH'$7KHPXOWLGHWHUPLQHGQDWXUHRILGLRPSURFHVVLQJ0HPRU\
&RJQLWLRQ
/RHQQHNHU5RGPDQ % 1DUD\DQDQ 6 &RPSXWDWLRQDO DSSURDFKHV WR ILJXUDWLYH
ODQJXDJH ,Q 0 6SLYH\ 0 -RDQQLVVH . 0F5DH (GV &DPEULGJH +DQGERRN RI
3V\FKROLQJXLVWLFVSS1HZ<RUN&DPEULGJH8QLYHUVLW\3UHVV
0DQQLQJ&'6FKXW]H+)RXQGDWLRQVRI6WDWLVWLFDO1DWXUDO/DQJXDJH3URFHVVLQJ
&DPEULGJH0$0,73UHVV
0LNRORY 7 &KHQ . &RUUDGR * 'HDQ - D (IILFLHQW HVWLPDWLRQ RI ZRUG
UHSUHVHQWDWLRQVLQYHFWRUVSDFH,Q3URFHHGLQJVRI:RUNVKRSLQ,&/5
0LNRORY 7 6XWVNHYHU , &KHQ . &RUUDGR * 6 'HDQ - E 'LVWULEXWHG
UHSUHVHQWDWLRQV RI ZRUGV DQG SKUDVHV DQG WKHLU FRPSRVLWLRQDOLW\ $GYDQFHV LQ 1HXUDO
,QIRUPDWLRQ3URFHVVLQJ6\VWHPV
0LOOHU *$ :RUG1HW $ OH[LFDO GDWDEDVH IRU (QJOLVK &RPPXQLFDWLRQV RI WKH $&0
0LOOHU * &KDUOHV : &RQWH[WXDO FRUUHODWHV RI VHPDQWLF VLPLODULW\ /DQJXDJH DQG
&RJQLWLYH3URFHVVHV±
0LWFKHOO-/DSDWD0&RPSRVLWLRQLQGLVWULEXWLRQDOPRGHOVRIVHPDQWLFV &RJQLWLYH
6FLHQFH±
0RRQ5)L[HGH[SUHVVLRQVDQGLGLRPVLQ(QJOLVK7RURQWR2[IRUG8QLYHUVLW\3UHVV
0XQNUHV -$OJRULWKPV IRUWKHDVVLJQPHQW DQGWUDQVSRUWDWLRQSUREOHPV -RXUQDORI WKH
6RFLHW\IRU,QGXVWULDODQG$SSOLHG0DWKHPDWLFV±
1XQEHUJ*6DJ,$:DVRZ7,GLRPV/DQJXDJH
3DO61DVNDU6.%DQG\RSDGK\D\6$K\EULGZRUGDOLJQPHQWPRGHOIRUSKUDVH
EDVHG6WDWLVWLFDO0DFKLQH7UDQVODWLRQ,Q3URFHHGLQJVRIWKH6HFRQG:RUNVKRSRQ+\EULG
$SSURDFKHVWR7UDQVODWLRQ±
3HFLQD 3 $ PDFKLQH OHDUQLQJ DSSURDFK WR PXOWLZRU[ H[SUHVVLRQ H[WUDFWLRQ ,Q
3URFHHGLQJV RI WKH /5(& :RUNVKRS 7RZDUGV D 6KDUHG 7DVN IRU 0XOWLZRUG
([SUHVVLRQV
3HGUHJRVD)HWDO6FLNLWOHDUQ-RXUQDORI0DFKLQH/HDUQLQJ5HVHDUFK
3HQJ-)HOGPDQ$9\ORPRYD(&ODVVLI\LQJLGLRPDWLFDQGOLWHUDOH[SUHVVLRQVXVLQJ
WRSLFPRGHOVDQGLQWHQVLW\RIHPRWLRQV3URFHHGLQJVRIWKH&RQIHUHQFHRQ(PSLULFDO
0HWKRGVLQ1DWXUDO/DQJXDJH3URFHVVLQJ(01/3
5HVQLN36HOHFWLRQDQG,QIRUPDWLRQ$&ODVV%DVHG$SSURDFKWR/H[LFDO5HODWLRQVKLSV
'RFWRUDO 'LVVHUWDWLRQ 'HSDUWPHQW RI &RPSXWHU DQG ,QIRUPDWLRQ 6FLHQFH 8QLYHUVLW\ RI
3HQQV\OYDQLD
5RQJ;:RUGYHFSDUDPHWHUOHDUQLQJH[SODLQHGDU;LYSUHSULQWDU;LY
5XVVHOO6:)LVFKHU,'RUPH\HU57KHFDWDQGWKHEURFDGHGEDJ8VLQJPHWDSKRU
DQDO\VLV WR FRPSXWDWLRQDOO\ SURFHVV FUHDWLYHO\ PRGLILHG LGLRPV ,Q % 6KDUS 0 =RFN
(GV 3URFHHGLQJV RI WKH WK ,QWHUQDWLRQDO :RUNVKRS RQ 1DWXUDO /DQJXDJH 3URFHVVLQJ
DQG&RJQLWLYH6FLHQFH
6DKOJUHQ07KHGLVWULEXWLRQDOK\SRWKHVLV5LYLVWDGL/LQJXLVWLFD
6DOWRQ * 5RVV 5 .HOOHKHU - (YDOXDWLRQ RI D VXEVWLWXWLRQ PHWKRG IRU LGLRP
WUDQVIRUPDWLRQ LQ 6WDWLVWLFDO 0DFKLQH 7UDQVODWLRQ ,Q 3URFHHGLQJV RI WKH :RUNVKRS RQ
0XOWLZRUG ([SUHVVLRQV 0:( DW WK &RQIHUHQFH RI WKH (XURSHDQ &KDSWHU RI WKH
$VVRFLDWLRQIRU&RPSXWDWLRQDO/LQJXLVWLFV
6DQJDWL ) YDQ &UDQHQEXUJK $ 0XOWLZRUG H[SUHVVLRQ LGHQWLILFDWLRQ ZLWK UHFXUULQJ
WUHHIUDJPHQWVDQGDVVRFLDWLRQPHDVXUHV,Q3URFHHGLQJVRI1$$&/+/7
6FKXOGHU0+RYH\(0HWDSKRUGHWHFWLRQWKURXJKWHUPUHOHYDQFH,Q3URFHHGLQJVRIWKH
6HFRQG:RUNVKRSRQ0HWDSKRULQ1/3±
6HJDO'0D\7LPHWRKLWWKHEUDNHVRQWKDWFOLFKp:DVKLQJWRQ3RVWS&5HWULHYHG
IURPKWWSZZZZDVKLQJWRQSRVWFRPZS
G\QFRQWHQWDUWLFOH$5KWPO
6KXWRYD ( D $XWRPDWLF PHWDSKRU LQWHUSUHWDWLRQ DV D SDUDSKUDVLQJ WDVN ,Q +XPDQ
/DQJXDJH7HFKQRORJLHV7KH$QQXDO&RQIHUHQFHRIWKH1RUWK$PHULFDQ&KDSWHURI
WKH$&/
6KXWRYD(E0RGHOVRIPHWDSKRULQ1/3,Q3URFHHGLQJVRIWKHWK$QQXDO0HHWLQJRI
WKH$VVRFLDWLRQIRU&RPSXWDWLRQDO/LQJXLVWLFV
6KXWRYD ( 9DQ GH &UX\V 7 .RUKRQHQ $ 8QVXSHUYLVHG PHWDSKRU SDUDSKUDVLQJ
XVLQJDYHFWRUVSDFHPRGHO,Q3URFHHGLQJVRI&2/,1*3RVWHUV
6RFKHU 5 3HUHO\JLQ $ :X - &KXDQJ - 0DQQLQJ &' 1J $ 3RWWV & 5HFXUVLYH GHHS PRGHOV IRU VHPDQWLF FRPSRVLWLRQDOLW\ RYHU D VHQWLPHQW WUHHEDQN ,Q
&RQIHUHQFHRQ(PSLULFDO0HWKRGVLQ1DWXUDO/DQJXDJH3URFHVVLQJ(01/3
6SRUOHGHU&/L/8QVXSHUYLVHGUHFRJQLWLRQRIOLWHUDODQGQRQOLWHUDOXVHRILGLRPDWLF
H[SUHVVLRQV ,Q 3URFHHGLQJVRI WKHWK&RQIHUHQFHRI WKH(XURSHDQ&KDSWHURI WKH$&/
7LNX11HZWRQ&)HEUXDU\7ZLWWHU&(2µ:HVXFNDWGHDOLQJZLWKDEXVH¶7KH
9HUJH 5HWULHYHG IURP KWWSZZZWKHYHUJHFRPWZLWWHUFHRVHQW
PHPRWDNLQJSHUVRQDOUHVSRQVLELOLW\IRUWKHRQ)HEUXDU\
7LWRQH '$ &RQQLQH &0 2Q WKH FRPSRVLWLRQDO DQG QRQFRPSRVWLRQDO QDWXUH RI
LGLRPDWLFH[SUHVVLRQV-RXUQDORI3UDJPDWLFV
7LWRQH'$ /LEEHQ 07LPHGHSHQGHQW HIIHFWV RIGHFRPSRVDELOLW\IDPLOLDULW\DQG
OLWHUDOSODXVLELOLW\RQLGLRPPHDQLQJDFWLYDWLRQ7KH0HQWDO/H[LFRQ
7XUQH\ 3' 3DQWHO 3 )URP IUHTXHQF\ WR PHDQLQJ 9HFWRU VSDFH PRGHOV RI
VHPDQWLFV-RXUQDORI$UWLILFLDO,QWHOOLJHQFH5HVHDUFK
8WVXPL $ &RPSXWDWLRQDO H[SORUDWLRQ RI PHWDSKRU FRPSUHKHQVLRQ SURFHVVHV XVLQJ D
VHPDQWLFVSDFHPRGHO&RJQLWLYH6FLHQFH
9DQ GH &UX\V 7 3RLEHDX 7 .RUKRQHQ $ /DWHQW YHFWRU ZHLJKWLQJ IRU ZRUG
PHDQLQJ LQ FRQWH[W ,Q 3URFHHGLQJV RI WKH &RQIHUHQFH RQ (PSLULFDO 0HWKRGV LQ
1DWXUDO/DQJXDJH3URFHVVLQJ±
:XOII65HWKLQNLQJLGLRPDWLFLW\$XVDJHEDVHGDSSURDFK1HZ<RUN&RQWLQXXP
=ZHLJ * %XUJHV &-& $ FKDOOHQJH VHW IRU DGYDQFLQJ ODQJXDJH PRGHOLQJ ,Q
:RUNVKRSRQWKH)XWXUHRI/DQJXDJH0RGHOLQJIRU+/71$$&/+/7
11 Appendices
$/,672),',20686(',17+,6:25.
1RWHWKDWWKHWRNHQVLQ&2&$DUHDOOWRNHQVLQWKHFDQRQLFDOIRUPQRWRQO\WKHILJXUDWLYHRQHV
Idiom
bear one’s cross
bite one's lip
bite one's tongue
bite the bullet
bite the dust
blow a fuse
blow the whistle
break someone's heart
break a leg
break the ice
bury the hatchet
buy the farm
call the shots
call the tune
change one's tune
chew the fat
clear the air
clear the deck
cover one's tracks
crack a joke
crack a smile
crack the whip
drop a bomb
drop a line
drop a bundle
drop the ball
face the music
fan the flames
flood the market
# tokens in COCA
37
848
306
157
104
29
509
1212
164
212
44
59
537
44
160
34
239
70
188
53
108
92
259
63
15
294
109
159
120
Idiom
follow one's nose
force someone's hand
get the axe/ax
get the message
get the sack
have a ball
hit a wall
hit the books
hit the ceiling
hit the hay
hit the road
hit the roof
hit the sack
hold the fort
kick the habit
kick the bucket
know the score
lick one's wounds
lose one's grip
lose one's head
lose one's marbles
lose one's mind
lose one's nerve
lose one's shirt
lose the thread
lose one's touch
make a bundle
make a scene
miss the boat
miss the mark
pack one's bags
pass the buck
pave the way
play ball
pull someone's leg
pull the plug
ring a bell
scratch one's head
shoot the breeze
shoot the messenger
# tokens in COCA
50
132
51
1140
7
323
136
73
48
24
702
61
57
48
121
37
96
134
313
192
48
792
152
102
56
97
76
153
169
179
473
107
1485
679
193
624
243
681
90
56
Idiom
show one's cards
show one's face
show one's stuff
smack one's lips
smell a rat
spill the beans
spread the word
straddle the fence
take heart
tie the knot
trim one's sails
turn the tables
use one's head
wear the pants
# tokens in COCA
31
340
79
156
52
115
593
33
324
259
28
362
160
49
%0$18$//<$1127$7(',',206
$VH[SODLQHGLQ6HFWLRQIRU0HWKRGWKHDXWKRUUHDQQRWDWHGWKHWUDLQLQJFRUSXVIRU
WKH LGLRPV OLVWHG EHORZ VXFK WKDW RQO\ WKH WUXO\ ILJXUDWLYH XVHV ZHUH DQQRWDWHG DV EHLQJ
ILJXUDWLYH7KHQXPEHURIWRNHQVLVWKHVDPHDVWKHQXPEHUUHSRUWHGLQ$SSHQGL[$DERYH7KH
ULJKWPRVWFROXPQJLYHVWKHQXPEHUOHIWDIWHUWKHPDQXDODQQRWDWLRQZDVFRPSOHWH
Idiom
blow a fuse
break the ice
buy the farm
change gears
drop a bomb
drop a line
hit a wall
lose one's marbles
lose one's shirt
miss the mark
ring a bell
tie the knot
trim one's sails
wear the pants
# tokens retagged
as literal
11
21
33
42
259
41
35
3
8
8
64
18
10
16
# tokens tagged
as figurative
18
191
26
61
60
22
101
45
94
171
179
241
18
33
&(;$03/(6(17(1&(6)5207+(),*85$7,9(/,7(5$/6(17(1&(
3$,5'$7$6(7
7KHVHVHQWHQFHSDLUVLQWKLVVHWDUHVLPSOLILHGYHUVLRQVRIWKRVHGHYHORSHGE\3URI7LWRQH¶V
ODE6HH6HFWLRQIRUDFRPSOHWHGHVFULSWLRQ
x 3HQHORSHKLWWKHERRNVZLWKKHUKDQGZKHQVKHVDWGRZQWRRTXLFNO\DWKHUGHVN
x 3HQHORSHKLWWKHERRNVEHIRUHWKHGUHDGHGILQDOH[DP
x 5XSHUWIORRGHGWKHPDUNHWZKLOHIL[LQJWKHOHDNLQWKHVWRUHSOXPELQJ
x 5XSHUWIORRGHGWKHPDUNHWZLWKKLVDEXQGDQWVR\EHDQFURSODVW\HDU
x &LQG\ZRUHWKHSDQWVDIWHUGHFLGLQJLWZDVWRRFROGWRZHDUDVNLUW
x &LQG\ZRUHWKHSDQWVLQKHUIDPLO\WKHZHHNDIWHUKHUIDWKHUGLHG
'&203/(7(5(68/76)25&203$5,621:,7+)$=/<(7$/
7KHIROORZLQJDUHWKHEHVWSUHFLVLRQDQGUHFDOOVFRUHVUHSRUWHGE\)D]O\HWDOIRUWKHLUWKUHH
PRGHOVRQQLQHLGLRPV7KH)VFRUHVZHUHFDOFXODWHGXVLQJWKRVHVFRUHV7KHLUVFRUHVDUH
IROORZHGE\VDPHWKUHHVFRUHVREWDLQHGE\RXUPRGHODYHUDJHGRYHUUXQV
literal instances
figurative instances
Fazly et al.
Idiom
Prec
Recall
take heart
hit roof
hit road
lose head
make scene
get sack
blow whistle
pull leg
pull plug
take heart
hit roof
hit road
lose head
make scene
get sack
blow whistle
pull leg
pull plug
1.00
0.65
0.80
0.56
0.66
0.86
0.78
0.75
0.90
1.00
0.00
0.00
0.50
0.86
0.00
0.78
0.88
0.94
0.97
1.00
1.00
0.90
0.96
1.00
0.52
0.55
0.98
0.60
0.00
0.00
0.12
0.30
0.00
0.92
0.95
0.75
target method
F1
Prec
Recall
0.98
0.79
0.89
0.69
0.78
0.92
0.62
0.63
0.94
0.75
0.00
0.00
0.19
0.44
0.00
0.84
0.91
0.83
1.00
0.78
0.90
0.71
0.85
0.98
0.67
0.37
0.91
0.74
0.35
1.00
0.65
0.82
0.96
0.28
0.77
0.98
0.96
0.87
0.84
0.93
0.99
0.97
0.93
0.71
0.93
1.00
0.25
0.14
0.38
0.23
0.48
1.00
0.83
0.89
F1
0.98
0.82
0.86
0.80
0.91
0.97
0.77
0.48
0.91
0.85
0.29
0.25
0.47
0.36
0.64
0.44
0.80
0.94
(&203/(7(5(68/76)25&203$5,621:,7+3(1*(7$/
7KH IROORZLQJ DUH WKH EHVW SUHFLVLRQ DQG UHFDOO VFRUHV UHSRUWHG E\ 3HQJ HW DO IRU WKHLU WZR
PRGHOVIROORZHGE\RXUVFRUHV
Idiom
blow whistle
take heart
lose head
make scene
Prec
0.62
0.93
0.76
0.80
Peng et al.
Recall
0.60
0.99
0.97
0.99
Acc
0.83
0.92
0.78
0.80
Target method
Prec
Recall
0.61
0.93
1.00
0.97
0.74
0.93
0.85
0.97
Acc
0.87
0.97
0.75
0.83
)(;$03/(6(17(1&(6)5207+(6(17(1&(&203/(7,217$6.
5HGEHUJ VDLG WKH PHWDDQDO\VLV ZDV PRUH HYLGHQFH WKDW WKH )'$ KDG BBBBBBBBBB E\
IDLOLQJWRUHTXLUHPRUHVH[VSHFLILFGDWDRQERWKGUXJVDQGGHYLFHV
x GURSSHGWKHEDOO
x WDNHQKHDUW
x FUDFNHGDVPLOH
x IORRGHGWKHPDUNHW
<RXPLJKWWKLQNKHZDVBBBBBBBBBBDQGZKRFRXOGEODPH\RXLWLVTXLWHDFODLP
x SXOOLQJ\RXUOHJ
x W\LQJWKHNQRW
x FKDQJLQJJHDU
x FUDFNLQJWKHZKLS
7KH/LEHUDO'HPRFUDWOHDGHU±BBBBBBBBBBIRU3ULPH0LQLVWHU'DYLG&DPHURQZKLOHKHLV
RQ KROLGD\ ± DGPLWWHG WKDW KH KDG HQMR\HG D OXFN\ XSEULQJLQJ ZLWK VXSSRUWLYH SDUHQWV
ZKRZHUHDEOHWRVHQGKLPWRIHHSD\LQJ:HVWPLQVWHU6FKRRO
x PDNLQJDVFHQH
x KROGLQJWKHIRUW
x KLWWLQJDZDOO
x IDQQLQJWKHIODPHV
*$'',7,21$/5(68/76)257+(6(17(1&(&203/(7,217$6.
7KHIROORZLQJWDEOHSURYLGHVWKHVDPHGDWDDV 7DEOHH[FHSWWKDWLQVWHDGRIUHSRUWLQJWKH
UHVXOWVRIWKHILYHEHVWSHUIRUPLQJYHFWRUEDWFKHVLWUHSRUWVWKHILYHEHVWSHUIRUPHUVDPRQJWKH
EDWFKHVWUDLQHGRQWKHPDQXDOO\DQQRWDWHGFRUSXV$VEHIRUH73WUXHSRVLWLYHVLVWKHQXPEHU
RI FRUUHFW DQVZHUV JLYHQ IRU WKH JLYHQ LGLRP RXW RI ZKLOH )3 IDOVH SRVLWLYHV LV WKH
QXPEHURIWLPHVWKHPRGHOLQFRUUHFWO\FKRVHWKHJLYHQLGLRP7KHODVW FROXPQJLYHVWKH)
VFRUH UHSRUWHG LQ 7DEOH IRU FRPSDULVRQ SXUSRVHV 5RZV KLJKOLJKWHG LQ JUH\ DUH WKH LGLRPV ZKLFK ZHUH DQQRWDWHG PDQXDOO\ 6HH 6HFWLRQ (PSLULFDO 5HVXOWV IRU D GLVFXVVLRQ RI
WKHVHUHVXOWV
idiom
bear cross
bite bullet
bite dust
bite tongue
blow fuse
blow whistle
break heart
break ice
bury hatchet
buy farm
call shot
change gear
change tune
chew fat
clear air
cover track
crack smile
crack whip
drop ball
drop bomb
drop bundle
drop line
face music
fan flame
flood market
TP
4
5
9
10
6
6
6
5
5
6
7
9
6
10
5
4
10
0
2
0
10
7
10
9
10
manual annotation
FP precision recall
5
0.444 0.400
6
0.455 0.500
3
0.750 0.900
5
0.667 1.000
2
0.750 0.600
5
0.545 0.600
1
0.857 0.600
3
0.625 0.500
13
0.278 0.500
3
0.667 0.600
0
1.000 0.700
0
1.000 0.900
1
0.857 0.600
2
0.833 1.000
1
0.833 0.500
0
1.000 0.400
2
0.833 1.000
0
0.000 0.000
0
1.000 0.200
1
0.000 0.000
9
0.526 1.000
0
1.000 0.700
2
0.833 1.000
0
1.000 0.900
5
0.667 1.000
F1 auto-tag F1
0.421
0.435
0.476
0.526
0.818
0.667
0.800
0.769
0.667
0.769
0.571
0.556
0.706
0.706
0.556
0.824
0.357
0.476
0.632
0.000
0.824
0.824
0.947
0.947
0.706
0.667
0.909
0.909
0.625
0.667
0.571
0.588
0.909
1.000
0.000
0.333
0.333
0.462
0.000
0.000
0.690
0.720
0.824
0.462
0.909
0.833
0.947
1.000
0.800
0.800
idiom
force hand
get ax
get sack
have ball
hit book
hit hay
hit road
hit sack
hit wall
hold fort
kick bucket
kick habit
know score
lose marble
lose shirt
make scene
miss boat
miss mark
pass buck
pave way
play ball
pull leg
pull plug
ring bell
shoot breeze
shoot messenger
show card
smell rat
spill bean
straddle fence
take heart
tie knot
trim sail
turn table
wear pant
AVERAGES
TP
8
9
7
9
10
9
10
10
5
2
4
10
7
9
10
10
7
1
10
9
5
8
10
10
7
8
7
5
5
5
5
8
7
10
10
7.12
manual annotation
FP precision recall
0
1.000 0.800
7
0.563 0.900
13
0.350 0.700
4
0.692 0.900
0
1.000 1.000
4
0.692 0.900
2
0.833 1.000
11
0.476 1.000
8
0.385 0.500
0
1.000 0.200
9
0.308 0.400
9
0.526 1.000
1
0.875 0.700
3
0.750 0.900
1
0.909 1.000
4
0.714 1.000
1
0.875 0.700
0
0.000 0.100
0
1.000 1.000
1
0.900 0.900
5
0.500 0.500
1
0.889 0.800
2
0.833 1.000
0
1.000 1.000
0
1.000 0.700
0
1.000 0.800
3
0.700 0.700
1
0.833 0.500
0
1.000 0.500
3
0.625 0.500
0
1.000 0.500
1
0.889 0.800
5
0.583 0.700
4
0.714 1.000
1
0.909 1.000
2.89
0.729 0.712
F1 auto-tag F1
0.889
1.000
0.692
0.696
0.467
0.467
0.783
0.667
1.000
1.000
0.783
0.800
0.909
0.909
0.645
0.667
0.435
0.417
0.333
0.500
0.348
0.593
0.690
0.833
0.778
0.900
0.818
0.667
0.952
0.909
0.833
0.833
0.778
0.889
0.000
0.000
1.000
0.909
0.900
0.889
0.500
0.500
0.842
0.842
0.909
0.833
1.000
0.824
0.824
0.889
0.889
1.000
0.700
0.750
0.625
0.667
0.667
0.526
0.556
0.588
0.667
0.947
0.842
0.706
0.636
0.800
0.833
0.842
0.952
0.952
0.691
0.702