3URF
RI
7KLUG
,QWHOOLJHQFH
3DFLILF$VLD
1LQJ
&RQIHUHQFH
=KRQJ
DQG
/L]KX
3$.''
=KRX
(GV
/HFWXUH
1RWHV
0HWKRGRORJLHV
LQ
IRU
$UWLILFLDO
.QRZOHGJH
'LVFRYHU\ DQG 'DWD 0LQLQJ SS
7KHGDWDSUHSURFHVVLQJPHWKRGIRUGHFLVLRQWUHH XVLQJDVVRFLDWLRQVLQWKHDWWULEXWHV
0DVDKLUR7HUDEH2VDPX.DWDL7HWVXR6DZDUDJL7DNDVKL:DVKLR DQG+LURVKL0RWRGD
5HVHDUFK &HQWHU IRU 6DIHW\ 6FLHQFH 0LWVXELVKL 5HVHDUFK ,QVWLWXWH ,QF
2KWHPDFKL &KL\RGDNX 7RN\R -DSDQ
(PDLO WHUDEH#PULFRMS
7HO )D[
*UDGXDWH 6FKRRO RI ,QIRUPDWLFV .\RWR 8QLYHUVLW\
<RVKLGD+RQPDFKL 6DN\RNX .\RWR -DSDQ
(PDLO NDWDL#SUHFN\RWRXDFMS
*UDGXDWH 6FKRRO RI (QJLQHHULQJ .\RWR 8QLYHUVLW\
<RVKLGD+RQPDFKL 6DN\RNX .\RWR -DSDQ
(PDLO VDZDUDJL#SUHFN\RWRXDFMS
,QVWLWXWH RI 6FLHQFH DQG ,QGXVWULDO 5HVHDUFK 2VDND 8QLYHUVLW\
0LKRJDRND ,EDUDNLVKL 2VDND -DSDQ
(PDLO ^ZDVKLRPRWRGD`#VDQNHQRVDNDXDFMS
$EVWUDFW
7KH GHFLVLRQ WUHH LV RQH RI WKH PDMRU NQRZOHGJH UHSUHVHQWDWLRQV RI GDWD PLQLQJ UHVXOWV EHFDXVH RI LWV
FRPSUHKHQVLYHQHVV DQG XVHIXOQHVV IRU GHFLVLRQ PDNLQJ E\ WKH KXPDQ DQDO\VWV 7KH GHFLVLRQ WUHH LV D
JRRG RQH LI WKH WUHH FRXOG FODVVLI\ WKH QHZ GDWD DFFXUDWHO\ DQG LI WKH VL]H RI WUHH ZHUH VPDOO
2QH RI WKH DSSURDFKHV WR JHQHUDWH VXFK D JRRG GHFLVLRQ WUHH LV SUHSURFHVVLQJ WKH WUDLQLQJ GDWD RI
GHFLVLRQ WUHH JHQHUDWLRQ LH WR DGG WKH QHZ DWWULEXWHV DQG WKHVH DWWULEXWH YDOXH IRU H[WHQGLQJ WKH
GHVFULSWLRQ RI WKH WUDLQLQJ GDWD 7KHUH DUH PDQ\ ZRUNV RQ GDWD SUHSURFHVVLQJ VXFK DV DWWULEXWH
JHQHUDWLRQ DQG DWWULEXWH VHOHFWLRQ PHWKRG +RZHYHU PRVW RI WKHP DUH EDVHG RQ ORJLF SURJUDPPLQJ
VR WKDW LW WDNHV PXFK WLPH IRU GDWD SUHSURFHVVLQJ $GGLWLRQDOO\ WKH\ QHHG D SULRUL NQRZOHGJH 7KHVH
IHDWXUHV DUH GLVDGYDQWDJH DV GDWD PLQLQJ GHDOW ZLWK ODUJH YROXPH GDWD 2Q WKH FRQWUDU\ RXU SURSRVHG
QRYHO PHWKRG LV GDWD GULYHQ DSSURDFK WKDW NQRZOHGJH RQ WKH UHOHYDQFH RQ DWWULEXWHV DUH JHQHUDWHG DV
DVVRFLDWLRQ UXOHV IURP WKH WUDLQLQJ GDWD DQG D SULRUL NQRZOHGJH LV QRW QHFHVVDU\ 7KH QHZ DWWULEXWHV
DQG DWWULEXWH YDOXHV DUH JHQHUDWHG RQO\ IURP WKH DVVRFLDWLRQ UXOHV ,Q WKLV SDSHU ZH SURSRVH WKH
PHWKRG DQG FODULI\ LWV IHDWXUH 7KH HIIHFWLYHQHVV RI RXU DSSURDFK LV HYDOXDWHG WKURXJK H[SHULPHQWDO
UHVXOWV
.H\ ZRUGV GDWD PLQLQJ DVVRFLDWLRQ UXOH GHFLVLRQ WUHH DWWULEXWH JHQHUDWLRQ GDWD SUHSURFHVVLQJ
,QWURGXFWLRQ
$GHFLVLRQWUHHLVRQHRIWKHPDMRUNQRZOHGJHUHSUHVHQWDWLRQVRIGDWDPLQLQJUHVXOWV
7KLVLVEHFDXVHWKDWLWLVHDV\WRXQGHUVWDQGLWVPHDQLQJIRUKXPDQDQDO\VWV7KHUHIRUH
KXPDQDQDO\VWVFDQH[SORLWHDVLO\LQWKHLUGHFLVLRQPDNLQJ
7R PDNH WKH GHFLVLRQ WUHH PRUH FRPSUHKHQVLYH DQG XVHIXO RQH WKHUH DUH WZR
GHPDQGVRQGHFLVLRQWUHH2QHLVWKHDFFXUDF\RIWKHGHFLVLRQWUHHLQRWKHUZRUGVWKH
UDWLR WKDW UXOHV LQ WKH GHFLVLRQ WUHH FDQ SUHGLFW DFFXUDWH FODVV IRU WKH QHZ VDPSOH GDWD
7KH RWKHU LV WKH FRPSOH[LW\ RI WKH GHFLVLRQ WUHH LH WKH VL]H RI WKH GHFLVLRQ WUHH WKDW
GHILQHGDVWKHQXPEHURIWKHUXOHVLQWKHGHFLVLRQWUHH)RUKXPDQDQDO\VWVWKHVPDOOHU
GHFLVLRQ WUHH LV WKH EHWWHU RQ LWV FRPSUHKHQVLYHQHVV DQG XVHIXOQHVV +RZHYHU PDNLQJ
VXFKDVPDOOGHFLVLRQWUHHIURPRULJLQDOWUDLQLQJGDWDLVQRWHDV\EHFDXVHWKHDWWULEXWHVLQ
WKH GDWD DUH QRW DOZD\V GHVFULEHG DGHTXDWHO\ LH DWWULEXWH VHWV LQFOXGHV UHGXQGDQW
LQIRUPDWLRQ RI FODVVLILFDWLRQ VKRXOG EH LQWHJUDWHG WR VPDOOHU QXPEHU RI QHZ DWWULEXWHV
7KHLQDGHTXDWHDELOLW\RIGDWDGHVFULSWLRQWHQGVWRPDNHWKHJHQHUDWLQJGHFLVLRQWUHHEDG
RQH
,Q JHQHUDO WKHUH DUH WZR W\SHV RI DSSURDFKHV WR JHW PRUH SUHIHUDEOH GHFLVLRQ WUHH
2QH LV GHYHORSLQJ QHZ GHFLVLRQ WUHH DOJRULWKP WKDW FDQ JHQHUDWH EHWWHU GHFLVLRQ WUHH
HYHQLIWKHWUDLQLQJGDWDLVVDPHRQHLHLPSURYLQJWKHGHFLVLRQWUHHJHQHUDWLRQSURFHVV
DV PRUH SRZHUIXO RQH 7KH RWKHU LV SUHSURFHVVLQJ WKH WUDLQLQJ GDWD WR H[WHQG LWV
GHVFULSWLRQVXFKDVDWWULEXWLRQJHQHUDWLRQDQGDWWULEXWHVHOHFWLRQ,QWKLVSDSHUZHIRFXV
RQWKHODWWHUDSSURDFKGDWDSUHSURFHVVLQJ
7KHUHDUHQXPEHUVRIUHVHDUFKHVRQGDWDSUHSURFHVVLQJVXFKDVDWWULEXWHVJHQHUDWLRQ
DQG DWWULEXWHV VHOHFWLRQ +RZHYHU PRVW RI WKH DSSURDFKHV QHHG D SULRUL NQRZOHGJH RQ
WKH DWWULEXWHV DQG WKHLU UHOHYDQF\ LQ WKH WUDLQLQJ GDWD DQG WKH NQRZOHGJH LV XVHG LQ
DWWULEXWHV JHQHUDWLRQ SURFHVV $GGLWLRQDOO\ PDQ\ RI WKH PHWKRG DUH EDVHG RQ ORJLF
LQWHQGHG DOJRULWKPV VXFK DV ,QGXFWLYH /RJLF 3URJUDPPLQJ ,/3 )RU H[DPSOH /DYUDF
XVHV ,/3 LQ WKHLU DWWULEXWLRQ JHQHUDWLRQ PHWKRG >@ 7KHVH PHWKRGV DUH HIIHFWLYH LQ WKH
FDVH WKDW WKH GDWD ZDV VPDOO RU D SULRUL NQRZOHGJH RQ DWWULEXWHV ZHUH JLYHQ 2Q WKH
FRQWUDU\ WKHVH DOJRULWKPV KDYH VRPH SUREOHP LI WKH GDWD YROXPH ZDV ODUJH )LUVW WKH
ORJLFEDVHGDWWULEXWHJHQHUDWLRQDQGVHOHFWLRQQHHGVPXFKFDOFXODWLRQFRVWZKHQWKH\DUH
DSSOLHGWRODUJHGDWD6HFRQGO\WKH\DOZD\VGHPDQGDSULRULNQRZOHGJHRIWKHDWWULEXWHV
DQGWKHLUUHOHYDQF\EXWWKHVHGHPDQGVDUHQRWVRHDVLO\VDWLVILHGEHFDXVHGDWDPLQLQJLV
DWRROQRWRQO\IRUH[SHUWRIWKHGDWDEXWDOVRIRUQRYLFHDQDO\VWV
,QWKLVSDSHUZHSURSRVHQHZGDWDSUHSURFHVVLQJDWWULEXWLRQJHQHUDWLRQPHWKRGWKDW
LV HIIHFWLYH IRU PDNLQJ JRRG GHFLVLRQ WUHH HYHQ IURP ODUJH YROXPH GDWD GHDOW LQ GDWD
PLQLQJ7KHNH\LGHDRIWKLVPHWKRGLVWKDW³WKHDWWULEXWHVDQGWKHLUDWWULEXWHYDOXHVWKDW
DSSHDUHG DV SDWWHUQ RI DWWULEXWH YDOXHV LQ GDWD WKDW LQFOXGHV VDPH LQIRUPDWLRQ IRU
FODVVLILFDWLRQ FDQ UHSODFH E\ RQH DWWULEXWH DQG LWV DWWULEXWH YDOXHV DV D UHVXOW RI
JURXSLQJWKHDWWULEXWHV)XUWKHUPRUHLIWKHQHZDWWULEXWHVDUHDGGHGWRWKHRULJLQDOGDWD
WKHGDWDZLOOJLYHDJRRGHIIHFWWRWKHGHFLVLRQWUHHRQWKHWUHHVL]HDQGDFFXUDF\´,QWKH
PHWKRG WKH DSSHDUHG SDWWHUQV DPRQJ WKH DWWULEXWHV DWWULEXWH YDOXHV DUH H[WUDFWHG DV
DVVRFLDWLRQ UXOHV WR UHSUHVHQW HDFK SDWWHUQ DV D QHZ DWWULEXWH DWWULEXWH YDOXHV 7KLV
DOJRULWKPLVGDWDGULYHQVRWKDWZHGRQ¶WQHHGDSULRULNQRZOHGJHRQDWWULEXWHVDQGWKHLU
UHOHYDQF\ 7R JHQHUDWH DVVRFLDWLRQ UXOHV ZH DGRSW DSULRUL DOJRULWKP SURSRVHG E\
$JUDZDO>@WKDWFDQJHQHUDWHDVVRFLDWLRQUXOHVPXFKIDVWHUHYHQLIWKHGDWDYROXPHZDV
ODUJH :LWK WKH DVVRFLDWLRQ UXOHV WKH FDQGLGDWHV RI QHZ DWWULEXWH DQG LWV YDOXH DUH
JHQHUDWHGDQGWKH\DUHHYDOXDWHGE\WKHLQGH[FDOFXODWHGZLWKWKHWZRLQGH[HVRIHDFK
DVVRFLDWLRQUXOH
7KHFRQWHQWVRIWKLVSDSHUDUHDVIROORZV,QVHFWLRQZHJLYHDVKRUWH[SODQDWLRQRI
WKHDVVRFLDWLRQUXOHVDQGDSULRULDOJRULWKPXVLQJIRUQHZDWWULEXWLRQLQSURSRVHGPHWKRG
,Q VHFWLRQ 7KH QRYHO GDWD SUHSURFHVVLQJ PHWKRG LV SURSRVHG ,Q WKH VHFWLRQ WKH
HIIHFWLYHQHVVRISURSRVHGPHWKRGLVFRQILUPHGWKURXJKH[SHULPHQWV,QWKHVHFWLRQZH
GLVFXVV WKH DGYDQWDJH FRPSDUHG ZLWK WKH IRUPHU PHWKRG DQG WKH OLPLWDWLRQ RI RXU
DSSURDFK)LQDOO\ZHFRQFOXGHLQVHFWLRQ
7KHDVVRFLDWLRQUXOHDQGLWVJHQHUDWLRQDOJRULWKP
,Q WKLV VHFWLRQ ZH JLYH D VKRUW H[SODQDWLRQ RQ WKH DVVRFLDWLRQ UXOHV DQG WKH
DSULRUL DOJRULWKP WKDW DGRSWHG DV DVVRFLDWLRQ UXOH JHQHUDWLRQ DOJRULWKP LQ WKH SURSRVHG
PHWKRG
,QWKHDVVRFLDWLRQDQDO\VLVDQGLWVGDWDIRUPDWWKHPLQLPDOXQLWRIGDWDLVFDOOHG
DVLWHPDQGDGDWDFDOOHGWUDQVDFWLRQLVDVHWRILWHPV7KHDVVRFLDWLRQUXOHLVGHILQHGDV
WKHUXOHVWKDWSUHVHQWWKHFRUHODWLRQFRRFFXUUHQFHEHWZHHQWZRVXEVHWVRILWHPVLQWKH
VDPHWUDQVDFWLRQ7KHDVVRFLDWLRQUXOHLVGHVFULEHGDVIROORZV
% !+
%%RG\FRQGLWLRQSDUWRIWKHDVVRFLDWLRQUXOH
++HDGFRQFOXVLRQSDUWRIWKHDVVRFLDWLRQUXOH
7KH³%RG\´DQG³+HDG´DUHVHWRILWHPV7KLVUXOHPHDQVWKDW ,I % RFFXUV DSSHDUV WKHQ + RIWHQ FRRFFXUV DOVR DSSHDUV LQ WKH VDPH
WUDQVDFWLRQ
7KHUHIRUHWKHDVVRFLDWLRQUXOHVUHSUHVHQWWKDWIUHTXHQWDSSHDUHGSDWWHUQLQWKHGDWDVHW ,WKDGEHHQPXFKWLPHWRJHQHUDWHDVVRFLDWLRQUXOHIURPWKHODUJHYROXPHGDWD
+RZHYHU WKH DSULRUL DOJRULWKP $JUDZDO SURSRVHG PDNH LW SRVVLEOH WR JHQHUDWH WKH
DVVRFLDWLRQUXOHVPXFKIDVWHUHYHQLIWKHYROXPHRIGDWDZHUHODUJHKHQFHLWLVIHDVLEOHWR
JHQHUDWHDVVRFLDWLRQUXOHVIURPWKHODUJHGDWD>@>@ ,QWKHDSULRULDOJRULWKPVXSSRUWYDOXHDQGFRQILGHQFHYDOXHDUHJHQHUDWHGIRU
HDFK DVVRFLDWLRQ UXOH DV RXWSXW LQGH[HV 7KH VXSSRUW YDOXH LV WKH UDWLR RI LWHPV LQ WKH
FRQGLWLRQDO SDUWV DSSHDUHG LQ WKH ZKROH RI WKH WUDQVDFWLRQV )RU H[DPSOH WKH VXSSRUW
YDOXHRIDVVRFLDWLRQUXOH5LVGHILQHGDVHTXDWLRQ
Q % ∪ + 1
Q % ∪ + = QXPEHU RI WUDQVDFWLR QV LQFOXGLQJ LWHPVHW RI %RG\ DQG +HDG
1 = QXPEHU RI WUDQVDFWLR QV
VXS 5 =
7KH KLJKHU VXSSRUW YDOXH LV WKH PRUH SDWWHUQV UHSUHVHQWHG E\ WKH DVVRFLDWLRQ UXOH LV
LQFOXGHGLQWKHGDWDVHW
2QWKHRWKHUKDQGVFRQILGHQFHYDOXHLVWKHUDWLRWKHQXPEHURIWKHWUDQVDFWLRQV
LQFOXGLQJ ERWK RI WKH FRQGLWLRQ DQG UHVXOW SDUW LWHPV WR WKH QXPEHU RI WKH WUDQVDFWLRQ
LQFOXGLQJ WKH FRQGLWLRQ SDUW LWHPV 7KH FRQILGHQFH YDOXH RI WKH UXOH 5 LV GHILQHG DV
HTXDWLRQ
VXS (% ∪ + )
FRQI (5 % ⇒ + ) =
VXS (% )
7KH FRQILGHQFH YDOXH UHSUHVHQWV WKH GHJUHH RI WKH FRRFFXUUHQFH EHWZHHQ %RG\ DQG
+HDGLQDWUDQVDFWLRQ7KHKLJKHUFRQILGHQFHYDOXHLVWKHPRUHDVVRFLDWLRQUXOHSUHGLFWV
WKHULJKWFRQFOXVLRQ
,QDSULRULDOJRULWKPWKHXVHIXOQHVVRIWKHHDFKDVVRFLDWLRQUXOHLVHYDOXDWHGE\
WKHVH LQGH[HV 7KH PLQLPDO VXSSRUW WKUHVKROG DQG PLQLPDO FRQILGHQFH WKUHVKROG DUH
GHWHUPLQHG DV WKH FRQGLWLRQ RI DVVRFLDWLRQ UXOH JHQHUDWLRQ 7KH WKUHVKROGV ZHUH VHW
KLJKHU WKH QXPEHU RI H[DPLQHG DVVRFLDWLRQ UXOH K\SRWKHVLV LV VPDOOHU +RZHYHU WKH
FDOFXODWLRQFRVWLVUHGXFHG:HQHHGWRVHWWKHVHYDOXHVWDNLQJWKHWUDGHRIILQWRDFFRXQW
,Q RXU PHWKRG WKH DVVRFLDWLRQ UXOHV RI WKH DWWULEXWHV DUH JHQHUDWHG DV NQRZOHGJHRIDVVRFLDWLRQDPRQJDWWULEXWHVLQWKHGDWDDQGXVHWKHPWRJHQHUDWHWKHQHZ
DWWULEXWHDQGWKHLUYDOXHV7KHUXOHVPHDQWKDWIUHTXHQWDSSHDUHGSDWWHUQRIWKHDWWULEXWH
DQGDWWULEXWHVYDOXHV
7KHSURSRVHGPHWKRG
2XWOLQHRISURFHVV
7KHRXWOLQHRIRXUSURSRVHGSUHSURFHVVLQJPHWKRGLVGHSLFWHGLQILJXUHDQG
VXPPDUL]HGDVIROORZ 6WHS
7KHWUDLQLQJGDWDIRUGHFLVLRQWUHHLVWUDQVIRUPHGLQWRWKHWUDQVDFWLRQGDWD
IRUPDWXVLQJLQWKHDVVRFLDWLRQUXOHJHQHUDWLRQ 6WHS
7KHDVVRFLDWLRQUXOHVRIDWWULEXWHVDQGWKHLUYDOXHDUHJHQHUDWHGIURPWKH
WUDQVDFWLRQGDWD 6WHS
7KHFDQGLGDWHVRIQHZDWWULEXWLRQDQGLWVDWWULEXWLRQYDOXHVDUHJHQHUDWHG
IURPWKHDVVRFLDWLRQUXOHV
6WHS
7KH XVHIXOQHVV RI FDQGLGDWHV RI QHZ DWWULEXWHV LV HYDOXDWHG ZLWK WKH
FULWHULDRILQIRUPDWLRQJDLQ 6WHS
7KH HYDOXDWHG YDOXH LV VDWLVILHG SUHGHILQHG FULWHULD WKH QHZ DWWULEXWH LV
DGGHGWRWKHWUDLQLQJGDWD 7KHHDFKSURFHVVLVH[SODLQHGPRUHH[DFWO\LQWKHIROORZLQJVHFWLRQV 6TCKPKPI
&CVC
6TCPUHQTOCVKQP
QH HQTOCV
6TCPUCEVKQP
&CVC
#UUQEKCVKQP TWNG
IGPGTCVKQP
CRTKQTK CNIQTKVJO
CUUQEKCVKQP TWNG
QH CVVTKDWVGU
ECPFKFCVGU QH PGY
CVVTKDWVG
IGPGTCVKQP CPF
GXCNWCVKQP
PGY CVVTKDWVGU
2TGRTQEGUUGF
&CVC
)LJXUH 7KH SURFHVV RI SURSRVHG SUHSURFHVVLQJ PHKRGRORJ\
'DWDIRUPDWWUDQVIRUPDWLRQ
)RUDSSO\LQJWKHDVVRFLDWLRQUXOHJHQHUDWLRQDOJRULWKPWRWKHWUDLQLQJGDWDZH
QHHGWRWUDQVIRUPWKHWUDLQLQJGDWDRIGHFLVLRQWUHHWRWUDQVDFWLRQW\SHGDWDIRUPDW7KH
IRUPDWRIWUDLQLQJGDWDIRUGHFLVLRQWUHHLVGHVFULEHGDVDVHWRIVDPSOHGDWDDQGVDPSOH
GDWDDUHDVHWRIDWWULEXWHYDOXHVIRUH[SODQDWLYHDWWULEXWHVDQGDFODVVDWWULEXWHDVIROORZV
³GDWDBM YBM«YBQMFBM!´ 2Q WKH FRQWUDU\ WKH WUDQVDFWLRQ GDWD LV D VHW RI WUDQVDFWLRQV ³WUDQV´ DQG HDFK
WUDQVDFWLRQLVFRPSRVHGE\LWHPVDVIROORZV
³WUDQVBM LWHPBM«LWHPBQMLWHPBQM!´ 7KHUHIRUH ZH QHHG WR WUDQVIRUP DWWULEXWH ³DL´ DQG DWWULEXWH YDOXH ³YLM´ LQ WKH
WUDLQLQJ GDWD LQWR LWHP ³DL YLM´ )RU H[DPSOH WKH WUDLQLQJ GDWD KDV WZR H[SODQDWLYH
DWWULEXWHV³FRORU´³VKDSH´DQGWKHFODVVDWWULEXWH³W\SH´
GDWD EODFNWULDQJOHJRRG!
WKHGDWDWUDQVODWHVLQWRWUDQVDFWLRQV
WUDQV ^³FRORU EODFN´³VKDSH WULDQJOH´³W\SH JRRG´`
:LWKWKLVGDWDWUDQVIRUPHGWRWKHWUDQVDFWLRQW\SHIRUPDWWKHDVVRFLDWLRQUXOHV
DUH JHQHUDWHG ZLWK WKH DSLRUL DOJRULWKP H[SODLQHG LQ WKH ODVW VHFWLRQ ,Q WKH DWWULEXWHV
FDQGLGDWHVJHQHUDWLRQSURFHVVZHXVHRQO\WKHDVVRFLDWLRQUXOHVWKDWVDWLVI\WKHQH[WWZR
FRQGLWLRQ
7KHFRQGLWLRQSDUWGRHVQ¶WLQFOXGHLWHPIURPWKHFODVVYDOXH
7KHUHVXOWSDUWLQFOXGHVRQO\RQHLWHPIURPWKHFODVVYDOXH
7KHVH DVVRFLDWLRQ UXOHV WKDW VDWLVI\ WKHVH FRQGLWLRQV UHSUHVHQW WKDW WKH FORVHO\ UHOHYDQW
DWWULEXWHV VHWV LQ WKH FRQGLWLRQ SDUW DQG WKH FODVV LQ WKH FRQFOXVLRQ SDUW 7KHUHIRUH WKH
DWWULEXWHVJURXSZRXOGLQFOXGHHIIHFWLYHLQIRUPDWLRQWRFODVVLI\WKHGDWD
*HQHUDWLRQRI1HZ$WWULEXWH&DQGLGDWHV
:LWK WKH JHQHUDWHG DVVRFLDWLRQ UXOHV WKH FDQGLGDWHV RI QHZ DWWULEXWLRQV DUH
JHQHUDWHG7KHDOJRULWKPLVVXPPDUL]HGDVEHORZ ,QSXW
n
n
56 ^5«5Q` $ VHW RI DVVRFLDWLRQ UXOHV JHQHUDWHG IURP WUDLQLQJ GDWD
5 $5956XS5&RQI5!
$5 FRQMXQFWLRQ RI DWWULEXWH QDPHV WKDW DUH WKH DWWULEXWH SDUW RI LWHPV LQ WKH FRQGLWLRQ SDUW
95 YHFWRU RI DWWULEXWH YDOXHV WKDW DUH DWWULEXWH YDOXH SDUW RI WKH LWHPV LQ WKH FRQGLWLRQ SDUW
6XS5
VXSSRUW YDOXH RI WKH DVVRFLDWLRQ UXOH 5
&RQI5
FRQILGHQFH YDOXH RI WKH DVVRFLDWLRQ UXOH 5
2XWSXW
n
n
56QHZ ^51«51Q` D VHW RI FDQGLGDWHV RI DWWULEXWLRQ 51
51 $V51951 6XS51&RQI51!
$51 DWWULEXWH QDPH RI WKH LWHPV LQ WKH FRQGLWLRQ SDUW
9651 VHW RI DWWULEXWH YHFWRUV
6XS51
&RQI51
VXSSRUW RI WKH DVVRFLDWLRQ UXOH 51
VHW RI FRQILGHQFH RI WKH DVVRFLDWLRQ UXOH 51
$OJRULWKP
:KLOH5 QRW HT QXOO^
LI$5 HT $51 51 LQ 56QHZ^
3XVK 95 WR 9651
6XS51 6XS5
3XVK &RQI5 WR &RQI51
`
HOVH^
*HQHUDWH 1HZ 51
$51 $5
9651 95
6XS51 6XS5
&RQI51 &RQI5
3XVK 51 WR 56QHZ
`
`
7DEOH1HZDWWULEXWLRQJHQHUDWLRQDOJRULWKP
#VVTKDWVG 8CNWG
P
Ý
8
0GY
#VVTKDWVG
40
nnn
8P
nnn
4
)LJXUH 8888P
CVVTKDWVG
XCNWG P
4P CUUQEKCVKQP
TWNG
45PGY4WNG5GV
nnnn
7KH FDQGLGDWHV RI QHZ DWWULEXWLRQ DQG LWV YDOXHV
7KH FDQGLGDWHV RI DWWULEXWHV JHQHUDWHG IURP WKLV DOJRULWKP KDV _951_
DWWULEXWHV YDOXHV DV GHSLFWHG LQ ILJXUH 7KH DWWULEXWH KDV _951_ 7KH VXSSRUW YDOXH
DQGFRQILGHQFHYDOXHLVXVHGLQHYDOXDWLRQRIWKHFDQGLGDWH
7KLVSURFHVVFDQUHJDUG DVWKH ³DWWULEXWH JURXSLQJ´WKDWVRPHRIWKH DWWULEXWHV
LQFOXGLQJ UHOHYDQF\ LQ WKH RULJLQDO WUDLQLQJ GDWD DUH JURXSHG DV D FDQGLGDWH RI D QHZ
DWWULEXWH
(YDOXDWLRQRI1HZ$WWULEXWH
7R HYDOXDWH WKH FDQGLGDWHV RI QHZ DWWULEXWHV ZH XVH WKH LQIRUPDWLRQ JDLQ
FULWHULDWKDWXWLOL]HLQGHFLVLRQWUHHDOJRULWKPVXFKDV&7KHLQIRUPDWLRQJDLQIRUHDFK
FDQGLGDWHRIQHZDWWULEXWHLVGHILQHGDVHTXDWLRQ>@>@
*DLQ 56 S
S Q
Q
≥ − ORJ − ORJ 1
1 1
1
56
− ∑ VL ⋅ {− FL ORJ FL − − FL ORJ − FL }
L =
Q


−  − ∑ VL 
 L = 
7KLVLQIRUPDWLRQJDLQLVHTXDOWRWKHRQHLQWKHFDVHWKDWWKHWRSQRGHZDVGLYLGHGE\WKH
FDQGLGDWHRIDWWULEXWLRQ7KHUHIRUHLIWKHLQIRUPDWLRQJDLQLVELJJHUWKDQWKHFDQGLGDWH
DWWULEXWHPD\XVHIXOIRUGHFLVLRQWUHHDOJRULWKP
1HZ$WWULEXWH
,IWKHHYDOXDWHGYDOXHRIWKHDWWULEXWHFDQGLGDWHVLVELJJHUWKDQWKHDWWULEXWHV
DUHDGGHGWRWKHWUDLQLQJGDWD%\VXFKSUHSURFHVVLQJWKHWUDLQLQJGDWDRIGHFLVLRQWUHH
DUHH[WHQGHGIURPWKHRULJLQDORQH
(PSLULFDOH[SHULPHQW
7KHGDWDVHWV
7RFRQILUPWKHHIILFLHQF\ RIRXUSURSRVHGPHWKRGZHPDNHVRPHH[SHULPHQW
ZLWK WKH VDPSOH GDWD VHWV 7KH SURSRVHG PHWKRG FDQ DSSO\ WR GDWD VHWV LQFOXGLQJ RQO\
GLVFUHWHDWWULEXWHVVRWKDWGDWDVHWVVDWLVILHGWKH FRQGLWLRQDUHSLFNHGXSIURPPDFKLQH
OHDUQLQJGHSRVLWRU\LQWKH8QLYHUVLW\RI&DOLIRUQLD,UYLQH>@7KHFKDUDFWHUVRIWKHGDWD
VHWVDUHVXPPDUL]HGLQWKHWDEOH
:H FRPSDUH WKH ³JURXSLQJ´ HIIHFW RI DWWULEXWH JURXSLQJ LQ RXU PHWKRG DQG
DWWULEXWHYDOXHJURXSLQJLQ&WRFRQILUPWKHIHDWXUHRIWKHSURSRVHGPHWKRGIURPWKH
JURXSLQJSRLQWRIYLHZ
7DEOH'HVFULSWLRQRIGDWDVHWV
7HVW 'DWD
FHQVXVGDWD
RI WUDLQLQJ GDWD
FDU
PRQN
PRQN
PRQN
PXVKURRP
QXUVHU\
VWLFWDFWRH
YRWHV
RI DWWULEXWHV
RI FODVVHV
PLVV
QR
QR
QR
QR
QR
\HV
QR
QR
\HV
7KHUHVXOWV
2QO\DWWULEXWLRQJHQHUDWLRQ
)LUVWZHPDNHWZRH[SHULPHQWVWRFRQILUPWKHHIIHFWRIRXUSURSRVHGPHWKRG :H
FRPSDUHWKHVL]HDQGDVVXUDQFHRIWKHGHFLVLRQWUHHVJHQHUDWHGIURP
RULJLQDOWUDLQLQJGDWDDQG
SUHSURFHVVHGGDWD
7KHH[SHULPHQWDOUHVXOWVDUHVXPPDUL]HGLQWDEOHDQGWDEOH
$V \RX FDQ FRQILUP LQ WKH WDEOH DIWHU SUXQLQJ WKH VL]H RI JHQHUDWHG IURP
SUHSURFHVVHGGDWDLVVPDOOHUWKDQWKDWRIWUHHJHQHUDWHGIURPRULJLQDOGDWDLQWKHRI
FDVHV DQG WKH RWKHU FDVHV DUH HTXDO VL]H )XUWKHUPRUH WKH SUHGLFWLRQ HUURU RI WKH WUHH
JHQHUDWHGSUHSURFHVVHGGDWDLVUHGXFHGLQRIFDVHVDQGWKHRWKHUFDVHVDUHHYHQ
2QO\ GLVFUHWH DWWULEXWHV DUH SLFNHG XS
7DEOH 5HVXOWVRIQRDWWULEXWHJURXSLQJVL]H
'DWD
RULJLQDO WUDLQLQJ GDWD
SUHSURFHVVHG WUDLQLQJ GDWD
%HIRUH
$IWHU
RI
%HIRUH $IWHU
RI
3UXQLQJ 3UXQLQJ DWWULEXWHV 3UXQLQJ 3UXQLQJ DWWULEXWHV
6L]H
FHQVXVGDWD
6L]H
6L]H
6L]H
PRQN
PRQN
PRQN
PXVKURRP
QXUVHU\
WLFWDFWRH
FDU
YRWHV
7DEOH 5HVXOWVRIQRDWWULEXWHYDOXHJURXSLQJSUHGLFWLRQHUURU
'DWD
FHQVXVGDWD
FDU
PRQN
PRQN
PRQN
PXVKURRP
QXUVHU\
WLFWDFWRH
YRWHV
RULJLQDO WUDLQLQJ GDWD
SUHSURFHVVHG WUDLQLQJ GDWD
%HIRUH
$IWHU 3UXQLQJ
%HIRUH
$IWHU 3UXQLQJ
3UXQLQJ
3UXQLQJ
(UURUV
(UURUV (VWLPDWH (UURUV
(UURUV (VWLPDWH
:LWKDWWULEXWHYDOXHJURXSLQJ
1H[WZHFRPSDUHGWKHHIIHFWRIDWWULEXWHYDOXHJURXSLQJDGRSWHGLQ&DVRSWLRQ
DQG WKH SURSRVHG PHWKRG 7KH DWWULEXWH YDOXH JURXSLQJ JLYHV DV VDPH DQ HIIHFW IRU
GHFLVLRQWUHHVDVSURSRVHGPHWKRGVXFKDVLQFUHDVLQJSUHGLFWLRQDFFXUDF\DQGUHGXFLQJ
WUHHVL]H>@7KHPDLQHIIHFWRIRXUSURSRVHGPHWKRGLVNLQGRIWKH³DWWULEXWH´JURXSLQJ
FI ³DWWULEXWH YDOXH´ JURXSLQJ 7KH WZR NLQGV RI H[SHULPHQW DUH H[HFXWHG 2QH LV
PDNLQJGHFLVLRQWUHH ZLWKRULJLQDOWUDLQLQJGDWDWRFRQILUPWKHHIIHFWRQO\ RI DWWULEXWH
JURXSLQJ,QWKHRWKHUH[SHULPHQWWKHGHFLVLRQWUHHVDUHJHQHUDWHGIURPSUHSURFHVVHG
GDWD WR FRQILUP WKH HIIHFW LQ FRPELQDWLRQ RI DWWULEXWH YDOXH JURXSLQJ DQG DWWULEXWH
JURXSLQJ7KHH[SHULPHQWDOUHVXOWVDUHVXPPDUL]HGLQWDEOHDQGWDEOH
7DEOH :LWKDWWULEXWHYDOXHJURXSLQJVL]H
'DWD
RULJLQDO WUDLQLQJ GDWD
SUHSURFHVVHG WUDLQLQJ GDWD
%HIRUH
$IWHU
RI
%HIRUH $IWHU
RI
3UXQLQJ 3UXQLQJ DWWULEXWHV 3UXQLQJ 3UXQLQJ DWWULEXWHV
6L]H
FHQVXVGDWD
6L]H
6L]H
6L]H
PRQN
PRQN
PRQN
PXVKURRP
WLFWDFWRH
YRWHV
FDU
QXUVHU\
7DEOH :LWKDWWULEXWHYDOXHJURXSLQJSUHGLFWLRQHUURU
'DWD
FHQVXVGDWD
FDU
PRQN
PRQN
PRQN
PXVKURRP
QXUVHU\
WLFWDFWRH
YRWHV
RULJLQDO WUDLQLQJ GDWD
SUHSURFHVVHG WUDLQLQJ GDWD
%HIRUH
$IWHU 3UXQLQJ
%HIRUH
$IWHU 3UXQLQJ
3UXQLQJ
3UXQLQJ
(UURUV
(UURUV (VWLPDWH (UURUV
(UURUV (VWLPDWH
$V VXPPDUL]HG LQ WKH WDEOH WKH VL]H RI SUXQHG GHFLVLRQ WUHH IURP
SUHSURFHVVHGGDWDLVVPDOOHUWKDQWKDWRIWUHHJHQHUDWHGIURPRULJLQDOGDWDLQWKHRI
FDVHVDQGHYHQLQFDVHVDQGELJJHULQFDVH)XUWKHUPRUHWKHSUHGLFWLRQHUURURIWKH
WUHH JHQHUDWHG SUHSURFHVVHG GDWD LV UHGXFHG LQ RI FDVHV HYHQ LQ WKH FDVHV DQG
LQFUHDVHGLQRQHFDVH
'LVFXVVLRQ
%DVLFFKDUDFWHULVWLFRISURSRVHGPHWKRG
)LUVW ZH GLVFXVV RQ EDVLF FKDUDFWHULVWLF RI SURSRVHG PHWKRG ZLWK WKH H[SHULPHQWDO
UHVXOWV RI 0RQN¶V GDWD VHWV 7KHUH DUH WKUHH GLIIHUHQW GDWD VHWV LQ WKH 0RQN¶V GDWD WKDW
KDYHDVVRFLDWLRQDPRQJWKHH[SODQDWLYHDWWULEXWHVDQGWKHFODVVDWWULEXWH7KHDVVRFLDWLRQ
LVVXPPDUL]HGDVIROORZV
− 0RQND DRUD !F − 0RQN([DFWO\WZRRI^D D D D D D `!F − 0RQND DQGD RUD DQGD !F :HGLVFXVVRQHDFKRIWKHH[SHULPHQWDOUHVXOWRI0RQN¶VGDWD
n0RQN
7KHGHFLVLRQWUHHVWKDWJHQHUDWHGIURPWUDLQLQJGDWDDQGSUHSURFHVVHGRQHDUH
GHSLFWHGLQILJXUH7KHGHFLVLRQWUHHSUHSURFHVVHGRQHLVGLYLGHGE\QHZDWWULEXWHD
D DQG D DQG WKH VHFRQG OHYHO QRGH LV GLYLGHG E\ D D DQG D DQG LV PXFK
VPDOOHUWKDQRULJLQDORQHDVWKHUHVXOWVRIWKHQHZDWWULEXWHD7KLVJRRGUHVXOWLVIURP
WKHIHDWXUHRISUHSURFHVVHGPHWKRGWKDWFDQJHQHUDWHWKHFRQMXQFWLRQRIDWWULEXWHVDVD
QHZDWWULEXWH
n0RQN
,QWKLVFDVHZHFDQILQGERWKDGYDQWDJHDQGOLPLWDWLRQRISURSRVHGPHWKRG7R
GHVFULEH WKH DVVRFLDWLRQV DPRQJ WKH DWWULEXWHV JLYHQ DERYH WKH GHVFULSWLYH DELOLW\ RQ
QHJDWLRQRURSHUDWLRQLVQHHGHG(YHQLQWKHOLPLWDWLRQRIGHVFULSWLYHDELOLW\RISURSRVHG
WKHVH DVVRFLDWLRQV FDQ GHVFULEH DV D DQG D DQG D DQG D DQG D DQG D DQG
D +RZHYHU EHFDXVH RI WKH UDUHQHVV RI WKHVH
SDWWHUQVLQFOXGLQJFRQMXQFWLRQRIPDQ\DWWULEXWHVWKHVHUDUHSDWWHUQVGRQ¶WJHQHUDWHGDV
DVVRFLDWLRQUXOHVRWKDWVXFKQHZDWWULEXWLRQZRXOGQRWJHQHUDWHG
n0RQN
7KHGHFLVLRQWUHHVWKDWJHQHUDWHGIURPWUDLQLQJGDWDDQGSUHSURFHVVHGRQHDUH
GHSLFWHG LQ ILJXUH ,Q WKLV GHFLVLRQ WUHH SUHSURFHVVHG GDWD RQH ZH FDQ VHH WKH
DGYDQWDJHRIWKLVPHWKRGWKDWDOVRVHHQLQ0RQN¶VFDVH7KHQHZDWWULEXWHDDDQG
DDUHDGRSWHGDVDQRGHDQGWKLVDGRSWLRQHQDEOHVWKHWUHHWRSUHGLFWZLWKQRHUURUDQG
WRPDNHWKHWUHHVPDOOHURQH
7KHFRPSDULVRQZLWKWKHDWWULEXWHYDOXHJURXSLQJ
1H[W ZH FRPSDUH WKH HIIHFW RI DWWULEXWH YDOXH JURXSLQJ SURSRVHG E\ 4XLQODQ
DQGRXUSURSRVHGPHWKRG C
C
C
^
^
^
^
C
^
^
^
^
^
^
^
^
^
C C ^ C ^ C C C ^ C ^ C ^ C C ^ C ^ C ^ C C 1TKIKPCN VTCKPKPI FCVC
CC CC CC CC GNUG
^ CC GNUG ^ CC ^ CC D 2TGRTQEGUUGF
VTCKPKPI UFCVC
)LJXUH7KHGHFLVLRQWUHHRI0RQN
7KHFRPELQHGDGRSWLRQRIDWWULEXWHYDOXHJURXSLQJDQGSURSRVHGZRUNVZHOOLQ
WKHPRVWFDVHV$VPHQWLRQHGDERYHWKHWZRPHWKRGRORJLHVKDYHGLIIHUHQWIHDWXUHDQG
DUHVXSSOHPHQWDODVJURXSLQJPHWKRGVWKDWLV³DWWULEXWH´JURXSLQJDQG³DWWULEXWHYDOXH´
JURXSLQJ2QWKDWUHDVRQWKHFRPELQHGDGRSWLRQPD\ZRUNVZHOO%RWKRIWKHPHWKRGV
FDQJHWWKHLUHIIHFWWRWKHUHVWULFWHGGDWDVHWV7RFODULI\WKHOLPLWDWLRQRIIHDVLELOLW\ZH
QHHGWRGRPRUHH[SHULPHQWDODQGORJLFDOLQYHVWLJDWLRQ
&RQFOXVLRQV
,Q WKLV SDSHU ZH SURSRVHG D QHZ GDWD SUHSURFHVVLQJ PHWKRG IRU PDNLQJ D JRRG
GHFLVLRQ WUHH XVLQJ DVVRFLDWLRQ RI DWWULEXWHV 7KLV PHWKRG GRHVQ¶W QHHG D SULRUL
NQRZOHGJHRIGDWDDQGFDOFXODWLRQFRVW7KHHIIHFWLYHQHVVRIWKHPHWKRGZDVFRQILUPHG
WKURXJK WKH HPSLULFDO H[SHULPHQWV $GGLWLRQDOO\ ZH FRPSDUHG WKH HIIHFW RI DWWULEXWH
YDOXHJURXSLQJLQ&DQGDWWULEXWHJURXSLQJLQSURSRVHGPHWKRGDQGLQYHVWLJDWHGWKH
FRPELQDWLRQHIIHFWLYHQHVVRIWKHWZRJURXSLQJPHWKRGV
$VIDUWKHUVWXG\ZHDUHJRLQJWRLQYHVWLJDWHWKHWRSLFVDVIROORZV
− 7KH DGHTXDWH VHWWLQJ RI WKUHVKROGV LQ WKH DVVRFLDWLRQ UXOH JHQHUDWLRQ $V
PHQWLRQHG LQ 6HFWLRQ WKH VHWWLQJ RI WKUHVKROGV PLQLPDO VXSSRUW DQG PLQLPDO
FRQILGHQFHPDNHDQHIIHFWWRWKHJHQHUDWHGDVVRFLDWLRQUXOHV
− ,PSOHPHQWLQJWKHSURSRVHGPHWKRGWRDSDUWRIWKHGHFLVLRQWUHHDOJRULWKP7KH
SURSRVHG PHWKRG FDQ LPSOHPHQW DV D SDUW RI WKH GHFLVLRQ WUHH DOJRULWKP VXFK DV
WKHDWWULEXWHYDOXHJURXSLQJ
5HIHUHQFHV
>@ 5$JUDZDO HW DO ³)DVW $OJRULWKPV IRU PLQLQJ DVVRFLDWLRQ UXOHV´ ,Q
3URFHHGLQJVRIWKHWK9/'%&RQIHUHQFHSS >@ 0-$%HUU\ HW DO 'DWD 0LQLQJ 7HFKQLTXHV )RU 0DUNHWLQJ 6DOHV DQG
&XVWRPHU6XSSRUW:LOH\ >@ ( %ORGHP HW DO ³'DWD'ULYHQ &RQVWUXFWLYH ,QGXFWLRQ $ 0HWKRG DQG ,WV
$SSOLFDWLRQ´,(((,QWHOOLJHQW6\VWHPVWKHLUDSSOLFDWLRQV9RO1RSS
>@ 1 /DYUDF HW DO ³$ UHOHYDQF\ ILOWHU IRU FRQVWUXFWLYH LQGXFWLRQ´ ,(((
,QWHOOLJHQW6\VWHPVWKHLUDSSOLFDWLRQV9RO1RSS
>@ û†:o2Q>f#æÅ̕onMùª^o^éý´Ù¸ãØ
és]Ø*,9S±Ž'+*fP+S¶SS
>@ &-0HU] HW DO 8&, 5HSRVLWRU\ RI PDFKLQH OHDUQLQJ GDWDEDVHV
>KWWSZZZLFVXFLHGXaPOHDUQ
0/5HSRVLWRU\KWPO@ >@
54XLQODQ & 3URJUDPV IRU 0DFKLQH /HDUQLQJ 0RUJDQ .DXIPDQQ
>@
65XVVHOO HW DO $UWLILFLDO ,QWHOOLJHQFH $ 0RGHUQ $SSURDFK 3UHQWLFH+DOO