81'(567$1',1*0$&+,1(6)52008/7,0(',$$1'
+<3(50(',$35(6(17$7,216
0DU\+HJDUW\
1+DUL1DUD\DQDQ
3DP)UHLWDV
'HSDUWPHQWRI3V\FKRORJ\8QLYHUVLW\RI&DOLIRUQLD6DQWD%DUEDUD&$
'HSDUWPHQWRI&RPSXWHU6FLHQFH6RIWZDUH(QJLQHHULQJ$XEXUQ8QLYHUVLW\
$XEXUQ$/
What is the best way to communicate to someone how a complex system, such as a
mechanical or biological system, works? 7KHWUDGLWLRQDOZD\LVWRSURGXFHDSULQWHGERRN
WKDWFRPELQHVWH[WXDOGHVFULSWLRQVZLWKGLIIHUHQWNLQGVRILOOXVWUDWLRQVSUHVHQWHGRQ
VHTXHQWLDOSDJHV:LWKWKHDGYHQWRIPXOWLPHGLDFRPSXWHUVDQGK\SHUPHGLDDXWKRULQJWRROV
KRZHYHUZHQRORQJHUQHHGWREHFRQILQHGWRWKLVWUDGLWLRQDOPHGLXP)RUH[DPSOHZHFDQ
QRZFKRRVHWRSUHVHQWYHUEDOLQIRUPDWLRQDVZULWWHQWH[WRUDVDQDXGLRFRPPHQWDU\
6LPLODUO\ZHFDQSUHVHQWGLDJUDPVDVVWDWLFRUDQLPDWHGDQGLPDJHVDVVWLOOSKRWRJUDSKVRU
YLGHRFOLSV0XOWLPHGLDSUHVHQWDWLRQVFRPELQLQJVRPHRUDOORIWKHDERYHPHGLDDUH
EHFRPLQJLQFUHDVLQJO\DYDLODEOHDVHGXFDWLRQDO&'520VDQGRQWKH:RUOG:LGH:HE
)XUWKHUPRUHZLWKWKHDYDLODELOLW\RIQHZDXWKRULQJWRROVZHGRQRWKDYHWREHFRQVWUDLQHG
E\WKHOLQHDUIRUPDWRIWKHSULQWHGERRNLQZKLFKLQIRUPDWLRQLVSUHVHQWHGDQGW\SLFDOO\
UHDGLQDVHTXHQWLDORUGHU,QK\SHUPHGLDV\VWHPVLQIRUPDWLRQLVSUHVHQWHGLQDFROOHFWLRQ
RIK\SHUOLQNHGGRFXPHQWVVRWKDWLQIRUPDWLRQFDQEHEURZVHGLQDQ\RUGHU
These technological advances present a tantalizing spectrum of choices to authors of
instructional materials. However, there are few empirically validated guidelines for how to
choose among the various capabilities for optimal design of hypermedia presentations.
7KHUHDUHVRPHZLGHO\KHOGEHOLHIVDERXWWKHHIIHFWLYHQHVVRIWKHVHQHZGHYHORSPHQWVIRU
H[DPSOHWKDWGLDJUDPPDWLFUHSUHVHQWDWLRQVDUHEHWWHUWKDQVHQWHQWLDOUHSUHVHQWDWLRQVWKUHH
GLPHQVLRQDOUHSUHVHQWDWLRQVDUHEHWWHUWKDQWZRGLPHQVLRQDORQHVDQLPDWHGGLDJUDPVDUH
PRUHHIIHFWLYHWKDQVWDWLFGLDJUDPVDQGLQWHUDFWLYHJUDSKLFVDUHEHWWHUWKDQQRQLQWHUDFWLYH
Appears as a chapter in J. Otero, J. A. Leon & A. C. Graesser (Eds.), The Psychology of Science Text Comprehension, Lawrence Erlbaum, 2001.
JUDSKLFV6FDLIH5RJHUV+RZHYHUPRVWRIWKHVHEHOLHIVKDYHQRWEHHQWHVWHG
V\VWHPDWLFDOO\
7KLVFKDSWHUGHVFULEHVDUHVHDUFKSURJUDPLQZKLFKZHGHVLJQHGSURWRW\SHGDQG
HYDOXDWHGVHYHUDOYHUVLRQVRIDK\SHUPHGLDSUHVHQWDWLRQH[SODLQLQJKRZDPHFKDQLFDO
V\VWHPZRUNV$OWKRXJKRXUSURWRW\SHVGHVFULEHDVSHFLILFPHFKDQLFDOV\VWHPWKHIOXVKLQJ
FLVWHUQRXUDSSURDFKWRWKHGHVLJQRIK\SHUPHGLDSUHVHQWDWLRQVFDQEHDSSOLHGWRH[SODLQLQJ
DQ\FRPSOH[V\VWHPIRUH[DPSOHDELRORJLFDOV\VWHPVXFKDVWKHKXPDQFLUFXODWRU\V\VWHP
,QDSDUDOOHOUHVHDUFKSURMHFWZHKDYHDSSOLHGRXUJXLGHOLQHVWRWKHGHYHORSPHQWRI
LQWHUDFWLYHDQLPDWLRQVRIFRPSXWHUDOJRULWKPVHJ+DQVHQHWDO+DQVHQ
6FKULPSVKHUDQG1DUD\DQDQDQGZHDUHFXUUHQWO\H[WHQGLQJRXUUHVHDUFKWRWKH
GRPDLQRIPHWHRURORJ\
Our design process (illustrated in Figure 1) begins with a review of research on
multimodal comprehension, leading to a preliminary model of this process. An initial
design for a hypermedia presentation is derived from this model. Then basic research is
conducted to refine and elaborate this model. At the same time, the design principles are
used to create an initial presentation, which is evaluated by assessing how people interact
with the presentation and how well they understand the mechanical systems it explains.
The experimental research and evaluation therefore proceed in parallel. The results of the
evaluation can be used to redesign the presentation and to suggest further experimental
studies (e.g., to examine the causes of particular comprehension failures). The results of the
experiments can refine the model that forms the basis of the next design cycle. This
process can be repeated to iteratively improve both the design of the hypermedia manual
and the model of comprehension upon which it is based.
This design process makes two types of prescriptions about instruction. First it
specifies the content of instruction -- what information should be communicated in the
hypermedia presentation and in what order. Second it makes prescriptions about the format
of instruction -- the media and modalities in which different types of information should be
presented. Many studies of the effects of new media and visualizations in education
investigate situations in which a new medium is used in the context of a novel method of
instruction, such as discovery or collaborative learning (e.g., Edelson, in press; White,
1993). Effects of instruction are compared to learning from traditional lecture and textbook
formats. In these situations it is difficult to evaluate whether differences in learning
outcomes are due to the new media used, the instructional method, or instructional goals
embodied in different pedagogies. In our research, we attempt to design instructional
treatments that systematically vary the content and the format of instruction, so that we can
evaluate the separate effects of each.
2XUUHVHDUFKEXLOGVRQUHFHQWUHVHDUFKH[DPLQLQJWKHFRQGLWLRQVXQGHUZKLFK
PXOWLPHGLDLQVWUXFWLRQLVPRVWHIIHFWLYHHJ0D\HU0RUHQRWKLVYROXPH0D\HU
)DUDGD\6XWFOLIIHDEMousavi, Low & Sweller, 1995). )RUH[DPSOH
0D\HUDQGKLVFROOHDJXHVKDYHVKRZQWKDWVWXGHQWVOHDUQEHWWHUIURPWH[WDQGGLDJUDPVLI
UHODWHGYLVXDODQGYHUEDOLQIRUPDWLRQLVSUHVHQWHGDVFORVHDVSRVVLEOHLQVSDFH0D\HU
*DOOLQLWKDWSHRSOHOHDUQPRUHHIIHFWLYHO\IURPDQLPDWLRQVLIWKH\DUHDFFRPSDQLHG
E\VLPXOWDQRXVFRPPHQWDULHV0D\HU6LPVDQGWKDWSHRSOHOHDUQPRUHHIIHFWLYHO\
IURPPXOWLPHGLDLQIRUPDWLRQLIYLVXDOLQIRUPDWLRQDQGYHUEDOLQIRUPDWLRQDUHSUHVHQWHGLQ
GLIIHUHQWPRGDOLWLHV0D\HU0RUHQR6LPLODUO\)DUDGD\6XWFOLIIHD
EKDYHVKRZQWKDWVWXGHQWVOHDUQPRUHHIIHFWLYHO\IURPPRYLHVDFFRPSDQLHGE\
FRPPHQWDULHVLIYLVXDOFXHVVXFKDVDUURZVDQGKLJKOLJKWLQJGUDZWKHLUDWWHQWLRQWRWKH
UHOHYDQWSDUWVRIWKHGLVSOD\DVWKH\DUHGHVFULEHGLQWKHFRPPHQWDU\2XUUHVHDUFK
DGGUHVVHVDGLIIHUHQWTXHVWLRQ:HDVNZKHWKHUPXOWLPHGLDSUHVHQWDWLRQVLQFOXGLQJ
DQLPDWLRQVFRPPHQWDULHVDQGK\SHUOLQNVOHDGWRGLIIHUHQWOHDUQLQJRXWFRPHVFRPSDUHGWR
WUDGLWLRQDOSULQWHGPHGLDZKHQERWKFRQWDLQWKHVDPHLQIRUPDWLRQDQGZKHQERWKDUH
GHVLJQHGDFFRUGLQJWRDYDLODEOHHPSLULFDOO\YDOLGDWHGJXLGHOLQHV
$0RGHORI0DFKLQH&RPSUHKHQVLRQIURP7H[WDQG'LDJUDPV
Explaining how machines work using the printed book is a well-established craft
since the 15th century (Ferguson, 1977). This medium consists of written text interspersed
with various kinds of illustrations, for example schematic diagrams, cross-sectional views,
exploded views and realistic depictions of machines. Popular books like How Things Work
(George Allen & Unwin Ltd., 1967) and The Way Things Work (Macaulay, 1988) are
excellent examples of this medium. Readers of such books typically have some specific
comprehension goal in mind. For example, a reader might want to understand how the
machine operates so that he or she can predict its behavior, operate it, or troubleshoot it.
2XUDSSURDFKWRGHYHORSLQJDK\SHUPHGLDPDQXDOZDVWRILUVWFRQVWUXFWDPRGHO
HVVHQWLDOO\DWDVNDQDO\VLVRIWKHSURFHVVRIXQGHUVWDQGLQJDPDFKLQHIURPWH[WDQG
GLDJUDPV1DUD\DQDQDQG+HJDUW\+HJDUW\4XLOLFL1DUD\DQDQ+ROPTXLVWDQG
0RUHQR7KLVPRGHOFDQEHVHHQDVDQH[WHQVLRQRIFRQVWUXFWLYLVWWKHRULHVRIWH[W
SURFHVVLQJHJ.LQWVFKZKLFKYLHZFRPSUHKHQVLRQDVDSURFHVVLQZKLFKWKH
FRPSUHKHQGHUXVHVKLVRUKHUSULRUNQRZOHGJHRIWKHGRPDLQDQGLQWHJUDWHVLWZLWKWKH
SUHVHQWHGLQIRUPDWLRQWRFRQVWUXFWDPHQWDOPRGHORIWKHREMHFWRUVLWXDWLRQGHVFULEHG,Q
DGGLWLRQWRWH[WFRPSUHKHQVLRQVNLOOVRXUPRGHOSURSRVHVWKDWFRPSUHKHQVLRQLVGHSHQGHQW
RQVSDWLDOVNLOOVIRULQWHJUDWLQJLQIRUPDWLRQLQWH[WDQGJUDSKLFVDQGHQFRGLQJDQGLQIHUULQJ
LQIRUPDWLRQIURPJUDSKLFGLVSOD\V+HJDUW\DQG6LPV+HJDUW\DQG.R]KHYQLNRY
$FFRUGLQJWRWKLVPRGHOSHRSOHFRQVWUXFWDPHQWDOPRGHORIDG\QDPLFV\VWHPE\
ILUVWGHFRPSRVLQJLWLQWRVLPSOHUFRPSRQHQWVUHWULHYLQJUHOHYDQWEDFNJURXQGNQRZOHGJH
DERXWWKHVHFRPSRQHQWVDQGPHQWDOO\HQFRGLQJWKHUHODWLRQVVSDWLDODQGVHPDQWLF
EHWZHHQFRPSRQHQWVWRFRQVWUXFWDVWDWLFPHQWDOPRGHORIWKHVLWXDWLRQ7KH\WKHQPHQWDOO\
DQLPDWHWKLVVWDWLFPHQWDOPRGHOWRFRQVWUXFWDNLQHPDWLFG\QDPLFPHQWDOPRGHORIWKH
V\VWHP:HSRVWXODWHWKDWPHQWDOPRGHOFRQVWUXFWLRQXQGHUWKHVHFLUFXPVWDQFHVUHTXLUHVWKH
IROORZLQJVWDJHV$OWKRXJKZHOLVWWKHPLQRUGHUZHDFNQRZOHGJHWKDWWKH\DUHQRWDOZD\V
DFFRPSOLVKHGLQWKLVRUGHU
$0DFKLQH'HFRPSRVLWLRQE\'LDJUDP3DUVLQJ'LDJUDPVRIPHFKDQLFDO
V\VWHPVDUHPDGHXSRIHOHPHQWDU\VKDSHVVXFKDVUHFWDQJOHVFLUFOHVDQGF\OLQGHUVZKLFK
UHSUHVHQWREMHFWVVXFKDVSLVWRQVJHDUVDQGWXEHV7KHILUVWVWHSLQFRPSUHKHQVLRQLVWR
SDUVHWKHFRQQHFWHGGLDJUDPLQWRWKHVHHOHPHQWDU\VKDSHVLHXQLWVLQWKHGLDJUDPWKDW
FRUUHVSRQGWRVXEFRPSRQHQWVRIWKHPHFKDQLFDOV\VWHP
%&RQVWUXFWLQJD6WDWLF0HQWDO0RGHOE\0DNLQJ5HSUHVHQWDWLRQDO
&RQQHFWLRQV7KHVHFRQGVWDJHLQPXOWLPRGDOFRPSUHKHQVLRQLQYROYHVPDNLQJ
DSSURSULDWHFRQQHFWLRQVLQPHPRU\DPRQJWKHFRPSRQHQWVLGHQWLILHGLQ6WDJH7KLVVWDJH
LQYROYHVPDNLQJWZRW\SHVRIFRQQHFWLRQVFRQQHFWLRQVWRSULRUNQRZOHGJHDQG
FRQQHFWLRQVWRWKHUHSUHVHQWDWLRQVRIRWKHUPDFKLQHFRPSRQHQWV
)LUVWWKHXVHUPXVWLGHQWLI\WKHFRPSRQHQWVWKDWLVPDNHFRQQHFWLRQVEHWZHHQWKH
GLDJUDPPDWLFHOHPHQWVLGHQWLILHGDW6WDJHDQGWKHLUUHDOZRUOGUHIHUHQWV)RUH[DPSOH
WKHXVHUPLJKWUHSUHVHQWWKDWDUHFWDQJOHUHSUHVHQWVDSLVWRQRUDFLUFOHUHSUHVHQWVDJHDU
3ULRUNQRZOHGJHFDQDOVRSURYLGHDGGLWLRQDOLQIRUPDWLRQDERXWFRPSRQHQWVVXFKDVZKDW
WKH\DUHW\SLFDOO\PDGHRIDQGZKHWKHUWKH\DUHULJLGRUIOH[LEOH7KLVLQIRUPDWLRQLV
YDOXDEOHLQPDNLQJLQIHUHQFHVDERXWKRZFRPSRQHQWVPRYHDQGFRQVWUDLQHDFKRWKHU©V
EHKDYLRUV
6HFRQGWKHXVHUPXVWUHSUHVHQWWKHVSDWLDOUHODWLRQVEHWZHHQGLIIHUHQWPDFKLQH
FRPSRQHQWVE\EXLOGLQJFRQQHFWLRQVEHWZHHQWKHUHSUHVHQWDWLRQVRIWKHVHFRPSRQHQWV
0D\HU6LPV,QXQGHUVWDQGLQJKRZDPDFKLQHZRUNVLQIRUPDWLRQDERXWWKH
VSDWLDOUHODWLRQVEHWZHHQPHFKDQLFDOFRPSRQHQWVIRUPVWKHEDVLVIRULQIHUHQFHVDERXWWKH
PRWLRQVRIFRPSRQHQWVEHFDXVHWKHVHVSDWLDOUHODWLRQVGHWHUPLQHKRZFRPSRQHQWVDIIHFW
DQGFRQVWUDLQHDFKRWKHU©VPRWLRQV
&0DNLQJ5HIHUHQWLDO&RQQHFWLRQV:KHQGLDJUDPVDUHDFFRPSDQLHGE\WH[W
DQDGGLWLRQDOVWDJHLQFRPSUHKHQVLRQLVWKDWRIUHVROYLQJFRUHIHUHQFHEHWZHHQWKHWZR
PHGLDLHPDNLQJUHIHUHQWLDOOLQNVEHWZHHQDQRXQSKUDVHLQWKHWH[WHJWKHSLVWRQ
DQGWKHGLDJUDPPDWLFXQLWWKDWGHSLFWVLWVUHIHUHQWHJDUHFWDQJOH1RYDN7KLV
VWHSLVFUXFLDOWRFRQVWUXFWLQJDQLQWHJUDWHGUHSUHVHQWDWLRQRIWKHFRPPRQUHIHUHQWRIWKH
WH[WDQGGLDJUDPLQPHPRU\DVRSSRVHGWRVHSDUDWHVXUIDFHOHYHOUHSUHVHQWDWLRQVRIWKHWH[W
DQGGLDJUDP
''HWHUPLQLQJWKH&DXVDO&KDLQRI(YHQWV:KHQDVNHGWRSUHGLFWWKHEHKDYLRU
RIPDFKLQHVIURPVWDWLFGLDJUDPVSHRSOHWHQGWRUHDVRQDERXWPDFKLQHRSHUDWLRQDORQJWKH
GLUHFWLRQRIFDXVDOSURSDJDWLRQLQWKHPDFKLQH+HJDUW\1DUD\DQDQ6XZD
0RWRGD7KHUHIRUHZHK\SRWKHVL]HDIRXUWKVWDJHRIFRPSUHKHQVLRQWKDW
LQYROYHVLGHQWLI\LQJWKHSRWHQWLDOFDXVDOFKDLQVRIHYHQWVLQWKHRSHUDWLRQRIWKHPDFKLQH
RUOLQHVRIDFWLRQLQWKHPDFKLQH
(&RQVWUXFWLQJD'\QDPLF0HQWDO0RGHOE\0HQWDO6LPXODWLRQDQG5XOH
EDVHG,QIHUHQFH7KHILQDOVWDJHRIFRPSUHKHQVLRQLVWKDWRIFRQVWUXFWLQJDG\QDPLF
PHQWDOPRGHORIWKHPDFKLQHLHDUHSUHVHQWDWLRQRIKRZWKHFRPSRQHQWVPRYHDQG
FRQVWUDLQHDFKRWKHU©VPRWLRQZKHQWKHPDFKLQHLVLQRSHUDWLRQ2XUSUHYLRXVUHVHDUFK
+HJDUW\1DUD\DQDQ6XZD0RWRGDVXJJHVWVWKDWSHRSOHFDQRIWHQ
LQIHUWKLVLQIRUPDWLRQIURPDVWDWLFGLDJUDPE\DSURFHVVZKLFKZHFDOOPHQWDODQLPDWLRQ
&RPSXWDWLRQDOPRGHOVDQGHPSLULFDOHYLGHQFHVXJJHVWWKDWWKLVLVDQLQFUHPHQWDOSURFHVVLQ
ZKLFKWKHUHDVRQHUFRQVLGHUVWKHFRPSRQHQWVRUVXEV\VWHPVLQGLYLGXDOO\DVVHVVHVWKH
LQIOXHQFHVDFWLQJRQHDFKLQIHUVWKHUHVXOWLQJEHKDYLRURIHDFKDQGWKHQSURFHHGVWR
FRQVLGHUKRZWKLVEHKDYLRUDIIHFWVWKHQH[WFRPSRQHQWRUVXEV\VWHPLQWKHFDXVDOFKDLQ,W
GHSHQGVRQERWKSULRUNQRZOHGJHHJUXOHVWKDWJRYHUQWKHEHKDYLRURIWKHV\VWHPLQ
TXHVWLRQDQGVSDWLDOYLVXDOL]DWLRQSURFHVVHV
,QLWLDO'HVLJQRID+\SHUPHGLD0DQXDO
,QDUHFHQWSDSHU1DUD\DQDQ+HJDUW\ZHLGHQWLILHGSRWHQWLDOVRXUFHVRI
FRPSUHKHQVLRQHUURUWKDWXVHUVPLJKWHQFRXQWHULQHDFKVWDJHRIFRPSUHKHQVLRQDQG
GHYHORSHGK\SHUPHGLDGHVLJQJXLGHOLQHVLQWHQGHGWRDPHOLRUDWHWKHVHGLIILFXOWLHV7KHVH
ZHUHDSSOLHGWRGHYHORSDSURWRW\SHK\SHUPHGLDPDQXDOWKDWH[SODLQVKRZDWRLOHWWDQND
IOXVKLQJFLVWHUQZRUNV$OWKRXJKWKLVLVDIDPLOLDUGHYLFHLWVLQQHUZRUNLQJVDUHQRW
LQWXLWLYHO\REYLRXV,WLVUHODWLYHO\FRPSOH[KDYLQJWZRPDLQVXEV\VWHPVDZDWHURXWSXW
V\VWHPWKDWIOXVKHVZDWHULQWRWKHWRLOHWWDQNDQGDZDWHULQOHWV\VWHPWKDWUHILOOVWKHWDQNIRU
WKHQH[WXVH([SODLQLQJDIOXVKLQJFLVWHUQSUHVHQWVLQWHUHVWLQJFKDOOHQJHVIRURXUWKHRU\
EHFDXVHLWVRSHUDWLRQLQYROYHVWZRFDXVDOFKDLQVRIHYHQWVWKDWRFFXULQWDQGHPEXWDUHDOVR
WHPSRUDULO\GHSHQGHQWRQHDFKRWKHU7KHSDUWLFXODUWRLOHWWDQNH[SODLQHGLQRXUPDQXDO
VKRZQLQ)LJXUHDOVRFRQWDLQVDVLSKRQUDLVLQJWKHLQWHUHVWLQJTXHVWLRQRIKRZWR
H[SODLQDEDVLFSK\VLFVSULQFLSOHLQWKHFRQWH[WRIH[SODLQLQJKRZDVSHFLILFPDFKLQHZRUNV
7KHLQLWLDOGHVLJQIRURXUK\SHUPHGLDPDQXDOFRQWDLQHGVHYHQVHFWLRQVGHVLJQHGWR
JXLGHXVHUVWKURXJKWKHVWDJHVRIFRPSUHKHQVLRQLQRXUPRGHO)LJXUHFRQWDLQVDQLPDJH
RIDVFUHHQIURPWKHILUVWVHFWLRQRIWKLVPDQXDO
6HFWLRQ¦3DUWV§7KHSULPDU\REMHFWLYHVRIWKLVVHFWLRQDUHWRKHOSWKHXVHULQ
GHFRPSRVLQJWKHPDFKLQHLQWRLWVFRPSRQHQWVDQGEXLOGLQJUHIHUHQWLDOFRQQHFWLRQVEHWZHHQ
HOHPHQWVRIWH[WDQGGLDJUDPV3UHYLRXVUHVHDUFKKDVSRLQWHGRXWWKDWGLDJUDPVDUHRIWHQ
XQGHUVSHFLILHGLQWKDWWKH\GRQRWFRQWDLQHQRXJKLQIRUPDWLRQIRUDXVHUWRLGHQWLI\ZKHWKHU
WZRRUPRUHFRQQHFWHGXQLWVLQDGLDJUDPUHSUHVHQWVHSDUDWHREMHFWVRUSDUWVRIDVLQJOH
REMHFW1RYDN)RUH[DPSOHDGLDJUDPHOHPHQWVXFKDVDOLQHPLJKWUHSUHVHQWWKH
HGJHRIDQREMHFWRUDQREMHFWLWVHOIDURSH)XUWKHUPRUHUHVROYLQJFRUHIHUHQFHEHWZHHQ
WH[WDQGJUDSKLFVFDQEHDVRXUFHRIFRPSUHKHQVLRQGLIILFXOW\0D\HU0D\HU6LPV
7RIDFLOLWDWHXVHUVLQWKHVHSURFHVVHVWKHILUVWVHFWLRQRIWKHPDQXDOSUHVHQWVD
FURVVVHFWLRQDOGLDJUDPRIWKHWRLOHWWDQNLQZKLFKGHFRPSRVLWLRQLVIDFLOLWDWHGE\ODEHOLQJ
WKHGLIIHUHQWIXQFWLRQDOFRPSRQHQWVE\SUHVHQWLQJWKHPLQGLIIHUHQWFRORUVDQGE\DOORZLQJ
XVHUVWRFOLFNRQWKHODEHORIDQ\FRPSRQHQWDQGKDYHWKHFRPSRQHQWKLJKOLJKWHGLQWKH
GLDJUDPVHH)LJXUH
6HFWLRQ¦6XEV\VWHPV§7KHGHFRPSRVLWLRQRIDPHFKDQLFDOV\VWHPLVRIWHQ
KLHUDUFKLFDOVXFKWKDWWKHV\VWHPEUHDNVGRZQLQWRIXQFWLRQDOVXEV\VWHPVZKLFKFDQ
WKHPVHOYHVEHEURNHQGRZQLQWRPRUHHOHPHQWDU\FRPSRQHQWV7KHREMHFWLYHRI6HFWLRQ
LVWRIDFLOLWDWHLGHQWLI\LQJWKHIXQFWLRQDOVXEV\VWHPVWRZKLFKWKHFRPSRQHQWVEHORQJ WKH
ZDWHURXWSXWV\VWHPDQGWKHZDWHULQSXWV\VWHP7KHILUVWSUHVHQWDWLRQLQWKLVVHFWLRQLVD
VFKHPDWLFGLDJUDPZLWKDFFRPSDQ\LQJWH[WWKDWRXWOLQHVWKHYDULRXVVXEV\VWHPVRIWKH
PDFKLQHWDQN7KLVSUHVHQWDWLRQDOVRDOORZVWKHXVHUWRDQLPDWHDQH[SORGHGYLHZRIWKH
WDQNLQZKLFKWKHLQSXWDQGRXWSXWV\VWHPVDUHVHSDUDWHGLQVSDFH,QWZRIXUWKHU
SUHVHQWDWLRQVWKHXVHUYLHZVGLDJUDPVKLJKOLJKWLQJWKHFRPSRQHQWVRIILUVWWKHRXWSXW
V\VWHPDQGWKHQWKHLQSXWV\VWHPHDFKDFFRPSDQLHGE\WH[WGHVFULELQJWKHIXQFWLRQRIWKH
UHOHYDQWVXEV\VWHP
6HFWLRQ¦&RQQHFWLRQV§7KHREMHFWLYHRI6HFWLRQLVWRIDFLOLWDWHFRQVWUXFWLRQ
RIDVWDWLFPHQWDOPRGHORIWKHPDFKLQH7KLVLQFOXGHVLGHQWLI\LQJWKHUHDOZRUOG
FRPSRQHQWVWRZKLFKGLDJUDPXQLWVUHIHUDQGUHWULHYLQJSULRUNQRZOHGJHDERXWWKHVH
FRPSRQHQWVZKDWWKH\DUHPDGHRIWKHLUSULQFLSOHVRIRSHUDWLRQHWF7KLVLQIRUPDWLRQLV
QRWLQFOXGHGLQKLJKO\VFKHPDWL]HGGHSLFWLRQVEXWLVQHFHVVDU\IRUFRUUHFWLQIHUHQFHVDERXW
KRZWKHPDFKLQHZRUNV6FKZDUW]%ODFN,QWKLVVHFWLRQWKHXVHULVVKRZQD
FURVVVHFWLRQDOYLHZRIWKHWRLOHWWDQNDQGWDNHQRQDJXLGHGWRXURIWKHFRPSRQHQWV7H[W
GHVFULEHVHDFKRIWKHFRPSRQHQWVLQWXUQSRLQWLQJRXWLWVOLQNDJHVWRRWKHUV\VWHP
FRPSRQHQWVDQGRWKHULQIRUPDWLRQWKDWLVQRWYLVLEOHLQWKHGLDJUDPVXFKDVLWVPDWHULDO
FRPSRVLWLRQDQGIXQFWLRQ2QO\WKHWH[WDERXWRQHFRPSRQHQWLVYLVLEOHDWDWLPHDQGWRDLG
WKHFRQVWUXFWLRQRIUHIHUHQWLDOFRQQHFWLRQVWKHFRPSRQHQWLQTXHVWLRQLVKLJKOLJKWHGLQWKH
GLDJUDPFRQFXUUHQWO\ZLWKWKHSUHVHQWDWLRQRIWKHWH[WXDOGHVFULSWLRQ
6HFWLRQ¦4XHVWLRQV§7KLVVHFWLRQLVGHVLJQHGWRHQFRXUDJHWKHXVHUWRUHDVRQ
DERXWWKHFDXVDOLW\DQGG\QDPLFVRIWKHPDFKLQH3UHYLRXVUHVHDUFKKDVVKRZQWKDWWKH
JHQHUDWLRQRILGHDVRUVHOIH[SODQDWLRQVLPSURYHVOHDUQLQJ&KLGH/HHXZ&KLX
/D9DQFKHU7KHUHIRUHRQHRIRXUJXLGHOLQHVLVWKDWXVHUVVKRXOGEHHQFRXUDJHGWR
PHQWDOO\DQLPDWHWKHPDFKLQHLHDWWHPSWWRSUHGLFWLWVEHKDYLRUEHIRUHWKH\DUHVKRZQ
DQLPDWLRQVRIWKHFDXVDOFKDLQDQGPRYHPHQWRIWKHPDFKLQH6HFWLRQGRHVWKLVE\
SUHVHQWLQJXVHUVZLWKWKHVWDWLFGLDJUDPDQGDVHWRIPXOWLSOHFKRLFHTXHVWLRQVLQZKLFK
XVHUVDUHDVNHGWRLPDJLQHWKDWDFRPSRQHQWRIWKHV\VWHPLVPRYLQJLQDJLYHQZD\DQG
KDYHWRSUHGLFWKRZDQRWKHUFRPSRQHQWRIWKHV\VWHPZLOOEHPRYLQJ8VHUVDUHJLYHQ
IHHGEDFNRQWKHLUDQVZHUV
6HFWLRQ¦&DXVDO&KDLQ§7KHSXUSRVHRIWKLVVHFWLRQLVWRKHOSWKHXVHULQ
XQGHUVWDQGLQJWKHSURSDJDWLRQRIFDXVDOLW\LQWKHHQWLUHPDFKLQH,WFRQWDLQVDQDXGLR
FRPPHQWDU\GHVFULELQJWKHRSHUDWLRQRIWKHWRLOHWWDQNH[SODLQLQJFDXVDOSURSDJDWLRQ
ZLWKLQDQGDFURVVWKHZDWHURXWSXWDQGZDWHULQOHWVXEV\VWHPV6\QFKURQL]HGZLWKWKH
FRPPHQWDU\WKHFRUUHVSRQGLQJFRPSRQHQWVDQGSDWKVRIFDXVDOSURSDJDWLRQDUH
KLJKOLJKWHGLQWKHVWDWLFVFKHPDWLFGLDJUDP$Q¦H[SODLQVLSKRQSULQFLSOH§EXWWRQJLYHV
XVHUVWKHRSWLRQWRDFFHVVDGHVFULSWLRQRIWKHIXQGDPHQWDOSK\VLFVSULQFLSOHXQGHUO\LQJWKH
RSHUDWLRQRIWKHZDWHURXWSXWV\VWHPWKHVLSKRQVHH6HFWLRQ
6HFWLRQ¦0RYHPHQW§7KHREMHFWLYHRIWKLVVHFWLRQLVWRFRQYH\KRZWKH
PDFKLQHDFWXDOO\RSHUDWHVE\GHVFULELQJDQGVKRZLQJWKHPRYHPHQWVRILWVFRPSRQHQWV,W
LQFOXGHVWKHVDPHDXGLWRU\FRPPHQWDU\DV6HFWLRQ$QDQLPDWLRQLQZKLFKDOOFRPSRQHQW
EHKDYLRUVDUHFRQFXUUHQWO\VKRZQFRQWLQXRXVO\F\FOHVWKURXJKWKHRSHUDWLRQRIWKHGHYLFH
GXULQJWKLVFRPPHQWDU\$WWKHHQGRIWKLVQDUUDWHGDQLPDWLRQXVHUVKDYHWKHRSWLRQRI
UHSOD\LQJLWRUYLHZLQJDVLOHQWDQLPDWLRQ$Q¦H[SODLQVLSKRQSULQFLSOH§EXWWRQDJDLQJLYHV
XVHUVWKHRSWLRQWRDFFHVVDGHVFULSWLRQRIKRZDVLSKRQZRUNV
6HFWLRQ¦7KH6LSKRQ§7KHSXUSRVHRIWKLVVHFWLRQLVWRH[SODLQDIXQGDPHQWDO
SK\VLFVSULQFLSOHXQGHUO\LQJWKHEHKDYLRURIWKHWRLOHWWDQNWKHVLSKRQ,WFRQWDLQVD
VFKHPDWLFGLDJUDPRIWKHPDFKLQHZLWKWKHVLSKRQEHOODQGSLSHKLJKOLJKWHG7KHWH[W
GHVFULEHVKRZDVLSKRQZRUNVDQGKRZLWDSSOLHVWRWKHRSHUDWLRQRIWKHZDWHURXWSXW
V\VWHPRIWKHWRLOHWWDQN8VHUVFDQDOVRYLHZDVLOHQWDQLPDWLRQRIWKHVLSKRQHIIHFW7KLV
VHFWLRQLVQRWSDUWRIWKHVHTXHQWLDOSDWKWKURXJKVHFWLRQVRIWKHPDQXDO,QVWHDGLWLV
UHDFKHGIURP6HFWLRQVRULQUHVSRQVHWRDXVHUFOLFNLQJWKH¦H[SODLQVLSKRQSULQFLSOH§
EXWWRQ
1DYLJDWLRQDQG*XLGDQFH7KHPRGHORIPDFKLQHFRPSUHKHQVLRQRXWOLQHGDERYH
VXJJHVWVDVHTXHQFHRIFRPSUHKHQVLRQVWDJHVVXFKWKDWWKHODWHUVWDJHVDUHDWOHDVW
VRPHZKDWGHSHQGHQWRQVXFFHVVIXOFRPSOHWLRQRIWKHHDUOLHURQHV,QSDUWLFXODU
UHSUHVHQWDWLRQRIWKHVSDWLDOUHODWLRQVEHWZHHQGHYLFHFRPSRQHQWV6WDJH%LVGHSHQGHQW
RQILUVWGHFRPSRVLQJWKHV\VWHPLQWRLQGLYLGXDOFRPSRQHQWV6WDJH$DQGWKHVWDJHVRI
ILQGLQJOLQHVRIDFWLRQDQGFRQVWUXFWLQJDNLQHPDWLFPHQWDOPRGHO6WDJHV'DQG(DUH
GHSHQGHQWRQVXFFHVVIXOFRQVWUXFWLRQRIDVWDWLFPRGHORIWKHPDFKLQH6WDJHV%DQG&
7KHUHIRUHLQWKHLQLWLDOYHUVLRQRIWKHK\SHUPHGLDSUHVHQWDWLRQZHFRQVWUDLQHGQDYLJDWLRQ
8VHUVVWXGLHGWKHVHFWLRQVRIWKHPDQXDOLQRUGHUVWDUWLQJZLWK6HFWLRQ¦3DUWV§$Q
H[FHSWLRQWRWKLVIRUZDUGWUDYHUVDOZDVVHFWLRQ¦WKH6LSKRQ§ZKLFKFRXOGEHDFFHVVHG
RSWLRQDOO\IURP6HFWLRQRU2QFHWKHXVHUKDGVHHQDVHFWLRQKHRUVKHZDVDOORZHGWR
UHWXUQWRLWDWDQ\WLPH8VHUVSURFHHGHGIURPVHFWLRQWRVHFWLRQXVLQJDQRYHUDOO¦PDS§RI
WKHK\SHUPHGLDPDQXDOZKLFKVKRZHGLFRQVIRUDOOWKHVHFWLRQVRIWKHPDQXDODQGZDV
FRORUFRGHGWRVKRZXVHUVWKHLUFXUUHQWSODFHLQWKHV\VWHPWKHVHFWLRQVWKDWWKH\KDG
DOUHDG\VWXGLHGDQGWKHVHFWLRQVWKDWWKH\ZHUHDOORZHGWRPRYHWRDWDQ\JLYHQWLPHVHH
WKHXSSHUOHIWFRUQHURI)LJXUH6HFWLRQVRIWKHPDSZHUHKLJKOLJKWHGDQGEHFDPHPRXVH
VHQVLWLYHEDVHGRQWKHXVHU©VKLVWRU\VRWKDWFOLFNLQJRQDVHFWLRQZDVRQO\VXFFHVVIXOLILW
ZDVFRQVLVWHQWZLWKWKHVHQDYLJDWLRQFRQVWUDLQWV
(PSLULFDO6WXGLHVRI0XOWLPHGLDDQG+\SHUPHGLD&RPSUHKHQVLRQ
:HQRZVXPPDUL]HWKHUHVXOWVRIVHYHUDOH[SHULPHQWVGHVLJQHGWRHYDOXDWHGLIIHUHQW
DVSHFWVRIWKHK\SHUPHGLDSUHVHQWDWLRQ)LUVWZHH[DPLQHZKHWKHUSHRSOHOHDUQGLIIHUHQWO\
IURPK\SHUPHGLDSUHVHQWDWLRQVLQFOXGLQJK\SHUOLQNVDQGDQLPDWLRQVFRPSDUHGWRSULQWHG
PDQXDOVFRQWDLQLQJWKHVDPHYLVXDODQGYHUEDOLQIRUPDWLRQ6HFRQGZHDVNZKHWKHUSHRSOH
OHDUQGLIIHUHQWO\IURPK\SHUPHGLDSUHVHQWDWLRQVWKDWGLUHFWWKHOHDUQHUWRYLHZWKH
LQIRUPDWLRQLQDVSHFLILFRUGHUDVRSSRVHGWRWKRVHWKDWDOORZIUHHQDYLJDWLRQ7KLUGZH
DVNZKHWKHUSHRSOHOHDUQPRUHIURPDQDQLPDWLRQRIDPHFKDQLFDOV\VWHPLIWKH\ILUVWWU\WR
PHQWDOO\DQLPDWHWKHV\VWHP
'RSHRSOHOHDUQGLIIHUHQWO\IURPK\SHUPHGLDSUHVHQWDWLRQVFRPSDUHGWRSULQWHG
PDWHULDOVFRQWDLQLQJWKHVDPHLQIRUPDWLRQ"
7KHLQLWLDOK\SHUPHGLDPDQXDOGHVLJQHPERGLHGJXLGHOLQHVDERXWERWKWKHIRUPDW
DQGWKHFRQWHQWRILQVWUXFWLRQ7KHUHIRUHLWZDVLPSRUWDQWWRHYDOXDWHWKHVHSDUDWHHIIHFWVRI
IRUPDWGLIIHUHQFHVDQGFRQWHQWGLIIHUHQFHVEHWZHHQRXUPDQXDODQGW\SLFDOLQVWUXFWLRQDO
PDWHULDOV)LUVWWKHK\SHUPHGLDPDQXDOGLIIHUHGIURPVWDQGDUGSULQWHGPDQXDOVLQWKH
IRUPDWRILQVWUXFWLRQLWLQFOXGHGG\QDPLFHOHPHQWVLHFRQVWUDLQHGQDYLJDWLRQ
K\SHUOLQNVDQGDQLPDWLRQVWKDWDUHQRWDYDLODEOHLQWKHSULQWPHGLXP,QWKHILUVWH[SHULPHQW
WKDWZHGHVFULEHKHUHZHFRPSDUHWKHOHDUQLQJRXWFRPHVRILQWHUDFWLQJZLWKWKH
K\SHUPHGLDPDQXDOVWRWKRVHRIVWXG\LQJSULQWHGWH[WDQGGLDJUDPVFRQWDLQLQJWKHVDPH
FRQWHQW
([SHULPHQWD
+HJDUW\4XLOLFL1DUD\DQDQ+ROPTXLVW0RUHQRFRPSDUHGOHDUQLQJIURP
WKHK\SHUPHGLDSUHVHQWDWLRQGHVFULEHGDERYHWROHDUQLQJIURPDSDSHUSULQWRXWRIWKHWH[W
DQGGLDJUDPVXVHGLQWKHPDQXDOZKLFKZHZLOOUHIHUWRDVWKHIXOOWH[WFRQGLWLRQDQG
OHDUQLQJIURPDSDSHUSULQWRXWRIWKHODEHOHGGLDJUDPRIWKHWRLOHWWDQNDFFRPSDQLHGE\
SULQWHGWH[WGHVFULELQJWKHPRYHPHQWRIWKHFRPSRQHQWVLQRUGHURIWKHFDXVDOFKDLQWKH
FDXVDOWH[WFRQGLWLRQ7KHPDLQGLIIHUHQFHVEHWZHHQWKHK\SHUPHGLDPDQXDODQGIXOOWH[W
FRQGLWLRQVZHUHWKHDEVHQFHRIK\SHUOLQNVDQGDQLPDWLRQVLQWKHIXOOWH[WFRQGLWLRQDQGWKH
SUHVHQWDWLRQRIWKHYHUEDOGHVFULSWLRQRIWKHFDXVDOFKDLQDQGPRYHPHQWRIFRPSRQHQWV
ZKLFKZDVSUHVHQWHGYLVXDOO\DVWH[WUDWKHUWKDQDXGLWRUDOO\7KHPDLQGLIIHUHQFHEHWZHHQ
WKHIXOOWH[WDQGFDXVDOWH[WFRQGLWLRQVZDVWKDWWKHFDXVDOWH[WGLGQRWLQFOXGHWKHVHFWLRQV
H[SOLFLWO\SRLQWLQJRXWWKHVXEV\VWHPVRIWKHWRLOHWWDQNWKHFRQQHFWLRQVEHWZHHQ
FRPSRQHQWVDQGWKHPDWHULDOFRPSRVLWLRQRIWKHFRPSRQHQWV
7ZHQW\XQGHUJUDGXDWHVWXGHQWVVWXGLHGKRZDWRLOHWWDQNZRUNVIURPWKHPDQXDO
PRUHVWXGHQWVVWXGLHGWKHIXOOWH[WDQGPRUHVWXGHQWVVWXGLHGWKHFDXVDOWH[W$OO
VWXGHQWVZHUHWLPHGDVWKH\VWXGLHGWKHPDWHULDOV$IWHUZDUGVWKHLUFRPSUHKHQVLRQZDV
WHVWHGZLWKWKHIROORZLQJW\SHVRITXHVWLRQV
0HQWDODQLPDWLRQTXHVWLRQVZKLFKUHTXLUHGWKHPWRSUHGLFWKRZDFRPSRQHQWRIWKH
V\VWHPZRXOGEHPRYLQJJLYHQWKDWDQRWKHUFRPSRQHQWZDVPRYLQJLQDVSHFLILHG
ZD\)RUH[DPSOHRQHTXHVWLRQDVNHG¦,PDJLQHWKHFRQQHFWLQJURGLVPRYLQJXS
:KDWLVKDSSHQLQJWRWKHIORDWDUP"§
)XQFWLRQ4XHVWLRQV4XHVWLRQVDERXWWKHIXQFWLRQRIDFRPSRQHQWLQWKHV\VWHPHJ
¦:KDWLVWKHIXQFWLRQRIWKHIORDWDQGIORDWDUP"
¦)DXOWEHKDYLRU§TXHVWLRQVZKLFKGHVFULEHGDSDUWLFXODUIDXOWLQDFRPSRQHQWRIWKH
V\VWHPDQGDVNHGSDUWLFLSDQWVWRSUHGLFWKRZWKHV\VWHPZRXOGEHKDYHHJ¦+RZ
ZRXOGWKHWDQNIXQFWLRQEHDIIHFWHGLIWKHLQOHWYDOYHZDVVWXFNLQWKHZDWHULQOHWSLSH"
OLVWDOOSRVVLEOHDQVZHUV§
¦7URXEOHVKRRWLQJ§TXHVWLRQVZKLFKGHVFULEHGIDXOW\EHKDYLRURIWKHV\VWHPDQG
DVNHGZKDWFRPSRQHQWVPLJKWEHIDXOW\HJ¦6XSSRVHWKDWDIWHUIOXVKLQJWKHWRLOHW\RX
QRWLFHWKDWZDWHULVFRQWLQXRXVO\UXQQLQJLQWRWKHWDQN:KDWFRXOGEHZURQJ"OLVWDOO
SRVVLEOHDQVZHUV§
$EDFNJURXQGTXHVWLRQQDLUHDVNHGSDUWLFLSDQWVWROLVWDQ\FRXUVHVWKH\KDGWDNHQLQ
SK\VLFVPHFKDQLFVRUPHFKDQLFDOHQJLQHHULQJWROLVWDQ\PHFKDQLFDORUHOHFWULFDOLWHPV
WKDWWKH\KDGDWWHPSWHGWRUHSDLUDQGVSHFLILFDOO\ZKHWKHUWKH\KDGHYHUWULHGWRIL[DWRLOHW
FKDQJHWKHRLOLQDFDURUXQEORFNDGUDLQ)LQDOO\WKH\ZHUHDVNHGWRUDWHRQDVFDOHRIWR
KRZLQWHUHVWLQJWKH\WKRXJKWWKHPDWHULDOZDVZLWKPHDQLQJQRWLQWHUHVWLQJDWDOODQG
PHDQLQJYHU\LQWHUHVWLQJ
5HVXOWV7KHPDLQGLIIHUHQFHEHWZHHQWKHWKUHHJURXSVZDVLQWKHLUVWXG\WLPHV7KH
FDXVDOJURXSVSHQWWKHOHDVWWLPHVWXG\LQJWKHPDWHULDOVPLQXWHV6' $OWKRXJK
WKHIXOOWH[WDQGK\SHUPHGLDPDQXDOJURXSVUHFHLYHGWKHVDPHFRQWHQWWKHODWWHUJURXS
VSHQWORQJHUPLQXWHV6' WKDQWKHIXOOWH[WJURXSPLQXWHV6'
7KLVPD\UHIOHFWWLPHOHDUQLQJDQGLQWHUDFWLQJZLWKWKHFRPSXWHULQWHUIDFHDQGWKH
IDFWWKDWSDUWLFLSDQWVKDGWRYLHZWKHZKROHDQLPDWLRQVLQ6HFWLRQVDQGRIWKHPDQXDO
7KHVHDQLPDWLRQVZHUHSOD\HGDWDFRQVWDQWUDWHWKDWZDVSUREDEO\VORZHUWKDQWKHWLPH
WDNHQWRUHDGWKHUHOHYDQWWH[WDQGLQWHJUDWHLWZLWKWKHGLDJUDPLQWKHIXOOWH[WFRQGLWLRQ,W
PLJKWDOVRUHIOHFWKLJKHUPRWLYDWLRQLQGXFHGE\WKHK\SHUPHGLDSUHVHQWDWLRQ+RZHYHUWKH
W\SHRILQVWUXFWLRQGLGQRWLQIOXHQFHSDUWLFLSDQWV©UDWLQJVRILQWHUHVWLQWKHPDWHULDOV7KH
PHDQUDWLQJIRUWKHK\SHUPHGLDJURXSIRUWKHIXOOWH[WJURXSDQGIRUWKH
FDXVDOWH[WJURXS
7KHVHGLIIHUHQFHVLQVWXG\WLPHVGLGQRWOHDGWRGLIIHUHQFHVLQFRPSUHKHQVLRQ7KHUH
ZHUHQRVLJQLILFDQWGLIIHUHQFHVEHWZHHQWKHWKUHHJURXSVRQDQ\RIWKHFRPSUHKHQVLRQ
PHDVXUHV7KHRQO\HIIHFWRISULRUNQRZOHGJHZDVRQWKHPHQWDODQLPDWLRQTXHVWLRQV
SDUWLFLSDQWVZLWKPRUHSUDFWLFDOH[SHULHQFHZLWKPDFKLQHVLHWKRVHZKRKDGDWWHPSWHGWR
UHSDLUPRUHPDFKLQHVKDGKLJKHUPHQWDODQLPDWLRQVFRUHV6' FRPSDUHGWR
WKRVHZLWKOHVVSUDFWLFDOH[SHULHQFH6' ) S1RQHRI
WKHPHDVXUHVRIWUDLQLQJRUH[SHULHQFHKDGVLJQLILFDQWLQWHUDFWLRQVZLWKWKHW\SHRI
LQVWUXFWLRQVXJJHVWLQJWKDWWKHGLIIHUHQWW\SHVRILQVWUXFWLRQZHUHQRWGLIIHUHQWLDOO\HIIHFWLYH
IRULQGLYLGXDOVZLWKGLIIHUHQWDPRXQWVRISULRUNQRZOHGJHRUH[SHULHQFH
'LVFXVVLRQ,QVXPPDU\([SHULPHQWDVKRZHGQRGLIIHUHQFHVLQOHDUQLQJ
RXWFRPHVEHWZHHQWKHWKUHHJURXSVGHVSLWHGLIIHUHQFHVLQVWXG\WLPHV7KHVLPLODULW\LQ
SHUIRUPDQFHEHWZHHQWKHIXOOWH[WDQGK\SHUPHGLDJURXSVLQGLFDWHVWKDWZKHWKHUSHRSOH
YLHZLQIRUPDWLRQRQDFRPSXWHUVFUHHQRURQSDSHULQWHUDFWZLWKDK\SHUPHGLDLQWHUIDFH
LQFOXGLQJK\SHUOLQNHGWH[WDQGGLDJUDPVUDWKHUWKDQUHDGLQJSULQWHGPDWHULDODQGYLHZ
DQLPDWLRQVZLWKFRPPHQWDULHVUDWKHUWKDQVWDWLFWH[WDQGGLDJUDPVGRHVQRWKDYHVLJQLILFDQW
HIIHFWVRQOHDUQLQJWKLVPDWHULDO7KLVUHVXOWVXJJHVWVWKDWWKHPHGLDDQGPRGDOLWLHVWKURXJK
ZKLFKLQIRUPDWLRQLVSUHVHQWHGGRQRWDIIHFWFRPSUHKHQVLRQZKHQWKHLQVWUXFWLRQLV
GHVLJQHGDFFRUGLQJWRRXUFRPSUHKHQVLRQPRGHO
$FRPSDULVRQRIWKHIXOOWH[WDQGK\SHUPHGLDJURXSVZLWKWKHFDXVDOWH[WJURXSV
DOVRUHYHDOVQRGLIIHUHQFHVLQOHDUQLQJRXWFRPHV7KLVVXJJHVWVWKDWH[SOLFLWO\GHVFULELQJWKH
IXQFWLRQVRIWKHWRLOHWWDQNVXEV\VWHPVWKHFRQQHFWLRQVEHWZHHQFRPSRQHQWVDQGWKH
PDWHULDOFRPSRVLWLRQRIWKHFRPSRQHQWVGRHVQRWDIIHFWSHRSOH©VDELOLW\WRXQGHUVWDQGWKH
FDXVDOFKDLQRUNLQHPDWLFVRIWKHGHYLFH2QILUVWJODQFHWKLVDSSHDUVWRFRQWUDGLFWWKH
SULQFLSOHVRIRXUWKHRUHWLFDOPRGHO+RZHYHULWLVSRVVLEOHWKDWWKHODEHOHGGLDJUDPRIWKH
GHYLFHSURYLGHGWRSDUWLFLSDQWVLQWKHFDXVDOWH[WFRQGLWLRQSURYLGHGVXIILFLHQWLQIRUPDWLRQ
DERXWWKHGHFRPSRVLWLRQRIWKHGLDJUDPDQGWKHVSDWLDOUHODWLRQVEHWZHHQFRPSRQHQWVVR
WKDWWKHDGGLWLRQDOWH[WGHVFULELQJWKHVHDVSHFWVRIWKHV\VWHPZDVVXSHUIOXRXV)XUWKHUPRUH
VLQFHDWRLOHWWDQNLVDKRXVHKROGLWHPSDUWLFLSDQWVPLJKWKDYHDOUHDG\EHHQIDPLOLDUZLWK
WKHIXQFWLRQVDQGPDWHULDOFRPSRVLWLRQRILWVSDUWVVRWKDWWKHDGGLWLRQDOLQIRUPDWLRQRQ
WKHVHWRSLFVLQWKHIXOOWH[WDQGPXOWLPHGLDFRQGLWLRQV6HFWLRQRIWKHPDQXDOPLJKW
KDYHEHHQVXSHUIOXRXV+HJDUW\HWDOUHSOLFDWHGWKLVUHVXOWLQWZRIXUWKHU
H[SHULPHQWVLQZKLFKVWXGHQWVOHDUQHGDERXWFDUEUDNHVDQGDELF\FOHSXPSLQDGGLWLRQWR
WKHWRLOHWWDQNLQGLFDWLQJWKDWLWJHQHUDOL]HVWRRWKHUUHODWLYHO\IDPLOLDUGHYLFHV
([SHULPHQWE
([SHULPHQWDLQGLFDWHGWKDWWKDWWKHPHGLDDQGPRGDOLWLHVWKURXJKZKLFK
LQIRUPDWLRQLVSUHVHQWHGGRHVQRWDIIHFWFRPSUHKHQVLRQZKHQWKHLQVWUXFWLRQLVGHVLJQHG
DFFRUGLQJWRRXUFRPSUHKHQVLRQPRGHO,QDUHFHQWH[SHULPHQWZHFRPSDUHGK\SHUPHGLD
DQGSULQWHGYHUVLRQVRIRXUPDQXDOWRK\SHUPHGLDDQGSULQWHGLQVWUXFWLRQRQWKHVDPHW\SH
RIIOXVKLQJFLVWHUQIURPFRPPHUFLDOO\DYDLODEOHK\SHUPHGLDDQGPXOWLPHGLDSUHVHQWDWLRQV
¦7KH:D\7KLQJV:RUN§ERRNDQG&'520E\'DYLG0DFDXOD\0DFDXOD\
7KLVH[SHULPHQWDOVRDOORZHGXVWRFRPSDUHOHDUQLQJIURPRXUPDWHULDOVWROHDUQLQJ
IURPPDWHULDOVWKDWZHUHGHVLJQHGDFFRUGLQJWRWKHLQWXLWLRQVRIDQDZDUGZLQQLQJGHVLJQHU
EXWZKLFKZHUHQRWLQIRUPHGE\RXUFRPSUHKHQVLRQPRGHO
,QWKLVH[SHULPHQWVRPHSDUWLFLSDQWVOHDUQHGIURPDYHUVLRQRIRXUK\SHUPHGLD
PDQXDOZKLFKZDVUHYLVHGDFFRUGLQJWRWKHUHVXOWVIRXQGE\+HJDUW\HWDO%HFDXVH
ZHIRXQGWKDWVWXGHQWVOHDUQHGDVPXFKIURPWKHFDXVDOWH[WDVIURPWKHIXOOWH[WLQ
([SHULPHQWDERYHZHRPLWWHG6HFWLRQRIWKHRULJLQDOSUHVHQWDWLRQLQWKHUHYLVHG
PDQXDO7KHUHYLVHGPDQXDOFRPELQHG6HFWLRQVDQGRIWKHRULJLQDOPRGHOVRWKDW
VWXGHQWVILUVWVDZWKHFURVVVHFWLRQDOGLDJUDPRIWKHZKROHV\VWHPDVLQ6HFWLRQRIWKH
RULJLQDOPDQXDOWKHQVDZLWGHFRPSRVHGLQWRWKHZDWHURXWSXWDQGZDWHULQOHWV\VWHPV
ZLWKDOOWKHIXQFWLRQDOLW\GHVFULEHGLQ6HFWLRQRIWKHRULJLQDOPDQXDODQGWKHQZHUH
DOORZHGWRFOLFNRQLQGLYLGXDOFRPSRQHQWVDQGKDYHWKHPKLJKOLJKWHGLQWKHGLDJUDPDVLQ
6HFWLRQRIWKHRULJLQDOPDQXDO7KHUHYLVHGPDQXDODOVRFRPELQHG6HFWLRQVDQGRI
WKHRULJLQDOPDQXDOVRWKDWVWXGHQWVVDZDQDQLPDWLRQRIWKHV\VWHPLQZKLFKVXFFHVVLYH
FRPSRQHQWVLQWKHFDXVDOFKDLQZHUHSRLQWHGRXWXVLQJDUHGDUURZDVWKH\ZHUHGHVFULEHG
LQWKHDFFRPSDQ\LQJFRPPHQWDU\WKHDUURZWKXVSHUIRUPHGWKHVDPHIXQFWLRQDVWKH
VXFFHVVLYHKLJKOLJKWLQJLQ6HFWLRQRIWKHRULJLQDOPDQXDO7KLVHPERGLHVDGHVLJQ
SULQFLSOHVXJJHVWHGE\)DUDGD\DQG6XWFOLIIHDEWRGLUHFWVWXGHQWV©DWWHQWLRQWR
WKHSDUWRIWKHYLVXDOSUHVHQWDWLRQWKDWLVEHLQJGHVFULEHGLQWKHDXGLWRU\FRPPHQWDU\7KH
PDQXDOWKHUHIRUHKDGVHFWLRQVDQLQLWLDOVHFWLRQWKDWVKRZHGWKHKLHUDUFKLFDO
GHFRPSRVLWLRQRIWKHV\VWHPVDTXHVWLRQVVHFWLRQLGHQWLFDOWR6HFWLRQRIWKHRULJLQDO
PDQXDODPRYHPHQWVHFWLRQLQZKLFKVXFFHVVLYHFRPSRQHQWVLQWKHFDXVDOFKDLQZHUH
LQGLFDWHGZLWKDQDUURZDQGWKHVLSKRQVHFWLRQLGHQWLFDOWR6HFWLRQRIWKHRULJLQDO
PDQXDO
2WKHUVWXGHQWVOHDUQHGIURPWKHGHVFULSWLRQRIWKHWRLOHWWDQNLQ7KH:D\7KLQJV
:RUNERRNRU&'5200DFDXOD\7KHGHVFULSWLRQLQWKHERRNVKRZVD
ODUJHODEHOHGGLDJUDPRIWKHWRLOHWWDQNZKLFKGLIIHUHGIURPWKHGLDJUDPLQ)LJXUHLQWKDW
LWVKRZHGWKHWKLUGGLPHQVLRQDQGLVDUDWKHUZKLPVLFDOGHSLFWLRQVKRZLQJILVKVZLPPLQJ
LQWKHWDQNDQGILVKHUPHQVLWWLQJRQWKHIORDWDUP6HYHUDOVPDOOGLDJUDPVDUHVKRZQDV
LQVHWVWRWKHPDLQGLDJUDP2QHVKRZVDVLGHYLHZRIDWRLOHWVKRZLQJWKHORFDWLRQRIWKH
WDQNWZRPRUHVKRZWKHRSHUDWLRQRIDVLSKRQDQGDUHDFFRPSDQLHGE\WH[WH[SODLQLQJKRZ
DVLSKRQZRUNVDQGWKUHHPRUHVKRZGLIIHUHQWVWDJHVLQWKHIOXVKLQJRIWKHFLVWHUQDQGDUH
DFFRPSDQLHGE\DGHVFULSWLRQRIWKHIOXVKLQJSURFHVV7KH&'520YHUVLRQVKRZVWKH
VDPHODUJHODEHOHGGLDJUDPRQDVLQJOHVFUHHQRIWH[W)URPWKLVWKHXVHUKDVWKHRSWLRQRI
FOLFNLQJRQWZR¦PRYLH§LFRQVRQHRIZKLFKEULQJVWKHPWRDQRWKHUVFUHHQVKRZLQJD
VFKHPDWLFGLDJUDPRIWKHWRLOHWWDQNWKDWLVDQLPDWHGLQUHVSRQVHWRDPRXVHFOLFN7KLV
DQLPDWLRQLVYHU\IDVWWDNHVQRPRUHWKDQDFRXSOHRIVHFRQGVDQGLVQRWDFFRPSDQLHGE\D
FRPPHQWDU\&OLFNLQJRQWKHRWKHU¦PRYLH§LFRQEULQJVWKHXVHUWRDVFUHHQGHVFULELQJKRZ
DVLSKRQZRUNVZKLFKLVDOVRDFFRPSDQLHGE\DGLDJUDPWKDWWKHXVHUFDQDQLPDWH
0HWKRGXQGHUJUDGXDWHVWXGHQWVVWXGLHGRXUK\SHUPHGLDPDQXDOVWXGLHGD
SDSHUSULQWRXWRIWKHWH[WDQGGLDJUDPVXVHGLQWKHPDQXDOVWXGHQWVVWXGLHGWKH
K\SHUPHGLDSUHVHQWDWLRQRQWKHWRLOHWWDQNIURP¦7KH1HZ:D\7KLQJ:RUN§&'520
0DFDXOD\DQGVWXGHQWVVWXGLHGWKHFRUUHVSRQGLQJPDWHULDOVIURP¦7KH:D\
7KLQJV:RUN§ERRN0DFDOXOD\
$IWHUZDUGVDOOVWXGHQWVZURWHDGHVFULSWLRQRIKRZWKHGHYLFHZRUNHG7KH\ZHUH
LQVWUXFWHGWRLPDJLQHWKDWWKH\SXVKGRZQRQWKHKDQGOHRIWKHWRLOHWWDQNDQGGHVFULEHVWHS
E\VWHSZKDWKDSSHQVWRHDFKRIWKHRWKHUSDUWVRIWKHWDQNDVLWIOXVKHV7KHQWKH\DQVZHUHG
IXQFWLRQTXHVWLRQVDQGWURXEOHVKRRWLQJTXHVWLRQVDVGHVFULEHGLQ([SHULPHQWDDERYH
)LQDOO\WKH\ZHUHDGPLQLVWHUHGDWHVWRIVSDWLDOYLVXDOL]DWLRQDELOLW\¤WKH3DSHU)ROGLQJ
7H[W(NVWURP)UHQFK+DUPDQ'HUPDQ
5HVXOWV
6WXGHQWV VSHQW RQ DYHUDJH PLQXWHV VWXG\LQJ WKH K\SHUPHGLD PDQXDO PLQXWHVVWXG\LQJWKHSULQWHGYHUVLRQRIWKHPDQXDOPLQXWHVVWXG\LQJWKH¦:D\7KLQJV
:RUN§ &'520 DQG PLQXWHV VWXG\LQJ WKH ¦:D\ 7KLQJV :RUN§ ERRN 7KHUH ZDV D
ODUJH HIIHFW RI LQVWUXFWLRQ RQ DELOLW\ WR GHVFULEH KRZ WKH WRLOHW WDQN ZRUNHG VXFK WKDW WKH
VWXGHQWVZKROHDUQHGIURPWKHPDWHULDOVGHYHORSHGDFFRUGLQJWRRXUFRPSUHKHQVLRQPRGHO
GHVFULEHGPRUHVWHSVLQWKHFDXVDOFKDLQVWHSVIRUWKHK\SHUPHGLDJURXSVWHSV
IRU WKRVH ZKR UHFHLYHG WKH SULQWHG PDWHULDOV WKDQ VWXGHQWV ZKR OHDUQHG IURP WKH ¦:D\
7KLQJV :RUN§ PDWHULDOV VWHSV IRU WKH &'520 JURXS VWHSV IRU WKRVH ZKR
UHFHLYHGWKHSULQWHGPDWHULDOVIURPWKHERRN+RZHYHUDVWKHGDWDLQGLFDWHWKHUHZHUHQR
VLJQLILFDQW GLIIHUHQFHV LQ WKLV PHDVXUH EHWZHHQ WKH SULQWHG DQG PXOWLPHGLD YHUVLRQV RI
HLWKHUSUHVHQWDWLRQ
7KHUHZHUHQRVLJQLILFDQWGLIIHUHQFHVEHWZHHQWKHJURXSVRQHLWKHUWKHIXQFWLRQ
TXHVWLRQVRUWKHWURXEOHVKRRWLQJTXHVWLRQV6SDWLDODELOLW\ZDVPDUJLQDOO\UHODWHGWRDELOLW\
WRGHVFULEHKRZWKHWRLOHWWDQNZRUNHGDQGKDGDVLJQLILFDQWHIIHFWRQWURXEOHVKRRWLQJ
SHUIRUPDQFH+RZHYHUWKHUHZHUHQRLQWHUDFWLRQVRIWKHIRUPDWRILQVWUXFWLRQZLWKHLWKHU
VSDWLDODELOLW\SULRUNQRZOHGJHRUSUDFWLFDOH[SHULHQFHRIPDFKLQHVLQGLFDWLQJWKDWWKH
LQVWUXFWLRQZDVHTXDOO\HIIHFWLYHIRUVWXGHQWVZLWKGLIIHUHQWEDFNJURXQGVDQGDELOLWLHV
'LVFXVVLRQ,QVXPPDU\WKHUHVXOWVRIWKLVH[SHULPHQWDUHFRQVLVWHQWZLWKWKRVHRI
([SHULPHQWDDERYHLQVKRZLQJQRGLIIHUHQFHEHWZHHQSULQWHGYHUVLRQVRILQVWUXFWLRQDO
PDWHULDOVDQGK\SHUPHGLDYHUVLRQVLQFOXGLQJDQLPDWLRQVDQGK\SHUOLQNVZKHQWKHFRQWHQW
RIWKHPDWHULDOVLVWKHVDPH,WDOVRVKRZHGWKDWIRUDWOHDVWRQHLPSRUWDQWPHDVXUHRI
FRPSUHKHQVLRQWKHPDWHULDOVWKDWZHUHGHVLJQHGDFFRUGLQJWRRXUFRPSUHKHQVLRQPRGHO
ZHUHVXSHULRUWRDZDUGZLQQLQJFRPPHUFLDOO\DYDLODEOHPDWHULDOVVXJJHVWLQJWKDWRXU
WKHRUHWLFDOO\GHULYHGDQGHPSLULFDOO\VXSSRUWHGJXLGHOLQHVDUHDVLJQLILFDQWLPSURYHPHQW
RYHUWKHFRQYHQWLRQDOZLVGRPLQPXOWLPHGLDGHVLJQ7KHVHVWXGLHVFOHDUO\LQGLFDWHWKDWLWLV
WKHFRQWHQWDQGVWUXFWXUHRILQVWUXFWLRQDOPDWHULDOVDQGQRWWKHPHGLDDQGPRGDOLWLHVLQ
ZKLFKWKH\DUHSUHVHQWHGWKDWLVLPSRUWDQWIRUFRPSUHKHQVLRQRIFRPSOH[GHYLFHV
'RSHRSOHOHDUQGLIIHUHQWO\IURPK\SHUPHGLDSUHVHQWDWLRQV
WKDWGLUHFWWKHOHDUQHUWRYLHZWKHLQIRUPDWLRQLQDVSHFLILFRUGHU
FRPSDUHGWRWKRVHWKDWDOORZIUHHQDYLJDWLRQ"
2XUPRGHORIPDFKLQHFRPSUHKHQVLRQVXJJHVWVWKDWDSUHVHQWDWLRQVKRXOGSURYLGH
LQIRUPDWLRQDERXWWKHGHFRPSRVLWLRQRIWKHGHYLFHWKHVSDWLDOUHODWLRQVEHWZHHQ
FRPSRQHQWVWKHFDXVDOFKDLQVRIHYHQWVLQWKHRSHUDWLRQRIWKHPDFKLQHDQGWKHEHKDYLRUV
RIWKHFRPSRQHQWV,QRXULQLWLDOK\SHUPHGLDGHVLJQZHFRQVWUDLQHGQDYLJDWLRQLQWKH
PDQXDOWRHQVXUHWKDWXVHUVZRUNHGWKURXJKHDFKVHFWLRQ,QDVHQVHWKLVPDGHRXU
K\SHUPHGLDPDQXDOPRUHOLNHDGLJLWL]HGERRNLQZKLFKLQIRUPDWLRQLVSUHVHQWHGRQ
VHTXHQWLDOSDJHVUDWKHUWKDQDWUXHK\SHUPHGLDV\VWHPWKDWDOORZVLQIRUPDWLRQWREHYLHZHG
LQDQ\RUGHU
2XUGHFLVLRQWRFRQVWUDLQQDYLJDWLRQLQWKHLQLWLDOYHUVLRQRIWKHK\SHUPHGLD
SUHVHQWDWLRQZDVEDVHGRQHYLGHQFHWKDWSHRSOHHVSHFLDOO\QRYLFHVLQWKHGRPDLQRI
LQWHUHVWGRQRWVHDUFKWKHVHPHGLDRSWLPDOO\)RUH[DPSOHXVHUVFDQVHDUFKKDSKD]DUGO\
JHWORVWDQGIDLOWRJHWDQRYHUYLHZRIKRZWKHLQIRUPDWLRQLQWKHGLIIHUHQWGLVSOD\VLV
LQWHJUDWHG+DPPRQG$OOLQVRQ6SRHKU2QWKHRWKHUKDQG6KDSLUR
IRXQGWKDWVWXGHQWVLQGLFDWHGDGHHSHUOHYHORIOHDUQLQJIURPDOHVVVWUXFWXUHG
K\SHUPHGLDGRFXPHQWFRPSDUHGWRRQHLQZKLFKWKHKLHUDUFKLFDOVWUXFWXUHZDVPDGH
REYLRXVDQGRQHLQZKLFKWKHSUHVHQWDWLRQZDVOLQHDU6KHLQWHUSUHWHGWKLVUHVXOWVLQ
UHODWLRQWRUHVHDUFKRQWH[WFRPSUHKHQVLRQVKRZLQJWKDWPRUHNQRZOHGJHDEOHUHDGHUVOHDUQ
PRUHIURPOHVVHODERUDWHDQGOHVVFRKHUHQWWH[WVFRPSDUHGWRWH[WVWKDWDUHHDVLHUWR
FRPSUHKHQG0F1DPDUD.LQWVFK6RQJHU.LQWVFKEHFDXVHWKHOHVVVWUXFWXUHG
WH[WVSURPRWHPRUHDFWLYHSURFHVVLQJ,QJHQHUDOWKHUHLVQRFOHDUFRQVHQVXVLQWKH
OLWHUDWXUHUHJDUGLQJWKHUHODWLYHHIIHFWLYHQHVVRIK\SHUPHGLDV\VWHPVFRPSDUHGWRPRUH
OLQHDUIRUPVRISUHVHQWDWLRQ)RUH[DPSOHLQDPHWDDQDO\VLVRIVWXGLHVFRPSDULQJ
K\SHUWH[WWROLQHDUWH[W&KHQDQG5DGDUHSRUWHGWKDWK\SHUWH[WZDVPRUHHIIHFWLYHLQ
FDVHVDQGWKHRSSRVLWHZDVWUXHLQWKHRWKHUFDVHV
,Q([SHULPHQWZHFRPSDUHGOHDUQLQJRXWFRPHVIURPRXULQLWLDOK\SHUPHGLD
PDQXDOWKDWFRQVWUDLQHGWKHRUGHURIQDYLJDWLRQDQGDYHUVLRQZKLFKFRQWDLQHGWKHVDPH
LQIRUPDWLRQEXWDOORZHGVWXGHQWVWRYLHZWKHVHFWLRQVLQDQ\RUGHU
([SHULPHQW
0HWKRG7KLUW\VL[VWXGHQWVWRRNSDUWLQWKLVH[SHULPHQW(LJKWHHQOHDUQHGIURPWKH
K\SHUPHGLDSUHVHQWDWLRQGHVFULEHGDERYHLQZKLFKWKH\ZHUHUHVWULFWHGWRYLHZWKHVHFWLRQV
RIWKHSUHVHQWDWLRQLQWKHRUGHUSUHVFULEHGE\RXUFRPSUHKHQVLRQPRGHO7KHRWKHU
VWXGHQWVOHDUQHGIURPDSUHVHQWDWLRQWKDWFRQWDLQHGWKHVDPHVHFWLRQVDQGLQIRUPDWLRQ
ZLWKLQWKHVHFWLRQVEXWWKH\ZHUHIUHHWRYLHZWKHVHFWLRQVLQDQ\RUGHU$IWHUZDUGVDOO
VWXGHQWVDQVZHUHGWKHPHQWDODQLPDWLRQTXHVWLRQVIXQFWLRQTXHVWLRQVIDXOWEHKDYLRU
TXHVWLRQVDQGWURXEOHVKRRWLQJTXHVWLRQVXVHGLQ([SHULPHQWDDERYH
5HVXOWV3DUWLFLSDQWVZKROHDUQHGIURPWKHUHVWULFWHGQDYLJDWLRQYHUVLRQRIWKH
PDQXDOVSHQWPRUHWLPHVWXG\LQJ0 PLQXWHV6' WKDQSDUWLFLSDQWVZKR
YLHZHGWKHIUHHQDYLJDWLRQPDQXDO0 PLQXWHV6' ,QJHQHUDOSDUWLFLSDQWV
YLHZLQJWKHUHVWULFWHGYHUVLRQUHYLVLWHGDVHFWLRQ0 6' PRUHRIWHQWKDQGLG
SDUWLFLSDQWVYLHZLQJWKHIUHHYHUVLRQ0 6' 7KHW\SHRISUHVHQWDWLRQVWXGLHGIUHHYHUVXVUHVWULFWHGQDYLJDWLRQGLGQRWDIIHFW
SHUIRUPDQFHRQDQ\RIWKHIRXUFDWHJRULHVRIFRPSUHKHQVLRQTXHVWLRQV)XUWKHUPRUHWKHUH
ZHUH QR VLJQLILFDQW HIIHFWV RI WKH LQWHUDFWLRQV EHWZHHQ SUHYLRXV H[SHULHQFH RU PHFKDQLFDO
WUDLQLQJZLWKW\SHRIPDQXDOSUHVHQWHGIRUDQ\RIWKHFDWHJRULHVRITXHVWLRQV
$OWKRXJKSDUWLFLSDQWVLQWKHIUHHQDYLJDWLRQFRQGLWLRQZHUHDOORZHGWRYLHZWKH
VHFWLRQVLQDQ\RUGHULQIDFWPRVWRIWKHPYLHZHGWKHPLQWKHRUGHUSUHVFULEHGLQWKH
UHVWULFWHGQDYLJDWLRQYHUVLRQRIWKHPDQXDO7ZHOYHRIWKHSDUWLFLSDQWVLQWKHIUHH
QDYLJDWLRQFRQGLWLRQYLHZHGWKHVHFWLRQVLQWKHRUGHUSUHVFULEHGLQWKHFRQVWUDLQHG
QDYLJDWLRQYHUVLRQRIWKHPDQXDODQGPRVWRIWKHRWKHUVGLGQRWGHSDUWPXFKIURPWKLV
RUGHU7KLVPD\EHEHFDXVHWKHLFRQVWKDWSHRSOHFOLFNHGWRJDLQDFFHVVWRWKHVHFWLRQVRIWKH
PDQXDOZHUHDOLJQHGIURPOHIWWRULJKWRQWKHVFUHHQVHHXSSHUOHIWFRUQHURI)LJXUH
7KHUHIRUHPRVWSDUWLFLSDQWVGLGQRWXVHWKHIUHHGRPWKDWWKH\ZHUHDIIRUGHGWRYLHZ
VHFWLRQVLQDQ\RUGHUDUHVXOWWKDWKDVEHHQUHSRUWHGHOVHZKHUH:KLWH
7KLVVWXG\WKHUHIRUHVKRZHGQRGLIIHUHQFHVLQHIIHFWLYHQHVVRIK\SHUPHGLD
SUHVHQWDWLRQVZLWKIUHHDQGUHVWULFWHGQDYLJDWLRQ,WZDVDOLPLWHGWHVWRIWKHGLIIHUHQFHV
EHWZHHQIUHHDQGUHVWULFWHGQDYLJDWLRQJLYHQWKDWWKHQXPEHURIVHFWLRQVLQWKHSUHVHQWDWLRQ
ZDVUHODWLYHO\VPDOO'LIIHUHQFHVPD\HPHUJHLQPRUHFRPSOH[K\SHUPHGLDV\VWHPVRU
LQV\VWHPVZKHUHWKHLQWHUIDFHXVHGIRUQDYLJDWLRQGRHVQRWVXJJHVWDQRUGHULQZKLFKWR
YLHZWKHLQIRUPDWLRQ+RZHYHUDVUHYLHZHGDERYHWKHLVVXHRIZKHWKHUK\SHUPHGLDDUH
PRUHHIIHFWLYHWKDQOLQHDUSUHVHQWDWLRQVLVFOHDUO\DQRSHQTXHVWLRQDQGSUREDEO\GHSHQGV
RQPDQ\DVSHFWVRIWKHGHVLJQRIVSHFLILFSUHVHQWDWLRQV6KDSLUR&KHQDQG5DGD
'R3HRSOH/HDUQPRUHIURPDQ$QLPDWLRQRID0HFKDQLFDO6\VWHPLIWKH\ILUVW7U\WR
0HQWDOO\$QLPDWHWKH6\VWHP"
([SHULPHQWVDDQGEGHVFULEHGDERYHVKRZHGQRDGYDQWDJHRIOHDUQLQJIURP
DQLPDWLRQVDFFRPSDQLHGE\FRPPHQWDULHVFRPSDUHGWROHDUQLQJIURPWH[WDQGVWDWLF
GLDJUDPVSUHVHQWLQJWKHVDPHLQIRUPDWLRQ7KLVUHVXOWLVFRQWUDU\WRPDQ\SHRSOH©V
LQWXLWLRQVDERXWWKHHIIHFWLYHQHVVRIFRPSXWHUDQLPDWLRQV6FDLIH5RJHUV:H
PLJKWH[SHFWDQLPDWLRQVWREHPRUHHIIHFWLYHWKDQVWDWLFGLDJUDPVIRUH[DPSOHEHFDXVH
WKH\SRUWUD\WKHWHPSRUDOFKDQJHVLQWKHRSHUDWLRQRIDPDFKLQHH[SOLFLWO\UDWKHUWKDQ
UHO\LQJRQWKHXVHUWRYLVXDOL]HWKHVHFKDQJHVDQGEHFDXVHWKH\PLJKWEHFORVHUWRDPHQWDO
PRGHORIDPHFKDQLFDOV\VWHPWKDQDVWDWLFGHSLFWLRQ/RZH
+RZHYHUWKHUHDUHDOVRVHYHUDOUHDVRQVZK\DQLPDWLRQVDFFRPSDQLHGE\
FRPPHQWDULHVPLJKWEHOHVVHIIHFWLYHWKDQVWDWLFWH[WDQGGLDJUDPV)LUVWDQLPDWLRQV
DFFRPSDQLHGE\FRPPHQWDULHVWDNHDVSHFLILHGDPRXQWRIWLPHWRGLVSOD\,QFRQWUDVW
UHDGLQJRIVWDWLFGLDJUDPVDQGWH[WLVVHOISDFHG7KHUHLVDTXHVWLRQWKHUHIRUHRIZKHWKHU
FRPSUHKHQVLRQSURFHVVHVFDQNHHSXSZLWKWKHSDFHDWZKLFKDQDQLPDWLRQLVSUHVHQWHGVHH
DOVR/RZH
6HFRQGDUHDOLVWLFDQLPDWLRQRIDPHFKDQLFDOV\VWHPVKRZVDOOFRPSRQHQWVPRYLQJ
VLPXOWDQHRXVO\,QFRQWUDVWZKHQSHRSOHDWWHPSWWRXQGHUVWDQGKRZDPDFKLQHZRUNVRU
DQVZHUTXHVWLRQVDERXWKRZDGHYLFHZRUNVWKH\PHQWDOO\DQLPDWHWKHFRPSRQHQWVRQHE\
RQHLQRUGHURIDFDXVDOFKDLQRIHYHQWV%DJJHWW*UDHVVHU+HJDUW\+HJDUW\
6LPV1DUD\DQDQ6XZD0RWRGD)XUWKHUPRUHLQDFRPSOH[
PDFKLQHVXFKDVWKHIOXVKLQJFLVWHUQWKHUHDUHWZRFDXVDOFKDLQVWKDWRSHUDWH
VLPXOWDQHRXVO\7KHUHIRUHWRXQGHUVWDQGDQDQLPDWLRQSHRSOHQHHGWRDWWHQGWRDQGUHODWH
FKDQJHVWKDWRFFXUVLPXOWDQHRXVO\LQGLIIHUHQWUHJLRQVRIVSDFH/RZH+RZHYHURXU
YLVXDODWWHQWLRQLVW\SLFDOO\IRFXVHGRQRQO\RQHORFDWLRQDWDWLPH
7KLUGYLHZLQJDQDQLPDWLRQLVDSDVVLYHSURFHVV3KHQRPHQDVXFKDVWKHVHOI
JHQHUDWLRQHIIHFW6ODPHFND*UDIDQGWKHVHOIH[SODQDWLRQHIIHFW&KLGH/HHXZ
&KLX/D9DQFKHUVXJJHVWWKDWSHRSOHOHDUQPRUHHIIHFWLYHO\LIWKH\DUHPRUHDFWLYH
LQWKHOHDUQLQJSURFHVVDQGDFWXDOO\JHQHUDWHLGHDVRUH[SODQDWLRQV6LPLODUO\SHRSOHOHDUQ
PRUHIURPLQIRUPDWLRQSUHVHQWDWLRQVLIWKH\ILUVWSDUWLFLSDWHLQDFWLYLWLHVWKDWDFWLYDWHWKHLU
NQRZOHGJHUHOHYDQWWRWKHWRSLFRIWKHSUHVHQWDWLRQ%ULWWRQ*UDHVVHU.LQWVFKHWDO
0F1DPDUD.LQWVFK6RQJHU.LQWVFKRUJHQHUDWHWKHLPSRUWDQWGLVWLQFWLRQV
UHOHYDQWWRXQGHUVWDQGLQJWKHWRSLF6FKZDUW]%UDQVIRUG
,QFRQWUDVWWRSDVVLYHO\YLHZLQJDQDQLPDWLRQSUHYLRXVUHVHDUFKKDVVKRZQWKDW
SHRSOHFDQEHTXLWHVXFFHVVIXOLQPHQWDOO\DQLPDWLQJPHFKDQLFDOV\VWHPV+HJDUW\
1DUD\DQDQ6XZD0RWRGDDQGWKDWSHRSOHZLWKKLJKVSDWLDODELOLW\DUHPRUH
VXFFHVVIXOWKDQSHRSOHZLWKORZVSDWLDODELOLW\+HJDUW\6LPV+HJDUW\
6WHLQKRII+HJDUW\.R]KHYQLNRY(YHQLISHRSOHDUHQRWVXFFHVVIXOLQ
PHQWDOO\DQLPDWLQJDPDFKLQHLIWKH\ILUVWDWWHPSWWRGRVREHIRUHYLHZLQJDFRPSXWHU
DQLPDWLRQWKH\PLJKWGLVFRYHUZKLFKPHFKDQLFDOOLQNDJHVWKH\FDQHDVLO\YLVXDOL]HDQG
ZKLFKOLQNDJHVWKH\FDQQRWYLVXDOL]H7KHPHQWDODQLPDWLRQSURFHVVPLJKWDOVRLQGXFHWKHP
WRDUWLFXODWHWKHLULQWXLWLRQVDERXWKRZWKHPDFKLQHZRUNVVRWKDWZKHQWKH\YLHZWKH
DQLPDWLRQWKH\FDQFRPSDUHWKHVHLQWXLWLRQVWRWKHDFWXDOSK\VLFDOSURFHVVVKRZQLQWKH
DQLPDWLRQ7KH\FDQWKHQXVHWKHDQLPDWHGGLVSOD\WRFKHFNWKHDFFXUDF\RIWKHLUPHQWDO
DQLPDWLRQVDQGWRHQFRGHLQIRUPDWLRQDERXWWKHPRWLRQVRIFRPSRQHQWVWKDWWKH\FRXOGQRW
LQIHUIURPWKHGLDJUDP:HUHFHQWO\WHVWHGWKHHIIHFWVRIOHDUQLQJIURPFRPSXWHUDQLPDWLRQV
YHUVXVOHDUQLQJE\PHQWDOO\DQLPDWLQJVWDWLFGLDJUDPVLQWKUHHH[SHULPHQWV
([SHULPHQWD
0HWKRG1LQHW\WKUHHXQGHUJUDGXDWHVWXGHQWVSDUWLFLSDWHGLQGLIIHUHQWFRQGLWLRQV
RIWKLVH[SHULPHQW7ZHQW\IRXUVWXGLHGDVWDWLFGLDJUDPRIWKHIOXVKLQJFLVWHUQVKRZQLQ
)LJXUH'LDJUDP*URXS7ZHQW\RQHVWXGHQWVVWXGLHGWKHVWDWLFGLDJUDPDQGWKHQ
DWWHPSWHGWRH[SODLQWRWKHH[SHULPHQWHURUDOO\KRZWKHV\VWHPZRUNHG0HQWDO$QLPDWLRQ
*URXS7ZHQW\VL[VWXGHQWVVWXGLHGWKHVWDWLFGLDJUDPDQGWKHQYLHZHGDQDQLPDWLRQ
DFFRPSDQLHGE\DFRPPHQWDU\H[SODLQLQJKRZWKHIOXVKLQJFLVWHUQZRUNHG&RPSXWHU
$QLPDWLRQ*URXS7ZHQW\WZRVWXGHQWVYLHZHGWKHVWDWLFGLDJUDPWKHQDWWHPSWHGWR
H[SODLQWRWKHH[SHULPHQWHUKRZWKHV\VWHPZRUNHGDQGILQDOO\YLHZHGWKHFRPSXWHU
DQLPDWLRQ&RPELQDWLRQ*URXS$IWHUZDUGVDOOVWXGHQWVDQVZHUHGPHQWDODQLPDWLRQ
TXHVWLRQVIXQFWLRQTXHVWLRQVDQGWURXEOHVKRRWLQJTXHVWLRQVVLPLODUWRWKRVHXVHGLQ
([SHULPHQWDGHVFULEHGDERYH)LQDOO\VWXGHQWVZHUHJLYHQWKH3DSHU)ROGLQJ7HVW
(NVWURP)UHQFK+DUPDQDVDWHVWRIVSDWLDOYLVXDOL]DWLRQDELOLW\
5HVXOWV9LHZLQJWKHDQLPDWLRQDQGFRPPHQWDU\VLJQLILFDQWO\LPSURYHG
SHUIRUPDQFHRQDOOWKUHHW\SHVRITXHVWLRQV7KDWLVVWXGHQWVLQWKH&RPSXWHU$QLPDWLRQ
*URXSDQGWKH&RPELQDWLRQJURXSDQVZHUHGPRUHTXHVWLRQVFRUUHFWO\WKDQVWXGHQWVLQWKH
RWKHUWZRJURXSV6WXGHQWVZLWKKLJKVSDWLDODELOLW\DOVRDQVZHUHGPRUHIXQFWLRQDQG
WURXEOHVKRRWLQJTXHVWLRQVFRUUHFWO\)RUWKHPHQWDODQLPDWLRQTXHVWLRQVWKHUHZDVDWUHQG
IRUVWXGHQWVLQWKHPHQWDODQLPDWLRQJURXSWRKDYHEHWWHUSHUIRUPDQFHWKDQWKRVHLQWKH
GLDJUDPVJURXSEXWWKLVWUHQGGLGQRWUHDFKVWDWLVWLFDOVLJQLILFDQFH
([SHULPHQWE
([SHULPHQWDLQGLFDWHGWKDWVWXGHQWVFDQOHDUQKRZDPHFKDQLFDOV\VWHPZRUNV
IURPDQDQLPDWLRQDFFRPSDQLHGE\DFRPPHQWDU\1RVLJQLILFDQWHIIHFWVRIILUVWDWWHPSWLQJ
WRPHQWDOO\DQLPDWHWKHV\VWHPZHUHREVHUYHG+RZHYHUWKHSDUWLFLSDQWVLQ([SHULPHQWD
PLJKWQRWKDYHEHHQJLYHQHQRXJKLQIRUPDWLRQWRJHQHUDWHDQH[SODQDWLRQRIKRZWKH
V\VWHPZRUNHG7KH\ZHUHVKRZQDVWDWLFGLDJUDPRIWKHV\VWHPDQGWKLVGLDJUDPZDVQRW
SUHVHQWZKHQWKH\DWWHPSWHGWRH[SODLQKRZWKHV\VWHPZRUNHGVRWKH\KDGWRPHQWDOO\
DQLPDWHWKHV\VWHPIURPPHPRU\,Q([SHULPHQWEVWXGHQWVLQWKHPHQWDODQLPDWLRQDQG
FRPELQDWLRQJURXSVZHUHVKRZQWKUHHVWDWLFGLDJUDPVVKRZLQJGLIIHUHQWVWDJHVLQWKH
RSHUDWLRQRIWKHGHYLFHDQGWKHVHWKUHHGLDJUDPVZHUHYLVLEOHZKLOHWKH\DWWHPSWHGWR
H[SODLQKRZWKHV\VWHPZRUNHG
:HZHUHDOVRFRQFHUQHGWKDWWKHPHQWDODQLPDWLRQTXHVWLRQVZHUHQRWGLDJQRVWLFRI
WKHGLIIHUHQFHVEHWZHHQWKHJURXSVEHFDXVHWKH\ZHUHPXOWLSOHFKRLFHTXHVWLRQVDQG
WKHUHIRUHVWXGHQWVKDGDJRRGFKDQFHRIDQVZHULQJWKHPFRUUHFWO\LIWKH\FRXOGHOLPLQDWH
RQHRUWZRRIWKHDQVZHUFKRLFHV7KHUHIRUHLQWKLVH[SHULPHQWWKHPXOWLSOHFKRLFH
TXHVWLRQVZHUHUHSODFHGE\DWDVNLQZKLFKWKHVWXGHQWVZHUHDVNHGWRGHVFULEHVWHSE\VWHS
KRZWKHIOXVKLQJFLVWHUQZRUNVDVLQ([SHULPHQWEDERYH
0HWKRG(LJKW\VWXGHQWVSDUWLFLSDWHGLQWKLVH[SHULPHQWLQHDFKRIWKH
FRQGLWLRQV'LDJUDPRQO\0HQWDO$QLPDWLRQ&RPSXWHU$QLPDWLRQDQG&RPELQDWLRQ
0HQWDO&RPSXWHU$QLPDWLRQ7KHSURFHGXUHZDVVLPLODUWRWKDWRI([SHULPHQWDZLWK
WKHFKDQJHVQRWHGDERYH
5HVXOWV6WXGHQWVZKRYLHZHGWKHDQLPDWLRQDQGFRPPHQWDU\ZURWHPRUHFRPSOHWH
GHVFULSWLRQVRIKRZWKHIOXVKLQJFLVWHUQZRUNHGWKDWLVWKH\LQFOXGHGPRUHVWHSVLQWKH
FDXVDOFKDLQLQWKHLUGHVFULSWLRQ)XUWKHUPRUHVWXGHQWVZKRPHQWDOO\DQLPDWHGWKH
PHFKDQLFDOV\VWHPDOVRZURWHPRUHFRPSOHWHGHVFULSWLRQVWKDQWKRVHZKRGLGQRWPHQWDOO\
DQLPDWHWKHV\VWHP7KHHIIHFWVRIWKHVHWZRPDQLSXODWLRQVZHUHDGGLWLYHVXFKWKDW
VWXGHQWVLQWKH&RPELQDWLRQFRQGLWLRQZKRILUVWPHQWDOO\DQLPDWHGWKHV\VWHPDQGWKHQ
YLHZHGWKHFRPSXWHUDQLPDWLRQKDGWKHEHVWSHUIRUPDQFHRIDOOJURXSVRQWKLVPHDVXUH
7KLVLQGLFDWHVWKDWVWXGHQWVOHDUQPRUHIURPYLHZLQJDQDQLPDWLRQRIDPHFKDQLFDOGHYLFHLI
WKH\ILUVWDWWHPSWWRPHQWDOO\DQLPDWHWKHGHYLFH6WXGHQWVZLWKKLJKHUVSDWLDODELOLW\DOVR
KDGEHWWHUSHUIRUPDQFHRQWKLVPHDVXUH
6WXGHQWVZKRYLHZHGWKHDQLPDWLRQDQGFRPPHQWDU\DOVRKDGEHWWHUSHUIRUPDQFHRQ
WKHWURXEOHVKRRWLQJTXHVWLRQV$OWKRXJKWKHUHZHUHWUHQGVIRUPHQWDODQLPDWLRQDQGVSDWLDO
DELOLW\WRLPSURYHSHUIRUPDQFHRQWKLVPHDVXUHWKHWUHQGVGLGQRWUHDFKVWDWLVWLFDO
VLJQLILFDQFH1RQHRIWKHH[SHULPHQWDOPDQLSXODWLRQVKDGVLJQLILFDQWHIIHFWVRQWKHIXQFWLRQ
TXHVWLRQV
([SHULPHQWF
([SHULPHQWVDDQGEVKRZHGWKDWSHRSOHDUHEHWWHUDEOHWRGHVFULEHKRZD
PHFKDQLFDOV\VWHPZRUNVDIWHUYLHZLQJDQDQLPDWLRQDFFRPSDQLHGE\DFRPPHQWDU\WKDQ
DIWHUYLHZLQJDVWDWLFGLDJUDP([SHULPHQWEDOVRVKRZHGWKDWWKH\DUHEHWWHUDEOHWR
GHVFULEHKRZDPDFKLQHZRUNVLIWKH\DWWHPSWWRPHQWDOO\DQLPDWHWKHPDFKLQHEHIRUH
YLHZLQJDQDQLPDWLRQ+RZHYHU([SHULPHQWVDDQGEDERYHLQGLFDWHWKDWVWXGHQWVOHDUQ
DVZHOOIURPVWDWLFGLDJUDPVDFFRPSDQLHGE\WH[WDVIURPDQLPDWLRQV,QWKLVH[SHULPHQWZH
H[DPLQHGWKHHIIHFWVRIWZRIDFWRUVRQVWXGHQWV©DELOLW\WRGHVFULEHKRZWKHIOXVKLQJFLVWHUQ
ZRUNV7KHILUVWIDFWRUZDVZKHWKHUWKH\OHDUQHGKRZWKHV\VWHPZRUNHGIURPDQDQLPDWLRQ
DFFRPSDQLHGE\DFRPPHQWDU\RUIURPDVWDWLFGLDJUDPDFFRPSDQLHGE\WH[W7KHVHFRQG
IDFWRUZDVZKHWKHUWKH\ILUVWDWWHPSWHGWRPHQWDOO\DQLPDWHWKHPHFKDQLFDOV\VWHPEHIRUH
UHFHLYLQJDQH[SODQDWLRQRIKRZLWZRUNHG
0HWKRG1LQHW\VL[VWXGHQWVSDUWLFLSDWHGLQWKLVH[SHULPHQWLQHDFKJURXS7KH
ILUVWJURXSILUVWVWXGLHGDGLDJUDPRIWKHIOXVKLQJFLVWHUQDQGWKHQUHDGDWH[WDFFRPSDQLHG
E\DGLDJUDPGHVFULELQJKRZLWZRUNHG7H[WDQG'LDJUDPJURXS7KHVHFRQGJURXS
YLHZHGDVWDWLFGLDJUDPRIWKHGHYLFHWKHQDWWHPSWHGWRH[SODLQWRWKHH[SHULPHQWHUKRZLW
ZRUNHGDQGWKHQUHDGDWH[WDQGGLDJUDPGHVFULSWLRQRIKRZLWZRUNHG0HQWDO$QLPDWLRQ
7H[WDQG'LDJUDPJURXS7KHWKLUGJURXSVWXGLHGWKHGLDJUDPDQGWKHQYLHZHGDQ
DQLPDWLRQDFFRPSDQLHGE\DFRPPHQWDU\H[SODLQLQJKRZWKHGHYLFHZRUNHG&RPSXWHU
$QLPDWLRQ*URXS7KHIRXUWKJURXSVWXGLHGDVWDWLFGLDJUDPRIWKHGHYLFHWKHQDWWHPSWHG
WRH[SODLQWRDQH[SHULPHQWHUKRZLWZRUNHGDQGWKHQYLHZHGDQDQLPDWLRQDFFRPSDQLHG
E\DFRPPHQWDU\H[SODLQLQJKRZWKHGHYLFHZRUNHG0HQWDO$QLPDWLRQ&RPSXWHU
$QLPDWLRQ*URXS$IWHUZDUGVDOOSDUWLFLSDQWVZHUHDVNHGWRGHVFULEHLQZULWLQJVWHSE\
VWHSKRZWKHIOXVKLQJFLVWHUQZRUNVDVLQ([SHULPHQWEDERYHDQGDQVZHUHGWKH
WURXEOHVKRRWLQJDQGIXQFWLRQTXHVWLRQV
5HVXOWV$VLQ([SHULPHQWEVWXGHQWVZKRPHQWDOO\DQLPDWHGWKHPHFKDQLFDO
V\VWHPLHDWWHPSWHGWRH[SODLQKRZLWZRUNVEHIRUHYLHZLQJDQDQLPDWLRQRIWKH
PHFKDQLFDOV\VWHPZURWHPRUHFRPSOHWHGHVFULSWLRQVRIKRZWKHGHYLFHZRUNVFRPSDUHGWR
WKRVHZKRYLHZHGWKHDQLPDWLRQZLWKRXWILUVWPHQWDOO\DQLPDWLQJ7KLVUHSOLFDWHVWKHUHVXOW
([SHULPHQWEDERYHWKDWVWXGHQWVOHDUQPRUHIURPYLHZLQJDQDQLPDWLRQRIDPHFKDQLFDO
GHYLFHLIWKH\ILUVWDWWHPSWWRPHQWDOO\DQLPDWHWKHGHYLFH+RZHYHUIRUWKHWH[WDQG
GLDJUDPFRQGLWLRQVWKHUHZHUHQRGLIIHUHQFHVRQWKHRXWFRPHPHDVXUHVEHWZHHQWKRVHZKR
PHQWDOO\DQLPDWHGEHIRUHUHDGLQJWKHWH[WDQGGLDJUDPDQGWKRVHZKRGLGQRW)XUWKHUPRUH
WKHUHZHUHQRGLIIHUHQFHVRQWKHRXWFRPHPHDVXUHVEHWZHHQWKRVHZKRYLHZHGDQLPDWLRQV
DFFRPSDQLHGE\FRPPHQWDULHVDQGWKRVHZKRYLHZHGWH[WDQGGLDJUDPV
+RZPLJKWZHH[SODLQWKHVHUHVXOWV")LUVWLWLVSRVVLEOHWKDWWKHSURFHVVRI
XQGHUVWDQGLQJDWH[WDFFRPSDQLHGE\DVWDWLFGLDJUDPLQYROYHVPHQWDODQLPDWLRQ+HJDUW\
-XVW7KDWLVZKHQRQHYLHZVDVWDWLFGLDJUDPRIDPHFKDQLFDOGHYLFHDQGUHDGVD
WH[WGHVFULELQJKRZLWRSHUDWHVWKHFRPSUHKHQVLRQSURFHVVLQYROYHVYLVXDOL]LQJKRZWKH
GLIIHUHQWSDUWVRIWKHGLDJUDPZLOOPRYHWRDFFRPSOLVKWKHIXQFWLRQRIWKHPDFKLQH,QWKLV
YLHZILUVWPHQWDOO\DQLPDWLQJDGLDJUDPEHIRUHUHDGLQJDWH[WDQGGLDJUDPGRHVQRWDIIHFW
FRPSUHKHQVLRQEHFDXVHWKHFRPSUHKHQVLRQSURFHVVLWVHOILQYROYHVPHQWDODQLPDWLRQ,Q
FRQWUDVWYLHZLQJDFRPSXWHUDQLPDWLRQGRHVQRWLQYROYHPHQWDODQLPDWLRQDQGWKHUHIRUH
OHDUQLQJIURPDQDQLPDWLRQLVHQKDQFHGLIRQHILUVWPHQWDOO\DQLPDWHVWKHGHYLFH6HFRQG
FRQVLVWHQWZLWK([SHULPHQWVDDQGEGHVFULEHGDERYHWKLVH[SHULPHQWLQGLFDWHVWKDWLWLV
WKHFRQWHQWDQGVWUXFWXUHRILQVWUXFWLRQDOPDWHULDOVDQGQRWWKHPHGLDDQGPRGDOLWLHVLQ
ZKLFKWKH\DUHSUHVHQWHGWKDWLVLPSRUWDQWIRUFRPSUHKHQVLRQRIPHFKDQLFDOV\VWHPV
6XPPDU\DQG&RQFOXVLRQV
,QVXPPDU\ZHGHYHORSHGDWKHRUHWLFDOPRGHORIFRPSUHKHQVLRQRIPHFKDQLFDO
V\VWHPVIURPWH[WDQGGLDJUDPV)URPWKLVZHGHULYHGGHVLJQJXLGHOLQHVDQGLPSOHPHQWHG
VHYHUDOYHUVLRQVRIDK\SHUPHGLDSUHVHQWDWLRQWKDWH[SODLQVKRZDPHFKDQLFDOGHYLFHWKH
IOXVKLQJFLVWHUQZRUNV:HGHVFULEHGGLIIHUHQWH[SHULPHQWVVRPHRIZKLFKHYDOXDWHG
GLIIHUHQWYHUVLRQVRIRXUK\SHUPHGLDSUHVHQWDWLRQDQGVRPHRIZKLFKZHUHIRFXVHGRQ
VSHFLILFDVSHFWVRIRXUPRGHO
:HREWDLQHGVXSSRUWIRURXUWKHRUHWLFDOPRGHOLQWKDWSHRSOHOHDUQHGPRUHIURP
PDWHULDOVGHVLJQHGDFFRUGLQJWRWKLVPRGHOWKDQIURPFRPPHUFLDOO\DYDLODEOHERRNVDQG
&'520VWKDWH[SODLQHGWKHVDPHPDWHULDOV2QHVSHFLILFGHVLJQJXLGHOLQHWKDWSHRSOH
VKRXOGEHLQGXFHGWRPHQWDOO\DQLPDWHDPHFKDQLFDOV\VWHPEHIRUHVHHLQJDFRPSXWHU
DQLPDWLRQRIWKHGHYLFH¤ZDVDOVRHPSLULFDOO\YDOLGDWHG7KHVHDGYDQWDJHVZHUHSULPDULO\
HYLGHQWLQSDUWLFLSDQWV©DELOLW\WRGHVFULEHVWHSE\VWHSKRZWKHPDFKLQHZRUNV,QFRQWUDVW
GLIIHUHQFHVEHWZHHQFRQGLWLRQVZHUHW\SLFDOO\QRWREVHUYHGIRUWURXEOHVKRRWLQJWDVNVWKDW
UHTXLUHWKHXVHUWRUHDVRQZLWKWKHLQIRUPDWLRQJLYHQLQWKHPXOWLPHGLDSUHVHQWDWLRQ
2XUH[SHULPHQWVSURYLGHQRHYLGHQFHWKDWWKH format of instruction -- the media and
modalities in which different types of information are be presented – has any effect on
comprehension and learning. The comprehension measures were not affected by whether
information was presented in static text and diagrams or as animations accompanied by
commentaries. Furthermore, there we no effects on comprehension of hyperlinking
information or allowing participants free rather than restricted navigation of the hypermedia
presentations. We have observed similar results in a parallel project on teaching computer
algorithms. These results are consistent with a growing body of research showing no
significant effects of animations over static media when they present the same information
(Morrison, Tversky & Betrancourt, 2000)
:KDWFDQZHFRQFOXGHDERXWWKHHIIHFWLYHQHVVRIDQLPDWLRQVDQGK\SHUPHGLDLQ
VFLHQFHLQVWUXFWLRQ"2XUUHVHDUFKVKRZVQRDGYDQWDJHVRIWKHVHQHZPHGLDRYHUWUDGLWLRQDO
SULQWIRUPDWV+RZHYHUWKHUHKDYHEHHQVHYHUDOH[FHOOHQWGHPRQVWUDWLRQVRIWKH
HIIHFWLYHQHVVRIDQLPDWLRQVLQVFLHQFHWHDFKLQJ1RWDEOHH[DPSOHVDUH:KLWH
)UHGHULNVHQ©VVWXGLHVRIWHDFKLQJPHFKDQLFVDQGHOHFWULFDOFLUFXLWVXVLQJLQWHUDFWLYH
PLFURZRUOGVWKDWDOORZVWXGHQWVWRUXQDQLPDWLRQVRIWKHSKHQRPHQDEHLQJH[SODLQHG
:KLWH)UHGHULNVHQ:KLWHDQG*XWZLOO+RZHYHULQWKLVFDVHWKHDQLPDWLRQV
DUHHPEHGGHGLQDZLGHUFXUULFXOXPLQZKLFKVWXGHQWVDUWLFXODWHWKHLULQWXLWLRQVDERXWWKH
SKHQRPHQDEHLQJVWXGLHGDQGDUHWDXJKWEDVLFVFLHQWLILFLQTXLU\VNLOOV6LPLODUO\LQ
(GHOVRQ©VLQSUHVVZRUNRQWHDFKLQJVWXGHQWVDERXWFOLPDWHXVLQJDQLQWHUDFWLYH
YLVXDOL]DWLRQV\VWHPVXFFHVVLYHLWHUDWLRQVRIWKHLQVWUXFWLRQDOSURJUDPLQFOXGHGPRUH
VWUXFWXUHGDFWLYLWLHVIRUVWXGHQWVEHFDXVHLWZDVREVHUYHGWKDWVWXGHQWVGLGQRWXVHPXFKRI
WKHIXQFWLRQDOLW\RIWKHV\VWHPLIWKH\ZHUHQRWJLYHQVSHFLILFOHDUQLQJJRDOVDQGVXSSRUWLQJ
FRQFHSWXDOLQIRUPDWLRQ
:HH[SHFWWKDWQHZPHGLDKDYHWKHSRWHQWLDOWREHSRZHUIXOLQVWUXFWLRQDOWRROVLQ
VFLHQWLILFFRPPXQLFDWLRQDQGWHDFKLQJ+RZHYHURXUUHVHDUFKKDVVKRZQWKDWPHUHO\
WUDQVODWLQJLQIRUPDWLRQIURPDWUDGLWLRQDOSULQWPHGLXPWRDK\SHUPHGLDV\VWHPLQFOXGLQJ
DQLPDWLRQVFRPPHQWDULHVDQGK\SHUOLQNVGRHVQRWDIIHFWFRPSUHKHQVLRQDQGOHDUQLQJZKHQ
WKHFRQWHQWRIWKHLQIRUPDWLRQLVKHOGFRQVWDQW%DVLFUHVHDUFKLVQHHGHGWRVWXG\WKH
FRQGLWLRQVXQGHUZKLFKWKHVHSRZHUIXOQHZPHGLDFDQEHXVHGLQWKHLQVWUXFWLRQDOSURFHVV
$FNQRZOHGJPHQWV
6HOPD+ROPTXLVWDVVLVWHGZLWKGDWDFROOHFWLRQDQGDQDO\VLVIRU([SHULPHQWDQG-LOO
4XLOLFL&KULVWLQD&DWHDQG1DULVD+RHYDWDQDNXDVVLVWHGZLWKGDWDFROOHFWLRQIRUWKHRWKHU
H[SHULPHQWV5R[DQD0RUHQRDVVLVWHGLQGHYHORSPHQWRIWKHK\SHUPHGLDSUHVHQWDWLRQV
7KLVUHVHDUFKLVVXSSRUWHGE\WKH2IILFHRI1DYDO5HVHDUFKXQGHUFRQWUDFW1
WR$XEXUQ8QLYHUVLW\DQG1WRWKH8QLYHUVLW\RI&DOLIRUQLD
5HIHUHQFHV
%DJJHWW : % *UDHVVHU $ & 4XHVWLRQ DQVZHULQJ LQ WKH FRQWH[W RI
LOOXVWUDWHG H[SRVLWRU\ WH[W 3URFHHGLQJV RI WKH WK $QQXDO &RQIHUHQFH RI WKH &RJQLWLYH
6FLHQFH6RFLHW\/DZUHQFH(UOEDXP$VVRFLDWHVSS
%ULWWRQ%.*UDHVVHU$&(GV0RGHOVRIXQGHUVWDQGLQJWH[W
0DKZDK1-/DZUHQFH(UOEDXP$VVRFLDWHV,QF
&KHQ&5DGD5,QWHUDFWLQJZLWKK\SHUWH[W$PHWDDQDO\VLVRI
H[SHULPHQWDOVWXGLHV+XPDQ&RPSXWHU,QWHUDFWLRQ
&KL07+GH/HHXZ1&KLX0/D9DQFKHU&(OLFLWLQJVHOI
H[SODQDWLRQVLPSURYHVOHDUQLQJ&RJQLWLYH6FLHQFH
(GHOVRQ'&LQSUHVV$GGUHVVLQJWKHFKDOOHQJHVRILQTXLU\EDVHGOHDUQLQJ
WKURXJKWHFKQRORJ\DQGFXUULFXOXPGHVLJQ-RXUQDORIWKH/HDUQLQJ6FLHQFHV
Ekstrom, R. B., French, J. W., Harman, H. H., & Derman, D. (1976). Kit of factorreferenced cognitive tests. Princeton, NJ: Educational Testing Service.
)DUDGD\36XWFOLIIH$D$QHPSLULFDOVWXG\RIDWWHQGLQJDQG
FRPSUHKHQGLQJPXOWLPHGLDSUHVHQWDWLRQV3URF$&00XOWLPHGLD&RQIHUHQFH
$&03UHVV
)DUDGD\36XWFOLIIH$E'HVLJQLQJHIIHFWLYHPXOWLPHGLDSUHVHQWDWLRQV
3URF$&0&RQIHUHQFHRQ+XPDQ)DFWRUVLQ&RPSXWLQJ6\VWHPV&+,¨
$&03UHVV
)HUJXVRQ(67KHPLQG
VH\H1RQYHUEDOWKRXJKWLQWHFKQRORJ\6FLHQFH
)UHGHULNVHQ - 5 :KLWH % < *XWZLOO - '\QDPLF PHQWDO PRGHOV LQ
OHDUQLQJ VFLHQFH 7KH LPSRUWDQFH RI FRQVWUXFWLQJ GHULYDWLRQDO OLQNDJHV DPRQJ PRGHOV
-RXUQDORI5HVHDUFKLQ6FLHQFH7HDFKLQJ
*HRUJH $OOHQ 8QZLQ /WG +RZ 7KLQJV :RUN 7KH 8QLYHUVDO
(QF\FORSHGLDRI0DFKLQHV*UHDW%ULWDLQ
+DPPRQG 1 $OOLQVRQ / ([WHQGLQJ K\SHUWH[W IRU OHDUQLQJ $Q
LQYHVWLJDWLRQRIDFFHVVDQGJXLGDQFHWRROV,Q$6XWFOLIIH/0DFDXOD\(GV3HRSOHDQG
&RPSXWHUV93URFHHGLQJVRI+&,©&DPEULGJH&DPEULGJH8QLYHUVLW\3UHVVSS
+DQVHQ651DUD\DQDQ1+6FKULPSVKHU'+HJDUW\0(PSLULFDO
VWXGLHVRIDQLPDWLRQHPEHGGHGK\SHUPHGLDDOJRULWKPYLVXDOL]DWLRQV7HFK5HS&6(
'HSDUWPHQWRI&RPSXWHU6FLHQFHDQG6RIWZDUH(QJLQHHULQJ$XEXUQ8QLYHUVLW\
+DQVHQ656FKULPSVKHU'1DUD\DQDQ1+/HDUQLQJDOJRULWKPVE\
YLVXDOL]DWLRQ$QRYHODSSURDFKXVLQJDQLPDWLRQHPEHGGHGK\SHUPHGLD,Q3URFHHGLQJVRI
WKH7KLUG,QWHUQDWLRQDO&RQIHUHQFHRQWKH/HDUQLQJ6FLHQFHV&KDUORWWVYLOOH9$
$VVRFLDWLRQIRUWKH$GYDQFHPHQWRI&RPSXWLQJLQ(GXFDWLRQ
+HJDUW\00HQWDODQLPDWLRQ,QIHUULQJPRWLRQIURPVWDWLFGLDJUDPVRI
PHFKDQLFDOV\VWHPV-RXUQDORI([SHULPHQWDO3V\FKRORJ\/HDUQLQJ0HPRU\DQG
&RJQLWLRQ
+HJDUW\04XLOOLFL-1DUD\DQDQ1++ROPTXLVW60RUHQR5
0XOWLPHGLD,QVWUXFWLRQ/HVVRQVIURP(YDOXDWLRQRID7KHRU\EDVHG'HVLJQ-RXUQDORI
(GXFDWLRQDO0XOWLPHGLDDQG+\SHUPHGLD
Hegarty, M. & Just, M.A. (1993). Constructing mental models of machines from
text and diagrams. Journal of Memory and Language, 32, 717-742.
+HJDUW\0.R]KHYQLNRY06SDWLDODELOLWLHVZRUNLQJPHPRU\DQGPHFKDQLFDO
UHDVRQLQJ,Q-*HUR%7YHUVN\(GV9LVXDODQG6SDWLDO5HDVRQLQJLQ'HVLJQ.H\
&HQWUHRI'HVLJQDQG&RJQLWLRQ6\GQH\$XVWUDOLD
+HJDUW\06LPV9.,QGLYLGXDOGLIIHUHQFHVLQPHQWDODQLPDWLRQ
GXULQJPHFKDQLFDOUHDVRQLQJ0HPRU\&RJQLWLRQ
+HJDUW\06WHLQKRII.8VHRIGLDJUDPVDVH[WHUQDOPHPRU\LQD
PHFKDQLFDOUHDVRQLQJWDVN/HDUQLQJDQG,QGLYLGXDO'LIIHUHQFHV
.LQWVFK:7KHUROHRINQRZOHGJHLQGLVFRXUVHFRPSUHKHQVLRQ$
FRQVWUXFWLRQLQWHJUDWLRQPRGHO3V\FKRORJLFDO5HYLHZ
.LQWVFK:%ULWWRQ%.)OHWFKHU&50DQQHV601DWKDQ0-
$FRPSUHKHQVLRQEDVHGDSSURDFKWROHDUQLQJDQGXQGHUVWDQGLQJ,Q'/0HGLQ(G7KH
SV\FKRORJ\RIOHDUQLQJDQGPRWLYDWLRQ9ROSS1HZ<RUN$FDGHPLF3UHVV
/RZH5.([WUDFWLQJLQIRUPDWLRQIURPDQDQLPDWLRQGXULQJFRPSOH[
YLVXDOOHDUQLQJ(XURSHDQ-RXUQDORI3V\FKRORJ\RI(GXFDWLRQ
0DFDXOD\'7KH:D\7KLQJV:RUN%RVWRQ+RXJKWRQ0LIIOLQ&RPSDQ\
0DFDXOD\ ' 7KH 1HZ :D\ 7KLQJV :RUN &'520 1HZ <RUN 1< '.
,QWHUDFWLYH/HDUQLQJ
0D\HU5(0XOWLPHGLD,QVWUXFWLRQ$UHZHDVNLQJWKHULJKWTXHVWLRQV"
(GXFDWLRQDO3V\FKRORJLVW
Mayer, R. E. & Gallini, J. (1990). When is an illustration worth ten thousand words?
Journal of Educational Psychology, 83, 715-726.
0D\HU5(0RUHQR5$VSOLWDWWHQWLRQHIIHFWLQPXOWLPHGLDOHDUQLQJ
(YLGHQFHIRUGXDOSURFHVVLQJV\VWHPVLQZRUNLQJPHPRU\-RXUQDORI(GXFDWLRQDO
3V\FKRORJ\
0D\HU5(6LPV9.)RUZKRPLVDSLFWXUHZRUWKDWKRXVDQG
ZRUGV"([WHQVLRQVRIDGXDOFRGLQJWKHRU\RIPXOWLPHGLDOHDUQLQJ-RXUQDORI(GXFDWLRQDO
3V\FKRORJ\
0F1DPDUD'6.LQWVFK(6RQJHU1%.LQWVFK:$UHJRRGWH[WV
DOZD\VEHWWHU",QWHUDFWLRQVRIWH[WFRKHUHQFHEDFNJURXQGNQRZOHGJHDQGOHYHOVRI
XQGHUVWDQGLQJLQOHDUQLQJIURPWH[W&RJQLWLRQ,QVWUXFWLRQ
0RUULVRQ-%7YHUVN\%%HWUDQFRXUW0$QLPDWLRQ'RHVLWIDFLOLWDWH
OHDUQLQJ",Q$%XW]$.UXJHU32OLYLHU(GV6PDUW*UDSKLFV3DSHUVIURPWKH
$$$,6SULQJ6\PSRVLXP0HQOR3DUN&$$$$,3UHVV
0RXVDYL6</RZ56ZHOOHU-5HGXFLQJFRJQLWLYHORDGE\PL[LQJ
DXGLWRU\DQGYLVXDOSUHVHQWDWLRQPRGHV-RXUQDORI([SHULPHQWDO3V\FKRORJ\
1DUD\DQDQ1++HJDUW\02QGHVLJQLQJFRPSUHKHQVLEOHK\SHUPHGLD
PDQXDOV,QWHUQDWLRQDO-RXUQDORI+XPDQ&RPSXWHU6WXGLHV
1DUD\DQDQ 1 + 6XZD 0 0RWRGD + $ VWXG\ RI GLDJUDPPDWLF
UHDVRQLQJIURPYHUEDODQGJHVWXUDOGDWD3URFHHGLQJVRIWKHWK$QQXDO&RQIHUHQFHRIWKH
&RJQLWLYH6FLHQFH6RFLHW\/DZUHQFH(UOEDXP$VVRFLDWHVSS
1DUD\DQDQ1+6XZD00RWRGD+'LDJUDPEDVHGSUREOHPVROYLQJ
7KH FDVH RI DQ LPSRVVLEOH SUREOHP 3URFHHGLQJV RI WKH WK $QQXDO &RQIHUHQFH RI WKH
&RJQLWLYH6FLHQFH6RFLHW\/DZUHQFH(UOEDXP$VVRFLDWHVSS
1RYDN*'LDJUDPVIRUVROYLQJSK\VLFVSUREOHPV,Q-,*ODVJRZ1+
1DUD\DQDQ % &KDQGUDVHNDUDQ (GV 'LDJUDPPDWLF 5HDVRQLQJ &RPSXWDWLRQDO DQG
&RJQLWLYH3HUVSHFWLYHV%RVWRQ0$0,73UHVVDQG0HQOR3DUN&$$$$,3UHVV
6FDLIH05RJHUV<([WHUQDOFRJQLWLRQ+RZGRJUDSKLFDO
UHSUHVHQWDWLRQVZRUN",QWHUQDWLRQDOMRXUQDORI+XPDQ&RPSXWHU6WXGLHV
6FKZDUW]'/%ODFN-%$QDORJLPDJHU\LQPHQWDOPRGHOUHDVRQLQJ
'HSLFWLYHPRGHOV&RJQLWLYH3V\FKRORJ\
6FKZDUW]'/%UDQVIRUG-'$WLPHIRUWHOOLQJ&RJQLWLRQDQG
,QVWUXFWLRQ
6KDSLUR$03URPRWLQJDFWLYHOHDUQLQJ7KHUROHRIV\VWHPVWUXFWXUHLQ
OHDUQLQJIURPK\SHUWH[W+XPDQ&RPSXWHU,QWHUDFWLRQ
6ODPHQND1-*UDI37KHJHQHUDWLRQHIIHFW'HOLQHDWLRQRID
SKHQRPHQRQ-RXUQDORI([SHULPHQWDO3V\FKRORJ\+XPDQ/HDUQLQJDQG0HPRU\
6SRHKU.7(QKDQFLQJWKHDFTXLVLWLRQRIFRQFHSWXDOVWUXFWXUHVWKURXJK
K\SHUPHGLD,Q.0F*LOO\(G&ODVVURRP/HVVRQV,QWHJUDWLQJ&RJQLWLYH7KHRU\DQG
3UDFWLFH&DPEULGJH0$0,73UHVV
:KLWH%<,QWHUPHGLDWHFDXVDOPRGHOV$PLVVLQJOLQNIRUVXFFHVVIXO
VFLHQFHHGXFDWLRQ",Q5*ODVHU(G$GYDQFHVLQ,QVWUXFWLRQDO3V\FKRORJ\+LOOVGDOH1-
/DZUHQFH(UOEDXP$VVRFLDWHV
3ULRU5HVHDUFKRQ
0XOWLPRGDO&RPSUHKHQVLRQ
3UHOLPLQDU\0RGHO
([SHULPHQWVRQ
'HVLJQ*XLGHOLQHV
0XOWLPRGDO&RPSUHKHQVLRQ
+\SHUPHGLD0DQXDO
(YDOXDWLRQ
)LJXUH+\SHUPHGLDSUHVHQWDWLRQGHVLJQSURFHVV
)LJXUH$VFUHHQRIWKHK\SHUPHGLDPDQXDO