Approximate Statistical Tests for Comparing Supervised

!"$#%&(')*+,-./
1023546378'9:;=<>.@?A-..
/+/&&BCD+
EGFHJILKNM6O PRQDSUTWVXVYT[ZXS]\^F
_;`;a;bdce-f:gNhRei_jflkJaJm
QDTWnRK5ZlVYIoT[pVqH5rtsjHJIunvVYTWZw;\[S]TWpd\[T
x Z:TWyJHJpzw;VYKNVYT,{
pSU|JTWZ:MXSUV}
sjHJZX|NK5~U~]S]MW x€ƒ‚„N…J…†
QDT[\[TWIA‡T[Z …Jˆ  †‰‚J‚„
6Š ‹ŒXlŽ^i%
‘’”“–•t—”˜%—N™Xš>š›™Xœ‰“™Xž• ŸNœ%™G˜¡—i—”š›¢Y£¤“–¥1˜¡¦›™j•§¦=˜^¦›“¨•©¦›“¨ªl˜¡«¦B™X•©¦B•t¬­¢Wšt®i™¦›™Xš›¥1“–¯i“–¯i°Dž’i™¦B’i™lš>¢W¯i™1«™:˜¡šB¯i“¯”°
˜¡«±
°W¢%šB“²¦B’i¥¢%³¤¦›±]—5™l𛬭¢%šB¥G• ˜%¯i¢¡¦B’i™lš¢%¯´˜q—N˜¡š›¦›“¨ª³i«¨˜¡š«™:˜¡šB¯i“¯”°G¦B˜W•©µJ¶‘’”™X•›™>¦›™:•§¦=•˜%š›™jª¢%¥1—”˜%š›™:®™£¤—N™Xš›“±
¥1™l£¤™X“–•©¯W¦B¦=•
˜¡«–º»«¦§·6·‰¦›—N¢q™6®¤¼-™l™X¦›™Xš›šBš›¢%¥1š“–½¯i¶™>‘¦›’”ž"™l¢,“–šž—i“–šB®¤¢%™X¸”«·[˜%±¾¸i³”“«–•›“™X¦§®A·q¢¡•§¦=¬d˜^“¦B¯N“–ª•©¦›¢%“¨šBªlšB˜%™X«¿ª¦B¦›«™:·
•§¦=®¤•*™¦B˜¡™XšBª™q¦B“•›¯”’i°6¢^ž˜1¯A®¤“²¦›¹J¢™lšB’”™l˜Y¯NœWª™G™*’iž“–°%’i’À™X¯D—i¯išB¢%¢q¸”®¤˜%“¸i¹5“–™X«“š›¦§™X·,¯”ª¢%™ ¬
‘"·‰—N™¼™lšBšB¢%š“–¯2ªl™lš›¦B˜¡“–¯,•›“²¦B³”˜^¦B“¢W¯”• ˜%¯”®,•›’i¢W³i«–®´¯i™lœW™lš ¸5™j³”•©™:®¶t‘’i™X•›™*¦B™X•©¦B•-˜¡šB™
ºÁ˜[½˜q¦›™X•©¦-¬­¢%š¦B’i™
®i“²¹J™lšB™l¯”ªl™¢%¬¦§ž3¢&—išB¢%—5¢%𛦛“–¢%¯”•3˜¡¯N®,º­¸5½˜—”˜¡“–šB™X®‰±Â®¤“¹5™Xš›™X¯”ª™:•ÃR¦B™X•©¦3¸N˜%•›™X®G¢W¯q¦B˜%µ‰“¯i°1•›™lœ%™XšB˜%«”š=˜¡¯”®i¢%¥
¦BšB˜%“¯JÄ:¦›™:•§¦t•©—”«“¦B•X¶>ÅÆ¦B’i“š=®D¦B™X•©¦Xǘq—N˜¡“–š›™:®‰±¾®i“²¹J™lšB™l¯”ªl™X• 曙:•§¦t¸”˜%•›™X®´¢%¯ È:É^±]¬­¢%«¨®ªšB¢W•B•§±¾œ^˜¡«–“–®i˜¡¦›“–¢%¯ÇN™£‰±
¦B’”¢´“¸i’”“¦B˜Y•-œW•›™G¢%«¥1¢^ž)™lž‘’”·‰˜¡¦—5™G™l«–™l¼tœ^™l˜^šB¦Bš›™X¢W®DšX¶—išB‘¢%’i¸N™1˜¡¸iŸi“–¬»«“¦›¦§’ ·q¦›¢%™X¬R•©¦&‘"·‰“¨•&—N™&˜D¼3¯i™X™Xž9š›šB¢%¦›š:™X¶"•©¦XÅÊÇ¿Î^¬­¢%£i³iÏ%š›ª¦›œJ’DÇ;¦B¸N™X˜%•©•›¦X™XÇN®ÐË,¢%ªl¯o̙XÎ¥G“¦›˜¡™Xš:šBÍ ˜¡•¦›¦B“–™X¢%•©¯”¦X•>Ç5“–¢%•¬•©Ï^’”±U¢^¬­¢Wž«–®¯
ªlš›¢[•›•©±¾œY˜%«“¨®i˜¡¦›“–¢%¯¶3ÑR£¤—N™X𛓖¥1™l¯[¦B••©’”¢^žÒ¦B’”˜^¦¦B’i“¨•¦B™X•©¦˜¡«¨•©¢G’N˜%• ˜Wªlªl™l—¤¦=˜¡¸i«–™*‘·‰—5™*¼™lšBšB¢%š:¶"‘’i™*—N˜¡—5™lš
¦B˜%™X«–•›•©¢¦B•X¶j¥1‘™X˜W’”•©™1³išBªl™Xš›•-¢[•›¦B•©’i±¾™6œY˜%—5«“¨¢^®iž3˜¡™l¦›™Xšq®´ºÁ˜%øi¦›“–™:«“•§¦§¦>·,“¨¦B•-¢´¦›’i®¤™1™¦B¥1™Xª¢W¦j•©¦>˜¡«–—N°%¢W¢^ž3š›“¦›™l’”𛬭¥Ó³i«]¶®¤“‘¹5™X’iš›™q™X¯”Î^ª£i™:Ï%•*ªœDž’i¦B™X™X•©¯A¦t¦›“¨•>’i™X•›·Ð’i¢^®¤ž¢¯´™l¦B£¤¢“–•©¦=¸5½t™G¢%•©¬«–“–¦›°%’i’[™:¦›•©«–· ™
¥1¢Wš›™
—N¢^ž3™l𛬭³i«¿¦B’”˜¡¯uË´ªlÌ ™l¥G˜¡š:Í •&¦›™:•§¦:¶Ð‘’”™
ª=’i¢W“–ªl™q¢¡¬¦B’i™
¸5™X•©¦&¦B™X•©¦j“¨•j®¤™¦B™lšB¥j“–¯i™:®À¸‰·2¦B’i™
ªl¢%¥j±
—”³¤¦B˜¡¦›“–¢%¯”˜%«"ª¢[•§¦*¢%¬š›³i¯”¯i“¯”°D¦›’i™
«™:˜¡šB¯i“¯”°D˜%«°W¢%šB“²¦B’i¥D¶Ô”¢%š&˜%«°W¢%šB“²¦B’i¥G•t¦›’N˜^¦ªX˜¡¯A¸N™6™£¤™Xªl³¤¦›™:®A¢%¯i«–·
¢W¯”ª™WÇË´ªX̙X¥1˜%šXÍ •t¦B™X•©¦*“–•t¦B’i™G¢%¯”«·,¦›™X•©¦*ž“¦›’ ˜%ªXª™X—¤¦B˜%¸i«™G‘"·‰—N™G¼-™lšBš›¢WšX¶qԔ¢%š*˜%«°W¢%šB“²¦B’i¥G•-¦›’”˜¡¦&ªl˜¡¯
¸5¸5™t™XªX™˜¡£¤³”™X•›ªl™t³¤“²¦›¦ ™:®q®i“¦BšB™l™X¯
ª¦›¦›«–·6“–¥1¥1™X•X™Xlj˜W¦›•©’i³i™*šB™XÎY•3£”œ^Ï%˜¡ªšBœj“¨˜^¦›¦›™:“–¢%•§¦¯D“¨®¤•"³išB™-™Xªl¦B¢%¢1¥1¦›¥1’i™*™l¯”ª=®i’i™X¢W®“–ªlǤ™t¸5¢¡™X¬;ªX˜¡¦BšB³”˜%•›“™¯”““¦¯i°G“–••›•›™«¦:“–°%¶ ’[¦B«·G¥1¢%šB™-—N¢^ž3™l𛬭³i«˜¡¯”®
Õ
!"#%$'&(*)+,.-+/1023*$'!+&4!"//5,.6.#*!"67-8-+9:0!+#;6.(<,7!";6.(=>!+,7=+-+670??9@-+A#*,.!+67B5#*!"67-8
!+ CD*$0!+3EGFIH( 6.-8J!"6. 9@-+JKL6.#;M !"6. 6N#*!+,02*(-D&J!"(*&(&%OQPR(/1H(/S-8 '-+96.T/1!"/S*R6.
-6.3)+ 67=8!" -8U-+9V(F3H(67-8*$(&(02-81 !" U!"JXW(6. 6.=!"6. 6.#*!+,02*-I&J!"6.!+&(FIH!" 9@-+6.UFIH( 67-8%$%!+&</-+/S-8 2!A*KY !"6. 6.#*!+,Q !"Z(-[KLU!+#*#X*/1!"\1,7?/S*9@-+0!+1#X6N<6.676.!+,
XWD/*;6.023*O
PV-<H&(*!+&](MFIH( 67-8^!+6.&>6N_6NZ/1!"/S*$`676.a(,7/9bH,c -d#X-816.&(*!!eWI-8(-80:E_-+9(
&6gfS*3hCD6.&1Q-+9V!"6. 6.#*!+,SFIH( 67-8T!"J!"6N '6Ni0!+#;6.(,7!"16.(=(O`jk67=8H( =86.)+J!:!eWD-8(-80:Ei-+9
6.!"6. 6.#*!+,%FIH( 6.-8*O
lm*hH`\S*=86.?!"c(R-3-+Q-+9S( *+OQPR(nB5 c6.H(JKTL0:H1 c#X-86N&(*c6.`K(*(*cKTL!"' H&(ED6.(=
-8,7Ed!A6N(=8,7!"//1,N6.#*!"67-8<&(-80!+6Nd-+ZKL(**Khi!"i H1&(EI6N(=M0:H,.67/1,7&(-80!+6.1*O]02-8 Z!"/1/1,.67&
!";#$(*a6.L!i6.(=8,7&-80!+6.A-+9Q6No **$!+&p-8H(L=+-8!+,6.L -?B51&M(Z\S #*,.!+6gB5*L-+LZ\S ,7!"16.(=?!+,7=+-+;670q -r!"/1/1,7EA6.p!"U&(-80!+6.O'sL-tKh*)+*[$m!?9bH&1!+02o!+,V=+-8!+,k-+9Q!"#<6.u0!+#16.(
,7!"16.(=6.` -:B5&2,.!"6.(=!+,.=+-+670c!"cKT-+CKT,.,-t)+*T!ZKL6.&(J!+=+L-+9%!"//1,N6.#*!"67-8&(-80?!+6.*Ov<
KL6.,N,5*H(r -6NJ6.H(U\S,7-tKwD9x-+(02-8023*$1,7*HJ#X-86.&*R(6.=8,7Xyz&(-80!+6.r#*!+ +O
v4676.M!i6.(=8,.&(-80!+6.$(*Z!":zKT-i&6gfS*3R *-+9kFIH( 6.-8R&(*/S&6N(=-8AKL(**RKha!"
!+!+,7EI{6.(=i#*,.!+6gB5*J-+!+,7=+-+670?*OJ|}#*,.!+6gB5*6.!9bH#X67-8p!"*$5=86.)+u!+6.(/1HJXWD!+02/5,7+$5!+67=8
!"RXW(!+02/1,.U --8(-+9k~#*,.!+ *Oh|€,7!";6.(=2!+,7=+-+6.06.J!29bH#X6.-8r!"*$=867)+A!? *R-+9XWD!+02/5,7
!+&^(67a#*,.!+ *$Q#X-81 H#Xa!u#*,N!+6gB1*[OM‚>!p/5!"6.#*H,.!":!"//1,.6.#*!"6.-8_ * 6.=($`-8H(Z/;6.0!"Ed=+-8!+,n6.
HH1!+,.,7E> -^B&ƒ(\S i#*,N!+6gB1*G!+&„ 6.0!" d67G!+#*#*H(!+#XE-8…9bH(H(uXWD!+0/1,7*O†j(-+rXWD!+0/1,7+$
H(/1/-8Kh!"aKh-+CI6.=?9@-+'!G02&6.#*!+,6. H023!"67-8p#X-80/1!+oEu!+&pKTaKL6.p -G0!+IH(9b!+#XH(!+&
,.,(!+?6. H102ok9@-+c#*,.!+679xED6.(='\1,7-I-D&2#X,.,.*Ok|nQ(R6N02JKhR!"L&(67=816.(=(R6. H023*$+Kh#X-8H,.&
=8!"(*!?,.!"=+:#X-8,.,7#X6.-8A-+9`\1,7-I-D&A#X,.,.!+&p![)+a!i3H10!+MXWD/*#*,.!+679xEG!+#d#X,.,‡OTv<#X-8H,.&A(
!"//1,.E]!,.!"6.(=!+,7=+-+6710ˆ -</1-I&1H#Xi!d#*,.!+6gB1*a9x-8016.:*a-+9'#*,.!+67B1&^#X,N,.*O]PR(r#*,.!+6gB1*
KT-8H,.&\Sa6.0/1,7023 &d6.(26. H102o!+&d -8,.&%Ov<Kn!+oU-8H(U6.1 H023' -A#X-8o!+6.d02-8 !+#*#*H(!" a#*,.!+67B1*JKh:#*!+rB5&O
PR*U!"U -802!"//1,N6.#*!"67-8*$D(-tKh*)+*$1KL(*'Kh0aH J ,.#XTU\T,7!";6.(=!+,7=+-+6710‰;!"(*
!+_B5&6N(=r(?\S a#*,.!+6gB5*OMj(-+XWD!+02/5,7+$H(//S-8 KT?Kn!+oa -d ,.,Q!+^,7#X -86N#?0!+6.,h ED 0
!"a,7!"U -u#X-+=867{*G!+&<B,7 *nŠH(Cd0!+6.,‹Oiv„(*)+*:(GH *#X67)+:!+^0!+6.,c02!"=+i!"
(*E<#X-86N&(*nŠH(Cd0!+6.,‹$%*E<KL6.,.,VŒ5!"=A1!":02!"=++Or*6.-I&6N#*!+,.,7E+$m!A,.!"6.(=A!+,.=+-+670Ž6N#*,.H&(&
6.M/-+=+!+0KL6.,.,S!+!+,7EI{*ZZ!+#*#*H0aH,.!" &AXW(!+02/1,.J-+95Š HCG!+&p(-8DygŠH(C?0?!+6.,%!+&uH(/&!" 67
B5,7 *;6.(=H,7OQUH(Š-+\u6.n -i&(* *;06.(KL6N#M,7!"16.(=2!+,7=+-+6710 -?6.#*,.H1&(U6.r(U/-+=+;!+0MO
PR(XWI:,7*)+,`-+9n(2!eWD-8(-80ZE]&16. 6.(=8H16.(\S*zKT*]zKT-p9bH&1!+02o!+,k!+CI*‘: 6.0!"6N(=A!+#’y
#*H(!+#XEM!+&p#(-I-86.(=2\S*“Kh*u#*,.!+6gB5*”@-+!+,7=+-+;670;•;O`v4(MKTZ0!"C+*R-8H(R\1,.-3-D&M#X,.,&6.!"=8-86.
ED 0M$kKTiKT-8H,.&],N67C+? -0!"C+M!p#*,.!+6.0ˆ!"\S-8H(:67:!+#*#*H(!+#XE+OsL-tK#*!+^KTG02!+H(?16.Z!+#*#*H(;!+#XE(–
|'&A-+9Q#X-8H(; +$5K(AKTa&67=8A: ED 0M$Kh:KT!+3' -G#;(-3-8:(:\S #*,.!+6gB1*R9@-80— -802a *-+9
!)"!+6.,.!"\5,7#*,.!+67B1**O
PRi,7-tKh 2,7*)+,h-+9R(G!eWD-8(-80ZE>#X-8#X*a(G!+02-8H3:-+9&!"!d!)"!+6.,.!"\1,7+Ou“9JKTr!)+M!,.!"=+
!+02-8H3Z-+9J&!"!D$k(_KhG#*!+^*: -802i-+9J67Z!+6.& -u *)+i!+a!A a *Z9@-+Z*)"!+,.H!"6.=p#*,.!+67B1**O
˜H#™6N02/1,7*a !"6. 6.#*!+,J0*(-I&1a#*!+š\SM!"//1,N67&_6.>6Na#*!++O^s-[KT*)+*$T6N>0-8 267H!"67-81*$`(
!+02-8H3-+9J&!"!A6N,.6.06. &!+1&<Khi(*&< -pH?!+,.,k-+9J67!+Z6N(/1H(' -p-8H(:,7!"6.=r!+,.=+-+670OPR6.
02!+k1!"`KhJ0aH `H J -802n9@-+0†-+91!+02/1,.6N(=a”b6‹O›+O7$3#X-8y‡)"!+,.6.&!"6.-8:-+`J\-I-+ !"/S•% -/S*9@-+0
(Z!"6. 6.#*!+,%!+1!+,7EI6.*O
œ-[Kˆ!"MKh^!)+]*)I67*KT&(<=+*!+, H1#XH(d-+9:]!eWI-8-80ZE+$,7**žŸp#X-86.&(*M]6.(
!"6.6.#*!+,kF3H(67-8*Ov<!+H02:!"U!+,.,&!"!?/S-86.3”@XWD!+02/5,7;•!"&(!Ku6.1&(*/S&(3,7Ei9@-80 !
BWD&M/-+\5!"\16.,.67“E?&6. 67\1H(6.-8?&(XB5&r\3Er(U/1!"6.#*H,.!"R!"//5,.6.#*!"67-8r/-+\1,70MO
¡ H 67-8 ‘`¢(H(//S-8 TKhR!"n=867)+2!U,N!"=+J!+02/1,7n-+9&!"!U!+&2!#*,.!+67B1*c£OkPR(J#*,.!+67B1*c£„0!E
Õ
Õ
¤
single domain
multiple domains
9
analyze classifiers
predict
classifier
accuracy
analyze algorithms
choose
between
classifiers
predict
algorithm
accuracy
choose
between
algorithms
large
sample
small
sample
large
sample
small
sample
large
sample
small
sample
large
sample
small
sample
1
2
3
4
5
6
7
8
Õ
jk67=8H( ‘L| !eWI-8(-80:Ep-+9h !"6. 6.#*!+,kFIH( 67-81'6Np0!+#;6.(,7!"6N(=(O'PL(Z\S-WD&d(-D&(r” ¡ H( 67-8
6.n(:H(\DŠ #XJ-+9k6.n/1!"/S*O
•
![)+\S*]#X-8 H1#X &]H6.=G/5!"U-+9h?&!"!D$m\5H(U(*6.-8H(=8]&!"!M0!+6.6.=i9@-+Z!A */5!"!" A **O‰sL#X+$KT<#*!+02!+H(d(<!+#*#*H(;!+#XE-+92£ˆ-84(d A *M!+&4#X-8 H1#Xr!š\56.(-806.!+,
#X-8DB5&#XG6No *)"!+,”‡¢D(&(#X-+T-D#(!+%$ wc9x-8
—PR67\167!+6‡$ w-8!)D6‹$ •;O_œ-+ !"6. ¡ H 67-8 $%(2#*,.!+6gB1*U#X-8H,.&d![)+2\*d/1-I&1H#X&p\IEd!+oE02*(-D&ƒ”@+O›=(O7$V6.3 *)D67*KL6.(=G!+
XWD/*;•;w167R(*&A-+R!)+:\*A/-D&H#X&G\3EM!2,7!";6.(=2!+,7=+-+6.0MO
¡ H 67-8 ¤ ‘ 67)+?!0!+,.,1&!"! **$ T$oH//S-8 JKh!"//1,7Ea,7!"6.=!+,.=+-+67
0 „ - < -:#X-8 H#X
#*,.!+6gB1*
£ Qwn(-[K!+#*#*H(;!" ,7E_KL6N,.,T
£ #*,N!+679@E_(*KXW(!+02/1,7;
– T#*!+H MKTM!)+A(-d */1!";!" M **$T(*A6.2-d&167#X2Kn!Eš -]!+ KT*6.FIH( 67-8%Oš|q9@F3Ho,7E3yz!"//1,N67&^ !" *=+E™6. -]#X-8o)+*
6.FIH( 67-8<6No - ¡ H(67-8
I
‘ J!+uKT2/&16.#X!+#*#*H(!+#XE-+9h!+,7=+-+6.
0 KL6.6N ;!+6.(&-8
!+&-80,7E3yz ,7#X &A&!"!i *-+9J”b!"/1/-W(6.0!" ,.E(•T:!+02:67{*Z!+
Q–u‚9k-($1(pKhZ#*!+u/&16.#XJ(
!+#*#*H(!+#XEM-+9Q£ K6.#rKn!+R-+\1!+6.(&M9@-80 !+6N6.(=-8
TO
¡ H 67-8 ‘ 67)+?zKT-:#*,.!+6gB1*T
£ p!+&r£ <!+&-8H(=8?&!"!9x-+T! */1!"!" h**$I&(* *;06.(
KL6N#_#*,N!+6gB1*:KL6.,.,Q\G02-+r!+#*#*H(!" r-8š(*K :XW(!+02/1,.*OuPR16.ZFIH( 67-8>#*!+>\Si!+ KT*&^\3E
02!+H6.(=Ai!+#*#*H(!+#XE^-+9J!+#™#*,.!+6gB1*:-8_(G */1!"!" r *!+&_!"/1/1,7ED6.(=A˜#*œL0!"[žŸ *$
KL6N#rKL6.,N,5\S&(#X67\S&G\,.-[KO
¡ H 67-8
!(
‘ 67)+_zKT-u#*,.!+6gB1*:£ !+&^£ /-D&H#X&d\IEd9x*&6N(=A!p0?!+,.,`&!"!p*"
4 -uzKT,7!"16.(=i!+,.=+-+670#
‰!+
& $G$K6.#d#*,.!+6gB1*KL6.,N,%\S202-+2!+#*#*H(!" 6.d#*,.!+679@EI6.=i(*K†XW(!+02/1,7;–
|=8!+6.$R\S#*!+H dKT1!)+](-š */1!"!" < *r-+9 r&1!"!D$LKTd#*!+1(-+M!+ KT*G16.iFIH( 6.-8„&167#X,7E+O
|=8!+6.$( -802U!"#(*;R!)+U!"C+p(U!"//-8!+#;M-+9V#X-83)+*6.=26.T/-+\1,.0 6No -2!2FIH( 67-8r!"\S-8H(
,7!"16.(=]!+,7=+-+6.0p” ¡ H 67-8 •;O4‚9Kh#*!+&(* *;06.(MKL16.#ƒ!+,.=+-+670 HH!+,.,.E_/-D&H#X?02-+
!+#*#*H(!" :#*,.!+6gB5*”@KL(r !+6N(&r-8A&!"!2 *L-+9!"/1/-W(6.0!" ,.E?(!+0Z67{*t•;$((AKh#*!+A,7#X
Õ
Õ
Õ
Õ
(Z#*,N!+6gB1*”‡£^-+'£ c•n#X!" &A\IEi!"!+,7=+-+670AO
¡ H 67-8 ‘ 67)+!i,7!";6.(=?!+,.=+-+670 }!+&!i,.!"=+:&!"!r *T$KL!"6NL(a!+#*#*H(!+#XEu-+9Q(
#*,.!+6gB1*'/-D&H#X&u\IE KL(d !+6.&u-8<*K} !+6.6.=G *-+9h!M /S#*6gB1&d6.{*
–„PL6.F3H 67-8
!+T(-+c#X67)+&i0aH#;i!" 367-8r6.2(',.67 *!"H+Oc(L!"//1-8!+#$D!+&()+-D#*!" &r\3E2#cl ^/-eŠ #X
”bs'6.o -8%$œ!+,‹$kPR67\1167!+6‹$ #hl # !+0 020:\*;*$ w !+0aH $ 8•;$Q6.U -H(\&67)D6.&( 6.3 -:!Z T*T!+&i *)+*!+,&6.@Š -86.3Q !+6.16.(= *h-+9%67{* OkPL( 6.c !+6.(&?-8?!+#;G-+9( ;!+6.6.(=
*n!+&i(LH,76.=Z#*,.!+67B1*Q!"' &G-8G(L T **OhPR(![)+*!"=+U/S*9@-+0!+#XR-8G( n *
6.0?!" R(Z!+#*#*H!+#XEr-+9k*KH*O
¡ H 67-8
I‘ 6.)+p!?,.!"6.(=!+,.=+-+670 }!+&p!i0!+,.,%&1!"!? *
T$1K!"L6.L(:!+#*#*H(;!+#XEA-+9`(
#*,.!+6gB1*/-I&H1#X&d\3E KL( 6.U !+6N(&d-8_(*K !+6.6.=r*-+9T(?!+02i67{*?!+ c– -8!)D6
” •?(-[K!"? !"67B1& y‡9x-8,N&ƒ#X-8y‡)"!+,.6.&1!"67-8š/-D&H#X9b!+67,7E_=+-3-D&™ 6.0?!" ?6.š6N2#*!+ +O
œ-+ !"R6.r!+3Ei!+02/1,.6.=:!"//-8!+#$(KT#*!+(-+R ;!+6. -8r !+6N6.(= *J-+9VXWD!+#X,7Er(U!+067{*
!+ TO` !+&$DKT !+6.G-8r&!"! *n!"J!)+,.67=83,7E29x*KT*JXW(!+02/1,7U”@+O›=(O7$ -+9V(U67{*'-+9 ^6.
y‡9@-8,.&]#X-8 y‡)e!+,.6N&!"67-85•!+&d,.Ep-8<(2!+H02/67-8d1!"U(2/S*9x-+;0!+#X-+9T,.!"6.(=r!+,7=+-+6.0
#1!+(=+02-I-+,7EpKL67d#!+=+6.(267{*-+9ca ;!+6.6.(=r&!"!DOaPR6.!+H102/67-8#*!+<\Sa#;(#C+&
XWD/*;6.023!+,.,7Eu”@\IEr/*9x-+0?6.(=!+&&6767-81!+,#X-8y‡)"!+,.6.&!"67-8MH&67;•cKL6.r*)+p0!+,N,7*J !+6.6N(=a**$
\1H(T67h#*!+1(-+n\'#(#C+&A&67#X,7E9x-+n !+6.16.(=Z *T-+9%'6.{*L-+9 T
O LH,7h-8G(!"/SL-+9m,.!"6.(=
#*H()+?(-[K 1!"6.™ -802A#*!+ 6.202-I-+(2!+H02/167-8>KL6.,.,h\SG)D67-8,.!" & ”bs'!+H,7*
$ U!";*$
¢IH=($ }PR6.3\IE+$ !3•;Ohœ-8(*(,7$167R6.J-+\5 *)+&M -?-8,.&rXWI/S*6N02o!+,N,7Ei6.M0-8 R!"//1,.6N#*!"67-8*O
¡ H 67-8
3‘ 67)+dzKT-p,7!";6.(=G!+,7=+-+670? ‰!+
& $—!+&d!M,.!"=+2&!"!M * T$K6.#d!+,7=+-+;670
KL6.,N,o/-I&H1#XT02-+n!+#*#*H!" L#*,.!+6gB1*KL(a !+6N(&-82&!"!U *`-+9!U /S#*6gB1&67{* –rPL6.kF3H 67-8
!+`(-+`#X67)+&0aH#2!" 367-8$I!+,7(-8H(=8(
#hl #< !+0Y!+QH&67&:6.`FIH( 67-8a9@-+Q*=+67-8
/-+\5,70*O`PR(*EG&67)D6.&(
š6.3 -*)+*!+,%&6.bŠ-86.3T !+6.6.=a *L!+&M!26.(=8,. **
O c!+#;A!+,7=+-+6.0 6.
!+6.&p-8u!+#d !+6N6.(=?**$S!+1&u!+,.,VH1,76.(=?#*,N!+6gB1*;L!"a &d-8u(: **O'|'u!+1!+,7EI6.L-+9
)"!"6.!+#X'#*!+ii\SR/S*9@-+02&?!"J6.#*,.H1&(` *0h9@-+h(#-86.#X-+9V,7!"6.=Z!+,7=+-+6710M$o(#-86.#X
-+9(' !+6.6N(= **$(!+&i!+#r6.&6.)I6.&1H!+,I hXW(!+02/1,.+OkPL( ¡ H1!+6gy ”blV6.&0!+%$ +¤ •h6.c!"/1/1,.67&
-?&(* *;06.(UKL*(*J(XfS#X&HU -(#-86.#X-+9`,7!"6N(=2!+,7=+-+670?J6.J67=867B5#*!+o,.Ei(-8Dy‡{**-(O
¡ H 67-8 ‘ 67)+a“Kh-U,7!"16.(=!+,.=+-+670 ƒ!+
& $ !+&!'0!+,.,3&!"! * T$+KL6.#;:!+,.=+-+670€KL6.,.,
/-D&H#X02-+!+#*#*H!" #*,.!+67B1*TKL(G !+6.&r-8M&!"!2 *n-+9(U!+0267{*U!+ Q–pPR(/1H/S-8 -+9
6.n/5!"/*J6.n -i&(#X67\S!+&M#X-802/1!" *)+*!+,m!"6. 6.#*!+,S L9x-+R!+1 Kh*;6.(=6.JFIH( 67-8%O h#*!+H1 >6.J0?!+,.,‹$(67nKL6.,.,5\SU(#X!"EG -H U(-8,N&(-8H(J!+&r!+0/1,.6.(=a02*(-D&*OQ|'R02367-8(&M6. ¡ H 67-8
i!"\-t)++$%6.02!+'!"KTa#*!+-+!+1 Kh*6NFIH( 67-8uXW(!+#X,7EpKL6.(-8H(0?!"CI6.=?(!+H0/67-8
!"(/S*9@-+0!+#X-+9k(:zKT-i,7!";6.(=!+,7=+-+;670R#;!+(=+'02-I-+,7EGKL67A#;!+(=+'6.M(a67{*-+9
( ;!+6.6.(=2 **On¢I/S#*6gB5#*!+,.,7E+$DKTKL6.,N,*&M -i!+H02U!"R,.!"67)+:&6gfS*#X6.r/S*9@-+0!+#XU-+9
(“Kh-i!+,7=+-+670?R#!+=+L,7-tKL,7EiKL67M#1!+(=+6Nr(67{*U-+9k( ;!+6.6.(= **O
¡ H 67-8 ‘ U67)+GzKT-,.!"6.(=:!+,7=+-+670? !+& $Y!+&G&!"! *T9@-80‰*)+*!+,S&(-80!+6.$oKL6N#
!+,7=+-+6.0 KL6.,.,+/1-I&1H#XT0-+T!+#*#*H!" R#*,.!+67B1*VKL(a !+6.(&:-8XW(!+02/1,79@-80†(*K™&(-80!+6.’–GPR6.
6.c/*;!"/1`(02-8 h9@H1&!+023!+,1!+&i&6 i#*H1,7QF3H 67-8G6.0!+#;6.(',7!"6.=(O`¢I-80L !";#(*n![)+
!"//1,N67&!6.02/5,7R67=8i U”@-+Tv46.,.#X-[WI-8r67=8&Dy‡!+(CDQ ;•c - E2 -!+ KT*n6.TF3H(67-8$o\5!+ &
-8ƒ6N(=8,7rHa-+?#X-8y‡)"!+,.6.&!"67-8(y‡\1!+ &š 6.0?!" *$n\1H(( A ?&(-](-+2!"C+6.3 -]!+#*#X-8Hoi(
H#X*!+6.o“E-+9(6.&6.)I6.&1H!+,3#X-80/1!"6. -81*O `fS#X67)+,7E+$DKh'Kn!+oT -a#X-80:\16.L('H,.Q9@-80‰*)+*!+,
!+ KT* - ¡ H 67-8 $cKL*i!+#;ƒ!+ KT*!+2!+™!+-I#*6.!" &…H#X*!+6.3zE+O>PR6.6.!+™6.0/-+!+o
FIH( 67-8M9@-+R9bH(H(!"#O
¡ H 67-8
3$ $D!+1& !"(02-8 J6.02/S-+!+3c9@-+nXWI/S*6N02o!+,5 !";#G-8r,7!"6.=:!+,7=+-+6.0*O
v4(™ -802*-8(A&*)+,7-+/12!<(*K,7!";6.(=d!+,7=+-+6710 ”@-+i!<02-I&16gB5#*!"67-8> -]!+šXW(6. 6.(=d!+,7=+-+6.0i•;$
!+ KT* -?(aFIH( 67-81L#*!+&(* *06.KL(*(*(:*K !+,7=+-+;670 6NR\S* *1!+AXW(6. 6.(=i!+,7=+-"y
6710*
O (9@-+H1!" ,7E+$(0!+oEG&!"!2 *JH&G6N?XWD/S*6.023!+, !"#;M!" -3-0?!+,.,1 -2!+,.,.-[K4/S-86.(=
¡ H(67-8
3OQs#X+$316.`/1!"/S*c9@-D#*H Q-8i&(*)+,7-+/S&=+-I-D&? !"6.6.#*!+, h9@-+ ¡ H(67-8 O`v<&(XB5(
Õ
Õ
Õ
Õ
Õ
Õ
!
Õ
!+&A#X-80/1!"UB1)+!"6. 6.#*!+, L9@-+R6.JFIH( 6.-8O
T*9@-+/-D#X*&6.(=iKL67Aa&(*;67)e!"6.-8A-+9Q(!"6. 6.#*!+,V *$676.Kh-+(-+6.=?!"!+#d-+9
(FIH( 67-81/S-8 &<!"\S-t)+#*!+<\SaXWD &(&d\S*E+-8&<#*,N!+6gB5#*!"6.-8!+,.=+-+670!+&<0?6.#*,.!+6gB5#*!"67-8
!" *Oj(-+UXW(!+02/1,7+$S6.0!+3EA&(#*6.67-8(yz0!"CI6N(=? * 6.=8*$676.'6.02/S-+!+3L -G 6.0?!" aa#X-8&16767-8!+,
/-+\5!"\16.,.67“E?!"'!?(*K€XWD!+0/1,7Z\S,7-8=8J -?!+#;u-+9`Z~ #*,.!+*OL(Z02!+H(Z-+9k(a!+#*#*H(!+#XEA-+9
/-+\5!"\16.,.67“E? 6N0!" 6NR(a,7-+=G,.-8*$1!+& ¡ H( 6.-8 $ ¤ $ $!+
& ?#*!+u!+,.,m\S*/1(;!+ &A6.M *;0L-+9
&(* *0?6.6.(=2ZXWD/#X &d,7-+=r,7-8-+9Q!G#*,.!+6gB5*R-+'!+!+,7=+-+6.0MO¢D6.0?6.,.!",7E+$ ¡ H( 67-8 $ !($ 3$!+&
#*!+d\S:*/5(!+ &6. *0'-+9h&* *06.6N(=KL6N#d#*,.!+67B1*-+U!+,7=+-+;670 !+'(20!+,.,.*',7-+=A,.-8*O
v<!"UH![KT!"U-+9V!+3Ei !"6.6.#*!+,S !"#;M /S#*6gB5#*!+,.,.E!+&&(6.(=Z F3H 67-8n6.i(#*!+ -+9k,7-+=
,7-8$(-[KT*)+*O
…0!+3E™(H(!+,L(*zKT-+C…!"//5,.6.#*!"67-8$c(u!+ Cƒ6.2 -^/&6.#Xi!]#X-8o6N3H(-8H1 /S-8A)e!";6.!"\1,7+O
_(i/-+\1,70*$(GF3H1!"&_*-+a6N:HH!+,.,7E(r!"H(!+,h,7-8:9@H1#X67-8$k!+& ¡ H( 67-8 $ ¤ $ $
!+& A#*!+^\2*/1(!+ &^6.< *0?-+9R&(* *0?6.6.(=r(iXWI/S#X &>02!+>F3H1!"&<*-+:-+9R!A/&6.#X -+
-+-+9T!+]!+,7=+-+6710MO2¢(6.06.,.!";,7E+$ ¡ H 67-8 $ !(
$ 3$V!+&
#*!+<\S*/1!+ &d6.d *0-+9n&(* *0?6.6.(=
KL6N#r/&16.#X -+J-+R!+,7=+-+6710 !+J(0?!+,.,7*J02!+AFIH!"&r*-+O ¡ H 67-8
#*!+M\SU!+&&( &r\3E
#X-8 ;H#X6.(=?!i#X-8DB5&(1#X:6.3 *)"!+,%\1!+ &p-8p:(-+0?!+,%-+ &16. 67\5H(67-8<”b&(*/S&6.=2-8A(a67{*Z-+9
( U *;•;O ¡ H( 67-8 #*!+d\!+&&( &u\3E#X-8 H#X6N(=G!r#X-8DB&(#X6.o *)e!+,9@-+'aXWD/S#X &
&6gfS*#X+O`|'h0o67-8&i!"\S-[)++$(PR #cl #>/-eŠ #XT1!+c&(*)+,7-+/S&G!+!+,7ED6.`-+9)"!"6.!+1#XL #16.FIH(
9@-+ ¡ H( 6.-8 !+& 3Oh|//-+/;6.!" U !"6. 6.#*!+,m R9@-+R(:0!+,.,!+0/1,7FIH( 67-8:” ¤ $ !($ I$1!+1& •
!"Z6.,.,S(-+RKT,.,S !"\1,N6.(&O
PRa !"6N 6.#*!+, U9@-+*=+6.-80*(-I&1'0?!EuH(=+=+ KT![EI-+9T&(67=816.(=i!"6. 6.#*!+, 9@-+
(Z,.-+=?,7-8R#*!++OhPR6NT6NR!+A6.02/S-+!+3R!"!29@-+R9bH(H(!"#O
PV-2&(67=8G!+&G*)e!+,NH!" U !"6. 6.#*!+, $((LB5 n */M6.T -6N&(o6.9xE(U -8H(;#Xh-+9V)e!";6.!"67-8i!"
0aH J\S#X-8o -8,.,7&M\IEi!+#;p *Ocj(-+LZ#*!+ KTZ!"Z#X-816.&(*6N(=($D(*!"9@-8H(R6.0/-+!+oR -8H#X
-+9k)e!";6.!"67-8O
jk67*$V(*i6.U?!+&-80Ž)"!"6.!"67-8^6.<(G ,7#X67-8_-+9J(? a&!"!A1!"a6.ZH &] -u*)e!+,NH!" (,7!"6N(=i!+,7=+-+;670*O!+oEp/1!"6.#*H,N!"!+1&(-80,7E3yz&(![KLA U&!"!r **$-8a#*,.!+6gB1*0!Ep-8H(y
/S*9@-+0Y!+(-+(*h*)+?-8H(=8-8?(RKL(-8,.J/-+/5H,.!"67-8(zKT-:#*,.!+6gB1*QKh-8H,N&2/S*9x-+;0}6.&(36.#*!+,.,7E+O
PR6NJ6.J!2/1!"6N#*H,.!",7E?/6N(=:/1-+\1,709@-+L0!+,N,S &!"!? **O
PRZ #X-8&u -8H(#X-+9k!+1&(-80)"!"6.!"67-8AH,7R9x-80 (Z ,.#X67-8A-+9k( ;!+6.6.(=&1!"!DOJu!+oE
/1!"6N#*H,.!"'!+&(-80,.Eoyz&(;!KLp !+6.6.=? **$-8(2!+,7=+-+6.0q0?!Ep-8Hy‡/*9x-+0 !+(-+*'*)+d-8H(=8$S-8
(?![)+*!"=++$`(2“Kh-!+,7=+-+670?!)+?(?!+02i!+#*#*H(!+#XE+O Q)+_0!+,.,`#1!+(=+ -u(2 !+6.6N(=M *
”bH#;d!+U!+&&6.(=i-+U&(,7*6.(=G!G9x*K}&!"!G/S-86.3;•R0?!Eu#*!+H1 2,.!"=+#!+(=+U6Np(2#*,.!+6gB1*'/-D&H#X&
\IEu!M,7!"16.(=r!+,7=+-+670AO T6.0!+ƒ” !($ 8•U!+U#*!+,N,7&d6.'\S!)D67-+
6.1 !"\16.,.6.zE D$S!+1&<(2!+
(-tKL>!"26.a6.!d *67-8H1:/1-+\1,70ˆ9x-+2r&#*6.67-8^ *A!+,7=+-+6.0*$QH1#š!+ n| PŽ” T6N0!+$
j(6.&0!+$5,N($ ¢I -8+$ !3•;O
|q67&> -8H(;#XM-+9)"!"6.!+#XM#*!+™\Sr6.3 *!+,n!+&-80(6.>(A,7!";6.(=d!+,7=+-+6710M
O T-816.&(*$
9@-+hXW(!+02/1,7+$3('KL6.&(,7E3yzH &\5!+#CI/-+/1!"=8!"67-8r!+,7=+-+;670Y9@-+T !+6.6.=U9x*&Dy‡9@-+Kn!"&G(H(!+,5(*zKT-+CD*O
PR6N:!+,7=+-+;670 6.HH!+,.,.Ed6N676.!+,.6.{*&<KL67š!d *-+9R!+&-80ˆKT67=83KL6.#;^6.:(™6N02/-t)+*O<PR(
H1,76.(=Z,.!"(&G(*zKT-+Ci&*/1&h#X;676.#*!+,.,7E-8G('!+&(-80 !"6N(=a!" ?” U-8,7 €-8,.,N!+#C$ •;O
6.h#*!+ +$D*)+r679S(L !+6N6.(=&!"!a!"'(-+h#;!+(=+&$D('!+,7=+-+670‰6.c,N67C+,7E -Z/-I&H1#XL!Z&67f*3
3E3/S-+(6NJ679V67R6NJXWI#*H &p!"=8!+6.M9@-80!?&6gfS*oJ;!+&(-80 !"6N(=? !" +O
PRL,.!+ T -8H(#X-+9!+&(-80})"!"6.!"6.-8!"n0:H1 Q\SL!+&1,7&2\IE !"6. 6.#*!+,1 T6.Q!+1&(-80 #*,.!+6gBy
#*!"67-8r*-+Ok“9m!:BWD&G9x;!+#X67-8
-+9m(' R&!"!a/S-86.3h!"!+1&(-80,7E06N,.!"\S,7&$8G-,.!"6.(=
!+,7=+-+6.0 #*!+p!+#;67*)+:!+A*-+R!" Z-+9k,.n!+
O
| =+-I-I& !"6. 6N#*!+,R r(-8H,N&™(-+?\SM9@-3-8,7&…\3E™ p -8H(;#X?-+9U)"!"6.!"67-8%O„PRA i(-8H,.&
#X-8#*,.H1&(c!"(hzKT-!+,.=+-+670!"T&6gfS*o679!+&:-8,7EU679((6.m/S*#X3!"=+T-+95#X-+#Xk#*,.!+6gB#*!"67-8
KT-8H,.&A\SZ&16gf*o*$5-8p(!)+*;!"=++$KL(A ;!+6.(&A-8!i !+6.6.= *-+9Q!?=867)+pBWI&67{*a!+&A &
Õ
Õ
Õ
Õ
Õ
Õ
Õ JÕ
-8A!+,.,m&!"!2/S-86.3J6.r(U/S-+/1H,.!"6.-8O
PV-i!+#*#X-802/5,.6.p6.*$1!i !"6. 6N#*!+,m 6N(=?/-I#X&H0:H1 !+#*#X-8H3'9x-+'( a -8H(#X-+9k)e!"6N!"67-8O
PV-?!+#*#X-8H3L9@-+R yz&!"!?)e!"6N!"67-8A!+&M/S-867\16N,.67zE2-+9!+&-80#*,.!+67B5#*!"67-8r*-+[$( !"6.6.#*!+,
/-D#X&H(0:H#X-86.&(*Z(?67{*2-+9n(? : *:!+&<(?#X-81 F3H#XZ-+9n#1!+(=+:6N<(2 a **O
PV->!+#*#X-8H3r9x-+G !+6.16.(="yz&!"!_)e!"6N!"67-8…!+&„6.3 *!+,L!+&(-80*$J(d !"6. 6N#*!+,L/-I#X&Hp0aH XWD#*H( J(n,7!"6.=!+,7=+-+6.0€0aH,76./1,7Q6.0V!+&02!+H(T(h)"!"6.!"6.-8a6N:!+#*#*H(;!+#XEZ-+9(nH,76.(=
#*,.!+6gB1**O
PR16.'/1!"/S*U\*=86N'\3Ed&(#X6.\16.(=iB1)+ !"6.6.#*!+,` U\S!"6N(=r-8 ¡ H( 6.-8 ‘˜u#*œ0!"žŸZ *$
!i 9x-+(&6gfS*#X:-+9QzKT-r/-+/S-+67-8$1(a!+02/5,7& *$(a#X-8y‡)"!+,.6.&!" & *$!+&d!
(*K R#*!+,N,7&G( W ¤ #X)G *OTPR('/1!"/S*J(M&(#X6.\T!26.0aH,.!"6.-8i H1&(E!"R **CDJ -02!+H(
(2/-+\1!"\16.,.6.zEM!"U!+#;] UKL6N,.,V6N#X-+#X,7E&(* #XZ!M&67f*1#XaK(d(-A&6gfS*#XXW(6. ”bPhE3/S
L*-+’•;O2PR(H1,7'-+9h(26N0:H,N!"67-8d H&(E(-[K}1!"U-8,7Eu˜#*œ0!"žŸ *$m(2#X-8 y‡)e!+,.6N&!" &
*$%!+&u( W ¤ #X)p U!)+2!+#*#X*/!"\1,.aPhE3/SaJ*-+O'PR(aPcEI/SZR*-+-+9QZ!+0/1,7& 6.)+*Ep\5!+&u!+&(a U6.)+*EAXWD/S67)+#X-802/1H(!"6.-8!+,.,7E+$S -GKh2&(-r(-+#X-86.&(*67'9@H(*O'PR(
PhE3/SR`*-+h-+9('&6gfS*#XXy‡-+9xy‡/-+/S-+67-81` n6.cH1!+#*#X*/!"\1,7'6. -802#*!+ *$I\1H(T67c6.c)+*E#!"/
-*)"!+,.H!" +$1-KhZ*!+6.(&A6.J9x-+R9bH(*R H&(E+O
PR6.0:H1,.!"67-8d H&(Eu6.U -802*KL1!"6.&(!+,.67{*&d!+&<&-3U(-+!+&1&(U!+,.,k!+ /S#XU-+9T !+6.16.(="yz&!"!
)"!"6.!"67-8OTPm-i-+\!+6.u!02-+:!+,.6. 6N#*)e!+,.H1!"67-8M-+9`(9@-8H(0!+6.6N(= *$KhZ#X-81&H#X &p!i *L-+9
XWD/*;6.023H6.=M!+,Q,7!"16.(=A!+,7=+-+6.0U-8_!+,.6. 6N#2&!"!p *Oiv<G02!+H(&_\S-+<(iPhEI/2
*-+!+&p(:/S-tKh*'-+9`(Z *OPRZH1,7L-[K !":#X-8 y‡)e!+,.6N&!" & !+'#X-86. o,7E
,7*)"!" &PhE3/STm*-+[OPRn&16gf*#XXy‡-+9Ny‡/1-+/S-+67-8V Q!+`!+#*#X*/!"\5,7JzEI/STm*-+$8\5H(,.-[K™/S-[KT*O
T-+-+9Q(a0!+6.6N(=zKT-r U![)+a=+-I-D&PhEI/:R*-+!+&u!+ -8!"\5,7:/S-tKh*OPR( W ¤ #X)u U6.
,.6.=8o,7E02-+L/-tKh*9@H,!+i˜#*œ0!"žŸh *$D\1H(Q!+,. -: 6.0c02-+XWI/S67)+R -Z/S*9@-+0MOks#X+$
(2/1!"/S*Z#X-81#*,.H&(!"Z( W ¤ #X)< Z6.(2 :-+9T#;(-86.#X9@-+Z6.XWI/S6.)+a,7!";6.(=M!+,7=+-+6.0*$
\1H(Z!":˜#*œL0?!"žŸa a6NU\S* *:9@-+:02-+?XWD/167)+?!+,7=+-+;670*Orlm6N /_#X-D&(i6.0/1,70236.(=M\S-+
6.R!)"!+6.,.!"\5,79x-80(Z!+H((-+[O
;J%
v<KL6N,.,5!+H02U!"J*UXWD6N R!2 *ˆ-+9V/S-867\1,7'&!"!2/S-86.3*$(#*!+,.,7&r( !#"OJPR*U!+,. XW(6. a -802$
%#& ')(*,+"-. !#"
$ /š1!"a#*,.!+6gB1Z!+#10324 6.3 --8(?-+9L~ #*,.!+ *Ouv„6.(-8H(:,.-8Z-+9
=+(*!+,N67zE+$KT2KL6.,.,V!+H0:!"6
~ 5 ¤ $%!+,7(-8H(=8d(-8(-+9h(H,7'6.6.'/1!"/S*&(*/S&u-8d6.
!+H102/67-8$36.#XR-8Hh-8,.E#X-81#X*?K6.,.,D\SRKL(**h!+iXWD!+02/5,7L6.h#*,.!+6gB5&#X-+#X,7E2-+n6.#X-+#X,7E+O
2!+2!"//1,.6N#*!"67-8:* 6.(=($o!U!+0/1,7 6.`&(![KLa!+&(-80,.E9x-87
0 —!+#*#X-+&6.=' -!'B1WI&a/-+\5!"\16.,.67“E
&6. 67\1H(6.-89
8pO'|Y#X-8,.,7#X67-8u-+:
9 &;#<"=>")'?(A@B#CDE(F6NL#X-8 ;H#X &\3Ep,N!"\,N6.(=2!+#$
0G2 !+#*#X-+&16.(=
9
- /`<” 0S•;O c!+#;p ;!+6.6.(=?XWD!+02/5,7(**9@-+!+RZ9@-+I
0 H<0KJA/`<” 0S;• L;ORp-802a!"/1/1,.6.#*!"67-81*$((*a0!E
\S!2 -8H(#XZ-+9k#*,N!+6gB5#*!"6.-8M(-86. U!"L!+&(-80,.E? *RZ,.!"\S, -i!+A6.1#X-+#XJ)"!+,.H(+O
|},7!";6.(=!+,7=+-+;67
0 †!"C+!+'6.(/1HR!? *'-+9` !+6.6N(=2XWD!+0/1,7:
M !+&p-8H /1H(L!i#*,.!+67B1*O
/N O
N KL6.,N,`06.#*,.!+679@Eu!+]XW(!+02/1,.i&(![KL
PR(? H(?*-+Z!" i-+9J!":#*,N!+6gB1*a6.U(i/-+\1!"\56.,.67“Ep!" /š
!+&-80,7EG9x-8P
0 !+#*#X-+;&6.(= Q
- 8pOh‚p/!+#X6.#X+$56.J*-+R!" a6.J 6.0?!" &p\IEr!"CI6.=-8H(![)e!+6N,.!"\1,7
N -81
!+02/5,7 †!+&™H(\&6.)I6.&16.(=A676.3 -]!d !+6.16.(=< *R
M !+&™!< *T
SZO™PR(M*-+2!" A-+9 /
S
N -8AU/-+/5H,.!"67-8)$ _O
/-t)I6N&(J!+A 6N0!" -+9( H(U*-+R!" -+9 /u
PRZ3H1,.,%3E3/S-+(6NJ -i\S &u6.L!"L9@-+!?!+&(-80?,7EM&(!KM !+6.6N(= *U
M}-+9kBWD&p67{*+$5(
zKT-r,7!"6.=!+,.=+-+670RKL6.,.,![)+Z(a!+02:*-+R!" :-8u! LXW(!+02/1,7Z!+&(-80,.Eoyz&(;!KLM9@-8V
0 ^$
KL(*Z!+,.,%;!+&(-80—&(!KL!"a0!+&(:!+#*#X-+;&6.(=i -G&6N 67\1H67-8W
8pOLlm*3
/ N ^\SZ(a#*,.!+6gB5*R-8H( /1H(\3E
!+,7=+-+6.
0 !+6.&A-8u !+6.6.= *D
M$!+&p,7* /N ™\S(a#*,.!+6gB5*R-8H( /1H(\3EM!+,7=+-+;67
0 $ ;!+6.(&
-8MOcPL(r(ZIH,.,S3E3/S-+(6NJ#*!+A\SKR67 A!+
/ N T”<0S•5
/`”<0S•
5 /N R”<0•53/`”<0S•
%J
KL(*U((-+!"67-8 6N&6.#*!" n(Z/-+\1!"\16N,.67zE?!"C+AKL67r /S#XJ -(!+1&(-80
!+6.16.(=2 *:
M}!+&r RXW(!+02/1,7 0VO
&(![KLJ-+9k(
]D
U TSA v<(-[K„&(#X67\SJ(' !"6. 6N#*!+, h!"h!"L(L0?!+6.?H\DŠ#Xc-+96.`/1!"/S*Ov<'\*=86N2KL676.02/1,7
(-8,.&-8H(n L!+&A(A#X-86N&(*R L\1!+ &M-8M!+02/1,.6N(=:9@-80 Z!)"!+6.,.!"\1,.&!"!DO
!#"%$'&(*),+-#)+
Õ
PV-!"/1/1,7E˜#*œL0!"[žŸJ ” Q)+*6. *$ +•;$KhU&6.)I6.&L-8H(J!)"!+6.,.!"\5,7'!+02/1,7-+9V&!"!T$(6.3 -2!: ;!+6.6.(=
*U
M‰!+1&u!i U *
SZOLv< !+6Np\S-+d!+,7=+-+6710 Y!+1&$ -8u: !+6N6.(=?*ED67,.&6N(=?#*,.!+6gB1*
/ N >!+& /N TOnv<a(A '( :#*,.!+6gB5*J-8u(Z ' **ORj(-+'!+#XW(!+02/1,7 0$2 SZ$KTZ#X-+;&u(-tK
67JKn!+L#*,.!+6gB1&r!+&A#X-81 H#XR9x-8,N,7-[KL6N(=a#X-836.(=+1#XEG!"\5,7+‘
3 H10Z\S*J-+9kXWD!+0/1,7
0?6.#*,.!+6gB1&G\3EG\S-+ / N !+& / N 3H10Z\S*J-+9kXWD!+0/1,7
06.#*,.!+6gB5&i\IE /N š\1H(J(-+R\IE / N 3 H10Z\S*J-+9kXWD!+0/1,7
06.#*,.!+6gB5&?\IE / N \1H(J(-+L\3E / N 3H10Z\S*J-+9kXWD!+0/1,7
06.#*,N!+6gB1&G\IEG67(* / N ]-+ /N TO
v<:KL6.,.,H U(:(-+!"67-8
./0/ . /21
.1/ . 101
KL(* . 5 . /0/43 . /2153 . 1/63 . 101 6.J(U -+!+,IH0Z\S*J-+9XWD!+0/1,7R6.r L *SZO
&(*V(hIH,.,+3E3/S-+(6.*$e(c“Kh-!+,7=+-+6710%(-8H1,.&![)+h(T!+02h*-+m;!" +$eK6.#:02!+m!"
'
! 759' J9x-+J=+-I-D&(y‡-+9xy‹B1n!"R#X-802/5!"n(&6. 67\1H(6.-8
./21 5 .1/ OQ˜#*œL0!"[žŸJ R6.n\1!+&r-8M8
-+9h#X-8H1o'XWD/#X &<H1&(*'(aIH,.,3E3/S-+(6NL -r(a-+\1 *)+&<#X-8H1o*OPL(:XWD/S#X &<#X-8H3'H1&(*
(ZIH,.,S3E3/S-+(6.J!"
/2153 . 1/ •0: ¤
.101
PR9x-8,N,7-[KL6N(=? !"6.6.#6.&6. 6.\1H( &]”b!"//-W(6.0!" ,7E•J!+;7<9UK67 Õ &*=+*:-+9`9@*&(-80Aw567L6N#X-+ y
/S-+!" ! #X-836.IH67zEd#X-+#X6.-8 M *0 ”@-+9>= Õ 6.d(IH02*;!" -+’• -A!+#*#X-8H39x-+Z(29@!+#XU1!"(
!"6.6.#U6.R&6N#X* UKL6N,7?7 9 &6N 67\1H67-8?6.L#X-8o6.IH(-8H‘
”@ . /21 = . 1/ @= Õ • 9
./21 3 .1/
A
‚9(IH,.,(3EI/-+6.c6Nc#X-+#X*$D(?(L/1-+\1!"\16.,N67zEa!"c16.cFIH!+367zE6.c=+!" *n!+B7 9 1 /2C D0E 5
! Õ ! 6NT,.T!+ O On¢I-KT0?!E?‚Š#XJ(UIH,.,oEI/S-+(6.T6.i9@![)+-+R-+9m3EI/-+6.h!"J(
zKTA -i!+,.=+-+670J1!)+:&16gf*on/S*9@-+0!+#XUKLr !+6.(&M-8A(/1!"6N#*H,.!"n !+6.6N(=2 *:MO
” ./21
. /0/
3 .1/ •0:
”.
¤
œ -+ M(-[KT*)+*$`!"a6.Z a1!+Z“Kh-d(-+#X-80?6.(=8:KL67]*=8!"&_ - ¡ H(67-8 Opjk67; *$`67Z&(-I
(-+R&6.#X,7Ei0!+H(U)"!"6.!"\16N,.67zE&1H( -(#-86.#X-+9(U !+6.6N(=a*J-+R(6.3 *!+,S!+1&(-80(n-+9
(,7!";6.(=M!+,7=+-+6.0MO2|‰6.(=8,7 !+6.16.(=r * M6.U#-8 $V!+1&<(!+,7=+-+6.0U!"-81,7Eu#X-802/5!"&
H6N(=2!"' !+6.6.= **OLs#X+$˜#*œ0!"žŸ' (-8H,.&A-8,7Er\S:!"//1,N67&M679kKha\S,.67*)+ : -8H#X
-+9%)"!"6.!"\16N,.67zE2!"U0?!+,.,‹Oc¢I#X-8&$D67n&(-IT-+T&6.#X,7E#X-802/1!"'/S*9@-+0!+#X'-+9%(U!+,7=+-+;670h-8
!+6.16.(= *'-+9Q67{*,
@ @ $\1H(R-8,.EM-8u *-+9Q67{*,
@ M @ $5K6.#u0:H1 L\SZH\1 !+36.!+,.,7Er0!+,.,.*L!+ @ @
-H(!H i#*673,7Ei,.!"=+U **OhsL#X+$1KhZ0aH !+H02U1!"R(U,.!"6.)+Z&6gfS*#XU-+\1*)+&
-8M !+6N6.(=2 *L-+96.{*B
@ M @IKL6.,.,S 6N,.,S(-8,.&r9@-+R !+6.6N(=2 *R-+9k67{*B
@ @O
+-#)+ & + 4
&
?+
&
& + 6)
|
#X-8&„6.0/1,7A !"6.6.#*!+,R r6.?\1!+ &ƒ-8„02!+H6.(=]u&6gfS*#Xp\*“Kh*„(u*-+i!" u-+9
!+,7=+-+6.0 !+& (d*-+A!" <-+9!+,7=+-+6.0 $ ”‡¢((&(#X-+ T-D#!+$ •;O—¢I/S#*6gB5#*!+,.,.E+$L,7*
$5}” ./0/ 3
./21 •0: . \SL('/-+/S-+6.-82-+9% TXWD!+0/1,7T6.#X-+#X,.E#*,.!+6gB5&\IE!+,7=+-+670 !+&G,7*
15 ” ./0/ 3 .1/ 0• : . \SZ/-+/S-+6.-8M-+9` 'XWD!+02/5,7L6.1#X-+#X,7EM#*,.!+6gB1&M\IEA!+,7=+-+6710$GOJPR(
!+H102/67-8H&(*;,7EI6N(=26. !"6.6.#*!+, 6.!"KL(d!+,7=+-+6710 ‰#*,.!+6gB5!+dXW(!+02/1,7 0<9@-80
(U R* SZ$D(U/-+\1!"\56.,.67“E2-+9V06.#*,.!+6gB#*!"67-8i6N QOQs#X+$(UIH0:\*n-+9k06.#*,N!+6gB5#*!"6.-8c-+9
. RXW(!+02/1,.R6.J!2\16N(-806.!+,!+&-80 )e!";6.!"\1,7KL67M0!+ . !+1&M)e!";6.!+#X ” = • . O
PRi\16N(-806.!+,h&6. 6.\1H(67-8]#*!+š\SrKT,.,gyz!"//-W(6.0!" &^\3E>!-+0!+,n&6. ;67\1H(67-8]9x-+!+ -81!"\1,7
)"!+,.H(a-+9 . O^jH((*;02-++$Q(M&67f*1#XG\S*“Kh*š“Kh- <"B( ( " B( " J-+0!+,.,7E^&6. 6.\1H( &];!+&(-80
)"!"6.!"\1,7:6.Z6. ,79J(-+0!+,N,7E&16. 67\5H( &OrsL#X+$`iFIH!+o6.zE = #*!+^\S?)D67*KT&_!+(-+0!+,N,7E
&6. 67\1H( &?679%KT!+H02'!"n(02!+H(&G*-+T;!" <!+& ^!"U6.&(*/S&(3*
O &*h(UIH,.,
3E3/S-+(6N*$D6.JKL6N,.,51!)+:!?02!+M-+9k{**-?!+&M!? !+&!"&A*-+R-+9
Õ
Õ
5
!#"
¤ ”
$
Õ
.
=
•
KL(* 5}” 3 Q•0: ¤ 6Nn(Z!)+*;!"=+:-+9k(U“Kh-i*-+R/-+\1!"\16.,.6.67*O
j(-806.J!+1!+,7EI6.*$DKhZ-+\!+6.M(: !"6. 6.#
5
%
= ¤ ”
&
Õ
= 0• : .
KL6N#_1!+G”b!"//1-W(6.0!" ,7E•U! !+&1!"&_(-+;0!+,h&16. 67\5H(67-8Oiv<r#*!+^‚Š #X:GIH,.,Q3EI/-+6.Z6.9
@ % @ ')( /2C D+* E 5 <”@9@-+! ¤ yz6N&(&p 'K67p/1-+\1!"\16.,N67zEr-+9c6.1#X-+#X,7EA‚Š#X6.=i(IH,.,m3EI/-+6.
A
-+9 O •;O
PR16.R !+\S*pH &u\IEA0!+oEM !"#*6.#*,.H&16.(=(:!+H(-+” 67* *;6.#$s'6.,.&$ n!"CD676‹$
•;OsL-tKh*)+*[$J(*p!"u *)+*!+,R/1-+\1,70K67š6.2 *Ojk67 *$T\S#*!+H !+& }!"A!+#;
02!+H&-8:(h!+02c *K
SZ$t*E!"h(-+6.&(*/S&o*O¢D#X-8&$e(c k!"%(T&(!KL\1!+#CDm-+9
˜#*œL0!"[žŸQ *‘`67Q&(-I`(-+Q02!+H(J)"!"6.!"67-82&HJ -U(R#(-86N#XR-+95 ;!+6.6.(=U *`-+h6.o *;!+,D)e!";6.!"67-8
-+9T(?,7!"16.(=M!+,7=+-+6710 !+&<67&(-I(-+Z&167#X,7E02!+H(22/*9x-+0?!+#X2-+9h?!+,7=+-+6710-8
!+6.16.(=2 *J-+9`67{*B
@ @ $\1H(J!"*R-8M(Z0!+,.,7*J !+6N6.(=2 *J-+9`67{*B
@M @O
PRc,.!+#CU-+9(6.1&(*/S&(#Xk-+9 G!+& p#*!+Z\SQ#X-+#X &:\IEU#!+=86.(=R(h 6.0!" h-+9D(h !+1&!"&
*-+L -\S
Õ
Õ
5
!,".- 0/
. /21<3 . 1 /
.9 A
PR6Nu6.0!" ™9@-I#*Hd-8†(™/-+\1!"\56.,.67“E-+9i&6.!"=+*023<-+9?(™“Kh- !+,.=+-+670™”‡¢D&(#X-+
T-D#(;!+$ •;OTPL(UH,.6.(= % !"6.6.#U#*!+A\SUKR6. M!+
Õ
% -
5
@ ./21 = .1/ @=
. /21<3 . 1/
1
ÕJ
KL6N#rKTZ#*!+A#X-+=8167{*Z!+J(:F3H1!"U-I-+R-+9V?7 9 !"6N 6.#U6.M˜#*œL0?!"žŸL *O
6./1!"/S*$5KT2!)+XWD/S*6.023!+,.,7EM!+!+,7EI{*&d(H#X-+#X & % !"6. 6.#"$6.1#X:6N !"6N 6.#
6.R6NA#*H(3LHZ!+&MKTZKn!+o &u -?&(* *;06.(Z-[K \5!+&,7EG(i”b6.#X-+#X;•J6.1&(*/S&(#X!+H0/67-8
!efS#XL(!+#*#*H!+#XEM-+9V *O
j(-+0!+,N,Q!+02/1,.?67{**$(*G!"?XW(!+#Xa)+*67-8Z-+9n\-+^˜u#*œ0!"žŸa a!+&]? :9x-+a(
&6gfS*#XU-+9zKT-?/1-+/S-+67-8T!"L![)+-86.&A?
7 9 !+&M-+0!+,%!"//1-W(6.0!"67-81*O
4 & #)$" )
$ & 8+-#)+
PR(h(XWI !"6. 6N#*!+,o VKhT#X-86.&*V6.V#*H(3,7E'(h02-8m/S-+/1H,.!"V6.U(T0!+#6N(c,7!"16.(=L,.6. *!"H(+O
|…*67k-+9`”bHH!+,.,.E(• 6.!+,.k6.k#X-8&1H#X &Ok‚a!+#; 6.!+,‡$eR!)"!+6.,.!"\1,7J!+0/1,7 6.k!+&(-80?,7EZ&67)D6.&(&
6.3 -u!u !+6.6N(=A *
M-+9R!p/#*67B1&_6.{*”@+O›=(O7$Q“E3/16N#*!+,.,7EdzKT-u167&U-+9L(i&!"!o•Z!+&>!A * SZO
l%!";6.(=G!+,7=+-+670? Y!+& $!"a\S-+p ;!+6.(&u-8 M‰!+&p:H,76.(=i#*,.!+6gB5*L!"a &-8 SZO
l%* ”@/#X6.)+,7E+$ •R\SZ(a-+\1 *)+&u/-+/S-+6.-8A-+9` XW(!+02/1,7'06.#*,N!+6gB1&A\3EA!+,7=+-+;670
”@ /S#X67)+,7E $2•T&1H(6.(=: 6.!+, Oc“9VKh!+H02U!"J( &6gfS*#X 5 = KT*&(![KL
6.&*/1&(o,.EM9x-80 !M-+0!+,`&6. 67\1H(6.-8$5dKh#*!+^!"//1,7E¢DH&(3*žŸ *$V\3Ed#X-802/1H6.(=r(
!"6.6.#
1 .
J
1 !
5
/
Õ
KL(* 5 1 ! 1 O'&(*a(r3H,N,c3E3/S-+(6N*$k6.a !"6. 6.#r!+!
&6. 67\1H(6.-8<KL67 . =
&(*=+*L-+9V9@! *&(-8" 0AOcj(-+ 6.!+,.*$(3H,N,oEI/S-+(6.J#*!+M\SU‚Š #X &p6.9 @ @ ' 9 D /2C D+* E 5 ¤ ! +¤ O
PR*U!"Z0!+3Ei/S-+ 36.!+,&(![KR\1!+#CIJ-+96.J!"/1/-8!+#%Ocjk67 *$(!+J-+\1*)+&A!"\S-[)++$5(6.A &6.)I6.&1H!+,
&6gfS*#X KL6.,.,'(-+M![)+]!™(-+0!+,&6. 67\1H(6.-8$n\#*!+H1 !+& !"](-+M6.&*/1&(o*O
¢I#X-8&%$(( žŸR!"Z(-+R6N&(*/S&(3*$3\S#*!+HU(U L *L6.r(U 6.!+,Nn-[)+*,.!"/š”b!+&r( ;!+6.6.(=
*r6.( 6.!+,.?-[)+*,.!"/!+GKT,.,x•;O v<dKL6N,.,R *d\S,7-[K—!"G( )I6.-8,.!"67-8i-+9(d!+H0/67-8
H&*,7ED6.(=Z L#*!+H Z*)+*Z/-+\1,70T!"L0?!"C+Z6.n LH!"9@U -?H+O
#
4%$'& ( & <)))&+* $, 4$ +
6$ & -8+-#)+
0
PR6NR 6.L6N&(o6N#*!+,m -G(Z/1*)I6.-8HR-8(aXW(#X*/1!"'6.1 !+&p-+9Q#X-81 H#X6.=?!+#d/5!+67L-+9Q ;!+6.6.(=
!+&d *U\3E!+&-80,7Ep&6.)I6.&16.(= T$mKh6.1 !+&d!+&(-80?,7Ep&67)D6.&( 46.3 -/.d&6.@Š -86.3' *-+9cFIH!+,
67{*+$ S 1 $VO*OO*$=S,03OJv<:(u#X-8&H#X1.M 6N!+,.*OT‚p!+# 6.!+,‹$1(Z *'6. S $S!+&p(Z !+6N6.(= *'6.
(ZH167-8G-+9`!+,.,S-+9V-+(*
S'2#J43657OQPL(!+02 !"6.6.#6.J#X-802/5H( &O PRL!+&()"!+o!"=+-+96.c!"//-8!+#;i6.c!"c!+#;G T*h6.h6.&(*/S&(3k-+9(L-+**O`sL-tKh*)+*[$(6.
R 6N,.,1H(f*T9@-80(/-+\5,70 1!"T' ;!+6.6.(=a *J-t)+*,.!"/%O`r! y‡9x-8,.&M#X-8y‡)"!+,.6.&!"67-8%$3!+#;
/1!+67`-+9S !+6N6.(=U *T!" -+9S(RXW(!+02/1,7OkPL6.k-[)+*;,.!"/i0![E/*)+3c6.`!"6. 6.#*!+,1 Q9@-80
-+\!+6.16.(=i!r=+-I-D&u6.0!" 2-+9c!+0-8Ho-+9h)e!"6N!"67-8!"Kh-8H,N&u\Sa-+\1 *)+&<6.9`!+#< !+6N6.(=G *
KT*Z#X-802/1,.* ,7EG6N&(*/S&(3h-+9k/*)I67-8H1h ;!+6.6.(=2 *O
PV->6N,.,.H ;!" p6.?/S-86.o*$#X-86.&(*i(<(!" r(67=83\S-+i!+,7=+-+;670MOY¢DH(//S-8 p-8H(i ;!+6.6.(=^ *
#X-83!+6.nzKT-2#*,.H *;h-+9V/S-86.o‘!,.!"=+U#*,.H1 *h\S,7-8=86.(=Z --8(U#*,.!+T!+1&G!0?!+,.,5#*,NH *n\S,7-8(=86.(=
-p?-+(*#*,.!+*OM‚9nKhi/S*9x-+;0 ! ¤ y‡9@-8,.&^#X-8y‡)"!+,.6.&!"6.-8$VKTi0aH aH(\&67)D6.&(a6. !+6.6.=A&!"!
6.3 -zKT-r&6.@Š -86.3R **On“9`!+,.,%-+9Z/S-86.3R6.M(:0!+,.,7*#*,.H *=+-i6.3 --8(:-+9`(-8 “Kh-r **$5(
\S-+GHT-+9m!" J(67=83\S-+n!+,.=+-+670 KL6.,N,1![)+U,7*)e!" &A*-+n!" *$(\S#*!+H KLi0!+,.,
#*,.H *Z6.U6.<2 : **$V*)+*Ed/S-86.o6N<67UKL6.,.,\S06.#*,.!+6gB5&O:v4(_(0!+,.,`#*,NH *Z6.Z6.d(
!+6.16.(=:**$ -802-+9V67T/S-86.3n0!Ed”b6.#X-+#X,.E(•c\S !" &A!+J(!" J(6.=8o\S-+h-+9m R*T/S-86.3*$
KL6N#!+,. -M6.#X!+a*-+!" +"
O T-83)+* ,.E+$%679`(20!+,.,#*,.H *6N*)+1,7Ep&67)D6.&(&p\S*zKT*<(
zKT-:**$o((R*-+c!" QKL6N,.,D6.02/-t)++$o\S#*!+H J9@-+c!+#; h/S-86.ok(*RKL6.,.,I\SR!#X-+ /S-8&6.(=
Õ
( !"\IEA !+6N6.(=i/-86No!"KL6.,.,/1-[)D6.&(a(#X-+#X#*,.!+6gB5#*!"67-8O#c6.(*Kn!E+$mKh#*!+< *!"(
/S*9@-+0!+#XU-+9(“Kh-i9x-8,.&1n-+9V(:#X-8y‡)"!+,.6.&!"6.-8rKL6.,.,\SU#X-+,.!" &p!"(*R!+u6.&(*/S&o*O
v<U)+*67B1&G6.hXWI/S*6.0o!+,.,.E29x-+ y‡9@-8,.&M#X-8 y‡)e!+,.6N&!"67-8i-8r(Ul%* *R#X-+=816767-8r!+ Cd” -+!+,Q ;!+6.6.(=rXW(!+02/1,7’•6.<!+]XWD/S*6.023KL(*2?3H1,.,`oEI/S-+(6.Kn!+ ;H(p”b&(#X6.\&d\S,7-tK'•;O
v<202!+H(&p(#X-+,N!"67-8d#X-3 i#*6.o\S*zKT*(a&16gf*#X6Np*-+!" -8“Kh-r9@-8,.&RK676.
!#X-8y‡)"!+,.6.&!"6.-8$ !+& 2 OuPL(?-+\1*)+&>)e!+,.H?Kn!+ O $hKL6.#;_!+#*#X-+&16.(= -!
26.
1
/
67=816gB5#*!+3,7E&6gfS*3h9@-80 KL67 OhUi'-+*R!+&$(679%'*-+n!" !+1& 2 !"
&(![KL9x-80—6N&(*/S&(3 y‡9@-8,.&<#X-8y‡)"!+,.6.&1!"67-8”b6‹O›+O7$%-8d6.&(*/S&o'&!"!r *;•;$(2#X-+,N!"67-8
#X-I i#*673L6.
= ! JKL6N#M!+#*#X-+&6N(=2 -?! 6NJ(-+L67=816gB5#*!+3,7E?&6gfS*3J9x-80 {**-(O
Õ
4
Õ
A
*
)
Õ
Õ
$ & - +-#)+
2 -802L6.1676.!+,3XWI/S*6.0okKL6.( .3y‡9x-8,N&#X-8y‡)"!+,.6.&1!" &/1!+67& *$oKTL!" 02/1 & -Z&(* *;06.(
KL3E( !"6. 6N#JKT!+c -3-a,.!"=+L6N -802L#*!+*OQPR(LIH02*;!" -+c-+9S( !"6.6.#J 6.0?!" c(02!+
&6gfS*#Xd6.„(d/S*9@-+0!+#X-+9Z(d“Kh-ƒ!+,7=+-+;670d”@-[)+*u(.…9@-8,.&’•;$RKL6.,.p(<&((-80?6.!" -+
6.0?!" n()"!"6.!+1#X'-+9m( U&6gfS*#X*Okv467r EDo(*6N#&!"!D$(KTU#X-8 H1#X & .G(-8(y‡-[)+*,.!"/1/16.(=
!+6.16.(=:*J!+&r02!+H(&G(02!+M!+&G)"!"6.!+#X'-8r(-8 !+6N6.(=a **O`v<U9@-8H&i!"nKL6.,7(
)"!"6.!+#XQKn!+m,.67=83,7EH1&(* 6.0?!" &UKL(UQ !+6.6N(=J *%-t)+*,.!"//S&$e(c02!+1Kh*c-D#*#*!+67-8!+,.,.E
)+*Er/S-3-+;,7E? 6N0!" &$1!+&A6.JKn!+R(#*!+H1 -+9k(Z,.!"=+ )"!+,.H(OcPR(U/-+\1,70 #*!+A\SU !+#X&p (Z#X-+,.!"67-8n\S*“Kh*p(Z&6gfS*3J9x-8,N&J!+L&(#X67\S&r!"\-t)++O
v<U9@-8H&G!"R679mKh*/1,N!+#X&r(U3H0*!" -+J-+9m( !"6. 6.#'KL6.i'-+\5 *)+&M&6gfS*#X'9@-80
! F>" ' (9@-8,.&<-+9n( .3y‡9@-8,.&_#X-8y‡)e!+,N6.&!"67-8$m(? !"6.6.#2\S#*!+02KT,.,gy‡\S![)+&O?PR6NU,7&]HU -u(
$
W ¤ #X)M/1!+67& *O
Z16.% *$+KTc/S*9@-+0 */5,.6.#*!"67-8%-+9 ¤ y‡9x-8,.&a#X-8 y‡)e!+,.6N&!"67-8OV‚Z!+#*/1,N6.#*!"67-8$t(T![)e!+6N,.!"\1,7
&!"!!"G!+&(-80,.E/5!"6767-8(&^6.3 -pzKT-dF3H1!+,gyz67{*&^ *
1 !+
& 9 O h!+#š,7!"6N(=p!+,7=+-+6.0 ” -+
$•J6. !+6.(&u-8!+#< *!+&u &d-8(a-+(*U **OUPR6.L/-D&H#XR9@-8H('*-+ 6.0?!" *‘ 1 !+& ( 1 ”@ !+6.(&™-8 1 !+&™ &…-8 9 •!+& 9 !+1& 9 ”@ !+6.&™-8 9 !+1&™ &ƒ-8 1 •;O
¢DH(\1 !+#X6.(=2#X-+/-81&6.(=*-+J6.0!" R=86.)+LHJ“Kh-i 6.0!" &p&6gfS*#X 1 5 1 = 1 !+&
9
9
( zKT-a&67f*1#X*$8(R6.0!" &i)e!"6N!+#XL6. ! 9 5}” 1 = • 9 3 ” 9 = • 9 $
9 5
= O`j(-80
KL(* 5” 1 3 9 0• : ¤ OUl%* ! 9 \:(a)e!"6N!+#Xa#X-80/1H( &u9x-80q( ‹y‡*/1,.6N#*!"67-8$5!+1&u,7* 1 1 \SU( 1 9@-80 (U)+*EiB5 J-+9 (UB1)+U*/5,.6.#*!"67-8OkPL(A&(XB5'9x-8,N,7-[KL6N(=2 !"6. 6.#
5
1 1 1E E 1 ! 9 J
" KL6N#UKTcKL6N,.,+#*!+,.,+( W ¤ #X) !"6N 6.#"Ov<T#*,.!+6N0!"VH1&(*%(h3H,N,+oEI/S-+(6.*$ !+V!"/1/-W(6.0!" ,.E
! &6. ;67\1H(67-8iKL67 &*=+*R-+9k9@*&(-80MOQPR!"=8H023R=+-IL!+R9@-8,.,7-tKL*O
lm* 4\SR!Z !+1&!"&?-+0!+,(!+1&(-80Y)"!"6.!"\1,.R!+& $†\SR!?7 9 !+&-80})e!";6.!"\1,7RK67 . = &(*=+*
-+9k9x*&(-80MOQPR(r\IEM&(XB56.67-8$I(ZFIH!+o6.zE
Õ
&
Õ
Õ
Օ
$ :D” . =
” •
!+R! &6N 67\1H67-8?K67 . =
&(*=+*L-+9k9x*&-80679 †!+& $!":6.&(*/S&(3*O
PRGHH!+, !"6. 6.#r6.a&(*6.)+&^\IE_ !"6.(=KL67>!*a-+9!+&(-80ˆ)"!"6.!"\1,7 1 J
1!)D6.(=
A 02! !+^!+&
!A(-+;0!+,Q&6. 67\1H(6.-8pKL6.]02!+…!+1&<)e!";6.!+#X
9"O?lm* 5 1 \S2(?!+02/1A A ,7
9 5 <” = ]• 9 \SU(H0-+9FIH!"&p&(*)I6N!"67-8n9@-80 (Z0!!+OQ PR( A&XB5(
$
5
5
1
.
=
”
9: 9
Õ
V•0:
A
v<,.,gy‡CD(-tKL<H1,79x-80 /-+\1!"\56.,.67“EA(*-+Ed !" ?1!" ‰1!+!A !+&1!"&](-+0?!+,`&6. 6.\1H(67-8!+&
$ !+R! 7<9'&6. 67\1H(6.-8KL67 . = &(*=+*R-+9V9@*&(-80MOQ| 02-+U0!"C"!"\1,7H,.h9@-80/-+\5!"\16.,.67“E
(*-+Ei6.T!"
!+& $Y!"U!+,. -6.&*/1&(o*$3/-t)I6.&&?('-+67=86N!+, žŸTKh*&!KLG9@-80 !a-+0!+,
&6. 67\1H(6.-8OVs#X+$1KT#*!+A/5,.H(=(06.3 - hF3H!"6.-8]” •J!+J9@-8,.,7-tKL*‘ Õ
Õ
5
$ D: ” . =
&
.
1
5
9
&
1
5
&
.
”
:D” ”
9 D: ” .
Օ
=
V•0:
Օ•
=
9 ”. =
V•
=
ՕA
PR6Nn=867)+J(ZH1H!+,&(XB516767-8G-+9V !"6. 6.#KL 5 O
v<<#*!+#X-8 ;H#X \3E…!+1!+,7-+=+E!+G9@-8,.,7-[K*O '&(*r(dIH,.,L3EI/-+6.*$J(dIH02*!" -+M-+9 $
1
1 $n6.2(p&16gf*#XM-+9zKT-^6.&(36.#*!+,.,.E^&6. 67\1H( &^/-+/S-+6.-8*$h-]Khu#*!+…!"9@,7E> !"i67!+i!+
!"//-W(6.0!" ,7Ep-+0!+,V!+&(-80—)"!"6.!"\1,7:KL67u{**-M02!+<!+&dH(CD(-tKLu!+&!"&&(*)D6.!"67-8 ^6.9Q(
H&*,7ED6.(=Z *R#X-83!+6.(&u!"L,7!+ /-86No*OQs#X+$1,.* 5 1 1 :
kO
|',. -ZH1&(*Q('3H1,.,(oEI/S-+(6.$ ! 9 :
9!+c! 7<9R&6N 67\1H67-8ZK67 &*=+*-+9S9@*&(-80 679Kh'0!"C+
(U!+&&6.67-8!+,!+H02/167-8G!" 1 !+ & 9 !"6.&(*/S&(3*OkPR6.T!+H0/67-8G6.n9@!+,N +$(!+nKh![)+
*M!"\S-[)++$(\S#*!+H('“Kh-&16gf *#Xy‡ -+9xy‡/-+/S-+67-8h!"U02!+H(&i-8G(-+//S-86. 9@-8,.&h-+9m! ¤ y
9@-8,.&?#X-8y‡)e!+,N6.&!"67-8Oc¢I6N,.,‹$+('!+H02/167-8-+9%6.1&(*/S&(#XR6.c/-+\1!"\5,7E202-+!"/1/-+/6N!" R9x-+ ¤ y‡9@-8,.&
#X-8 y‡)e!+,.6N&!"67-8d!+d9@-+ y‡9@-8,.&]#X-8 y‡)e!+,.6N&!"67-8$\S#*!+H 6Nd( ¤ y‡9x-8,N&]#*!+ +$m2 !+6.6.=G *:!"
#X-802/1,.* ,7EG-8Dy‡-[)+*,N!"//16.(=A”b!+&%$1!+R!+,7Kn!EDL6.M#X-8y‡)"!+,.6.&!"67-8%$((U *R!"Z(-8(y‡-[)+*,.!"/1/16.(=3•;O
”@v<Z#;(-8 ¤ y‡9@-8,.&M#X-8 y‡)e!+,.6N&!"67-8G\S#*!+H U67n=867)+R,.!"=+' R *R!+&M&6N@Š-86Noh !+6.6.=a *OQPR(
,.!"=+r i *6.(*&(&$c\#*!+H1 MKhA!"G-81,7E^H6.(=-8(r/1!+67&š&6gfS*#X 1 1 6. O>PRr&16.@Š -86.o
!+6.16.(= *T(,7/G0!"C+ 1 !+& 9 02-+'6.&(*/S&(3*OV|&!KR\5!+#C$3-+9%#X-8H(; +$I6NQ!"T(,.!"6.(=
!+,7=+-+6.0`!"J !+6N(&-82 !+6.16.(= *Q!+,795-+95R67{*J-+9(J !+6.6N(='*`9x-+cKL6.#;$8H&(* ¡ H 67-8
$KTZ **Cr(67J,.!"67)+/S*9@-+0!+#X+O •
v<2#X-8H,.& *#
$ 5 ! 9 1 :
9 $5\5H(LKLpKT: &u16.LXWD/S*6.023!+,.,7E+$5Kha9@-8H&u!":H,76.(=
6.0?!" T-+95(n)e!";6.!+#XnKT!+k)+*E:(-86.E+$8!+&a-+9x {**-(Ok6.06.,N!"V67H1!"67-8*$"-+(*`1!)+J9@-8H&:!"
#X-80:\16.6.=M(?H,7-+9R0aH,76./1,72#X-8y‡)"!+,.6.&1!"67-8#*!+>(,7/> !"\16.,.6.{*!+_ 6N0!" +$` -pKT?/S*9@-+0
B1)+ ¤ y‡9@-8,.&A#X-8y‡)"!+,.6.&!"67-81J!+&A&(XB5
Õ
Õ
$
‚9kKha!+H02:!"L
9
5
E
9
1
"
!
9
A
9x-80!+# ¤ y‡9@-8,.&p#X-8y‡)"!+,.6.&!"67-8u!":6N&(*/S&(3n-+9`!+#-+(*$5(
$Ž6.Z(iH0 -+9 6.&(*/S& oU!+1&(-80 )"!"6.!"\5,7!+#;>1!)D6.(=u! 7<9i&6. 6.\1H(67-8KL67 Õ &*=+*i-+9
9@*&(-80MOT
E(H00?!"67-8d/-+/S*“Ep-+9n( 7<9&6N 67\1H67-8$6.U02!+ $q!+Z! 7<92&6. 67\1H(6.-8
KL67 &(*=+*a-+9R9@*&(-80MOPR6.:,.!+ 6.&(*/S&(1#X!+H02/167-8^6NZ!+,. -9b!+,. +$`\S#*!+H i!+# ¤ y‡9@-8,.&
#X-8 y‡)e!+,.6N&!"67-86.k#X-802/1H( &9@-80†(J!+0J !+6.6.=&!"!DO`s-[KT*)+*$3XWD/S*6.023!+,I Q-[KT&!"
6.J6NJ(,7!+ L/-+\1,70?!"6.#'-+9U)e!"6.-8HR6.&*/1&(#X!+H02/167-8nH&(*;,7EI6N(=: W ¤ #X)M *O
jk6.!+,N,7E+$1 -rH c
FIH!"67-8š” Õ •;$KT0:H'0!"C+a(!+H0/67-8A1!"(a)"!"6.!+#X: 6.0!" !"
!
6.&*/1&(o2-+9 (1 1 O PL6.?0aH ?\!+H0&€”@!"(*G!+/-t)+&!+r6.™uHH1!+, &6. 67\1H( 6.-8
&(*6.)e!"67-8•;$\#*!+H1 2Kh!"H6.(=G-8,7Eu-8(2-+9T(2-+\1 *)+&_&6gfS*#X-+9T/-+/S-+67-8'!"*U!+
(d02!+„-+9Z!+,.,-+9(-+\1 *)+&4&67f*1#X*O |'r&6.#*H1 &…!"\S-t)++$L(d02!+4&6gfS*#X &i -[)+* 6.0!" :( H(U&6gfS*#X+$D\S#*!+H U-+9V(,.!+#CG-+9V6N&(*/S&(#X\S*zKT*M(&6gfS*3n9x-8,.&1h-+9
(Z#X-8y‡)"!+,.6.&!"67-8%O
!
Õ¤Õ
•
4
•
-2
Observed t value
0
2
••
•••••••••
•
•
•
•
••••
•••••••
•••••
•
•
•
••••••
•••••
•
•
•
•
•••••
••••
•
•
•
•
••••
•••••
•
•
••••••
••••
•
•
•
••••
••••
•
•
•••••
••••••
•
•
•
•••••••
•••
•
•
•
•••••
•••••
•
•
•••••
••
• ••
•
-4
•
•
•
•
-6
• •
-4
-2
0
2
Predicted t value (based on quantile)
4
6
Õ
jk67=8H( ¤ ‘ ¡¡ /5,7-+U#X-802/1!"6.=?(2&6. 67\1H(6.-8M-+9 $ )"!+,.H('-+9 -M(:)"!+,.H('(*E(-8H,N&u![)+
H&*:!
&6. 6.\1H(67-8dKL6. &*=+*a-+9R9x*&(-80MOu|',.,c/-86NoZKh-8H,N&]9b!+,.,Q-8^(r,.6.(
5P0™679J(
&6. 67\1H(6.-8h0?!"#(&O
Õ
v467A!+,.,S-+9k(Z!+H0/67-8*$DKT#*!+A/1,NH(=26.r - cFIH!"67-8^” •T --+\!+6. O
lm*:HH00!"6.{*2(?!+H102/67-8U!+1&]!"//-[WD6N0!"67-86No)+-8,7)+&^6.<6NU&(*67)"!"67-8OGj6. *$%KT
02/1,.-[Er(Z(-+;0!+,%!"//-W(6.0!"67-8M -i(\16N(-806.!+,&6N 67\1H67-8Oc¢I#X-8&$5Kh:!+H02Z/1!+67KL6N U6.&(Xy
/S&(#X-+9 1 !+& 9 9@-+!+,., O2PR67;&$5KT?!+H0226.&(*/S&#X:\S*“Kh*< ! žŸ*O2jk6.!+,.,7E+$SKT
!+H1026.&(*/S &#X\S *“Kh*p(ZIH02*!" -+L!+1&A&((-806N!" -+R-+9( !"6. 6.#"O LKn!E? -*)"!+,.H!" '( W ¤ #X)G !"6. 6.#XWD/*;6.023!+,.,7E26.h -0?!"C+'!FIH!+o6N,7XyzF3H1!+o6.,.R/1,7-+” ¡¡
/1,7-+;•;$!+J(-[KLM6NGjk67=8H ¤ OhPR( ¡¡ /1,7-+J(-tKL $ #X-802/5H( &r)e!+,NH(n-+9m( W ¤ #X)r !"6. 6.#9@-+R!
#*!+ KL(*L('3H,N,13E3/S-+(6Nc6.cCI(-tKLi -!"/1/1,7EA”@(#
 Z!+ C?!+n&(#X6.\&\,.-[K'•;O`PV-:=+(*;!" ! ¡¡ /1,7-+*$`( $ )"!+,.H(a!"M -+ &š!+&š!+67=8&^FIH!+36.,7M”b6‹O›+O7$Q(6.:!+C_6.^(G-+ &š,.6. &67)D6.&(&A\3E $ •;OPRp(6.3)+* a#*H0:H1,.!"67)+ &6. 6.\1H(67-8]”@KL6. &(*=+*-+9`9@*&(-80i•L6.'H &
-2#X-802/1H( 9@-+n!+#AFIH!+36.,7+$I()"!+,.H('!"J! yz&16. 67\5H( &!+1&(-80 )"!"6.!"\5,7KT-8H,.&G![)+'!"C+p679%67
!+&d!+&!"!+(COPR6NL)"!+,.H(a\S#X-802(
0_#X-I-+&6.!" +$!+&(-+6.=86.!+,V)e!+,.H \S#X-802
#X-I-+&6.!" +O`r-+(*RKT-+&$(9x-+R!+#;A-+\1*)+&M/S-86.3*$I\5!+ &r-8A67n-+&16.!+,/S-86767-8GKL6716.? $ /S-86.3*$%KT?#*!+_#X-802/5H( KL!"Z)e!+,.H67Z(-8H1,.&d!)+G!+&]679T( $ /S-86.oU1!+&<\S*< H,7Ed&(![KL
9@-80‰! &6N 67\1H67-8O%‚9( $ /-86NoT!)+U! &6. ;67\1H(67-8K67 &(*=+*T-+9%9@*&(-80A$3(G(*E
Õ
Õ
Õ
Õ
Õ
Õ¤
Õ
(-8H1,.&r,.67-8M(,.6N( 5O0VOQPR(B1=8H((-[KLL!9@!+67;,7E=+-I-I&rB5J -(,.6.+Oks-[KT*)+*$!"R(U!+6.,.n-+9
(Z&16. 67\5H(67-8$ 6.J-802*KL!"L0-+#X-8 *)"!"67)+:!+A67R-8H,.&G\S+O
H'#;(-86.#Xa-+9`B1)+*/1,.6N#*!"67-8R-+9h#X-8y‡)"!+,.6.&!"6.-8p6.'(-+!"\167 !"E+O QWI/1,.-+!" -+EA H&167(-[KT&
!"MH6N(=]9@*Kh*r-+A02-+!+B1)+*/1,.6N#*!"67-8?6N#X!+ &(d6. Cš-+9aPcEI/Sp*-+O |Ž/-867\1,7
XWD/1,.!+!"67-89x-+h6.`6.`1!"c(*!"RzKT-:#X-80/*6N(=/-+\5,70*OVv„67?9x*KT*c*/1,.6.#*!"67-81*$+(L(-86N R6.
(n02!+H(02o-+9( ! žŸ\S#X-802 -8H(\1,7-802+OVv467a02-+T*/1,.6N#*!"67-8*$t(T,N!+#C-+916N&(*/S&(#X
!+02-8(=( ! žŸT\S#X-802n -8H\1,7 -802+O`v4(*(*TB1)+U6.T(\S T)e!+,NH(9@-+n(U3H10Z\S*T-+9m*/1,N6.#*!"67-8
6.J!+p-+/MFIH( 6.-8O
PR16.%#X-802/1,.* V-8HV&(#X;67/67-8U-+9(cB1)+T !"6. 6N#*!+,o -+9((T3H1,.,83E3/S-+(6N%!"VzKT-,.!"6.(=
!+,7=+-+6.0T!)+U(!+02*-+T;!" '-8r!a=867)+G/S-+/1H,.!"6.-8u”@\1!+ &r-8r!!+02/5,7L-+9V!"n/-+/5H,.!"67-85•;O
Q
Q%A
:qTATS
c'
v<(-[K H(p -G!+uXWI/S*6N02o!+,m*)e!+,NH!"67-8A-+9c( B5)+:02*(-D&*OPR(/1H/S-8 U-+9Q(a6.0aH,.!"67-8
Kn!+R -?02!+H((Z/-+\1!"\16N,.67zE?-+9PhE3/SUn*-+L-+9!+,7=+-+670?*Oc| PcEI/SUT*-+R-D#*#*H(JKLM(
IH,.,Q3E3/S-+(6.Z6.: H(d”b6‹O›+O7$`(*r6.:(-&16gf*#X?\S*zKT*>(?“Kh-<,7!";6.(=p!+,.=+-+670’•U!+&^(
,7!"16.(=2!+,7=+-+6710‚Š#XL(ZIH,.,S3E3/S-+(6N*O
PV-?02!+H(U/-+\1!"\16.,.6.zE-+9`PhE3/SUT*-+$KTZ#X-8 H#X &A -80Z6.0aH,.!" &A,.!"6.(=a/-+\5,70*O
PV-…H&(*; !+&„( ]/1-+\1,70$67A6.MH1 *9@H1, -™6.(C„!"\1 !+#X,7E!"\S-8H(M(<\S![)I6.-+M-+9,.!"6.(=
!+,7=+-+6.0*O
" $ + + 6 4$ * & $'& $, & + 6")
T-86.&(*!i/S-+/1H1,.!"67-8u-+9 &1!"!r/-86No'!+&<H(/1/-8Z!"U( !+6N6.(=G *67{*6.BWD&OZPR(9@-+
!=86.)+u,7!";6.(=!+,7=+-+;670:$S&(XB5( h”<0•n -G\Z(Z/-+\1!"\16.,.6.zEG!"L(a#*,.!+67B1*L/-I&H1#X&M\3E KL(A !+6.(&M-8!2!+1&(-80,7E3yz&(![KLr !+6.6N(= *:”@-+9`(ZBWD&A67{*t•JKL6.,.,S06.#*,.!+679@E 0VOh‚9 T”<0S•5 $
( 0š6.!+,.KT![EI:#X-+#X,7E]#*,.!+6gB1&d\IE#*,N!+6gB1*;U/-D&H#X&d\IE :O?‚9 ”<0S• 5 Õ $m 0š6.U!+,7Kn!ED
06.#*,.!+6gB5&O
jk67=8H( (-tKLa(M02!+H(&^)"!+,.H(a-+9[ ”<0•U9@-+2!/-+/5H,.!"67-8_-+9 3$ /S-86.o:KL67^ /S#Xa ( !(O &(#*6N67-8 *!+,7=+-+6.0 ” ¡ H6.,N!+$ Õ •U !+6.(&-8d!+1&(-80,7E3yz&(![KLp ;!+6.6.(=r *-+9 Õ
XW(!+02/1,7*OPR(2/S-86.3'Kh*? -+ &d\IEu67U
)"!+,.H(*O67)+d )"!+,.H(*$mKh#X-8H,N&<6.0aH,.!" (
\S!)D67-+n-+9 !(O -8M!!+&(-80?,7E?&(![KLr R *J-+9V/S-86.on\IE?!"CD6.(=!+#;A/S-86.3 0p!+1&M06.#*,.!+679@EI6.=
67`KL6./-+\1!"\16N,.67zEt ”<0S•;OQPR16.Kh-8H,N&2(-+`XWD!+#X,.Ea*/-D&H#Xn(R\S![)I67-+`-+9 !(O $3\#*!+H1 L67`!+H102
!"(206.#*,.!+6gB#*!"67-8p*-+0!+&(\IE !(O !"26.1&(*/S&(3L9@-+'!+#;< UXW(!+02/1,7+$K(*!+6.
#4
9b!+#Xi(d#*,.!+6gB5#*!"67-8?-+9&!"!^/-86Noi!"r!"u#*,7-8u -+=+*(*MK6.,.,n & -^\S67=8,7Eš#X-+,N!" &O
s-[KT*)+*$V16.6.0aH,.!" & !(O /-I#X&H:KT-8H,.&d![)+(2!+02![)+*!"=+?*-+U!" ?!+(2!+,!(O !+,7=+-+6.0M$!+&M67JK6.,.,XWD167\167J!26.0?6.,.!"J&(*=+*-+9k)"!"6.!"67-8r9@-80 -8!+&(-80 ;6.!+, -(XWI*O
v<#*!+M6N0:H,N!" ',7!";6.(=!+,7=+-+6710cK67G)e!"6.-8HT/-+/S*67h\3Ei&(XB56.=:!/S-+/1H,.!"6.-8-+9m/S-86.3
!+&r!+67=86.(=!a)e!+,NH( t<” 0S•c -2!+#A/-86No*Ok‚9%KhKn!+oR!,7!";6.(=a!+,7=+-+670 -2![)+67=8G)e!";6.!+#X+$
KTZ#*!+p!+67=8M)"!+,.H(J-+9 (!" O $5KL6.#;M6.J()"!+,.H(U=86.)I6.=2(Z0!eW(6.0aH0)e!";6.!+#X9@-+L!\56.(-806.!+,
!+&-80})e!";6.!"\1,7+OV“9KhLKT!+3QzKT-:,.!"6.(=!+,7=+-+6.0` -Z1!)+R(!+02J*-+c!" R-82(L/-+/5H,.!"67-8$
KTZ#*!+AH(!"R(:!)+*!"=+a)e!+,NH(-+9 R-[)+*'(U/S-+/1H,.!"6.-8W
ˆ6.J!+029@-+R\S-+A!+,7=+-+6.0*O
Z-8H H1&67*$eKhTKn!+o & -#X-8 H1#X6N0:H,N!" &:,7!";6.(=R/1-+\1,70%!"kKh-8H,N&/-t)I6N&(h
! Kh-+; y
#*!+ ?9x-+-8H( !"6N 6.#*!+, OTPV-i!+#*#X-802/5,.6.M16.*$(KTZ -8H(=83L -?0?!eWD6.0?67{*(“Kh-G0!+6.A-8H(#XR-+9
!+&-80 )"!"6.!"67-8%‘)"!"6.!"6.-8?H,76.(=Z9x-80 (#-86.#X-+9% J&!"! *J!+&i)e!";6.!"67-8iH,76N(=9@-80
(a#(-86.#Xa-+9` !+6.16.(= **OJv<67=8(-+&A(a6.H(Z-+9Q#*,.!+6gB#*!"67-8A(-86.+O h#*!+H1 a6.L!ef#X ;!+6.6.(=
!+&u U&!"!GF3H1!+,.,7E+$S67'#*!+\S:6N#X-+/S-+!" &6.o -G(-[)+*;!+,.,*-+'!" +OUv<2!+,. -r67=8(-+&6.3 *!+,
Õ
1
0.9
0.8
epsilon(x) for C4.5
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
0
jk67=8H
1000
2000
3000
4000
5000
Data Point x
6000
‘c˜u!+H(&M)"!+,.H(R-+9 C E ”<0•T9@-+!/S-+/1H,.!"6.-8i-+9
7000
8000
&!"!2/S-86.3*O
!+&-80(6.(2,7!"16.(=r!+,7=+-+670?*$6.#X6.'KL6N,.,m0!+1679x6. ,79c6.(2!+02KT![E!+U ;!+6.6.(=
*R)"!"6.!+1#X\3Er#*!+H6N(=2(!+02:,7!"6.=!+,.=+-+670 -/1-I&1H#X&6gfS*3n#*,N!+6gB1*;*O
j(-+L n-+9VPhE3/S'Q*-+[$IKTU&(67=8&i“Kh-*n-+9 T)"!+,.H(*‘ T<” 0S•c9@-+R!+,7=+-+670 !+1& R”<0S•c9@-+
!+,7=+-+6.0 'O8v<L !"\5,.6.(&2!!"=+*Q*-+c!" c!+&#-8 R-8,7EazKT-Z&6N 6.#Xk)"!+,.H(` -ZH1 J9x-+ h<” 0•
!+& J<” 0•;‘ 1 !+1& OUv<=+(*!" &<!G/S-+/1H,.!"67-8p-+9c/S-86.3*Oj(-+U(aB1 !+,79`-+9c(/-+/5H,.!"67-8$
KT'!+67=8(& 9 T<” 0S
• 5 9 1 n!+1& R<” 0S• 5 *O`j(-+J('0!+6.16.(=:1!+,79-+9%('/S-+/1H,.!"67-8%$8Kh*)+*; &G6.
!+&d!+6.=8(& <” 0
• 5 9 !+& <” 0• 5 9 1 *OUjk67=8H( !M(-tKL'6.'#X-8DB1=8H(;!"67-8p-+9 ')e!+,NH(*OPR(67{*
-+9Q(a/S-+/1H,.!"6.-8p6.'679 ,.*)e!+3*$5\S#*!+HZ9 KT!"a!+0/1,.6.(=?KL67u*/1,.!+#X023!+&p*a!"a-81,7EMzKTCD6.&'-+9T/-86No*OaPR(6.0/-+!+o'6.(=M6.1!"U(2/S-+/1H,.!"67-8d6.'*)+,.E&6.)I6.&&p\S*zKT*<(zKTCD6.&T-+9k !+6.6N(=/-86No*O
PRZXfS#X-+9Q16.L6.L!"L!+#;<!+,7=+-+6710 !+'!+p-t)+*!+,.,V*-+'!" Z-+9 *$5!+1&p!+#;!+,.=+-+670q!+
(2!+022 -+!+,Q)e!"6N!+#X+O:s-[KT*)+*$9x-+!+3Ep=867)+] UXWD!+0/1,7+$(2!+,7=+-+6.0!)+?)+*Eu&67f*3
*-+Z!" *OiPL6.U0!"C+:(2XfS#X:-+9T(!+1&(-80Ž#-86.#X-+9n :&1!"!A *)+*Ed!"/1/1!"3*O?&(*&$
H,.(p i&!"!^ *?6.2XW(!+#X,7E™FIH!+,.,7Eš&67)D6.&(&š\S*“Kh*…(AB1; !+,79'-+9(p/S-+/1H,N!"67-8™!+&
(a #X-8&!+,79-+9`(Z/S-+/1H,N!"67-8$((*:KL6.,.,S\SZ!+!"//1!"o'!+&()e!+3!"=+a9x-+'-8(:!+,.=+-+670 -[)+*(
-+(*OH( !"6N 6.#*!+,J iKL6.,N,T*&™ -_!)+-86N&ƒ\S6.(=d9@-3-8,7&ƒ\3Eš6.2!"/1/1!"3?&6gfS*#XA6.š*-+
!" *O `WD/S*6.023!+,k02!+H(0o:#X-8DB102&]!"( i#(-86.#XZ-+9 c<” 0•U!+& R<” 0S•&16.&d(2\S Š -+\-+96.0:H1,7!+(*-8H,.EZ0!eW(6.067{6.=KL6.6.Dyz!+,7=+-+;670 )e!";6.!+#XR!+&\S*zKT*Dyz!+,7=+-+;670Y)"!"6.!"6.-8KL6.,7
!+#167*)I6N(=(J&(67&a-t)+*!+,.,D*-+`;!" J-+9 *O202-8 k-+95-8HXWD/S*6.023*$+KTJH &2!')e!+,NH(n-+9 5 $
A
!+,7(-8H=8MKh:!+,. -6.3)+ 67=8!" & 5 ¤ $ 5 $S!+& 5 ! O
A
A
Õ!
A
Õ
0.25
0.2
Algorithm B
epsilon(x)
0.15
0.1
0.05
Algorithm A
0
1
N/2
Data Point x
jk67=8H(!(‘ 67=8(&A)"!+,.H(n-+9 h ”<0S•J!+&
#4
+-$ 4 ) ?+ 6 ) "
$ +
8$
c!+#;46.0aH,.!"67-86.i&167)I6N&(&ƒ6.3 ->!> *6.?-+9
N
R”<0•T9@-+L!+M-[)+*;!+,.,%*-+R!" -+9 5 A Õ O
)+-$ + ) + $ + ) +
Õ
6N!+,.*O !+#4 6.!+,‡$R!^&!"!> * -+9Z67{* 6.G!+&-80,7Eƒ&(![KLKL67*/1,.!+#X023M!+&„(!+!+,7EI{*&4H6.=_!+# -+9(B1)+< !"6. 6N#*!+, &(#X;67\S&]!"\S-[)++OAPR(?=+-8!+,h6.Z -u02!+H(??/-+/S-+6.-8<-+9n 6.!+,.:6.<KL6N#<(G3H1,.,Q3E3/S-+(6N6.
‚Š #X &O
j(-+c(nB1 zKT- $o˜u#*œ0!"žŸc Q!+&(J(-+;0!+,I `9x-+`R&6gfS*#Xn-+91“Kh-/1-+/S-+67-8*$
(a&!"!? *
ƒ6NR!+&-80,7Er&67)D6.&(&M6.3 -?! ;!+6.6.(=*D
M #X-83!+6.6N(=zKT-G67&n-+9 …!+&u! ' *
S #X-8o!+6N6.(=a(0!+6.16.(=Z-86.&?-+9 TOQPR(' !+6.16.(=:*n6Nh67=8-+&$D\S#*!+H 'KhU&(-2(-+n!+#XH!+,.,7E
XWD#*H( u,7!";6.(=]!+,7=+-+;670*O !"(*$Rp/S*9@-+0!+#XA-+9Z!+#4!+,7=+-+6710 -8(p r *r6.
6.0aH,.!" &u\IEp#*,.!+6.9xED6.(=i!+#d XW(!+02/1,7 0]!+&-80,7EM!+#*#X-+&6.=G -M67')"!+,.H(:-+9 T<” 0S•!+& J<” 0•;O
|}!+1&(-80 3H0:\S*6.u(!+(=+,
$ •'6.'&(![KLO'“9c6.'6.',7!+ <” 0S•;$(?
0]6.'#X-86N&(*&u -G\S
06.#*,.!+6gB5&\IEi!+,7=+-+6.0 :Oc| #X-8&r!+&(-80IH0:\*n6.n&(![KL$(!+Q
& 0A6.n#X-86N&(*&r06.#*,N!+6gB1&
\IE>!+,7=+-+;670 $ 679'!"3H10Z\S*?6.2,.!+ J<” 0S•;O…PL(MH,.-+9' A#*,.!+67B5#*!"67-8!"M(
/-D#X &ƒ\3E>(A!"//-+/6.!" A !"6N 6.#*!+,J *O|,.,n KT*M/S*9x-+;02&™H6.= ¤ yz6.&(&š ?KL67
#X-8DB5&#X,7*)+, O
j(-+`h!+02/1A ,7&Z/1!+67& *$86N%/-D#X-+9!+1&(-80,7EU /1,N67 6.(=RT&!"!' * pzKT-"y‡67& t-8(Xy
67;&JKn!+L*/S!" & 6.0*O c!+#;p *KT!+'#*,.!+67B1&A!+'&(#X67\S&A6NM(Z/1*)I6.-8HJ/1!"!"=+;!"/1O
PR( &6gfS*#X:6.<(?*-+Z;!" Z-+9J(?zKT-!+,7=+-+;670UKT*G#X-8,.,7#X &_!+1&]02/1,.-[E+&>6.<(
*O
j(-+n( .3y‡9@-8,.&?#X-8y‡)e!+,N6.&!" &i/1!+67& *$D(&!"!a * <Kn!+n&67)D6.&(&6No - .R5 ;!+&(-80 H(\(y
*T-+9mFIH!+,67{*+O h!+#G-+9m( J *nKT!+T(r#*,.!+6gB1&?\3E\S-+G!+,7=+-+6710hH16.(=:(;!+&(-80
/-D#X&H(:&(#X;67\S&u!"\S-[)++O:sL-tKh*)+*$V&H(6.=(2#*,.!+67B5#*!"67-8u/-D#X*$S -A6.0aH,.!" :!+&(-80—)"!"6.!ey
67-8M6.G(UFIH!+,.67“E?-+9V!+#M !+6N6.(= **$(KTU=+(*!" &A!!+1&(-80 )e!+,.H
]6NiU!+(=+ = 8¤ J 3 8¤ Õ
Õ
Õ
A
Õ
A
0.3
Probability of Type 1 Error
0.25
0.2
0.15
0.1
0.05
0
McNemar
Prop Test
Resampled t
Xval t
5x2cv t
j 67=8H( ‘hQ-+\1!"\56.,.67“E-+9`PcEI/SUh*-+R9x-+L!+# !"6. 6.#*!+,% *OhPRU9x-8H(!+&eŠ !+#X3R\1!";J9x-+L!+#u k
*/ o(a/-+\5!"\16.,.67“Er-+9hPhE3/SaR*-+9@-+ 5 O $ O ¤ $ O $m!+& O ! O c-+'\1!"(-tK #X-8DB5&#XL6.3 *)"!+,.k9x-+h( R/1-+\1!"\16.,N6767*O%PRL(-+67{*-83!+,,.6N(J(-[K`(R!"=+*h/-+\1!"\56.,.67“E-+9 O O
Õ
+! &A!+&1&(&A6Nn)e!+,.H _ -*)+*E h”<0•n!+& J”<0S•J\S*9x-+U=+*!"6.(=i(Z#*,.!+6gB5#*!"67-8OcPR(UH,7
-+9k(#*,.!+6gB5#*!"67-8nKT*A#X-8,.,7#X &u!+&AH(\(Š#X &M -?( *O
j(-+T( W ¤ #X) *$3B1)+R*/1,.6.#*!"67-81`-+9 ¤ y‡9x-8,.&i#X-8y‡)"!+,.6.&1!"67-82KT*R/S*9@-+02&$o!+1&( !"6N 6.#
Kn!+L#X-8 ;H#X &A!+L&#X67\S&r!"\S-[)++O
‚h6.h6.02/S-+!+3` -a-+ L!"h6.QXWD/S*6.023c&(-Ic-+h6.0aH,.!" L !+6.6.= *c)"!"6.!+#X+O`/5!"6.#*HDy
,.!"$I67c&(-IT(-+h02-D&(,(LXfS#Xh-+9-t)+*,.!"//16N(= !+6.16.(= *T-8L\S!)D67-+h-+9(#X-8y‡)e!+,N6.&!" &
Q-+h( W ¤ #X)2 *O`v<RKL6.,N,I#X-+#Xc6N`(-+#X-806N(=U\S,7-[Kƒ6.-8HQXWI/S*6N02okK672!+,,.!"6.(=
!+,7=+-+6.0*O
T jk67=8H( (-[KL'(a/-+\5!"\16.,.67“Ei-+9h0!"CI6N(=i!GPcEI/SZJ*-+'9@-+'!+#d-+9Q(:B1)+a/-D#X&H(LKL(u(
&!"!* š#X-83!+6. XWD!+0/1,7J!+&M-[)+*!+,N,*-+R;!" JKn!+J)"!"67&M9@-80 O - O ! O
PJKT-r-+9h:B5)+a !)+!r/1-+\1!"\16.,N67zEr-+9nPcEI/S:*-+U!"XW(#X*&U(2!"=+*U)"!+,.H(-+9 O ‘
(?&16gf*#XXy‡-+9Ny‡/1-+/S-+67-8U :!+&<!+02/5,7&
*OMPR(20?!+6.6.(=M :![)+i!+#*#X*/!"\5,7
/-+\5!"\16.,.67“E-+9k0!"CI6N(=!PhE3/SUT*-+L!+#*#X-+&16.(=2 -6NR6.0aH,.!"67-8%O
PR!+02/5,7& '!+L!i0:H#;u167=8(*R/-+\1!"\16.,.6.zE?-+9QPhEI/ZT*-+L!+p(Z-+* ORPR6.
H1,79@-80 (u9b!+#Xr!"G(p!+1&(-80,7E3yz&(![KL…&!"!> * ‰6.?,.6.C+,7Eš -š#X-83!+6.„!+46.0Z\5!+,.!+#Xp-+9
/S-86.3U9x-80Ž(?B1 !+,.9h-+9J(?/-+/5H,.!"67-8<#X-802/5!"&_ -u(? #X-81&^!+,.9h-+9J(?/-+/5H,.!"67-8OiPR(
!+0/1,7& T#*!+i&(* #Xn!+&i0!"=8679@E:16.Q&6gfS*#XHo6N,67c6. !"6. 6.#*!+,N,7E267=86gB#*!+o*O &(*&$
(/1-+\1!"\16.,N67zE-+9k0!"CD6.(=!PhEI/UT*-+LKL67M6Nn #*!+M\S6.1#X!+ &M\IEG6N#X!+6.=2(3H10Z\S*
-+9k!+02/5,7&M !+6.16.(= t L/1,.67*Ohj6.=8H( 2(-[KLL(ZXfS#XL-+9`)"!"67-8H1R3H10Z\S*J-+9k!+02/1,7&A/1,.67
Õ
Õ
0.25
Probability of Type 1 error
0.2
Resampled t
0.15
0.1
0.05
Xval t
0
0
20
40
60
Number of replications
80
100
jk67=8H(I‘Tc-+\1!"\16.,N67zEi-+9kPhEI/ZT*-+L9@-+L(:!+02/5,7& !+1&A( 3. y‡9@-8,.&u#X-8y‡)"!+,.6.&!" &
!+R3H0:\S*J-+9k!+02/1,N6.(=a*/1,.6.#*!"6.-8J6.n)"!"67&OQPRU*-+R!" 5 O
A Õ
9@-+\S-+A:!+02/1,7& '!+&u(a#X-8y‡)"!+,.6.&!" & *OLœ-+6.#X!"(a#X-8y‡)"!+,.6.&!" & &(-IR(-+RXW(67\567h16.n/-+\1,.0MO
PR&6gfS*#XXy‡-+9xy‡/-+/S-+67-8T HDfS*n9x-80 36.!+,.,7Ei(Z!+02U/1-+\1,70MOkv4(M(!+02/1,7
_6.cH*/ 3!"67)++$I(02!+H(&i&6gfS*#X'6.(zKT-:/1-+/S-+67-8`K6.,.,(\SL,.!"=++$I/#*6N!+,.,7EaKL(
n6.T(!" O Oc‚T6Nh6.3 * 6N(=:1!"T˜#*œL0?!"žŸJ J&(-In(-+T!"'6.h/-+\1,70AOkPL(C+*Ei&6gfS*#X
6.L!"L(:&6gfS*#XXy‡-+9xy‡/-+/S-+67-8R -81,7EG,.-3-+CDL!"(:&67f*1#X\S*zKT*p“Kh-G/-+/S-+67-81R!+&
(-+L!"L(67R!"\1 -8,NH( U)e!+,NH(*O T-86.&(*J(U9@-8,.,7-tKL6.(=zKT- ¤
¤ #X-8o6N(=+#XEr!"\1,7*‘
!
!
¤
!
T-+r-+9%( '!"\1,7n![)+('!+02U&6gfS*#X6N?*-+h!" n-+9 O ¤
$ -:&16gf*#XXy‡-+9Ny‡/1-+/S-+67-8
G !"G(0 6.&(36.#*!+,.,7E”b!+&‚Š#XG(3H1,.,R3E3/S-+(6N •;O†sL-tKh*)+*$˜#*œL0!"[žŸ
B51&(-<67=86gB#*!+o&6gfS*#XM6.>(M,7*9@!"\1,7+$c\5H(ZB&!+šXWI A 02,7E^67=86gB5#*!+3&6gfS*#X
”
•6.A(Z6.=8oR!"\5,7+ORPR6.L6.R\S#*!+H a6.M(a,7*9@R!"\1,7+$˜u#*œ0!"žŸ' !+ CDL(aF3H 67-8
v4!"RA 6.h(/-+\1!"\16N,.67zE6N -8n-+9V!a9@!+6.n#X-86Ni1!"TKTKL6.,.,1#X67)+ ! (!+&J!+1& !+6.,N;– d
('67=83h!"\5,7+$(67T!+CIT(UFIH( 67-8 v„1!"R6.c'/1-+\1!"\16.,N67zE26. ¤ -8 n-+9m!a9b!+67T#X-86Ni1!"TKTKL6.,.,
#X67)+ ¤ (!+&'!+& !+6.,.;– ƒPRZKn!EAKTa1!)+#X-8 H#X &-8H(6.0aH,.!" &u,7!"6N(=2/-+\1,.0*$KT
!"02-+',.67C+,7E -/-D&H#XR!"\5,7h,N67C+L(-8('-8?(,7*9@*$3/#*6N!+,.,7EKL( n6.c(!" O Ocœ-+ !"T6.9
KTa!+&uH &u(:#X-+#X &d)+*67-8p-+9Q(a&6gfS*#XXy‡-+9xy‡/-+/S-+67-8L *$67LKT-8H,.&u(-+1!)+2HDfS*&
9@-80 6.n/1-+\1,70MO
T#*!+H J-+91(n/-I-+\1!)D67-+R”b!+&67=8a#X-8 ;•V-+9(n!+0/1,7& *$8KTnXWD#*,.H1&(&:6.V9@-80 9@H(*
XWD/*;6.023*O
Õ
Õ
Õ
P A\167=+=+ i&(!KL\1!+#Cš-+9(p6N0:H,N!"67-8™6.2!"G67&(-I?(-+i#*!"/H(A-+G02!+H(A !+6.6N(=< *
R
)"!"6.!+#Xr-+)"!"6.!+#XrH,76.(=9@-80(M6.3 *!+,T\S!)D67-+2-+9'(M,7!"6.=!+,7=+-+;670MO>Pm-]!+&&
( 2/1-+\1,70$5KT#X-8&H#X &<!M#X-8&< *U-+9cXWD/S*6.023'KL67!+,`,7!";6.(=r!+,7=+-+6710!+&!+,
&!"!**O
Q
Q%
c' †
+
;1S
T )
PV-u*)"!+,.H!" G(rPcEI/S*-+a!" a-+9J-8H(a9x-8Ha !"6N 6.#*!+,h aKL67]!+,h,7!";6.(=p!+,.=+-+670$VKT
(*&(&a -B51&Z“Kh-U,7!"16.(=!+,.=+-+670!"`!+&6.&o6.#*!+,3/S*9@-+0!+#XTKL(a !+6.&Z-8 !+6N6.(=' *
-+9%!=86.)+?67{*+O`v<!+,. -:(*&(&,7!"16.(=!+,7=+-+6.0Q -:\SR)+*E2 i#*673*$D -Z!"h(LXWD/S*6.023
#X-8H,.&r\S*/1,N6.#*!" &M0!+3E?6.02OQPm-?!+#;67*)+16.*$DKh#;(-8 !(O R,7!+ ” ¡ H6.,.!+%$ •R!+&r(
B1k!" yz(67=83\S-+”bœ'œ•k!+,.=+-+670” !+!"!"3E+$ •;OQv<R(,7#X &2(*&6 i#*H,.m/-+\5,70*‘
(
 /-+\1,70†&(*)+,7-+/S&a\3E U-8(=” •;$3nlm* *
R#X-+=816767-82&!"! *L”bj(*"
E ¢D,.!" +$ •;$
!+&?(h6.0!&6N!+ 6.!"\S* hP!+ Cu”b˜u*
{ ˜H(/13E+$ 8•;OJ'9#X-8H( +
$ !(O !+&?œ'œ &(-a(-+h![)+
(G!+02/S*9@-+0!+#X?-8]( G&!"!p**OM‚  A!+1&_l%* * R#X-+=816767-8$kœœ/S*9@-+0Z0:H#;
\S* *R!+ !(O w(*)+* Z6.n ;H(6.r(Zh6.0!&!"!**O
HQ(XWIQ */KT!+c - &!+0!"=+ Z(R,7!"16.(=U!+,7=+-+6710` -U!"Q67`/S*9x-+;0!+#XnKT!+h6.&(36.#*!+,‹O
<( '
 M!+&]lm* * R#X-+=816767-8]!+CI*$VKhi02-I&16gB1&d(&6N !+#X?02* ;6.#02/1,7-tE+&]\IEœ'œ \S!2KT67=8o & cH1#*,.6.&(!+r&6.!+#XKL67r\5!+&AKT67=8oOhPR6NJ!+,.,7-[KT&pH1J -&H1#X(/S*9@-+0!+#XU-+9
œ'œHo6N,167c0?!"#(& !(O -8?-8 &!"!a **OQPV-ZFIH!+,.6.{*L(/*9x-+0?!+#XL-+9('!+,7=+-+6710Q-8i(
c6N0!M&!"!p**$KT?02-D&6gB5& !(O -u!"KL_#*,.!+679xED6.(=M(*K‰6. !+#X$67ZKh-8H,N&]0!"C+?;!+&(-80
#*,.!+6gB5#*!"67-8r*-+R!"L!/#*67B1&r!" +O
˜u-+/#*6.,7E+$5!+#;]&!"!r *Kn!+/1-I#X &d!+9x-8,.,.-[KL*Oj(-+
'
 I$Kh=+(*!" &<!r#*!+,.6.\!"67-8
*-+9 ¤+¤ $ XW(!+02/1,7a”b /1!+#X&u-8p!H1679x-+;0=+6.&M-+9k -8,NH(67-8 O •;Onv<ZA=+(*!" & &!"!
**$n!+#;…-+9U67{* $J -_6.0aH,.!" */1!"!" u 6.!+,.O…j(-80 !+#-+9' &!"!] **$nKT
!+&-80,7E<&(*K H(\1 *Z-+9L67{* ¤ $ ¤ $c!+& OuPR i6.{*ZKT*r#(-8 >\S#*!+H i(*E]!"G(
67{*?-+9 !+6.6.=d*iH&ƒ6.™( y‡9x-8,.&#X-8y‡)"!+,.6.&1!" &
*$J(p˜#*œ0!"G!+1&…&6gfS*#XXy‡-+9xy
/-+/S-+6.-8n *$S!+&A W ¤ #X) *$1/#X6.)+,7E+$1KLA(-8 : '!"Z=867)+u!+u6N676.!+,S&!"!i *
-+9 XWD!+02/5,7*OQj(-+!+#A6.{*U-+9m !+6.16.(= *Z” ¤ $ ¤ $1!+1& •;$1KT!+&eŠ H1 &G(&6. !+1#X0* 6.#
9@-+Qœœ„ -U!"`(R![)+*!"=+L/S*9@-+0!+1#XU”@-t)+*T!+,N, &!"! **$302!+H(&-8( ¤+¤ $ #*!+,.6.\!"67-8
XW(!+02/1,7;•T0?!"#(&p(Z!)+*;!"=+:/S*9@-+0!+#XU-+9 !(O -?KL676. O O
j(-+lm* * R#X-+=8676.-8$SKha!+&(-80?,7EAH(\&67)D6.&(&G( ¤ $ XW(!+02/1,7'6.3 -G!G#*!+,.67\1!"67-8p*'-+9
$ !+&i!+?XWD/S*6.023!+,5*c-+9 $ Ohv<(G&(*K &!"!a **$I!+#r-+9%67{* $D!+&(-80,.E
9@-80†(JXWD/*;6.023!+,I*Q-+9 $ XWD!+02/5,7*O`|L=8!+6N$+9x-80}!+#2-+9( R&1!"! **$8KTR&(*K™;!+&(-80
H(\5 *n-+967{* ¤ $ ¤ $1!+& Ocj(-+L!+#;p6.{*'-+9 !+6.16.(= **$KT!+&eŠ H1 &r(&6.!+#X02* 6.#'9@-+
œ'œ -r!"(!)+*!"=+i/S*9x-+;0!+#XG”@-t)+*:!+,., &!"!M *$S0!+H(&-8d( $ #*!+,.6.\!"67-8
XW(!+02/1,7;•T0?!"#(&p(Z!)+*;!"=+:/S*9@-+0!+#XU-+9 !(O -?KL676. O O
j(-+k(Tc6.0?!R&6.!+ 6.!"\S* k&!"! **$+Khn&(*K &!"!L *-+96.{* 9x-80€(
![)e!+6N,.!"\1,7
XW(!+02/1,7*Oj(-+!+#;<-+9T( &!"!M *$%(20!+6N6.(= ! XWD!+0/1,7KT*2*!+6.(&d9@-+#*!+,.67\1!"67-8O
c!+#;d-+9c( &!"!M *'-+9T67{* KT!+9bH((*H(\5!+02/1,7&p -r/-D&H#XZ;!+&(-80 H(\1*L-+9T67{*
¤ $ ¤ $D!+1& Ocj(-+T!+#G67{*R-+9S !+6N6.(=U **$IKh02!+H(&?(L![)+*!"=+*-+Q!" -+9 !(O !+&iœœ
”@-[)+* &!"!:**$(KL(i &r-8G( ! #*!+,.67\!"6.-8XW(!+02/1,7n#X-+/-81&6.(= -!+#;M&!"! *;•;O
v<(r!+&eŠH &G('!+&(-80 (-86. '!" -+9 !(O -a!"n(!)+*!"=+Z*-+n!" TKh-8H1,.&?\S6.&(36.#*!+,‹O
Õ JÕ
Õ
Õ
Õ
Õ
Õ
Õ
Õ
Õ
Õ
Õ
Õ
Õ
Õ
Õ
Õ
Õ
Õ
Õ
Õ
Õ
Õ
Õ
Õ JÕ
Õ
Õ
Õ
Õ
Õ
Õ
0.2
Probability of Type 1 Error
0.15
0.1
0.05
0
McNemar
Prop Test
Xval t
5x2cv t
jk67=8H( 3‘LPhE3/SaJ*-+!" 9@-+9x-8H( !"6.6.#*!+, *OUPR(a(*a\1!"KL676NA!+#d #X-+ /S-8&
-?  I$l%* * R#X-+=8676.-8$1!+&uc6.0!2&1!"!? * *O Q-+R\1!"!" #X-8DB5&(1#XZ6.3 *)e!+,NJ-8
( ;H(PcEI/SUh*-+R!" +O
& & &#) + )
Õ
jk67=8H( u(-tKLa(M02!+H(&šPhE3/SG*-+!" -+9(r9x-8H!"6. 6.#*!+,T *O^PR( y‡9x-8,.&™#X-8y
)"!+,.6.&!" & 6.(-8,7Eu UKL(-8 2PhE3/SaL*-+UXWD#X*& O O|',.,-+9T(-+(* *$m*)+<(
&6gfS*#XXy‡-+9xy‡/-+/S-+6.-8n :” c-+/uPV o•;$5(-[K !+#*#X*/1!"\1,7ZPhE3/SUT*-+R;!" *O
& $) 4& "
+)
PhE3/SaL*-+U6.(-+(2-81,7Ep6.02/S-+!+3#X-86.&(*;!"67-8u6N#;(-I-86.(=M!r !"6. 6N#*!+,k *OZ‚U6.'(202-8 6.02/S-+!+3J#X67 *6.-8M679m-8(+žŸL=+-8!+,m6.n -?\U#X-8DB&(oL!"R!+M-+\1 *)+&A/S*9@-+0!+#XU&6gfS*#X6Nn!+,‹O
nH(U679c-8+žŸ=+-8!+,Q6.' -A&(* #XKL*(*(*6.!r&6gfS*#X\S*zKT*<zKT-A,7!"16.(=r!+,7=+-+670?*$S(
(R
( &Z-+9J(i !"6N 6.#*!+,Q a6.Z6.02/S-+!+3*OrPL(/S-[KT*Z-+9R!p a6NU(?/1-+\1!"\16.,N67zEu!":67:KL6.,.,
‚Š #XR(ZIH,.,S3E3/S-+(6.nKL(r(:3H,N,3EI/-+6.J6.J9b!+,. +O
PV-a0!+H((L/S-[KT*T-+9(L $IKT'#*!+,.67\;!" &('&6. !+#X'02* 6.#L9x-+n(!"h(67=83\S-+'”@9@-+
(  !+&™lm* * R#X-+=86.67-8š!+ CD;•a!+&>(r!+&(-80 #*,N!+6gB5#*!"6.-8_*-+2!" _”@9x-+h6.0!o• !+#167*)+2)e!";67-8H'&6gfS*#X6.u(/*9x-+0?!+#X:-+9 !(O !+&dœœ:Om¢I/S#*6gB5#*!+,.,.E+$1KT&16.&p:9@-8,.,7-tKL6.(=(O
jk67 *$KT?02!+H(&](i/*9x-+0?!+#X9@-+"
!(O !+&_œ'œ KL(] ;!+6.(&<-8 XW(!+02/1,7:!+&] &
-8<(!"/1/-+/6N!" #*!+,N67\!"67-8dXW(!+02/1,.U!+\S*9x-+M”b&(-+ ( ?*-+U!" !+&
'•;OPR(
KT#;(-8 2)"!"67-8H'!"=+*Z*-+U!" \S*zKT*<(aXWD 02Oaj(-+U!+#;<!"=+**-+U!" $KT
#X-802/1H &ua9@!+#X67-8 ]H#!"
5 C E = V” C E =
'•;Ov<2(d#*!+,.67\1!" &(:*-+
!" h-+9 !(O !+&?œ'œ49x-+T !+6.16.(= *h-+9%67{* $ ¤ $(!+& ¤ -Z!"T(!+02L)"!+,.H(R-+9 G!"/1/1,.67&O
A-+(*'Kh-+;&*$1KT:!+&"Š H &A(a,7!"6.=#*H()+a9x-+(:(!" (6.=8o\S-+!+,.=+-+670 ”@KL67A&!+0?!"=+&
&6.!+#XL02* 6N#[•k -!"h67QKn!+Q/S-86767-8&2!"h!BWD&29@!+#X67-8?-+9RKT![E2\S*zKT*(L,.!"6.(=#*H)+
Õ
Õ
0.65
0.6
Error rate
0.55
C4.5
λ=0.1759
0.5
λ=0.3500
λ=0.5420
0.45
NN (λ=1)
0.4
150
jk67=8H(
200
250
Training set size
300
‘clm!"6.=a#*H()+R6.3 */S-8,.!" &r\S*zKT*!(O !+&Mœ'œ 9@-+R(U/5H(/S-8 '-+9V02!+H6.(=/S-[KT*O
9@-+ !(O !+&G9x-+Rœ'œ‰”@K67M!+MH&!+0?!"=+&G&16. !+#XU02* 6N#[•;OQj6.=8H( (-[Kn(U#*!+,.67\!" &M,.!"6.(=
#*H()+L9x-+L(l%* * R#X-+=8676.-8A!+ Cr9@-+R)e!";67-8Hn)"!+,.H(J-+9 O
jk67=8H( $ $V!+&
/5,7-+/-tKh*:#*H()+U9@-+U(29@-8H( !"6. 6N#*!+,k *O2PR( 2#*H()+(-tK}!"
(U#X-8 y‡)e!+,.6N&!" & J6.n0aH#M02-+/S-[KT*9bH,!+G(-+(*J(* OcsL1#X+$679(=+-8!+,6.T \Sa#X-8(B5&(3'1!"'*26.(-M&6gfS*#Xa\*“Kh*]zKT-A!+,7=+-+670?*$S((2#X-8y‡)"!+,.6.&!" & 6.
( R-+9#;(-86.#X+$5*)+A(-8H(=8p67RPhE3/SUT*-+L6.JH1!+#*#X*/!"\1,7+O
'9V(U LKL67r!+#*#X*/!"\1,7:PcEI/S'T*-+[$I W ¤ #X) L6.T(02-8J/-tKh*9@H,‡Oks-[KT*)+*$S67J6.
-+\S*6.=L -(-+ J!"k*)+2K(a(n/*9x-+0?!+#Xn-+91(J,7!"16.(=!+,.=+-+670k&67f*\IE /S*#X3!"=+
/S-86.3”b!+R6.r(lm* * R#X-+=86.67-8M!+ C•;$( !"6. 6.#*!+, R!"-8,.E?!"\1,7U -&(* #X6.n!"\S-8H(
-8(167&G-+9k(U6.02+O
Õ
Õ¤Õ
Õ
Q PR(JXWD/S*6.023QH=+=+ c!"c( W ¤ #X)2 h6.Q(L02-8 c/-tKh*9@H,(!+0-8(=Z(-8L !"6. 6N#*!+, c!"
![)+i!+#*#X*/!"\1,.?PcEI/S2*-+O?PR16.' :6.U!+,N -M(?02-8Z!"6. 9@ED6.(=($%\S#*!+H 6.!+ Z(2XfS#X
-+9k\S-+A(Z#;(-86.#X:-+9` !+6.6.=*a”@\IEMH6N(=:Z,7!"6N(=!+,7=+-+6710J-8p*)+*!+,V&16gf*oR ;!+6.6.(=
*;•J!+&M(#;(-86.#XZ-+9 L*:”@\3EM02!+H(6.(=(Z/*9x-+0?!+#X-8p *)+*!+,% *’•;O
' /16. Z(a9@!+#X'!"'˜u#*œ0!"žŸ' &(-I(-+'!+ '(:Xf#X'-+9Q)"!"ED6.(=(: !+6.6N(= **$S67
6.,N,m/S*9@-+0')+*EKh,N,‹O&(*&$%6.d!+,.,-+9T-8H(U)"!"67-8H'XWD/*;6.023*$Kh2(*)+*Z-8#X!K (2PcEI/Sa
*-+h!" '-+9S˜#*œ0!"žŸT TXWD#X*&i(L!"=+*T,7*)+,” O •;Oc?#X-83 !+ *$DKT&6.&-+\1 *)+'#*!+ cK(*
\S-+M( W ¤ #X)A!+&A&67f*1#Xy‡-+9Ny‡/1-+/S-+67-8T LKh*Z9x-I-8,7&O
PR W ¤ #X)š K6.,.,h9@!+6.,J6N™#*!+ 2KL(*M(M*-+;!" 02!+H(&ƒ6.š(M)"!"67-8H ¤ y‡9@-8,.&ƒ#X-8y
)"!+,.6.&!"67-8i*/1,N6.#*!"67-8c)e!"EiKL6.,.&1,7Er”@*)+rK(G( (*&;(*"-(J6.?*-+n!" J6.nH#;!+(=+&5•;O`v<KT*
!"\1,7Z -i-+\1*)+Z6NL6.A-802:6N0:H,N!" &p&!"!?XWI/S*6.0oLKL(*Z*-+!" Œ5H#XH!" &p\S*zKT*
¤
0.6
Xval t
Probability of Rejecting H0
0.5
0.4
0.3
5x2cv t
McNemar
0.2
Prop Test
0.1
0
0
1
2
3
4
Difference in performance between C4.5 and NN
5
j 67=8H( ‘-[KT*U-+9Q9@-8H( !"6. 6.#*!+, -8(" ?!+ COUPR((-+67{*-83!+,V!eW(6.'/1,7-+(3H10Z\S*
k
-+9c/S*#X3!"=+2/S-86.3\IEpK6.#u(zKT-A!+,7=+-+6.02” !(O !+&dœ'œ•'&67f*'KL( !+6.&u-8d ;!+6.6.(=
*L-+96.{* O
Õ
O !+& O Ocv<'&6N&?(-+h-+\1 *)+67h&1H(6.(=!+3E-+9-8H(TXWD/*;6.023c-8i!+,.6.6.#R&!"!DO`v46.,.&?)e!"6N!"67-8
*# !+H G\1!+&> 6.0?!" :-+9L(i)"!"6.!+1#X+Ou‚a6NZ(**9x-+r!+&()D6.!"\1,.? -d#(#C^(r02!+H&_*-+:;!" KL(M!"/1/1,7ED6.(=6.n *O
PR&6gfS*#XXy‡-+9xy‡/-+/S-+6.-8c hKL6N,.,(9b!+6.,56N?#*!+ nKL*L(“Kh-2,7!"16.(=Z!+,7=+-+;670T!)+)+*E
&6gfS*3 *=867-8 d-+9/-I-+/S*9x-+;0!+#Xr!+&šKL(*G(M*-+a;!" !"M#*,7-8 M - O O>v<A&6.&š(-+
#X-8H3 *a6./-+\5,70Ž6.<-8HXWI/S*6N02o-8]!+,.6N 6.#2&!"!D$*)+_-8H(=8 !(O !+&_œ'œ!"?)+*E
&6gfS*3Q!+,7=+-+6710*Ov<H/#Xc!"Q6.c6.`\S#*!+H L0-8 Q*-+;Q#X-80067 &?\3E2,7!";6.(=U!+,7=+-+6.0
!"J(!"k(n&(#*6.6.-8Z\S-8H&1!"67*Oms#X+$802-8k,.!"6.(=!+,7=+-+670?mKL6.:#X-802/5!"!"\1,7T*-+k!" `![)+
)+*EG6.06.,N!"h*=86.-8T-+9m/S-3-+n/S*9@-+0!+#X+$( -(/1!"-8,7-+=+E?!"JKT-+\1 *)+&M6Ni-8HR6.0aH,.!" &G&!"!
XWD/*;6.023T&-3n(-+J!"6.6N?/;!+#X6.#X+Ocœ-8(*(,7$I=86.)+r(UH(/S*67-+T/S*9x-+;0!+#X'-+9V˜#*œL0!"[žŸ
k!+&:(T6N#X-+#X!+H02/167-8mH1&(*,7ED6.(=n-8H()+*6.-8Z-+9T&67f*1#XXy‡-+9Ny‡/-+/-+67-8% *$+(*
#*!+A\SU(-ŠH 67B5#*!"67-8r9@-+R*)+*L02/5,7-[ED6.(=(ZH1#X-+#X &A&16gf*#XXy‡-+9Ny‡/1-+/S-+67-8n *O
PR y‡9@-8,.&u#X-8y‡)"!+,.6.&!" & '1!+6.=8APcEI/SZn*-+[ORsL-tKh*)+*$%6.!+,. -G!+L167=8A/S-[KT*$S!+&
(#X+$67L#*!+u\S#X-80021&(&u6NM(-8 a#*!+ 'KL(*ZPhEI/Zn*-+a”b6‡O›+O7$1(:9@!+6.,NH(U -i&* #X'!?!+,
&6gfS*#XU\S*“Kh*u!+,7=+-+670?;•h6.R02-+6N02/S-+!+o*O
Õ
4;
PR( !"6N(=r/-86No9x-+U6NL/1!"/S*Kn!+ ¡ H( 67-8 ‘67)+dzKT-A,7!"16.(=i!+,.=+-+670 !+1& $q!+&<!
0!+,N,&1!"!a* T$DKL6.#;G!+,.=+-+670 KL6.,N,1/-I&H1#X'02-+!+#*#*H!" #*,.!+67B1*TKL(G !+6.&i-8M&1!"!2 *
-+9m'!+067{*'!+ ^&(;!KLG9@-80 ('!+02/-+/5H,.!"67-85– (9@-+H1!" ,7E+$((-8('-+9%(U !"6. 6N#*!+,5 &(#X;67\S&M!+&A*)"!+,.H!" &6.M6NR/1!"/S*L#*!+!+ KT*16.RFIH( 67-8OJ|',.,-+9`(a !"6. 6N#*!+,% FIH67
JÕ
¤
0.8
Xval t
Probability of Rejecting H0
0.7
0.6
0.5
0.4
5x2cv t
McNemar
0.3
Prop Test
0.2
0.1
0
0
2
4
6
8
10
Difference in performance between C4.5 and NN
12
Õ
j 67=8H( ‘c-tKh*R-+99@-8H(J !"6. 6.#*!+,S J-8r(Ul%* * L#X-+=86767-8r!+C5OcPL(U(-+67{*-83!+,!eWD6Nn/1,7-+
k
(3H10Z\S*L-+9c/*;#Xo!"=+/S-86.3L\IEMKL6.#;A(azKT-M!+,7=+-+6710a” !(O !+1&uœ'œ•L&6gfS*LK(p ;!+6.(&
-8M !+6N6.(=2 *L-+96.{* O
H6N(=G-8,.&(-8H('-+!+0/1,.6.(=G02*(-D&*$KL6.u#X-8 FIH(#X2!"U(*E#*!+<-8,7Ep ,.,`HU!"\S-8H((
,.!"6.)+Z/S*9@-+0!+#X:-+9`(a,.!"6.(=!+,.=+-+670L-8p ;!+6.6.(=i *L-+9h67{* @ M @
@ @ $KL(*,@ M @6.L(
67{*U-+9k(U ;!+6.6.(=2 *L02/1,7-tE+&A6.M(!"6. 6.#*!+, *O
<!+&&676.-8u -M16.'9@H1&!+023!+,k/-+\1,70A$!+#;<-+9T( !"6. 6N#*!+,k !+U-+*(-+#X-806N(=8*O
PR(r&(*6.)e!"67-8>-+9( W ¤ #X)> FIH67!d,.!"=+MIH0Z\S*a-+96.1&(*/S&(#XG!+H02/167-8:!"!"
CD(-[KLd -M\S)D67-8,.!" &Oa˜u#*œ0!"žŸZ !+&d(2&6gfS*#XXy‡-+9xy‡/-+/S-+6.-8' &(-A(-+U02!+H2(
)"!"6.!"67-8^H,.6.(=p9@-80 (r#(-86N#XG-+9L !+6.6.=u *-+26.3 *!+,h!+&(-80?(a6._G!+,7=+-+;670!+&
(**9@-+<&(-™(-+M02!+H!+,N,-+9:(<6.02/S-+!+3G-8H(#XG-+9Z)"!"6.!"67-8%OYPR(d#X-8y‡)"!+,.6.&!" &
)D67-8,.!" J(:!+H02/167-8JH&*,7ED6.(=Z *$\S#*!+H U( ;!+6.6.(=2 *R-[)+*,.!"/mO
|'2!<#X-8 FIH(#XM-+9( M/-+\5,70*$`!+,.,n-+9(M !"6. 6.#*!+,n &(#X6.\&>6.>6.a/1!"/S*20aH \Si)D67*KT&™!+2!"//-[WD6N0!" +$h(H6. 6.#? *$h!"(*2!+™!+6.=+-+-8H,7E]#X-+#X? !"6.6.#*!+,n02*(-D&*O
PR6NU/1!"/S*:!+:(**9@-+i,.67&_-8_XWD/S*6.023!+,Q*)"!+,.H!"67-81-+9J( r02*(-D&*$k!+&_(?9@-8,.,7-tKL6.(=
#X-8#*,.H167-8h0:Hc\SL*=8!";&(&r!+c 3!"67)++$\S#*!+H ('XWI/S*6.0oh!"\5!+ &G-8?-8,.E2zKT-,.!"6.(=
!+,7=+-+6.0J!+&M*Z&!"!**O
PRMXWI/S*6.0o6N™6./1!"/S*?,.!+&ƒH2 -^#X-8002&ƒ67(*? W ¤ #X)
*$J9@-+?67H1!"67-8
6.™K6.#ƒ(u,7!";6.(=]!+,7=+-+;670!"p i#*673?-8H(=8ƒ -^H™ 6.02*$n-+G˜u#*œ0!"žŸG *$J9@-+
67H1!"67-8nKL(*U(Z,7!"16.(=!+,7=+-+670?R#*!+M\SUHr-8,.E?-8#X+
O h-+p L1!)+:6.06.,.!"T/S-[KT*O
HXWI/S*6N02oU1!)+i!+,N -A*)+!+,7&_(?-+#X-806.(=8-+9T-+(*:!"6. 6.#*!+,` *$-AKhG#*!+
#X-8DB5&o,7EM#X-8#*,NH&(Z1!"L(:!+02/1,.& (-8H,N&A(*)+*'\Z02/1,7-tE+&OLPR6NJ !+L)+*EA67=8
/-+\5!"\16.,.67“EU-+9PhEI/nV*-+$o!+1&H,7-+\!+6N(&2H6.=6NV c#*!+(-+`\Sn H &%OPRTXWD/S*6.023
!+,. -2H=+=+ R#*!+H(67-8M6.r6.3 */*6.(=a(H,7T-+9( y‡9x-8,N&M#X-8y‡)"!+,.6.&!" & *OcPR16.T R!+J!
,7*)"!" &</-+\5!"\16.,.67“Er-+9TPhE3/SaL*-+?”b!+U0:H#;<!+UzK6.#X(!"=+*Z,7*)+,x•;$%!+,.(-8H(=8<676.(-+(!",.E
Õ
¤+¤
0.18
Xval t
0.16
Probability of Rejecting H0
0.14
0.12
0.1
0.08
5x2cv t
0.06
McNemar
Prop Test
0.04
0.02
0
0
0.5
1
1.5
2
2.5
3
Difference in performance between C4.5 and NN
3.5
Õ¤Õ
j 67=8H(
k
‘R-[KT*'-+9`9@-8H( !"6. 6N#*!+,m -8u(ah6.0!2!+C5O'PR(Z(-+;67{*-8o!+,!eW(6.R/1,.-+L(a3H10Z\S*
-+9c/S*#X3!"=+2/S-86.3\IEpK6.#u(zKT-A!+,7=+-+6.02” !(O !+&dœ'œ•'&67f*'KL( !+6.&u-8d ;!+6.6.(=
*L-+96.{* O
!+L*)+*Z!+RU/-+\1,70K67r(!+02/1,7& *O
v<2(-+/S!"'(aH,.L6.p6.R/5!"/*KL6.,.,%\:H *9bH,% -r#*6736. '6.M(0!+#;6.(a,7!"6.=?!+&
(H(;!+,8(*zKT-+C#X-800aH6.67%!+V(*EU&(*)+,7-+/%$+H&* !+&$t!+&Z6.0/-[)+h0!+#6N(c,7!"16.(=R!+,7=+-+6.0*O
Z A
Q
:
c' PR(R!+H(-+`B1Q,7!"(&2-+9S( .3y‡9@-8,.&#X-8y‡)"!+,.6.&!" & c9x-80 R-83E U-8!)D6m”@/S* -81!+,#X-800aH6gy
#*!"67-85•;OQPL(!+H(-+h!+CI
86.0 U-8,. CIE-+9(UZ¢ ¢I!"6. 6.#* '*/1!"023T9x-+n !"6. 6N#*!+,1#X-8H1,76.(=
!+6N !+#X?!+& L!+&(9@-+&]œ!+,‹
$ R-83E -8!)D6‹$V!+1&]Pm-80ˆ˜u6.#(,.,Q9x-+a(,7/9bH,kH(=+=+ 6.-8*O?PR?!+HDy
(-+26.a!+,. -u)+*E_=+!" *9bH,h -d(i*9x**a9@-+(67#*!"*9@H1,c!+&16.(=($kKL6.#;_6.&o6gB5&_ *)+*!+,T*-+
6.d!+<!",N67*U)+*67-8d-+9T6.0!+IH#X;67/*OPR6.' !";#<Kn!+H(/1/-+ &u\IEp=+;!+oZ Lzy +¤ ! ¤ !+&
R“y ¤ !M9x-80q(œ!"6.-8!+,k¢(#*67#Xj(-8H&!"6.-8u!+&u=+!+3'IH0:\S*œ !"y y y A9@-80q(
 i#XZ-+9œ'!)"!+, L !"#%O
Õ
Õ
Õ
.
c
c T6.0!+%$1lQO7$1j67&0!+%$
(O1s:O7$SU,.($ Z O1|:O7$ ‰¢D -8(+$ 'O(OV” Õ !3•;O FAF -A !#" " (%' &;( FAF #"
-8H(/mO
&(A(F’Ocv]!+&Kh-+p‚3 *!"6.-8!+, '
T6.0!+%$SlcOQ” Õ 3! •;OsH(6N 6.#*-+9T6.1 !"\16.,.6.zEM!+&d !"\16.,N67{!"67-86.d02-I&,,7#X67-8OZPV#%O*/%O ! Õ I$
'*/1!"02o-+9`¢D!"6. 6.#**$ 6.)+*67“E?-+9n !+,.679@-+6N!D$ T *C+,7*E+$ n |:O
¤
T6.0!+%$lQOm” Õ 8•;O T!"=+=86N(=?/&6.#X -+*OG -B<" (K(#&"=<" '+$T” ¤ •;$ Õ¤
Õ ! O
!+!"!"3E+$ '
O:O” h &O •;O” ÕJÕ •;O (#&(F " (*'&U"#&C F1! (*&"-* FF -A !#" (A-B"=* ( F;O
T
-80/1H( *'¢I-D#*67*“EGc*$lm-8L|',.!+067 -8*$ J|ZO
67* *6.#;$`PUO O7$Qs6.,N&$Vs:O7$ n !"CD676‹$ a OJ” Õ •;O| #X-802/5!"6. -8^-+9J !+&]\5!+#CI/-+/1!"=8!"67-8>9@-+
(=8,.6.i XWDy‡ -"yz /S*#d0!"/1/16.(=(O B-B<" ( (A#&"=>")'+$+$ J Õ
O
h
Q9@-8$ O7$ Y
PR6.\167;!+6‹$ ZO
(O” Õ •;O"!" &; #=-. !#"G%W(#"D <F &;5On !"/10!+M!+1&ps'!+,.,‹$1œ*K
$Q-+C5$1œ%$O
Q)+*;67 *$ 'O¢Om” Õ +•;O (#"'& F>F + -A#" <" ')( "-(& %)* ( F;On1!"/10!+M!+&As'!+,.,‹$lm-8&(-8O
j(*E+$'OTv O7$
¢(,.!" +$ aO (O'” ÕJÕ •;O‰l%* *r#X-+=8676.-8„H16.(=_s-8,.,.!+1&Dyz zED,7!+&!"/67)+d#*,.!+67B1**O
G - >" (K(#&"=<" '+$+*e$ Õ Õ
Õ+¤ O
s'!+H,7*[$ aO7$U
!"1*$(˜_O7$5¢IH1(=($Is:O5¢O7$ PR6.3\IE+$Iœ:O%” Õ 3! •;OL67=+-+-8Hn,7!"6.=:#*H()+\S-8H&1Q9@-80
!"6. 6.#*!+,%0#!+6N#**O`‚!, & --/..0" "=1-2 3 &54 F5) " #CD 6- (&"=<" ' (A#&7&e$5//%O
3OS| n ˜Q*$œ*K8$Q-+C$1œ%$aO
s'6.o -8%$ a
O R O7$VœL!+,‹$ O%˜^O7$VPR67\567!+16‹$ ZO7$ #c l# !+0ˆ020:\*;” Õ •;O?| 6.=r,.!"6.(=
/1-I#X&1H(%H6.(=#c
l#R O+PV#Ot*/%O7$167)+*67“E-+9DPV-+-8o -($ '*/1!"0oV-+9n -802/1H( *¢(#*67#X+$
3 /%‘ t KRKLKOŸ#**OŸH( -+-83 -(OŸ#*! e (H(-8 e &(,7)+ e &(,.)++OŸo0,‡O
U-8![)I6‡$ OQ” Õ •;O939& ( &F:+*#&D ( &<+*#&CR#" -((*":)#"-( C ( "D#"?)* ; !#)F
6.*$5¢I!+9x-+& 6.)+*67“E+O
(-*>F!#" '&;< F’O2cO aO
U-8,7%$ OejTO7$ V-8,N,.!+#C$ (O 'OD” ÕJÕ •;O T!+#C:/1-+/1!"=8!"67-86. 6.67)+h -6.6.6.!+,o#X-8&16767-8*O3‚= >; #" -(F
<"? (*)&;#".+*#&CR "?, & -(FF<" 'A@<& F (*C F;$ `-8,‡O $%//%O B
¢D!+<j(!+#*6N#X-($ n|:O˜p-+=8!+
!+H(9@0?!+O
Z
U-8(=($ R O ' O7$ 67* *6.#;$DPUO aO5” Õ •;O Q-+yz#X-+#X6.(=Z-8H( /1H(c#X-D&6.=#X-+#Xh\16.!+Q!+&?)e!";6.!+#X+O
Q;67&676.$R|ZO7$ H ,N,‹$'¢O” c
&1*O •;$ ( (* + C" ( &" !#"# #" +*(*&;(*"-( #"D -EB>" (
K(#&"=<" '+$5//%O J Õ + ¤JÕ ¢D!+Aj(;!+#*6.#X-($ J|ZO1˜u-+=8!+!+H(9b0!+O
lm6N&0!+$IsZO ZO” Õ+ ¤ •;O"#F& F>F ;+HG #&!#"-( >"JI@ ( &>C (*" %#LK(F' "Ok¢D/6.(=+* `*,.!"=($œ*KM$Q-+CO
˜u*{+$ O O7$ ˜H(/5oE+$„VO]˜^Oƒ” Õ 8 •;O T */-867 -+EŽ-+9„0!+#6N( ,7!"6.=q&!"!"\1!+*O
N:OPOQSRUTPTVWVPVX5YPZP[\X]ZPY\XU^)_]`T1ab`cW^Wd)ef`TgPh)i:^Q:j<[)Y>OPj)ePklXNOb`c O
¡ H6N,.!+$ O ZOS” •;O <-m/, &;'&;#C F+*#&%I CD<&!-A#<K
(#&"=<" ')k- ˜u-+=8!+ !+H(9b0!+%$D¢D!+Gj(!+#*6N#X-($
J|ZO
Õ
L!+0:H $'OROZ” Õ 8•;OnI;## = !#" +9o#)FAF #" &;.-( FAF.(FG#"O(*& CR(.) FT+*#& "#"<p >" (#&
&(%' &;( FAF #"1OhO a
OS(6.*$ '67)+*67zEA-+9TPm-+-8o -($ */5!"023U-+9 T-802/5H( *Z¢D#*67#X+$%PV-+-83 -($
J!+!+&!DO
¢D(&#X-+$
O`v€O7$ T-I#;(!+$hv€O aO” Õ •;O8@
|'02*$1‚|:Oc
6.=8o h&6767-8O
¤
% F -#l (*)#F;Oƒ‚-[Kn!>¢I!" 6.)+*67“E^Q*$
!