Dynamic Programming

'\QDPLF 3URJUDPPLQJ
$VVHW 3ULFHV LQ DQ ([FKDQJH (FRQRP\
)DUKDQJ )DUD]PDQG
6SULQJ &RQWHQWV
3UHIDFH
,QWURGXFWLRQ
7KH 0RGHO
(TXLOLEULXP /LQNV (ODERUDWLQJ RQ WKH (TXLOLEULXP
([DPLQLQJ WKH DVVXPSWLRQV 7KH 3ULFH )XQFWLRQ &RQFOXVLRQ
5HIHUHQFHV
L
3UHIDFH
7KH SXUSRVH RI WKLV SDSHU LV WR HPSOR\ VRPH RI WKH WRROV RI G\QDPLF SUR
JUDPPLQJ WR VROYH D VSHFLnF WKHRUHWLFDO SUREOHP +HQFH DV WKH WLWOH LQGLFDWHV
WKLV SDSHU ZLOO DGGUHVV WKH LVVXH RI DVVHW SULFHV LQ DQ H[FKDQJH HFRQRP\ DV
GRQH LQ /XFDV +RZHYHU LW VKRXOG EH PHQWLRQHG WKDW RQO\ SDUWV RI WKH
DUWLFOH >@ ZLOO EH H[SORUHG 7KURXJKRXW WKH SDSHU DWWHPSWV ZLOO EH PDGH WR
OLQN WKH WKHRUHPV DQG UHVXOWV RI WKH DUWLFOH >@ WR WKRVH RI WKH ERRN
V >@
,QWURGXFWLRQ
:H H[DPLQH D SXUH H[FKDQJH HFRQRP\ ZLWK LGHQWLFDO FRQVXPHUV DQG RQH
VLQJOH JRRG 7KH VLQJOH JRRG LV SURGXFHG ZLWKRXW FRVWV LQ D QXPEHU RI
GLmHUHQW SURGXFWLYH XQLWV 7KH DVVHWV LQ WKLV HFRQRP\ DUH FODLPV WR DOO RU
SDUW RI WKH SURGXFWLYH XQLWV :H DVVXPH WKDW SURGXFWLYLW\ LQ HDFK XQLW
oXFWXDWHV VWRFKDVWLFDOO\ WKURXJK WLPH DQG WKHUHIRUH HTXLOLEULXP SULFHV ZLOO
oXFWXDWH WKURXJK WLPH 7KH PDLQ LGHD RI WKLV SDSHU LV WR XQGHUVWDQG WKH
UHODWLRQVKLS EHWZHHQ WKH H[RJHQRXV PRYHPHQWV UHVXOWLQJ IURP oXFWXDWLRQV
LQ SURGXFWLYLW\ DQG PDUNHW GHWHUPLQHG PRYHPHQWV LQ DVVHW SULFHV
7KH 0RGHO
:H FRQVLGHU DQ HFRQRP\ ZLWK D UHSUHVHQWDWLYH FRQVXPHU ZLWK UDWLRQDO H[
SHFWDWLRQV 7KH FRQVXPHU
V REMHFWLYH LV WR PD[LPL]H KLV H[SHFWHG GLVFRXQWHG
OLIHWLPH XWLOLW\ RI WKH VLQJOH FRQVXPHU JRRG LH
;
(
n W 8 FW ZKHUH n @ > DQG 8 # # LV FRQWLQXRXVO\ GLmHUHQWLDEOH ERXQGHG
W
LQFUHDVLQJ DQG VWULFWO\ FRQFDYH ZLWK 8 7KH FRQVXPSWLRQ RI WKH VLQJOH JRRG FW LV UHJDUGHG DV D VWRFKDVWLF SURFHVV
ZLWK WKH JRRG LWVHOI EHLQJ SURGXFHG RQ Q GLmHUHQW SURGXFWLYH XQLWV <W
<W <QW LV WKH RXWSXW YHFWRU LQ SHULRG W ZLWK <LW GHQRWLQJ WKH RXWSXW RI
XQLW L 7KHUHIRUH IHDVLEOH FRQVXPSWLRQ OHYHOV PXVW VDWLVI\
Q
;
› FW ›
<LW
L
DW DOO WLPHV $V PHQWLRQHG HDUOLHU SURGXFWLRQ LV H[RJHQRXVO\ JLYHQ DQG ZH
ZLOO DVVXPH WKDW WKH PRWLRQ DI <W LV JLYHQ E\ D PDUNRY SURFHVV ZLWK WKH
IROORZLQJ WUDQVLWLRQ IXQFWLRQ
) < < 3 I<W › < M<W
<J
ZKHUH ) ( Q d ( Q # LV FRQWLQXRXV ZLWK ) < :H IXUWKHU
DVVXPH WKDW ) KDV D VWDWLRQDU\ GLVWULEXWLRQ €c DQG WKDW IRU DQ\ FRQWLQXRXV
( Q EHLQJ D FRPSDFW DQG FRQYH[ VXEVHW RI #Q
IXQFWLRQ J< =
J< G) < < LV D FRQWLQXRXV IXQFWLRQ RI \ 1RWLFH WKDW LI J< KDG EHHQ D ERXQGHG FRQ
WLQXRXV IXQFWLRQ VR ZRXOG 7KDW LV ) SRVVHVVHV WKH )HOOHU SURSHUW\
7KH RZQHUVKLS LQ WKH SURGXFWLYH XQLWV LV GHWHUPLQHG HDFK SHULRG LQ D FRP
SHWHWLYH VWRFN PDUNHW ZKHUH HDFK XQLW KDV RXWVWDQGLQJ RQH XQLW RI SHUIHFWO\
GLYLVLEOH HTXLW\ VKDUH 7KH RZQHU RI D VKDUH DV RI WKH EHJLQQLJ RI SHULRG W
LV HQWLWOHG WR DOO WKH XQLW
V RXWSXW LQ WKDW SHULRG 7KH SULFHV RI WKH VKDUHV
DUH JLYHQ H[GLYLGHQG ZLWK SW SW SQW GHQRWLQJ WKH SULFH YHFWRU 7KH
FRQVXPHU
V EHJLQQLQJRISHULRG VKDUH KROGLQJV SRUWIROLR ZLOO EH GHQRWHG E\
WKH YHFWRU ]W ]W ]QW :H REVHUYH GXH WR WKH PDUNRY SURSHUW\ WKDW
DOO LQIRUPDWLRQ RQ WKH FXUUHQW DQG IXWXUH SK\VLFDO VWDWH RI WKH HFRQRP\ LV
DYDLODEOH LQ WKH FXUUUHQW < 7KH SULFH YHFWRU ZLOO WKHUHIRUH EH D IXQFWLRQ RI
<W )XUWKHUPRUH FW F]W <W SW DQG ]W J]W <W SW J EHLQJ D IXQFWLRQ
LH WKH FRQVXPHU
V FKRLFH RI FRQVXPSWLRQ DQG SRUWIROLR GHSHQG RQ FXUUHQW
YDOXHV RI ]W SW DQG <W (TXLOLEULXP
7KH IDFW WKDW 8 LV VWULFWO\ FRQFDYH DQG LQFUHDVLQJ LPSOLHV WKH H[LVWHQFH RI
VWURQJ PRQRWRQH SUHIHUHQFHV $V D UHVXOW LQ HTXLOLEULXP ZH ZRXOG KDYH
FW
Q
;
L
<LW
DQG ]W
7KH IRUPDO IRUPXODWLRQ RI RXU HTXLOLEULXP ZLOO EH DV IROORZV
7KH HTXLOLEULXP ZH ZLOO FRQVLGHU ZLOO EH D SDLU RI IXQFWLRQV 3 < DQG DQ
RSWLPDO YDOXH IXQFWLRQ y] < 7KH ODWWHU GHQRWHV WKH YDOXH RI WKH FRQVXPHU
V
REMHFWLYH IXQFWLRQ DV JLYHQ LQ ZKHQ KH EHJLQV LQ VWDWH < ZLWK D SRUWIROLR
]
'HnQLWLRQ $Q HTXLOLEULXP LV D FRQWLQXRXV IXQFWLRQ S< ( Q ( Q DQG D FRQWLQXRXV
ERXQGHG IXQFWLRQ y] < ( Q d ( Q # VXFK WKDW
) LV D GLVWULEXWLRQ IXQFWLRQ
WKH <LW V DUH GLYLGHQG SURFHVVHV
6LQFH
LH
L
y] < k
j
5
PD[ 8F n y[ < G) < < F[
VXEMHFW WR
F S< c [ › < c ] S< c ]
F w ›[›]
ZKHUH ] LV D YHFWRU ZLWK FRPSRQHQWV H[FHHGLQJ RQH
LL
IRU HDFK < y < LV DWWDLQHG E\ F
3
L
<L DQG [
7KH LQWHUSUHWDWLRQ LV DV JLYHQ LQ >@ *LYHQ WKH EHKDYLRXU RI SULFHV WKH
FRQVXPHU DOORFDWHV KLV EHJLQQLJRISHULRG UHVRXUFHV RSWLPDOO\ DPRQJ FXUUHQW
FRQVXPSWLRQ F DQG HQGRISHULRG SRUWIROLR [ 6LQFH › [ › WKH FRQVWUDLQW
[ › ] ZLOO EH QRQELQGLQJ +RZHYHU LW ZLOO SURYLGH XV ZLWK D FRPSDFW VHW WR
ZRUN RQ &RQVWUDLQW LL LV HYLGHQWO\ D PDUNHW FOHDULQJ FRQVWUDLQW 7KH LGHD
LV WR VWXG\ WKH PD[LPL]DWLRQ SUREOHP VWDWHG LQ IRU JLYHQ SULFH EHKDYLRXU
S< /LQNV
%HIRUH SURFHHGLQJ ZH ZLOO DWWHPSW WR OLQN WKH QRWDLRQ DQG WKHRU\ SUHVHQWHG
VR IDU WR WKDW RI WKH ERRN
V >@
8VLQJ WKH QRWDWLRQ LQ >@ WKH JLYHQV RI D VWRFKDVWLF G\QDPLF SURJUDPPLQJ
SUREOHP DUH ; ; = = 4 b ) DQG n 7KH PHDVXUDEOH VSDFHV DUH WKRVH
IRU WKH HQGRJHQRXV DQG H[RJHQRXV YDULDEOHV UHVSHFWLYHO\ 'XH WR PRQRWRQH
SUHIHUHQFHV F LV GHPRWHG WR D UHVLGXDO JLYHQ E\
F
< c ] S< c ] b S< c [
WKHUHIRUH RXU HQGRJHQRXV YDULDEOH LV ]W ZLWK > ]@Q Q M>] @3 DV LWV PHDVXU
DEOH VSDFH 7KH H[RJHQRXV YDULDEOH LV FOHDUO\ < ZLWK WKH PHDVXUDEOH VSDFH
( Q Q M( 3 +HQFH 6 $ > ]@Q d ( Q Q M>]@3 l Q M( 3 7KH WUDQ
VLWLRQ IXQFWLRQ DQG WKH GLVFRXQW IDFWRU KDYH IRUWXQDWHO\ WKH VDPH QRWDWLRQ
DV LQ >@ 7KH RQH SHULRG UHWXUQ IXQFWLRQ ) LV REYLRXVO\ WKH XWLOLW\ IXQF
WLRQ 8 :LWK WKHVH QRWDWLRQV LQ SODFH ZH FDQ GHnQH RXU FRUUHVSRQGDQFH
b > ]@Q d ( Q > ]@Q DV VHH WKH FRQVWUDLQW LQ L
b] < IZ > ]@Q S< c Z › < S< c ]J
ZKHUH ZH FOHDUO\ KDYH XVHG WKH IDFW WKDW FW LV UHVLGXDOO\ JLYHQ E\ WKH EXGJHW
FRQVWUDLQW 7KH JUDSK RI b LV GHnQHG DV
$
I] Z < > ]@Q d > ]@Q d ( Q Z b] < J
,W VKRXOG EH FOHDU IURP WKH DVVXPSWLRQV PDGH VR IDU WKDW 8 $ # +DYLQJ HVWDEOLVKHG WKH QRWDWLRQDO OLQN ZH UHWXUQ WR RXU H[DPLQDWLRQ RI WKH
HTXLOLEULXP
(ODERUDWLQJ RQ WKH (TXLOLEULXP
&OHDUO\ L EHDUV JUHDW UHVHPEODQFH WR D IXQFWLRQDO HTXDWLRQ )( RI D VWRFKDV
WLF G\QDPLF SURJUDPPLQJ 7KDW
V EHFDXVH GXH WR L LV HTXLYDOHQW WR
L
y] < VXS
[b]< k
j
5
8F n y[ < G) < < F
3
L <L
ZKLFK HYLGHQWO\ LV DQ )( %HIRUH VWDWLQJ WKH FRUUHVSRQGLQJ VHTXHQWLDO SURE
OHP 63 ZH ZLOO H[DPLQH WKH YDOLGLW\ RI VRPH RI WKH UHOHYDQW DVVXPSWLRQV LQ
>@ UHJDUGLQJ WKH H[LVWHQFH RI 63 &OHDUO\ $ $ DQG $ DUH YDOLG
7R VHH WKDW $ LV DOVR YDOLG ZH PXVW VKRZ WKDW b LV QRQHPSW\ FRPSDFW
YDOXHG DQG FRQWLQXRXV 7KLV LV HDVLO\ GRQH VLQFH LW LV FOHDU WKDW b IRU DOO
YDOXHV RI ] < ,W
V DOVR FOHDU WKDW b LV ERXQGHG b > ]@Q 7KH IDFW
WKDW b LV FORVHG DQG FRQWLQXRXV VWHPV IURP WKH IDFW WKDW WKH ›
FRQVWUDLQW
WKH YDOXHV LQ b PXVW IXOnO LV GHnQHG E\ D FRQWLQXRXV IXQFWLRQ *LYHQ WKHVH
IRXU DVVXPSWLRQV OHPPD >@ WHOOV XV WKDW WKH VHW RI IHDVLEOH SODQV g LV
QRQHPSW\ DQG WKHUHIRUH 63 LV ZHOO GHnQHG DQG JLYHQ E\
;=
b
c
b
c
VXS 8 F n W 8 {W> < W> {W < W <W xW < G< W
3
9
{gV W ( :KHUH < W ( Q W s ( Q d
c c d` ( Q DQG xW c ( Q W # LV GHnQHG DV
_ c^]
W WLPHV
LQ FKDSWHU LQ >@ )XUWKHUPRUH VLQFH WKH IXQFWLRQ y LQ )( LV ERXQGHG
DQG xW LV D SUREDELOLW\ PHDVXUH RQ ( Q W ZH KDYH WKDW
=
b
c b
c
OLP
n W y {W> < W> <W xW < G< W
W
( 3 9
IRU DOO { gV V 6 7KXV WKHRUHP >@ WHOOV XV WKDW WKH YDOXH RI WKH
63 DQG WKH VROXWLRQ WR )( FRLQFLGH 7KLV H[SDOLQV ZK\ ZH IURP QRZ RQ
VROHO\ FRQFHQWUDWH RQ WKH SURSHUWLHV RI WKH IXQFWLRQ y
:H ZLOO QRZ VKRZ KRZ SURSRVLWLRQV DQG LQ /XFDV
>@ FRUUHVSRQG
WR WKH WKHRUHPV LQ WKH ERRN >@ )LUVW ZH ZLOO H[DPLQH WKH YDOLGLW\ RI WKH
UHOHYDQW DVVXPSWLRQV
([DPLQLQJ WKH DVVXPSWLRQV
$ ,W LV REYLRXV WKDW F]W <W SW PXVW EH D VWULFWO\ LQFUHDVLQJ IXQFWLRQ RI ]W 7KHUHIRUH 8 FW 8F]W <W SW LV DOVR VWULFWO\ LQFUHDVLQJ LQ ]W $ )URP
S< c Z › < S< c ]
LW LV FOHDU WKDW bc < > ]@Q > ]@Q LV LQFUHDVLQJ
$ 8 LV DVVXPHG VWULFWO\ FRQFDYH VHH VHFWLRQ $ /HW ] ] > ]@Q < ( Q DQG t > @ 6XSSRVH WKDW Z b] < DQG Z b] < DQG FRQVLGHU tZ b tZ S< c tZ b tZ tS< Z b tS< Z
› t< S< ] b t< S< ] < S< c t] b t] ZKLFK VKRZV WKDW tZ b tZ bt] b t] < $ 8 LV DVVXPHG FRQWLQXRXVO\ GLmHUHQWLDEOH VHH VHFWLRQ 7KH DVVXPSWLRQV $ WKURXJK $ HQDEOH XV WR HPSOR\ WKHRUHPV DQG ZKRVH UHVXOWV FRUUHVSRQG ODUJHO\ WR WKRVH RI SURSRVLWLRQ DQG LQ >@ IRU JLYHQ S< LH
5HVXOW 7KHUH LV D XQLTXH ERXQGHG FRQWLQXRXV QRQQHJDWLYH IXQFWLRQ
y] \ S VDWLVI\LQJ L ] yc \ S LV DQ VWULFWO\ LQFUHDVLQJ DQG VWULFWO\
FRQFDYH IXQFWLRQ ZKLFK LV FRQWLQXRXVO\ GLmHUHQWLDEOH LQ ,QW> ]@Q $Q LQFUHDVLQJ IXQFWLRQ ZKLFK DOVR LV VWULFWO\ FRQFDYH PXVW EH VWULFWO\ LQFUHDVLQJ
IROORZV WULYLDOO\ IURP L
7KH 3ULFH )XQFWLRQ
5HPDUN 6LQFH WKH XWLOLW\ RI WKH FRQVXPHU LV LQFUHDVLQJ WKH FRQVWUDLQW
LQ L ZLOO EH ELQGLQJ LQ HTXLOLEULXP
6R
F
>] b [@ c S< < c ]
LH
#y] < S
#]L
8 F c ><L SL < @
L
Q
*LYHQ WKH SURSHUWLHV RI y ZH ZLOO FRQFHQWUDWH RQ WKH SUREOHP VWDWHG LQ L
7KH IRF
V ZUW [L DUH
u
t=
#
8 F c SL < b n
y[ < G) < < #[L
=
#y[ < 8 F c SL < n
G) < < L Q
#[L
ZLWK
F S< c [
< c ] S< c ]
F [ ! $FFRUGLQJ WR SURSRVLWLRQ LQ >@ LI QH[W SHULRG
V RSWLPXP FRQVXPSWLRQ F
LV REWDLQHG DW D SRVLWLYH YDOXH WKHQ
#y[ < #[L
8 F c ><L SL < @
L
Q
3
3
5HFDOOLQJ WKDW LQ HTXLOLEULXP ] [ F
<L < c ] DQG F
<L L
L
JLYHV XV
‚
‚
=
;
;
8
<M c SL < n 8
<M ><L SL < @G) < < L Q L
L
\ y c \ LV FRQWLQXRXVO\ GLmHUHQWLDEOH#DQG ERXQGHG :H FDQ FKRRVH 0 # VW
y [ \M › 0 IRU DOO [ 7KHQ ZH FOHDUO\ KDYH 0G) \ \ 0 7KLV HVWDEOLVKHV
WKH OHJLWLPDF\ RI GLmHUHQWLDWLQJ XQGHU WKH LQWHJUDO
JLYHQ
M
5HPDUN 6LQFH GRHVQ
W LQYROYH y] \ S $Q\ SULFH IXQFWLRQ Sa\
VROYLQJ ZLOO WRJHWKHU ZLWK y] \ Sa FRQVWLWXWH DQ HTXLOLEULXP
1RZ ZH GHnQH
‚
=
JL < n
8
;
<M
c <L G) < < L
Q
M
DQG
=
I < JL < n
I < G) < < L
Q
,W LV REYLRXV WKDW JLYHQ VROXWLRQV I < IQ < WR WKH SULFH IXQFWLRQ
SL < I < rL3 s
8
<M
M
L
Q
ZLOO VROYH DQG WKHUHIRUH EH DQ HTXLOLEULXP SULFH IXQFWLRQ 7KLV HVWDEOLVKHV
WKH UHODWLRQVKLS EHWZHHQ WKH HTXLOEULXP SULFH IXQFWLRQ DQG WKH HTXLOLEULXP
YDOXH IXQFWLRQ RI WKH FRQVXPHU
V SUREOHP
$ GHVLUDEOH SURSHUW\ RI WKH HTXLOLEULXP SULFH IXQFWLRQ LV JLYHQ LQ SURSRVLWLRQ
LQ >@ ZKLFK VWDWHV WKDW
5HVXOW 7KHUH LV H[DFWO\ RQH FRQWLQXRXV ERXQGHG IXQFWLRQ RQ ( Q VDWLV
I\LQJ ,H WKH HTXLOLEULXP SULFH IXQFWLRQ H[LVWV DQG LW LV XQLTXHO\ GHWHU
PLQHG
,Q RUGHU WR VHH WKLV FRQVLGHU WKH IROORZLQJ
6LQFH 8 LV FRQFDYH ZH KDYH WKH IROORZLQJ LQHTXDOLW\ IRU DQ\ JLYHQ F #
8a
F › 8 F 8 F c a
F b F
OHWWLQJ Fa
a
F
JLYHV XV
8 › 8 F 8 F c bF
VLQFH 8 LV ERXQGHG 8 F › . ZH JHW
8 F c F › 8 F › .
7RJHWKHU ZLWK WKH )HOOHU SURSHUW\ WKLV HVWDEOLVKHV WKH FRQWLQXLW\ DQG ERXQG
QHVV RI WKH JL IXQFWLRQV
1RZ LI I &E LV DQ\ FRQWLQXRXV ERXQGHG DQG QRQQHJDWLYH IXQFWLRQ RQ ( Q
WKHQ VR LV
=
n I < G) < < E\ OHPPD >@ 7KDW LV WKH IXQFWLRQ 7L I ( Q # JLYHQ E\
=
7L I< JL < n I < G) < < LV D PDSSLQJ IURP &E LQWR LWVHOI
7DNLQJ Ic DQG Kc IURP &E ZLWK I c › Kc ZH HDVLO\ VHH WKDW 7L I › 7L K
0RUHRYHU
=
7L I D< JL < n I < DG) < < =
JL < n I < G) < < n c D
7L I < n c D
I &E DQG D w %\ %ODFNZHOO
V VXpFLHQW WKHRUHP 7L I LV D FRQWUDF
WLRQ )LQDOO\ WKH UHVXOW LV JLYHQ DV D FRQVHTXHQFH RI WKH FRQWUDFWLRQ PDSSLQJ
WKHRUHP
&RQFOXVLRQ
,Q FORVLQJ LW LV ZRUWKZKLOH WR PHQWLRQ WKDW E\ DSSO\LQJ WKH WRROV RI G\QDPLF
SURJUDPPLQJ ZH KDYH ODUJHO\ DUULYHG DW WKH VDPH FRQFOXVLRQV DV LQ WKH nUVW
VHFWLRQV RI >@ +RSHIXOO\ QR LQMXVWLFH ZDV PDGH WR WKH UHVXOWV RI WKH RULJLQDO
DUWLFOH
5HIHUHQFHV
>@ /XFDV 5REHUW ( -U $VVHW SULFHV LQ DQ ([FKDQJH (FRQRP\
(FRQR
PHWULFD 9ROXPH ,VVXH 1RY b >@ /6WRNH\ 1DQF\ $QG /XFDV 5REHUW ( -U $QG 3UHVFRWW (GZDUG &
5HFXUVLYH 0HWKRGV LQ (FRQRPLF '\QDPLFV
+DUYDUG 8QLYHUVLW\ 3UHVV
nIWK SULQW