Ewa Kubicka http://www.math.louisville.edu/~ewa/
Course MLC/MFE seminars: http://www.math.ilstu.edu/actuary/prepcourses.html
Course MLC Casualty/Property Manual: http://www.neasseminars.com/registration/
Practice problem for Exam MLC for the week after 06/12/10
Suppose that whether or not it rains today depends on previous weather conditions
through the last two days.
Specifically suppose that if it has rained for the past two days, then it will rain tomorrow
with probability 0.7; if it rained today but not yesterday, then it will rain tomorrow with
probability 0.5; if it rained yesterday but not today, then it will rain tomorrow with
probability 0.4; if it has not rained in the past two days, then it will rain tomorrow with
probability 0.2.
Determine the states so that it would be a Markov Chain problem and find the transition
matrix.
1
Solution
We can transform the model into a Markov Chain by saying that the state at any time is
determined by the weather conditions during both that day and the previous day. Then,
we say that the process is in:
state 0
state 1
state 2
state 3
if it rained both today and yesterday
if it rained today but not yesterday
if it rained yesterday but not today
if it did not rain either yesterday or today
present
future
 
€
yesterday
today
tomorrow
r 

 r 

r
0
0
r 

 r 

 nr
0.3
nr 

 r 

r
0.5
nr 

 r 

 nr
0.5
r 

 nr 

r
0.4
r 

 nr 

 nr
0.6
nr 

 nr 

r
0.2
nr 

 nr 

 nr
0.8
0
€
€
€
€
€
€
€
€
prob.
0.7
2
1
1
2
0
2
1
2
3
3
3
1
3
Thus, the four-state Markov Chain can be represented by the transition probability
matrix:
0.7 0 0.3 0 
0.5 0 0.5 0 

P=
 0 0.4 0 0.6 


 0 0.2 0 0.8 
2