Leslie matrices and Markov chains. Example. Suppose a certain species of insect can be divided into 2 classes, eggs and adults. 10% of eggs survive for 1 week to become adults, each adult yields an average of 100 eggs per week, and 60% of adults survive an additional week. If a population of these insects currently consists of 100 eggs and 50 adults, find the “state” of the population one week from now, two weeks from now, and three weeks from now. Solution (by hand). .6(50) = 30 adults will survive, and .10(100) = 10 eggs will survive to adulthood, giving a total of 40 adults. The 50 adults will produce 50(100) = 5000 eggs. Better solution (using matrices). 0 100
100
Let ,
. 10 . 60
50
> L = matrix(c(0,100,.10,.60), nrow=2, ncol=2, byrow=TRUE) > L [,1] [,2] [1,] 0.0 100.0 [2,] 0.1 0.6 > x = matrix(c(100,50), nrow=2, ncol=1) > x [,1] [1,] 100 [2,] 50 > L %*% x [,1] [1,] 5000 [2,] 40 Not surprisingly, gives this information. The matrix L is a so‐called Leslie matrix. Since gives the state of the population after 7 days, will give its state 7 days after that. > L %*% (L %*% x) [,1] [1,] 4000 [2,] 524 After 14 days, we’ll have 4000 eggs, and 524 adults. Of course, this gets a little clumsy after a while. We’d like to be able to calculate
, but > L^2 %*% x [,1] [1,] 500000 [2,] 19 obviously is wrong. This is because L^2 computes L*L, and not the matrix product L %*% L. Fortunately, there is a package we can install that remedies this small annoyance. If you select Packages‐‐‐Load Package… from the taskbar, load the “expm” (matrix exponential) package, then from the command line, type > library(expm) Then, > (L %^% 2) %*% x [,1] [1,] 4000 [2,] 524 gives the proper result. In general, M %^% n will compute the n‐th power of matrix M. To find the state of the population after 21 days, > (L %^% 3) %*% x [,1] [1,] 52400.0 [2,] 714.4 does the trick. It should now be clear that this population, left unchecked, is about to explode. Example. See the previous problem. But, suppose instead each adult produces only 5 eggs per week. Find the state of the population after 7, 14, and 21 days. Solution. The calculations > L = matrix(c(0,2,.10,.60), nrow=2, ncol=2, byrow=TRUE) > L %*% x [,1] [1,] 0 [2,] 10 > (L %^% 2) %*% x [,1] [1,] 20 [2,] 6 > (L %^% 3) %*% x [,1] [1,] 12.0 [2,] 5.6 > (L %^% 4) %*% x [,1] [1,] 11.20 [2,] 4.56 > (L %^% 5) %*% x [,1] [1,] 9.120 [2,] 3.856 > (L %^% 6) %*% x [,1] [1,] 7.7120 [2,] 3.2256 > (L %^% 7) %*% x [,1] [1,] 6.45120 [2,] 2.70656 > (L %^% 8) %*% x [,1] [1,] 5.413120 [2,] 2.269056 suggests that the population is slowly going to die out. Markov chains. In the above example, there were two possible “states” of insect (eggs, adults), and there was a certain probability (possibly zero!) of transitioning from one state to the other. In general, given a collection of “states", and a collection of probabilities that give the probabilities of transitioning to and from the states, we call this a Markov Chain (or a Markov model). Example. A small downtown area consists of four street corners. Corner 1‐‐‐‐‐‐Corner 2 | | | | BAR‐‐‐‐‐‐‐‐‐‐‐‐‐‐HOME A drunken man is wandering the area. From corner 1, it is equally likely that his next move will be to either corner 2 or the bar. From corner 2, it is equally likely that his next move will be to corner 1 or home. Once the man reaches either the bar or his home, he will remain there with probability 1. We create a matrix M where column i gives the probability of transitioning from state i each of the other states in a single move. Let states 1, 2, 3, and 4 be corner 1, corner 2, the bar, and home respectively. 0 .5 0 0
.5 0 0 0
.5 0 1 0
0 .5 0 1
Note! If you encounter this topic again someday, you might see these matrices defined a bit differently, where the rows (not the columns!) tell you the probability of moving from a given state to another. In most applications, it makes little difference but it’s something to be aware of. In general, if you just swap rows and columns, one says you’ve “transposed” the matrix in question. Supposing the man begins at corner 1, let 1
0
0
0
Calculate and interpret the meanings of… a. b. c. > M = matrix(c(0,.5,0,0,.5,0,0,0,.5,0,1,0,0,.5,0,1), nrow=4, ncol=4, byrow=TRUE) > M [,1] [,2] [,3] [,4] [1,] 0.0 0.5 0 0 [2,] 0.5 0.0 0 0 [3,] 0.5 0.0 1 0 [4,] 0.0 0.5 0 1 > x = matrix(c(1,0,0,0), nrow=4, ncol=1) > x [,1] [1,] 1 [2,] 0 [3,] 0 [4,] 0 > M %*% x [,1] [1,] 0.0 [2,] 0.5 [3,] 0.5 [4,] 0.0 Not surprisingly, if you begin in state 1, then after 1 move, there is a 50% probability of being in state 2, and a 50% probability of being in state 3. > (M %^% 2) %*% x [,1] [1,] 0.25 [2,] 0.00 [3,] 0.50 [4,] 0.25 Note that if you begin in state 1, there is a… …25% probability you return to state 1 in 2 moves (move to 2 with probability .5 and back to 1 in the next move with probability .5). …0% probability of ending up in state 2 after 2 moves (check the assumptions to verify that this is impossible). …50% probability of ending up in state 3 (the bar) after 2 moves (you could go directly there on move 1 with probability .5, and if you move to state 2, it will be impossible to return to the bar on the next move). …25% probability of being home after 2 moves (move to state 2 then home is the only route). Thus it should not surprise you that will tell you the probability of ending up in each of the states after n moves. > (M %^% 100) %*% x [,1] [1,] 7.888609e‐31 [2,] 0.000000e+00 [3,] 6.666667e‐01 [4,] 3.333333e‐01 Note that after 100 moves, the probability of being in state 1 is for all practical purposes 0, and the probabilities of ending up at the bar or at home are approximately 2/3 and 1/3 respectively. We’d actually say the vector [0,0,2/3,1/3] is an eigenvector of the transition matrix. Example. Consider a population that contains two types of individuals, type A’s and type B’s. During each “move”, we do two things: 1. Pick one member of the population at random to die. 2. Pick one member of the population at random that is chosen to reproduce (possibly the same one as in step one!), making an identical copy of itself. This selection process is often referred to as a Moran Process. Let the states be the current number of A’s in the population (so the states are 0, 1, 2 and 3). Write down the transition matrix for this process. Column 1 (from the state with 0 A’s). If there are no A’s, none can appear, so we’ll remain in this state with probability 1. Column 2 (from the state with 1 A). Here, we start with the population looking like ABB. Three things are possible. After one move, there could be 0, 1, or 2 A’s. P(0 A’s after 1 move) = P(an A is selected to die) x P(a B is chosen to reproduce) = 1/3 x 2/3 = 2/9 P(2 A’s after 1 move) = P(a B is selected to die) x P(an A is chosen to reproduce) = 2/3 x 1/3 = 2/9 P(1 A after 1 move) = 1 – (2/9 + 2/9) = 1 – 4/9 = 5/9 Column 3 (from the state with 2 A’s). Here, we start with the population looking like AAB. Three things are possible. After one move, there could be 1, 2, or 3 A’s. P(1 A) = P(an A is selected to die) x P(a B is chosen to reproduce) = 2/3 x 1/3 = 2/9 P(3 A) = P(a B is selected to die) x P(an A is chosen to reproduce) = 1/3 x 2/3 = 2/9 P(2 A) = 5/9 Column 4 (from the state with 3 A’s). Here, the only possibility is that we will continue to have 3 A’s (one will die, but they’ll be replaced with another A). 1 2/9
0
0
0 5/9 2/9 0
Thus, the matrix of this process is 0 2/9 5/9 0
0
0
2/9 1
In R, > N = matrix(c(1,0,0,0,2/9,5/9,2/9,0,0,2/9,5/9,2/9,0,0,0,1), nrow=4, ncol=4) > N [,1] [,2] [,3] [,4] [1,] 1 0.2222222 0.0000000 0 [2,] 0 0.5555556 0.2222222 0 [3,] 0 0.2222222 0.5555556 0 [4,] 0 0.0000000 0.2222222 1 Note that we let byrow default to FALSE, and we filled the matrix by columns. As before, if you calculate a large power of N, that will tell you what the long‐term state of this system will be. > N %^% 100 [,1] [,2] [,3] [,4] [1,] 1 6.666667e‐01 3.333333e‐01 0 [2,] 0 6.088679e‐12 6.088679e‐12 0 [3,] 0 6.088679e‐12 6.088679e‐12 0 [4,] 0 3.333333e‐01 6.666667e‐01 1 For something a little cleaner to look at, let’s round all the entries of this matrix to 3 decimal places. > round (N %^% 100, 3) [,1] [,2] [,3] [,4] [1,] 1 0.667 0.333 0 [2,] 0 0.000 0.000 0 [3,] 0 0.000 0.000 0 [4,] 0 0.333 0.667 1 If we start with a population of zero or three A’s, we’ll stay in that state with probability 1. Starting with 1 A, we’ll eventually end up with 0 A’s with probability 2/3, and 3 A’s with probability 1/3. Starting with 2 A’s, the A’s will disappear with probability 1/3, and the B’s will vanish with probability 2/3. In the homework, one of the exercises will explore what happens if either A or B has a “reproductive advantage”.