Computing the Exact SecondOrder Moment of Markov Traffic
Assignment
Fan Yang
Transportation/Logistics
Environmental Systems Research Institute (ESRI)
380 New York Street, Redlands, California 92373
Email: [email protected]
Submitted for the 11th World Conference on Transportation Research
May 2007
1
COMPUTING THE EXACT SECONDORDER MOMENT OF MARKOV
TRAFFIC ASSIGNMENT
Fan Yang, Transportation/Logistics, ESRI Inc., 380 New York St, Redlands, CA,
92373, [email protected]
ABSTRACT
Markov traffic assignment has been recently proposed to model the temporal
traffic flow evolution under day-to-day variation, and to understand driver’s dayto-day route-choice behavior adjustment process. However, some recursive or
approximated methods are needed to estimate the second-order time-invariant
covariance matrix of the path flow random vector. This paper presents a novel
method of efficiently computing the exact solution of the covariance matrix with
less computational efforts. A numerical study is conducted on computing the
analytical formulation for the second-order covariance matrix, and using the
Markov Chain Monte Carlo (MCMC) algorithm to simulate the path flow
stationary distribution.
1. INTRODUCTION
The traffic equilibrium analysis has become the important advance in the field of
traffic network modeling. The concept of user equilibrium was first proposed by
Wardrop (1952), followed by the stochastic user equilibrium by Daganzo and
Sheffi (1977). The equilibrium traffic flows are considered as the best predicted
traffic flows in the long run and have been widely used for a variety of
transportation planning purposes.
2
Equilibrium analysis, however, only pays attention to a single ideal flow
distribution while ignoring day-to-day traffic flow variation from travel supply
and demand. From observation of real traffic networks, there is evidence of
significant day-to-day variation in travel demands, traffic volumes, and travel
times (May and Montgomery 1987; Hanson and Huff 1988; Mohammadi 1997).
In order to model the traffic assignment under this day-to-day variation, Cascetta
(1989) was one of the first to develop a stochastic process approach of modeling
day-to-day traffic flow evolution, namely the Markov traffic assignment. Markov
traffic assignment models have recently been extensively studied (e.g., Davis and
Nihan, 1993; Cantarella and Cascetta 1995; Hazelton, 2002; Watling and Hazelton,
2003; Hazelton and Watling, 2004). These models capture the traffic flow pattern
under day-to-day variation by means of a stationary probability distribution in
terms of a path flow random vector. Furthermore, day-to-day Markov assignment
models provide insight on drivers’ day-to-day behavior adjustment process such
as their day-to-day learning and information updating processes. This day-to-day
dynamic approach of modeling dynamic traffic network assignment has the
advantage that various individual behavior assumptions on a driver’s route choice
can be taken into account (e.g., Yang and Liu, 2006).
For the Markov traffic assignment models, computing the large transition
probability matrix is analytically difficult and computationally expensive. To
overcome this difficulty, Davis and Nihan (1993) showed that the overall behavior
of the traffic flow stochastic process could be approximated by the sum of a
nonlinear expected dynamical system and a vector autoregressive (VAR) linear
Gaussian process as travel demand grows large. Yang (2005) and Yang and Liu
(2006) generalized this conclusion to the situation where a driver’s inertia in
3
changing routes could be modeled as a flow-dependent random variable. In all of
their models, the stationary path flow distribution will be characterized by the
mean (e.g., stochastic user equilibrium) and covariance structures of multivariate
normal distribution.
The second-order covariance structure of a random path flow vector is of
particular interest in Markov traffic assignment models because it measures the
variation of traffic flow over time. Davis and Nihan (1993) provided a recursive
formulation in computing the time-dependent covariance matrix. Later on,
Hazelton and Watling (2004) used an approximated method of estimating the
equilibrium covariance matrix. However, their approaches are either iterative or
approximated ones, and none of them could obtain the analytical solution of the
stationary covariance matrix efficiently.
In this paper, we propose a novel approach to solve the equilibrium
covariance matrix exactly and analytically. Our conclusion is that under the mild
assumption, i.e., that the transportation network is symmetric (i.e., the Jocobian
matrix of link travel time function with respect to link flow is symmetric), the
analytical solution of the covariance can be computed with much less
computational burden. The underlying path flow stochastic process is based on the
work of Yang (2005) and Yang and Liu (2006).
The remainder of this paper is organized as follows. In section 2, we will
study the underlying stochastic process and its expected deterministic counterpart.
An iterative method to compute the time-dependent covariance matrix will be
presented in section 3. Section 4 will present how to obtain the exact solution for
the equilibrium covariance matrix. Section 5 presents some computational results
4
of computing the mean and the covariance matrix for the stationary multi-normal
distribution. Finally, we discuss concluding remarks in Section 6.
2. CONVERGENCE OF THE PATH FLOW STOCHASTIC
PROCESS
In this study, it is assumed that the travel demand is fixed and day-to-day static,
while the elastic demand can be extended by adding a dummy route indicating not
traveling (Cascetta, 1989). To reflect the fact that travel demand and traffic flows
can take non-negative integer values, we would consider the situation where travel
demand is a large and finite positive integer, and the path flow state space is a
finite fine grid.
Let us denote the total number of travel demand for an O-D pair w  W by
a large integer N w  Z  where Z denotes the set of positive integers, the set of
routes between O-D pair w by P w , and the dimension of P w by n r . To simplify
the notation, the superscript w is removed in the study below. The path flow
random vector at “day” t is denoted as X tN where
X
iP
N
i ,t
 N . Since the
demand is large but finite, the route flow state space is a finite and fine grid. The
integer-valued random route flow vector X tN is a vector in the finite state space
K 0N  {x  Z n :  xi  N} . The normalized state vector can be expressed by
r
i
1
N
{ X tN }
and
K N  {x  R n : x p 
r
therefore
the
normalized
space
will
be
k
, k  0,1,2,, N , p  P} .
N
5
In the literature of transportation network analysis, it is usually assumed
that the route cost and link cost functions can take real-valued flow variables.
When the demand goes to infinite, the finite state space K N will converge to the
widely used real-valued feasible route flow set K , i.e., K N  K as N  
where the set K  {y  R n : y  0, and  y p  1} . Let the route cost function be
r
pP
defined as F : K  R n and the link cost function  : X  R n where X is the
r
a
feasible real-valued link set X  {h : h  Ax and x  K} , A the link-route
incidence matrix, and n a the number of links.
Yang (2005) and Yang and Liu (2006) show that the stochastic process
1
{ X tN } can be closely approximated by its expected deterministic process
N
provided that the travel demand goes to infinite. Let u p ,t denote the
reconsideration percentage of drivers on path p who will reconsider their route
choices at day t , and  p,t the choice probability to switch to path p at day t
conditional on the traffic flow pattern at day t 1 ,
1 N
X t 1  x tN1 . Then the
N
expected deterministic process x(t ) can be expressed by
x p ,t  v p ( xt 1 )   p ( xt 1 ) ui ( xt 1 ) xi ,t 1  (1  u p ( xt 1 )) x p ,t 1
(1)
iP
where x p ,t is the path flow on path p at day t for the deterministic process x(t ) .
The convergence theorem 4.3 and 4.4 in Yang (2005) show that the
Markov chain
1
{ X tN } will converge to the deterministic process (1) in
N
probability as the travel demand goes to infinite. However, this result does not
provide any insight to judge the extent of the difference between the stochastic
6
and deterministic processes for large but finite demand. We will address this
question in the paper.
Our first assumption is the convergence of the initial state
that the initial condition
surely, i.e.,
1 N
X 0 . Suppose
N
1 N
X 0 converges to a deterministic state x0 almost
N
1 N a.s.
X 0  x0 and that their augmented difference at the initial
N
condition converges to a multi-normal random variable in distribution, i.e.,
N(
1 N
d
x0  x0 ) 

N (0,  0 ) where  0 denotes the covariance matrix at day
N
t = 0.
Followed by the argument in Cascetta (1989), it is straightforward to show
that route flows X pN,t ’s are multinomial distributed random variables conditional
on road information at day t 1 , i.e.,
1 N 1 N
Xt
X t 1  xtN1  M (1, vt ( xtN1 )) . We
N
N
state the first and second moments of a general multinomial distribution in
Proposition 1.
Proposition 1 Suppose random variables Y1 , Y2 ,, Yk has a joint multinomial
distribution M (n, p1 , p2 ,, p k ) where the parameter n is the total number
n  Y1  Y2    Yk and each pi denotes the probability that one will go to the i th
bin for i  1,2, k . Then
(i) EYi  npi
(ii) VarYi  npi (1  pi )
(iii) Cov(Yi , Y j )  npi p j , i  j
7
Since at t = 1 , given the information
from the Central Limit Theorem that
N(
1 N
X 0  x0N at day 0 , it follows
N
1

N  X qN,1  vq ( x0N ) 
N

d

N (0,1) , that is,
N
N
vq ( x0 )(1  vq ( x0 ))
1 N
1
1
1
d
xq ,1  vq ( x0N )) 

N (0, vq ( x0N )(1  v q ( x0N ))) .
N
N
N
N
Now let us consider the rescaled difference between the stochastic motion
and the expected deterministic approximation at t = 1 .
N(
1 N
X1 - x1 ) =
N
N[
1 N
1
X1 - v( X 0N )] +
N
N
Our claim is that the first term N [
N [v (
1 N
X 0 ) - v( x0 )]
N
1 N
1
1 N
X 1  v( X 0N )] , given
X 0  x0N ,
N
N
N
is independent of the second one. This claim was shown in the appendix of
Lehoczky (1980) by computing the moment generation function and also in the
proof of proposition 3 in Davis and Nihan (1993) by using the characteristic
function and dominated convergence theorem. By the Central Limit Theorem, the
first term has asymptotically a multivariate normal distribution with mean 0 and
covariance matrix defined by ( x0 )   ij ( x0 ) where
v j ( x0N )(1  v j ( x0N ))
 vi ( x0N )v j ( x0N )

 ij ( x0 )  
The second term N [v(
i j
i j
(2).
1 N
X 0 )  v( x0 )] , can be expanded by the first order Taylor
N
series given the continuously differentiable function v(×) and the almost sure
convergence of
N [v (
1 N
1 N a.s
X 0 at t = 0 ,
X 0  x0 by assumption. Therefore,
N
N
1 N
1
X 0 )  v( x0 )]  N Dv ( x0 )( X 0N  x0 ) , where Dv ( x0 ) denotes the
N
N
8
Jacobian matrix of function v () with respect to the normalized path flow x
evaluated at x0 .
By our assumption we have N (
N [v (
computation, it can be shown that
normal distributed, i.e.,
where the superscript
T
N [v (
1 N
d
X 0  x0 ) 

N (0,  0 ) . After some
N
1 N
X 0 )  v( x0 )] is also multi-variate
N
1 N
d
X 0 )  v( x0 )] 

N (0, Dv ( x0 ) 0 Dv ( x0 ) T ) ,
N
denotes transpose.
Combining both terms above, we can conclude that the suitably rescaled
difference between the stochastic process and its deterministic approximation at
t = 1 is a multi-variate normal distribution provided that the total number of
drivers is large but finite,
N(
1 N
d
X 1  x1 ) 

N (0, ( x0 )  Dv ( x0 ) 0 Dv ( x0 ) T ) .
N
N(
From the statements above, we claim that
1 N
X 1  x1 ) is normal
N
distributed provided that demand N is large but finite. By iterating the above
many
N(
times,
for
any
fixed
finite
time
t
,
1 N
d
X t  xt ) 

N (0, ( xt 1 )  Dv ( xt 1 ) t 1 Dv ( xt 1 ) T )
N
Hence
N(
we
can
conclude
(3).
1 N
X t  xt ) can be served to measure how close the expected
N
deterministic process can approximate the underlying stochastic motion. If N
approaches infinite, we already know
1 N
X t  xt for any finite time t . In fact,
N
there also exist some higher order terms of order o( N
1
2
) when we do the Taylor
9
expansion for the term N [v(
1 N
X t 1 )  v( xt 1 )] . But these higher order terms will
N
vanish as population size N goes to infinite.
3. AN ITERATIVE METHOD TO SOLVE THE
COVARIANCE MATRIX
Conditional on the road information on the day t 1 , for any given time t ,
the difference between the normalized route flow in stochastic process and the
deterministic process is asymptotically normal distributed with mean 0 and
covariance  t . The covariance matrix  t satisfies a recursive form as follows
 t  ( xt 1 )  Dv ( xt 1 ) t 1 Dv ( xt 1 ) T
(4).
In the following, the reconsideration percentage ut in the deterministic
process (1) is assumed to be flow-independent for simplicity. We might consider
it as a time-independent constant, e.g., ut = u (Watling, 1999) or a timedependent parameter, e.g., ut = 1/ t (Yang, 2005). In order to compute the
Jacobian matrix of choice probability with respect to route flow vector, we apply
the chain rule to the deterministic process (1), i.e.,
Dv ( xt 1 )  (1  ut 1 ) I  ut 1 D ( F ( xt 1 )) DF ( xt 1 )
(5).
The choice probability to select path p at day t in the widely used logit
model can be expressed as  p ,t 
exp( F ( x p ,t 1 ))
 exp( F ( x
j ,t 1
))
. Denote the Jacobian matrix
j
of
choice
probability
vector
with
respect
to
route
travel
cost
D ( F ( x))  pij ( F ( x)) . It can be shown that this Jacobian matrix is symmetric
and negative semi definite in R n
r
10
  i ( xt 1 )(1   i ( xt 1 )), i  j
pij ( Ft ( xt 1 ))  
 i ( xt 1 ) j ( xt 1 ), i  j

(6).
The covariance matrix  t can be computed by iterating (4) over time. In
addition, one important question is whether matrix  t is bounded or blows up
over time. Although the recursive formulation was not new in the literature, the
boundness of  t has not been well investigated. If  t will blow up as t increases,
the fixed point of the deterministic process (1) will become unstable and the
stochastic motion is moving far away from the fixed point, thus the approximation
will be deteriorated. Otherwise, if  t is still bounded over time, the stochastic
motion continues being attracted to mean deterministic process (1).
Denote x* as the logit-based stochastic user equilibrium path flow. Then
xt 1  x *
xt  x
2
*

2
(1  ut )( xt  x * )  ut ( ( xt )   ( x * ))
xt  x
*
2
 (1  ut )  ut D ( x * )
2
.
2
After doing Taylor expansion for xt near x* , it is readily to show that for
deterministic approximation, xt  x *  xt 1  x * is a contraction map provided that
the Jacobian matrix D ( x * ) of choice probability function with respect to route
flow at fixed point x* satisfies D ( x * )  1 . If the 2-norm of the Jacobian
2
matrix D ( x * ) is less than 1, i.e., D ( x * )  1 , then  t is bounded as t  
2
after the mean process (1) reaches its fixed point.
In fact, if the covariance matrix  t converges to a stationary matrix  * ,
we must have
*  ( x * )  Dv ( x * )* Dv ( x * )T .
(7).
11
Theorem 2 states the sufficient condition for the existence of the stationary
covariance matrix  * as follows.
Theorem 2 If D ( x * )  1 , then the covariance matrix { t } converges to  * .
2
Proof:  t  ( x * )  Dv ( x * ) t 1 Dv ( x * ) T
  t   *  [ ( x * )  Dv ( x * ) t 1 Dv ( x * ) T ]  [( x * )  Dv ( x * ) * Dv ( x * ) T ]
 Dv ( x * )( t 1   * ) Dv ( x * ) T
Take the Euclidian norm (2-norm) on each side, then
 t  *
t  
2
 Dv ( x * )  t 1  *
2
 Dv ( x )  t 1  
*
2
*
2
2
2
*
2
Dv ( x * )T .
2
. That is,
 t  *
 t 1  
2
*
2
 Dv (x * ) .
2
2
Since Dv ( x * )  (1  u t ) I  u t D ( x * ) and Dv( x* )  (1  ut )  ut D ( x * ) .
2
Therefore one has D ( x* )  1  Dv( x * )  1   t  *
2
zt   t  *
2
2
2
2
  t 1  * . Let
2
and zt 1   t 1  * . Then {zt } is a strictly decreasing sequence
2
with lower bound 0, which implies the sequence {zt } converges to 0. Therefore,
the sequence { t } converges to  * . 
Theorem 2 also provides us an iterative method to compute the covariance
matrix  * provided D ( x* )  1 .
2
(1) Suppose the initial condition  0  0
(2) In the (k+1)th iteration,  k 1  ( x * )  Dv ( x * ) k Dv ( x * )T
(3) If k 1  k
2
  , where g is a predefined tolerance, e.g.,   10 5 , then
stop. Otherwise, go to step (2).
12
4. THE EXACT SOLUTION OF COVARIANCE MATRIX
Although the above iterative method provides us the linearly convergence rate to
compute the covariance matrix  * , our motivation is to give the exact analytical
solution of matrix  * in the following. To comply with the study in Hazelton and
Watling (2004), in the following we focus on the case ut  1 , i.e., v( xt )   ( xt ) ,
which implies that all drivers might reconsider routes every day.
A formulation of  * is given as *  ( x* )  Dv ( x* )* Dv ( x* )T . If such
 * exists, it is called the stationary covariance matrix. Recall the matrix
( x* )  tij ( x*) has the following structure
vti ( x * )(1  vti ( x * )) i  j

t ij ( x * )  
.
 v i ( x * ) v j ( x * )
i j
t
 t
Hence the matrix ( x* ) is symmetric, positive semi-definite in R n but
r
singular. The Jacobian matrix of the choice probability function  with respect to
normalized route flow at stochastic user equilibrium x * can be expressed by
D ( x * )  D ( F ( x * )) DF ( x * ) . For admissible choice probability function
(Sandholm 2003), we know that D ( F ( x * )) is symmetric, negative semi-definite
in R n and negative definite in R n 1 . It is assumed that the Jacobian matrix
r
r
DF ( x * ) is also symmetric for symmetric transportation networks.
First we claim that we can find the exact solution for the matrix X in the
equation XC  Y  DX as follows.
Proposition 3 The exact solution of matrix X exists for equation XC = Y + DX
if matrix C and D are diagonalizable.
13
Proof: Let C  PHP 1 and D  QKQ 1 where H and K are diagonal matrices.
Then XPHP 1  Y  QKQ 1 X . Multiple Q 1 from the left and P from the right,
then one has Q 1 XPH  Q 1YP  KQ 1 XP . Let R  Q 1 XP and S  Q 1YP , the
above equation can be written as RH  S  KR . Since H and K are diagonal
matrices, one has (h jj  k ii )rij  sij .
If h jj  k ii  0 , then rij 
sij
h jj  kii
. Otherwise, rij could be any arbitrary number if
sij  0 , or has no feasible solution if sij  0 . Finally, the exact solution for matrix
X can be expressed by X  QRP 1 . 
However, Proposition 3 can not be applied to calculate stationary
covariance matrix  * directly in (7) because D ( F ( x * )) is singular, even for
symmetric transportation networks. In fact, it is straightforward to show that
except one eigenvalues with value 0 , all other eigenvalues of matrix D ( F ( x * ))
are real and negative. Hence, the intuitive idea is to project matrix D ( x * ) from
r
the space R n to the subspace R n
r
1
such that the projected matrix might be non-
singular and diagonalizable.
Theorem 4 The exact solution to the equation (7) can be obtained analytically if
the link travel cost function is symmetric.
Before proving Theorem 4, first we need to do the projection such that the
projected matrix is non-singular and diagonalizable. One way to do this projection
is proposed by Sanholm (2006). Let the matrix
14
 n  n(n  2)

 n(n  1)

n n
 n(n  1)

R



n n

 n(n  1)

1


n
n n
n(n  1)

n n
n(n  1)






n n
n(n  1)


n n
n(n  1)
n  n(n  2)
n(n  1)

1 

n

 


 

1 

n
1 

n 

where n = dim( R ) represent the rotation on R n by cos 1 (
r
1
n
) which transforms
from the space R 0n  { y  R n : 1T y  0} to R n  { y  R n : y n r  0} . Since R is
r
r
r
a rotation, it is orthogonal. Next let J  R ( n
r
1)n r
r
and J  [ I 0] . The multiplication
of J with any y  R n will drop y ’s last element. Finally, define the matrix
r
Z  R ( n 1)n by Z = JR , that is, Z is the matrix R with the last row removed.
r
r
Followed by Lemma A.3 in Sandholm (2006), we have ZZ T = I and Z T Z  
1
where   I  11T . One remark is that matrix F has the property x  x for
n
any x  R 0n .
r
Multiple Z from the left and Z T from the right to the equation (7) and
denote T  ( x * ) ,   * and G  D ( x * ) , respectively. Then one has the
equation ZZ T  ZTZ T  ZGG T Z T .
Note T 1  0 and G T 1  0 , hence  t 1  0 for all t . Therefore,
     . Substitute  by  , then ZGG T Z T  ZG()G T Z T .
Since Z T Z   , it is readily to show that ZGG T Z T  ZG(Z T Z )( Z T Z )G T Z T
 (ZGZ T )( ZZ T )( ZG T Z T ) .
15
~
~
~
~
Let us define   ZZ T , T  ZTZ T , G1  ZGZ T , and G2  ZG T Z T . Then
(7) can be expressed by
~ ~ ~~~
  T  G1G2
(8).
~
Before showing the condition where G1 is diagonalizable , let us introduce
the following two Lemmas.
Lemma 5 (Sandholm, 2006) Suppose that M  R nn maps R 0n into itself and
x  R 0n . Then ( , x) is an eigenvalues/eigenvector pair for M if and only if
( , Zx) is an eigenvalues/eigenvector pair for ZMZ T .
Lemma 6 (Yang, 2005) If matrix A is symmetric positive definite and matrix
B symmetric, then matrix AB is a diagonalizable matrix.
~
Lemma 7 G1 is diagonalizable if the link travel cost function is symmetric.
~
Proof: Since G1  ZGZ T and G  D ( F ( x * )) DF ( x * ) , it can be concluded that
~
G1  ZD ( F ( x * )) DF ( x * ) Z T . Each row vector of matrix D ( F ( x * )) , d i satisfies
d i   d i , hence D ( F ( x * ))  D ( F ( x * ))  D ( F ( x * )) Z T Z . Therefore, one
~
has G1  ( ZD ( F ( x * )) Z T )( ZDF ( x * ) Z T ) . By Lemma 5, matrix ZD ( F ( x * )) Z T is
symmetric, and negative definite in R n 1 . In addition, matrix ZDF ( x* )Z T is
r
r
~
symmetric, from Lemma 6, we can conclude that matrix G1  R n 1 must be
diagonalizable. 
~
~
Since G1 is diagonalizable, we can write G1  S 1S where S is non-
singular and  is a diagonal matrix with real valued diagonal elements. Note that
~
G1 might be singular, and that  might have zero valued elements. Furthermore,

~
G2  ZG T Z T  ZGZ T

T
~T
 G1  S T S T , is also a diagonalizable matrix.
16
~
~
Subsequently, after substituting G1  S 1S and G2  S T S T into (8),
~ ~
~
one has   T  S 1S S T S T  . Multiply matrix S to the left and S T to the
~
~
~
~
~
right, therefore, SS T  ST S T  SS T  . Let X  SS T and M  ST S T . Then
(8) can be rewritten as
X  M  X
(9).
From the analysis above, matrix M is symmetric and positive definite in R n 1 ,
r
however, the diagonal matrix  might contain zero elements, and hence might be
singular.
Without loss of generality, we can partition the matrix  into two block
matrices, an upper one with all zero elements, and a lower one with all non-zero
0

elements, i.e.,   
 . Similarly, we can spell out the block matrices for
 1 
M
both M and X as M   11
M 21
M 12 
X
and X   11

M 22 
 X 21
X 12 
. Therefore, (9) can
X 22 
be expressed by
 X 11
X
 21
X 12  M 11

X 22  M 21
M 12  0



M 22   1 X 22 1 
(10).
Then one has the following equations,
 X 11  M 11
X  M
 12
12

 X 21  M 21
 X 22  M 22   1 X 22  1
(11).
For the last equation in (11), equivalently, X 2211  M 2211  1 X 22 since 1 is
non-singular. From Proposition 3, we can obtain the exact solution for matrix X 22
analytically.
17
The statements above show that we can obtain an analytical solution for
~
~
matrix X . Recall X  SS T , so can we for the matrices  and  as
~
~
  S 1 XS T , and  *  Z T Z  Z T S 1 XS T Z . We can address the analytical
solution to the equation (7). The above statements conclude the proof of Theorem
4. 
In summary, due to non-invertibility of matrix D ( x * ) in R n , we project
r
it into the subspace R n 1 by taking the operator Z () Z T , hence the projection of
r
D ( x * ) is diagonalizable in R n 1 . In the subspace R n 1 , we can solve the
r
r
~
~
projected covariance matrix  analytically. The final step is to project matrix 
r
r
~
from subspace R n 1 back to space R n to obtain the solution of  * as  *  Z T Z .
Compared with the solution algorithm of the iterative method, the direct
method to compute the exact solution has the advantage that it can provide the
exactly solution of stationary covariance matrix  * and it requires much less
computational efforts. Furthermore, in terms of convergence, the direct method
does not require the condition D ( x* )  1 from the iterative method. When the
2
link travel time function is symmetric and the choice probability is extreme value
distributed (e.g., logit and probit models), we show that how to compute the exact
covariance matrix  * analytically. In addition, it is worthwhile noting that we can
compute the analytical solution for the inverse matrix ( * ) 1 if it exists (Yang,
2005).
If the stationary covariance  * exists, the behavior of the Markov chain
1
{ X tN } can be characterized by the multi-variate normal distribution
N
18
1 N d
Xt 
 N ( x * ,  * ) . Our fist application can be to use Markov Chain MontoN
Carlo (MCMC) simulation, for example, Metropolis-Hastings algorithm (Wendy
and Angel, 2002), to simulate some realizations of the long term behavior. After
“burning in” the samples in the early stage, we can use the large number of further
samples to approximate the statistical characteristics. Suppose that there are no
external shocks (e.g., incidents, construction, and special events) imposed to the
transportation network. From our day-to-day analysis, one observation is that
through drivers’ learning process, the traffic flow may approach a “stationary”
state. With the fast development of ITS, link count data become more and more
feasible, providing the possibility to fit into the traffic data to our model.
5. EXPERIMENTAL RESULTS
In this section, we compute the mean and covariance matrix of the stationary
invariant distribution in Markov traffic assignment, and use MCMC to simulate
the realization in terms of the invariant distribution. The simple test network will
be the 3  3 grid network with 9 nodes, 12 links and 6 routes from origin 1 to
destination 9. The total O-D flows are 500 vehicles. The route and link
correspondence is shown in table 5.1.
19
Figure 5.1 The 3  3 Grid Network
Table 5.1 Route and Link Correspondence in the 3  3 grid network
Route
1
2
3
4
5
6
Links
1-2-3-6-9
1-2-5-6-9
1-2-5-8-9
1-4-7-8-9
1-4-5-6-9
1-4-5-8-9
The link travel time function is  a  1.5[1  0.15(
xa
Ca
)3 ] , where xa is the link
flow and Ca is the capacity for link a . All nine links have the same capacity 2400
vehicles. The initial point of normalized route flow vector is randomly chosen as
x  0  (0.125, 0.2, 0.25, 0.15, 0.125, 0.15) .
If we have the ability to compute the stationary covariance matrix  * , we
can apply some MCMC techniques to generate a Markov chain whose stationary
distribution is multi-variate normal N ( x * , * ) and all the examples of the Markov
chain are independent of the past conditioned on all information on the previous
day. In the following, we will apply the Metropolis-Hastings algorithm to produce
a Markov chain with stationary distribution N ( x * , * ) . More details about the
Metropolis-Hastings algorithm can be found in Wendy and Angel (2002).
20
Let us study the case where perception error parameter  = 1.5 , the
stochastic user equilibrium of the normalized route flow is x* = [0.1824 0.1538
0.1538 0.1924 0.1538 0.1538]' 0.1538 0.1924 0.1538 0.1538]' and the exact
solution of the stationary covariance matrix is




* = 




0.2576 0.0741 - 0.0291 - 0.1408 - 0.0291 - 0.1323 
0.0741 0.2344 - 0.0244 - 0.1339 - 0.0244 - 0.1254 
- 0.0291 - 0.0244 0.1574 - 0.0299 - 0.0488 - 0.0249
.
- 0.1408 - 0.1339 - 0.0299 0.2606 - 0.0299 0.0742 
- 0.0291 - 0.0244 - 0.0488 - 0.0299 0.1574 - 0.0249

- 0.1323 - 0.1254 - 0.0249 0.0742 - 0.0249 0.2335 
Our first step is to apply the Metropolis-Hastings algorithm to simulate the
multi-variate normal distribution with mean x* and covariance matrix  * . Figure
5.2 shows the Markov chain samplings of normalized flow on route 1 after
“burning in” the first 2000 iterations. The quantile- quantile plot (Q-Q plot) shows
that the MCMC samples statistically observe the normal distribution in figure 5.3.
Figure 5.2 MCMC Samples of Normalized Flow on Route 1
21
Figure 5.3 Q-Q plot of MCMC Samples versus Standard Normal
In fact, the normalized route flows should range from 0 to 1, which is the
probability distribution of total demand over the finite routes. However, the multivariate normal distribution does allow the case where the sample runs beyond this
range. In other words, we need to truncate the multi-variate normal to assure that
the samples lie in the range of [0,1] . After applying this restriction to the MCMC
r
samples, we are actually throwing away some samples which do not lie in [0,1]n .
Therefore, the mean route flow vector will be changed accordingly. If the
equilibrium normalized route flow x i* on route i satisfies xi*  0.5 , then the mean
normalized route flow x̂i becomes greater than xi* . Otherwise, we have xˆi < xi* .
In addition, the deviation from xˆi to xi* depends on the difference between xi* and
0.5. Figure 5.4 illustrates the case where the equilibrium normalized route flow is
0.1284. In the limiting case N   , the multi-variate normal distribution will put
22
all the probability mass into the single point x* . Therefore, the probability
distribution remains the same point mass characteristic regardless of the restriction.
0.14
0.12
0.1
0.08
0.06
0.04
0.02
0
-1
-0.5
0
0.1824
0.5
1
1.5
Figure 5.4 Normal Probability Distribution Function with Truncation
6. CONCLUDING REMARKS
In this paper, we have proposed a novel approach to solve the stationary
covariance matrix analytically and exactly in Markov traffic assignment. The
main difficulty in finding the analytical solution for the stationary covariance
matrix is that the Jacobian matrix of the choice probability function with respect
to the route flow might be singular and non-diagonalizable. Intuitively, our idea is
r
to project this Jocobian matrix from the space R n to the subspace R n
r
1
such that
the projected Jocobian matrix could be diagonalizable. The conclusion is that if
the transportation network is symmetric then the analytical solution of the
23
covariance and its inverse can be computed directly with much less computational
burden.
With the fast development of ITS, link count data become more easily to
obtain. One interesting direction for our future work is to use the real traffic data
to validate our day-to-day traffic flow evolution in discrete time. For instance, use
chi-square test to check whether the link count data really observe the multivariate normal distribution that our model predicts. In addition, we are also
interested in how to use the link count data to do some maximum likelihood
estimation, for example, to estimate the travel demand and the perception error
parameter over the population.
24
REFERENCES
Cantarella, G.E. and E. Cascetta (1995). Dynamic processes and equilibrium in
transportation network: towards a unifying theory. Transportation Science, 29,
305-329.
Cascetta, E. (1989). A stochastic process approach to the analysis of temporal
dynamics in transportation network. Transportation Research, 23B, 1-17.
Daganzo C. and Sheffi, Y. (1977). On stochastic models of traffic assignment.
Transportation Science, 11, 253-274.
Davis, G.A. and N.L. Nihan (1993). Large population approximations of a general
stochastic traffic assignment model. Operations Research, 41, 169-178.
Hanson, S., Huff, J. (1988). Repetition and day-to-day variability in individual
travel patterns. R. C. Golledge, H. Timmermans, eds. Behavioural Modelling in
Geography and Planning. Croom Helm, Kent, U.K.
Hazelton, M. L. (2002). Day-to-day variation in Markovian traffic assignment
models. Transportation Research, 36B, 61–72.
Hazelton, M.L. and D.P. Watling (2004). Computation of equilibrium
distributions of markov traffic assignment models. Transportation Science, 38,
331-342.
25
Lehoczky, J. P. (1980). Approximations for interactive markov chains in discrete
and continuous time. Journal of Math. Sociol., 7, 139 – 157.
May, A. D., Montgomery F. O. (1987). Factors affecting travel times on urban
radial routes. Traffic Engineering Control, 28, 452–458.
Mohammadi, R. 1997. Journey time variability in the London area. Traffic
Engineering Control, 38, 250–257.
Sandholm, W.H., 2003. Evolution and Equilibrium under Inexact Information.
Games and Economic Behavior, 44, 343-378.
Sandholm, W.H., 2006. Evolution in Bayesian Games II: Stability of Purified
Equilibrium. Journal of Economic Theory, forthcoming.
Watling, D. (1999). Stability of the stochastic equilibrium assignment problem: a
dynamical systems approach. Transportation Research, 33B, 281-312.
Watling, D., Hazelton M.L. (2003). The dynamics and equilibria of day-to-day
assignment models. Networks and Spatial Economics, 3, 349-370.
Wardrop, J. G. (1952). Some theoretical aspects of road traffic research. Proc. Inst.
Civil Eng., Part II, 1, 325-378.
26
Wendy L. Martinez and Angel R. M., (2002). Computational Statistics Handbook
with MATLAB, Chapman & Hall/CRC.
Yang, F. (2005). An evolutionary game theory approach to the day-to-day traffic
dynamics. Dissertation, University of Wisconsin-Madison.
Yang, F. and Liu, H. (2006). A General Modeling Framework for Traveler’s Dayto-Day Route Adjustment Processes. Accepted by the 17th International
Symposium on Transportation and Traffic Theory (now a journal of Elsevier),
London, United Kingdom.
27