2 June 1997
PHYSICS
ELSEVIER
LETTERS
A
Physics Letters A 229 (1997) 347-353
A general framework for non-equilibrium phenomena:
the master equation and its formal consequences
Bernard Gaveau a, L.S. Schulman b
a Uniuersite’ Paris 6. Gbomttrie des lquutions aux d&iue’es partielles et physique mathkmatique. Case courier
172. Tour 46. 5Sme Itage.
4. Pbce Jussieu. 75252 Paris C&dex 05. France
b Physics Department.
Received 26 August
Clarkson University, Potsdum. NY 13699-5820.
1996; revised manuscript
received
Communicated
IO February
USA
1997; accepted for publication
24 February
1997
by C.R. Doering
Abstract
We consider non-equilibrium systems defined by a state space, and by a stochastic dynamics and its stationary state. The
dynamics need nor satisfy detailed balance. In this abstract framework we do the following: (I) define and analyze “relative
entropy”, (2) study dissipation in the relaxation to the stationary state, as well as the extra dissipation to maintain the system
in its stationary state against some detailed balance dynamics, (3) extend the fluctuation-dissipation
theorem and the
Onsager relations, and (4) give a formula for the stationary state in terms of a summation over trees. 0 Elsevier Science
B.V.
PACS: 05.70.Ln;
05.40. + j
1. Introduction
Most natural systems are out of equilibrium and
are often far from equilibrium. Still, we do not know
general laws for them comparable to those of equilibrium statistical mechanics,
although many attempts have been made, for example, to define thermodynamic functions in non-equilibrium
situations
(see, e.g., Refs. [l-5]).
One reason for the difficulty in extending the
equilibrium formalism to non-equilibrium
situations
is the fact that non-equilibrium
situations are dynamical: things happen in non-equilibrium
systems
whether they be in stationary or non-stationary states,
while the theory of equilibrium statistical mechanics
and thermodynamics
can be entirely deduced from
the knowledge of the Boltzmann distribution.
Our approach [6] begins from the master equation,
in effect assuming the system can be described by a
Markovian stochastic process. This will be inadequate for systems in which quantum interference is
important or for systems with infinite memory; nevertheless, the domain of applicability
is still large.
This approach shares features with many others, not
surprising in view of the already wide application of
the master equation. For example, an early paper
with relative entropy results similar to ours is Ref.
[71.
In this Letter we summarize and extend Ref. [6].
The system is represented by a state space X, which
we take to be finite. The specification of a stationary
state j?(x) (as a probability distribution on X) does
not characterize the dynamics, because such a state
can always be considered to be a Boltzmann state
0375-%01/97/$17.00
0 1997 Elsevier Science B.V. All rights resen red.
PII SO375-9601(97)00185-O
B.Gaueuu,L.S.Schulman/PhysicsLe!tersA
229(1997)347-353
348
with energy proportional to -log p(x). For the
equilibrium case, 3 generally carries most of the
information one wants (although equilibration rates
may be of interest too), while for the non-equilibrium case fi generally tells you only a small
portion of the essential properties of the system.
In our framework, the dynamics are specified by a
stochastic matrix R,, which is the probability of a
transition from state-y to state-x in a time step Ar, so
that
c
c
Rxy= 1,
XEX
R,,p”(Y) =p”(x).
(1.1)
YEX
We assume that R is irreducible, but we do not
assume that it satisfies detailed balance, i.e., we do
not assume that R,,p"( y) = R,,v p”(x) for all x and
y. In fact, deviation from detailed balance seems to
be essential for systems exhibiting organization and
which, as a consequence, exhibit net currents, although they remain in their stationary state.
The matrix R includes all dynamical effects, those
taking place in the system, as well as those describing the exchange of matter and energy with external
reservoirs.
2. Relative entropy of two probability
distribu-
Then
- i<( 6’p,)*&-*
qP,G)-
(2.4)
where ( )b denotes the average with respect to the
distribution p. If fi is an equilibrium distribution
(with free energy F,), i.e., J?(X) = exp[-p(E(x)
-
&Jl, then
q? I a = Ppeq - Fq),
F,=(E),+(q)/&
Thus Fq can be interpreted as the free energy of the
probability distribution q. In general, S(q 1p) represents the dissipation during the relaxation of the state
q to 3.
Let us also consider the transpose ‘R of R. Defining the scalar product ( I Ip by
(hJ)p=
CF(+J(&+),
the adjoint of ‘R is
(‘R) * = diag f Rdiag( p).
( 1
Call 1,4,the eigenvector of (‘R)* (‘RI associated with
the maximal eigenvalue pmax = CL,< 1 of this operator. ’ For any distribution, p, that is close to the
stationary state 3, as in (2.31, we have
tions
a ;tr:X(1
-cLm,x).
Given two probability distributions p and q on
the state space X, we define
S(RPIP)-S(PIP)
S( P) = -
When R satisfies detailed balance, (‘R) * = ‘R, and
P max is the leading eigenvalue of R’.
To prove the inequality (2.5), define p = ps as in
Rq. (2.3). Then
c
P( x) log P( x)7
XCX
s(Plq) = -
c P(X) log-P(4(x)x>.
(2.1)
XEX
S(p) is the usual entropy and S(p I q) (which we
call relative entropy) was introduced in Ref. [8], and
used in Ref. [91 and in Ref. [lOI in information
theory. S( p 1q) is always negative and is 0 if and
only if p = q. For any distribution p, the product Rp
gives the evolution of p in one time step. It can then
be shown that
S(plq) G-<(RplRq).
(2.2)
Let us now assume that p,(x) is a probability
distribution close to the stationary state p
P&(4 = a( 4 exp[ cp( x, 6 >Iv
‘P(X,S)=6~,(x)+6*sp,(x)+....
t 2.3)
(2.5)
R,(x)
=P(x)
exp[a+,(x)
+ . ..I.
’ In genera1 R canhave complex eigenvalues of norm 1, and
these can be useful for studying the same sort of dynamical
situations that interest us here. We expect that our dynamical
statements will ultimately be phrased in terms of the currenfs,
namely J,, = R,,jT(y)R,,j'T(x).
It then becomes convenient
to redefine R by R+(l- E)R + cl for some small positive E.
This pushes the norm-oneeigenvalues
down a bit, while leaving
the essentials of the spectrum intact. The periodic behavior that
one might otherwise have studied using the norm-l (non-real)
eigenvalues can now be examined by looking at _rXY.which fully
reflects the periodic behavior, and which is essentially unchanged
by the transformation.
B. Gaueau, L.S. Schulman / Physics Letters A 229 (1997) 347-353
Comparison of order 6 terms in the preceding equation leads to I/J, = diag(l/j)
R(diag j?)cp,. Using
Eq. (2.4) it follows that
for all x and y. The relative path entropy is defined
by
P), w(, 4))
.Y(Tl(R>
N---
s*
2
c
=-
1
diag( F)
349
Rdiag( P) cpI
(3.2)
path to T
One maximizes the second member under the conditions ( cpF)a = 1, ( ‘p, >a = 0 (the last condition follows from Cp, = 1, so that Ccp,(x)j?(x) = 0). Details are in Ref. [6].
It is always negative. It is easy to see that one has
Y(Tl(K
P>>(WV4))
T-
=s(plq)
+
I
c
R’P~W),
A,p(R,
(3.4)
r=o
3. Path entropy
where we have denoted
(a) Absolute path enrropy. A path up to time
T=nAtisasequencey={x,,...,x,)ofpointsin
X. If p is an initial distribution, the R-dynamics
induces a probability measure $R*P’(Y> on the space
of paths by
A,p( R,
P”.“‘(Y)
=R+T*T_,R.~y_,~\.T_Z...RXIXD~(~~).
(3.1)
We define an absolute entropy
c+(TI R, p) = -
c
yup
/dRsP’(~)
9,
W) = -
c R,,,q( y)
(3.5)
log>
X,Y
.r.y
and R’ denotes R to the power t.
Cc) Analysis near equilibrium. Assume now that
p’(x) is close to equilibrium and that R is close to W.
-We write
P( x) = P,( 4 ed
4 xY 6
>I
= p,,(.x)exp[h,( ~1+ S*a(x) +
log /~(~*~)(y).
(3.6)
to T
It is easy to verify that
Rx,= Yy exp[f(x,
Y, a)]
= WxY exp(6fjl’+6Zf!Z,‘+
~(7-1 R, P) < i S(R”p).
n=O
(b) Relative entropy. The relative entropy is a
measure of the extra dissipation needed to maintain a
non-equilibrium state in a larger environment that is
at thermal equilibrium. We represent the action of
this environment on our system by a stochastic matrix W. Under the influence of W, the system X
would relax to an equilibrium state p”q and the
“thermal equilibrium” condition is taken to mean
that W satisfies detailed balance
I$
. ..).
(3.7)
Then we have
lA,~(R,B,W)l=~~W~y~~,(~)(fl:‘)2.
x.y
(3.8)
To prove Eq. (3.8), we derive the following identities,
350
B. Gaueau, L.S. Schulman / Physics Letters A 229 (1997) 347-353
which are obtained by writing .Y$,RyA = 0 and expanding Eq. (3.7) to order a*. Next, using Eq. (3.5),
IA,S(R,
p’, W) I = CW,,
exp(6$‘+
The proof of Eq. (3.10) was not included in Ref. [6],
so that we now give an indication of its derivation.
Write
S’f$))
W,,=R,,+Sl’,y+...,
&)(Sr::)
Cv,,=O*
x
+ s’fi’:‘).
Then
We write
S(Wkjl P>
13=P&)
+8p,(x)
+0(S*).
= - C [ F( x) + SVjT( x)] lim 1 +
x
(
TX))
(The term of 0 (6 3> is not needed.) Then
IA&R,
= %4,(
xy
P>W)I
Y)W&’
+ cK,Pl(Y)f:(:
XY
1
The term in S drops out because C,,V.v, jT( y) = 0. It
then follows that
+ ..f.
Using the previous identities, this reduces to Eq.
(3.8).
Now consider the system X in its R-dynamics
stationary state, jT A measure of the extra dissipation per unit time step to maintain it there is given by
I A, S( R, jj, W) I. If we switch off the external reservoirs that produce the R dynamics, then the state fi
starts relaxing under the W dynamics and the rate of
dissipation is measured by S(WjT 1 p,,) - S( ,iT1 p,,).
One has the inequality
(3.9)
This inequality is obtained by a lengthy variational
calculation [6].
The dissipation necessary to maintain j? can also
be measured by S(Wj5 I j5). This quantity is the dissipation, during the first time step, of the relaxation
from p under the W-dynamics. One verifies that, up
to higher order terms in 6,
But W=Rexp(-6f(“)=R-SRf(‘)+...,sothat
k+(x) = C,R,r,f,‘~‘~( y>_ It follows that up to terms
of order S
VP( x) = c WJ$‘P(
Y) = c W&)Pes(
Y
and using
detailed
balance
this is VjY(x) =
p,,( x)C,W,,f,(L).
Therefore, up to higher order terms
S(WP
I p) - -;
ZP&)(
x
EWJi:l)‘.
Y
This is Eq. (3.10). The convexity
inequality
cW&:)(
Y))2
Y)]* 2 (CW&‘(
Y
Y
(valid because the square function
C,W,, = 1) implies the inequality
IS(W$l
P)l Q
lS(WjIP)l
is convex
and
IA,,.S(R, p”,W)l.
Finally, comparing Eq. (3.9)
obtain the inequality
(3.10)
Y)
Y
Q lA,P’(R,
and Eq. (3.10),
P,w)I.
we
(3.11)
B. Gaveau. L.S. Schulman/
Physics Letters A 229 (1997) 347-353
4. Reduced description
and then
Usually, a system with a large state space X is
characterized by a small number of slow variables
A ,,. . . , A, (under the R-dynamics). The Aj are in
general complex valued functions on X, and we
assume
-Z(a+6a)
(Aj),i=O.
j= l,...,
n.
A point x in X is thus characterized
x=(A,(x),...,A,(x),
5. Fluctuation-dissipation
theorem
Call (~=(a,,...,
an) the conjugate
the A, and define a state
by
u),
&C(U).
(4.1)
where u denotes the other coordinates (the fast variables). Essentially, the slow variables A,, . . . , A, are
left eigenvectors of R with eigenvalues close to 1
(but not equal to 1). For given values a,, . . . , a, of
A,. . . . A,, we define
ka,Ai+O(r2)
~,(~)=$ij(~)exp
n
k
to
(5.1)
i= I
so that, in the linear
modulo O( LY‘I>,
(A;)P~=
variables
response
(AiAj)pajz
approximation
(i.e.,
(5.2)
(Xa),,
j=l
P(a ,....,u.)=~~~B(X),~,~(A~(X)-~~),
where x,, = ( A, Ajja and
(4.2)
Z(a I,...,
a,) = -limF(a,,...,a
(
”) .
The function 2 is the generalization of the Einstein
entropy to a non-equilibrium
situation.
If we prepare an ensemble in the stationary state
F and observe only those samples in the ensemble
that exhibit a fluctuation a = (a,, . . . , a,) of the slow
variables A,,..., A,,, we can consider this sub-ensemble to define a probability
distribution
q,(x)
given by
p’(x>
4,(x) = - ,,...,
fYa
n
a)
It is then easy to see that the relative entropy as
defined in Section 2 is exactly the opposite of the
Einstein entropy
(4.5)
This justifies taking Z(a) to be a Lyapunov function
for the evolution of the a,, . . . , a, [1,3]. Namely, if
we wait a long enough time so that the a vary from
a to a + Sa, while the fast variables u recover their
conditional stationary distribution, the state q, would
become q,,+ su in the R-dynamics (to a very good
approximation) so that
s(q (I+ fiti I b> - q 4, I D> 2=0
A,(O)),+,
=,~,a,~Ai(x)(R.,-6,,),-(Y)Aj(Y)~
.x.j
(5.3)
where ( jP, denotes the average in the state p, and
Ai
is the evolute A,R of A, in one time step
At. Moreover,
([A;(“)
,fiS(AI(X)eai).
(4e4)
S(q, I p”) = -2(u).
A;(At) -
(4.3)
= -
+(Ryr
[ Aj(At)
-Ai(
CA,(
.I. Y
4[(R,,
-A,(O)]),,,
- ky)F(
- ‘,v)P(x)]Aj(Y)
Y)
+O(aA’)*
(5.4)
We deduce easily
Ca;aj([
i.i
A;(“)
= -2&+A,(Ar)
-A;(O)]
[ Aj(At)
-A;(O)),,<,.
-A,(0)]),jt,
(5.5)
The elimination of the (Y~in terms of ( Ai),><,.using
Eq. (5.2) and the susceptibility x, gives the fluctuation-dissipation
theorem in our context. In particular, we can choose for the Ai left eigenvectors of R,
Ai R = Ai A;. In this case, the susceptibility matrix
B. Gaueau. L.S. Schulman/ Physics Letters A 229 11997) 347-353
352
becomes diagonal. We then obtain
statement of the fluctuation-dissipation
([ Ai( Ar) - A;(O)] [ Aj(
= -#<Ai
Ar) -
Ai(
the following
theorem
)p,
-Ai(0)),nSij
I
(5-6)
Pn
Ref. [12] for the classical
ments).
(see
equilibrium
state-
Consider
a stochastic N X N matrix and denote
the set of states. Let us mark one
point, say j, of this set and call a spanning tree of
root j an oriented tree of root j (the orientation
going from the root to the leaves) so that any 1 =Gk
G N is a vertex of the tree.
Consider an edge (k, I) of such a tree, T. We
assign to this edge the weight R,, and we assign to
the tree T the weight
by (I,...,
W(T)
6. Onsager reciprocity
librium states
relations
so
F)=
n
(k.l) an edge of 7
R,,.
(7.1)
for non-equiThe main statement is that the minor Mjj of - 1 + Rjj
in -I+R
isgivenby
The relative entropy of the state p, is
+,I
=
N)
-~C(Ak)p,(AkA,>6’(A,)p”
k.1
that the corresponding
Mjj=
(-I)+’
(7.2)
I
(6.1)
where the sum is taken over all spanning trees q
with root j as defined above. In particular, Mjj is a
sum of positive quantities, although it was defined as
a determinant.
The stationary state is given by the formula
forces are
a$ P, 1F)
Fk= - a(A,),_
= ~(AkA,>;‘(AI)P,=q.
Fw(TJ,
(6.2)
(7.3)
1
The current
J, for A, is defined by
This replaces the Boltzmann
rium statistical mechanics.
Jk = ( Ak( At) - A,(O)),_
=
x,.v
(6.3)
We can then write
Jk= cLkj4,
(6.4)
with
Lkj=
of equilib-
6,,)P(Y)Aj(Y).
CffjCAk(X)(R.rY-
j
distribution
CAk(x)(R,,-‘.~,)~(Y)Ai(Y).
(6.5)
.V.Y
In general, the matrix L,, is not symmetric.
It
becomes symmetric if we assume that R satisfies
detailed balance, R,, g( y> = R,, E(x) for all x, y.
7. A tree summation
state
formula
for the stationary
In this section, we give a tree integral formula for
the stationary state of a stochastic matrix R (see also
Refs. [13,14]).
8. Conclusion
In this Letter we have presented a summary of a
general framework for non-equilibrium
phenomena.
The basic concept is the stationary state, given as a
probability
distribution
on the state space of the
system, together with a stochastic dynamics. In this
abstract setting we obtain, in a straightforward manner, extension to non-equilibrium
situations of many
of the familiar identities and inequalities of equilibrium theory (decrease of potential, fluctuation-dissipation, Onsager relations). In some cases we find
additional new relations.
In particular, we can extend all the usual results to
dynamics that do not necessarily
satisfy detailed
balance. Such an extension is almost surely needed if
one is to understand systems with emergent organization (see, e.g., Ref. [15]). Moreover, this abstract
point of view is justified by particular models that
l3. Guueuu, L.S. Schulman/
Physics Lerrers A 229 (1997) 347-353
have been previously analyzed 1161 in various contexts.
Although our efforts, as described here, have been
mostly devoted to providing parallels in the nonequilibrium case of equilibrium structures, there is
one set of quantities which, by definition, do not
appear at all in the equilibrium case. These are the
currents, J.yy= R,r, jT( y) - R,, p’(x). It is these that
allow the stationary state to nevertheless be dynamical. Interestingly, it is analogous quantities that one
generally uses in the description of biological, chemical, meteorological, and many other processes. We
expect that these markers of non-equilibrium, the
currents, will be important for the description of
complex phenomena and even in the elucidation of
the elusive concept of complexity itself.
Acknowledgement
We thank S. Mitter, M. Moreau, C. Rockland and
L.J. Schulman for helpful discussions. This work
was supported in part by the NSF grant PHY 93
16681 (L.S.S.) and by a European Union grant
“Capital humain et mobilite” 930096 (B.G.).
References
[I] 1. Prigogine,
Non-equilibrium
statistical mechanics (Wiley,
New York, 1962);
G. Nicolis and I. Prigogine, Self-organization
in non-equilibrium systems (Wiley, New York, 1977).
353
[2] S.R. de Groot and P. Mazur, Non-equilibrium
thermodynamics (Dover, New York, 1984).
131 J. Keizer. Statistical thermodynamics
of non-equilibrium
processes (Springer, New York, 1987).
[4] R. Kubo, Statistical mechanics (North-Holland,
Amsterdam,
1988).
[Sl J. Ross, K.L.C. Hunt and P.M. Hunt, J. Chem. Phys. 96
(1992J618;
Q. Zheng, J. Ross, K.L.L. Hunt and P.M. Hunt. J. Chem.
Phys. 96 (1992) 630;
M.O. Vlad and J. Ross, J. Chem. Phys. 100 (1994) 7268,
7295.
161 B. Gaveau and L.S. Schulman, J. Math. Phys. I2 ( 1996).
[7] M. Moreau, J. Math. Phys. 19 (1978) 2494.
[8] R.C. Tolman.
The principles
of statistical
mechanics
(Clarendon, Oxford, 1938).
191 1. Procaccia. Y. Shimoni and R.D. Levine. J. Chem. Phys. 65
( 1976) 3284;
R.D. Levine, J. Chem. Phys. 65 (I 976) 3302;
1. Procaccia and R.D. Levine, J. Chem. Phys. 65 (1976)
3357.
[lOI T.M. Cover and J.A. Thomas, Elements of information theory (Wiley, New York, 1991).
[I 1I D. Forster, Hydrodynamic fluctuations, broken symmetry and
correlation
functions,
Frontiers
in physics,
no. 47
(Benjamin-Cummings,
Reading, MA, 1975).
[12] D.J. Aldous, SIAM J. Disc. Math. 3 (1990) 450.
1131 PG. Doyle and J.L. Smell, Random walks and electric
networks (Math. Assoc. of America, Washington, DC, 1984).
[I41 D.L. Stein, ed.. Lectures on the sciences of complexity
(Addison-Wesley.
New York, 1989).
[ 151R.J. McGraw and L.S. Schulman, J. Stat. Phys. I8 ( 1978)
293;
B. Gaveau and L.S. Schulman, J. Phys. A. 20 (1987) 2865; J.
Stat. Phys. 74 (1994) 607; 70 f 1993) 613;
P. Bak, Physica A 163 (1990) 403;
D. Ben Avraham, R. Bidaux and L.S. Schulman, Phys. Rev.
A 43 (1991) 7093;
B. Yu and D.A. Browne, Phys. Rev. E 49 (1994) 3496.