Statistical Mechanics in Compl
ex Networks with Power Law
Distribution
Xin, Huolin
School of Physics, Peking Univeristy
[email protected]
1.1.
Introduction
From the notion of statistical mechanics, it is known that distributions dominate the
nature phenomenon and the world structures. Some of them are discovered and well
interpreted; some of them are invisible to us or beyond our understanding. Barabási
and Albert in 1999 suggest a power law distribution which was found in a number of
complex networks. They are called scale free networks. Such kind of power law
distribution can describe a variety of systems both in the nature, society and
man-made high-tech visual world, such as the World Wide Web, which is an
enormous visual network of human intelligence. In this network, the nodes are the
documents and the edges are the hyperlinks that point from one document to another.
It is discovered that the degree distribution of web pages follows a power law
distribution [Albert, Jeong, and Barabási, 1999]. Both probability that a document has
k outgoing hyperlinks, pour (k ) , and the distribution of incoming edges, pin (k )
have power-law tails.
(1)
pour (k ) ~ k  our and pin (k ) ~ k  in
This can be interpreted as the more links one has today, the more links it will have
tomorrow. However, this is only on the surface level of the experiment understanding.
The deep physics meaning are not revealed.
In this paper, we try to understand such phenomenon from different perspective by
treating networks as thermodynamics system. At first glance, it sounds rude because
network is not highly thermalized. However, further reflection tells that it is actually
its intrinsic properties that we have ignored before. Take social networks for example.
As members of a community and citizens of a country, we never stop doing our effort
to keep our society stable against the “second law of thermodynamics”. Otherwise,
the social network will break up. This gives us a hint that, in most cases, the
“uncertainty” of a stable network will not change over time. To interpret it in physics
is that the mean entropy of a network is a constant. On this assumption, we will in the
following sections show how it works to demonstrate the power law distribution.
Chapters’
title
2
The paper is organized as follows: the next section briefly introduces the
information theory and maximum-entropy estimates as a mathematical tool in section
1.3. In section 1.3, the derivation of the power law distribution will be demonstrated.
1.2.
Information Theory and Maximum-Entropy Estimates
Information theory provides a constructive criterion for setting up probability
distributions on the basis of partial knowledge, and leads to a type of statistical
inference which is called the maximum entropy estimate [E.T. Jaynes 1957]. Where
the Shannon entropy are usually used [S. Kullback 1959]:
H [ P( x)]  k  Pi ln Pi ( x)
(3)
i
It is the least biased estimate possible on the give information. Suppose
f ( x)   Pi f ( xi )
(4)
i
The quantity x is capable of assuming the discrete values xi (i=1,2…, n). We are not
given the corresponding probabilities pi; all we know is the expectation value of
function f(x); and the normalization condition
(5)
 Pi  1
i
From the merely facts equations (3) and (4), we want to find the probability
assignment which avoids bias, while agreeing with whatever information is given.
Therefore, in making inferences on the basis of partial information we must use that
probability distribution which has maximum entropy subject whatever is know. The
entropy is defined by Shannon, which reads
H [ P( x)]  k  Pi ln Pi ( x)
(6)
i
To maximize (5) subject to the constraints (3) and (4), Lagrangian multipliers ,  are
introduced
F   Pi ln Pi ( x)   ( Pi f ( xi )  f ( x) )  (  1)( Pi  1)
(7)
i
i
i
F
 ln Pi ( x)  1  f ( xi )  (  1)  0
Pi
(8)
from equation (7) we could obtain the result
Pi  e λ μf ( xi )
(9)
the constants ,  are determined by substituting into equation (3) and (4). The result
can be written in the form

ln Z (  )

  ln Z ( )
f ( x)  
where
Z ( )   e
i
it is called the partition function.
 f ( xi )
(10)
(10)
(11)
3
Chapters’ title
1.3.
The Derivation of Power Law Distribution
In section 1, we consider the mean entropy of system as a constant. In the language of
information theory, it means the expectation value of entropy S(x) is known, namely
S ( x)   Pi i ( x)Si ( x)
(12)
i
Pi is the probability of finding a system in one of the states corresponding to the ith
element (or group). where the entropy of each element in the system is represented by
the Boltzmann entropy,
Si ( x)  k B ln i ( x)
(13)
where the I is number of states of the ith element (or group). And the normalization
condition reads
(14_
 Pi  1
i
The Shannon entropy is
H [ P( x)]  k  Pi i ( x) ln Pi i ( x)
(15)
i
maximize (15) subject to the constraints (12) and (14)




F   Pi i ( x)ln Pi i ( x)     k B Pi i ( x)ln i ( x)  S   (  1)  Pi  1
i
 i

 i

therefore
(16)
F
  ln Pi i     kb ln i ( x)   0
Pi (i ( x))
i
i
(17)
Pi i ( x)  exp(  kB ln i ( x))  Ai
(18)


where A = e ; use constraint (13), A could be determined
A
1
1



Z
 i
(19)
i
hence that
Pi 
 i
  i
(20)
i
Equation (20) shows us the power law we expect.
1.4.
Application on Networks
Networks are likely to be interpreted as graphs with nodes linked by edges. Simple
behavior of each node (identical or not) can give a complex collective behaviors in a
network. However, there are some important quantities that would generalize the
topology of a network.
The first one is the degree (the number of edges of a node) distribution. It
describes how links distributed among different nodes. Here, we can treat the number
of state of the ith element (i) as the degree of the ith node, because the more
Chapters’
title
4
connections one node have the more opportunity it can interact the other nodes,
namely
 i ~ ni
(21)
where ni is the degree of the ith node. Then substitute it into equation (20)
Pi 
ni
 ni
 Ani
(22)
i
This give us very clear physics meaning : if a network is dynamically stable (or
temporarily say stable), the degree distribution would follow the power law.
Remember, dynamically stable is very important here. If new nodes are inserted
randomly into the network continuously, the power law will break up gradually.
The second one is the aggregation. The aggregation in networks is termed as
clustering. Usually, the clusters in a network can be treated as sub-networks, which
have various amounts of nodes and are relatively isolated to the other notes of their
supplementary network. Take the telecommunication network for example. Local
telecom network, such as a network of city, can be treated as a sub-system of the state
network. Traffic in such networks are main local calls. However, the toughest thing to
deal with is how we count the number of state of a sub-network, if we want to take
advantage of equation (20). To think about it, we’d better trace back to the (21),
where we used the opportunity interpretation for a single node. Can we use the same
inference? The answer is no but a little bit similar. Suppose a sub-group with M nodes.
The maximum links the network could have is M(M1)/2, but it is not the number of
states of the sub-network because it only relates to the number of nodes in the
network and has nothing to do with the other properties of the network. It seems we
have to discover a new quantity to serve our need. However, predecessors of network
topology have already solved the problem. They suggest another quantity named
average path length which is the average of the shortest distance between any nodes
[Albert, Barabási 2002]. Therefore the number of states could be easily defined as
 i  C Ll i
(23)
Where L is total path length of ith network and l is the average path length; usually in
an effective network
(24)
L~M 0 2
Then
 i  C Ml  
M  ( M   1)  ( M   l  1) M l

l!
l!
(25)
where M >> l; substitute it into equation 20
Pi  BM  
(26)
equation (26) shows very clear physics meaning that the size distribution of
sub-network follow the power law.
References
Réka Albert, Albert-László Barabási, 2002, Statistical mechanics of complex networks, Rev.
Mod. Phys., 47, 1
Chapters’ title
S. Kullback, Information Theory and Statistics, Wiley, New York, 1959
E. T. Jaynes, Information Theory and Statistical Mechanics, Phy. Rev. 106, 4
5