s
t = no of steps of
length s
Figure 1: Schematic of the path of a diffusing molecule, for example, one in a gas or a liquid. The particle is
moving in steps of length s. For a molecule in a liquid the steps are of length approximately equal to the size
of a molecule, i.e., s ≈ 0.1nm. The time taken for one step is this length divided by the speed of a molecule.
1
Dissipation: Making entropy
In thermodynamics you learnt that entropy always increases, it never decreases. Now, we will look at how
the entropy increases in a specific system: a single diffusing molecule. We will see how and why diffusion
increases entropy. So, this section looks at dynamics, in particular at what are called dissipative dynamics.
Dissipative dynamics are, by definition, dynamics that increase the entropy, as opposed to reversible dynamics
which keep the total entropy constant.
To keep things simple let us consider a single particle that starts off at the origin, x = 0, at time t = 0. It
will then diffuse away from the origin and this diffusion will increase the entropy associated with the particle.
To keep the notation simple we will set the length of a single hop of the molecule s = 1, i.e., we will work in
units of length equal to that of a single hop. For a molecule in a liquid a single hop of a molecule is around
0.1 nm. Also, our units of time will be the time for a single hop, around a picosecond in a liquid. Thus t = 1
is the time elapsed for a single hop, t = 2 the time for two hops etc. We will also work in one dimension.
So, after a time t, the particle has hopped t times. Let l of these hops be to the left and r = t − l be to the
right. The position of the particle x is just the number of hops it has made to the right minus the number it
has made to the left x = r − l = t − 2l. So, the particle has made a total of t hops, each of which has equal
probability of being to the left or to the right, and we want to know what is the probability that l of these
hops were to the left. This probability is just the number of different possible sets (arrangements) of l hops
to the left and r to the right, divided by the total number of different sets of t hops. The total number is just
2t . For 1 hop there are 2 possibilities, left and right, for 2 hops there are 22 = 4 possibilities, left left, left
right, right left and right right, etc. The number of sets of t hops that contain l hops of one type, to the left,
and r = t − l of the other type of hop, is t!/l!r!. Thus the probability of the particle taking l hops to the left
and hence being at position x is
1 t!
p(x) = t
.
(1)
2 l!r!
You recognise this as being just the same (except with t substituted for n) as the expression for the probability
of l particles out of a total of n (equivalent to t here) being in the left half of a box. Thus the maths is just
the same as for the box divided into two. As before, we take the log of the probability, find that it has a
maximum at l = r = t/2, which corresponds to x = 0, and then Taylor expand at that point. The Taylor-series
1
expansion around x = 0 gives us
1
p(x) ∝ exp − x2
t
t ≫ 1.
(2)
Before the Gaussian held for large n, here it holds for large t. This is because it is the result of a Taylor series
in which they have dropped all terms beyond the quadratic. This is OK for t ≫ 1 but not for t that is not
much larger than 1. Now, we know that p(x) must be normalised, it must add up to 1
∞
X
p(x) = 1.
(3)
x=−∞
For t ≫ 1 there are many terms in this sum and we can approximate it by an integral
Z ∞
p(x)dx = 1.
(4)
−∞
Introducing an, as yet unknown, normalisation constant c,
1 2
p(x) = c exp − x ,
t
we find c by substituting this expression for p into the integral
Z ∞
1 2
exp − x dx = 1.
c
t
−∞
The integral is the standard Gaussian integral that is in your formula booklets
Z ∞
π 1/2
exp −αx2 dx =
α
−∞
(5)
(6)
(7)
and so we have that
c (πt)1/2 = 1,
(8)
1 2
1
exp − x .
p(x, t) =
t
(πt)1/2
(9)
and c = 1/(πt)1/2 . Thus, we have that
Note that the probability, p(x, t) of finding the particle at position x is a function of time. In Fig. 2 we have
plotted the probability as a function of x for 2 different times.
Now, p(x, t) is symmetric around x = 0, and so the mean value of x, hxi = 0. This is because the particle is
as likely to go left as right. However, the mean of the square of the distance from the origin, hx2 i, is non-zero.
Using standard integrals over Gaussian functions (given for example in the little booklet you get in exams) it
can be calculated and is
Z ∞
Z ∞
1
1 2
1
1 1
2
2
2
3 1/2
x p(x, t)dx =
hx i =
x
exp
−
πt
= t
x
dx
=
(10)
1/2
1/2
t
2
(πt)
(πt) 2
−∞
−∞
and so the root-mean-square distance moved is
p
t1/2
hx2 i = 1/2
2
2
(11)
0.2
p(x,t)
0.15
0.1
0.05
0
-10
0
-5
x
5
10
Figure 2: Plots of Gaussian p(x, t), Eq. (9), as a function of x, for two different times t. The solid curve is
for t = 10 and the dashed curve for t = 20. At the later time the probability distribution function is broader,
the particle can be found over a longer part of the x axis. It is this increase in the uncertainty of the position
(which defines its state here) of the particle that leads to the increase in its entropy.
i.e., hx2 i increases linearly in time — the root-mean-square displacement increases as the square root of time.
This is characteristic of diffusion, if the time elapsed is doubled the typical distance travelled increases only
by a factor of 21/2 ≃ 1.4.
Now, Shannon’s expression for the entropy is
X
S=−
pi ln pi
(12)
i
For the times t ≫ 1 that we are considering we can approximate this sum by an integral, i.e.,
Z
X
−
pi ln pi =⇒ − p(x) ln p(x)dx
(13)
i
and so the entropy at time t is given by
S(t) = −
Z
p(x, t) ln p(x, t)dx
(14)
where we have indicated that the probability p is a function both of position x and time t, i.e., the probability
that the particle is at position x depends on time t. As the entropy is an integral over x it does not depend
on x, only on t. Now, to do this integral we need to substitute the probability of Eq. (9) into this equation.
This can be done but we can get an p
approximate answerp
and simplify the maths a bit if we realise that at
1/2
2
time t the particle is mostly between − hx i = −t
and hx2 i = t1/2 . If we neglect p(x, t) outside of this
range then we can approximate the Gaussian function probability by a step function
1/(2t1/2 ) −t1/2 ≤ x ≤ t1/2
p(x, t) =
(15)
0
outside this range
3
Figure 3: Ink diffusing in water. N.B. The swirls indicate that there is also flow (which is different from
diffusion) going on. This makes the picture look pretty but is not taken into account in the problem and is
not covered by the course.
The probability is 1/(2t1/2 ) due to normalisation. The 2 in the denominator is needed to ensure that the
integral with respect to x over p(x, t) equals one. With this approximation for p(x, t)
S = −
=
Z
Z
t1/2
1
ln
2t1/2
1
2t1/2
p(x, t) ln p(x, t)dx = −
−t1/2
1/2
ln 2t
1
× 2t1/2 = ln 2t1/2 = ln 2 + ln (t) .
2
2t1/2
dx =
1
1/2
ln
2t
2t1/2
Z
t1/2
dx
−t1/2
(16)
So, we are interested in how the entropy S varies with time. We have found that it increases, as we should
expect — 2nd Law of Thermodynamics says it should. This is clear if we take the time derivative of the
entropy dS/dt = 1/(2t) > 0 — it is always greater than 0.
The entropy increases because as time passes the length (volume for 3-dimensional diffusion) over which
the particle may be found increases. Thus the uncertainty in the position of the particle increases, and so
its entropy is larger. Another way of saying this starts by noting that at t = 0, before diffusion occurs, the
particle is definitely at the origin. So it is definitely in one state and so has an entropy of S = 0. After a
time t it can be any one of order t1/2 positions, i.e., t1/2 states. If we assume that these states are all roughly
equally likely then we can use the fact that the entropy S = ln Ω for a system with Ω equally likely states and
again get that S = ln Ω = ln(t1/2 ) = (1/2) ln t.
1.0.1
Equipartition
Before we consider diffusion in a gas we need to use the Equipartition Theorem. This tells us the average
kinetic energy of, amongst other things, molecules in a classical gas. We will need the average kinetic energy
in order to estimate how rapidly molecules diffuse. Also, once we know the average kinetic energy we can
calculate the average velocity and momentum.
Now, we know that the probability that an atom has a velocity component v along one direction, say the
4
x axis, is given by
K.E.
mv 2
p(v) ∝ exp −
= exp −
kT
2kT
(17)
Thus the energy is a quadratic function of velocity. Equipartition then tells us that the mean K.E. equals
(1/2)kT .
1
1
(18)
hK.E.i = h mv 2 i = kT
2
2
As the mass m is constant we can take it out of the average
1/2
kT
2
2 1/2
(19)
mhv i = kT
hv i
=
m
the typical, root-mean-square, velocity is proportional to the square root of kT and inversely proportional
to the square root of the mass m. Let us consider the Earth’s atmosphere. Here N2 is the most abundant
1/2
molecule. For N2 , m = 5 × 10−26 kg and kT = 4 × 10−21 J at room temperature. Thus hv 2 i
= 300 m s−1
— which is about the speed of sound. In fact the speed of sound in a gas is approximately set by the speed
of molecules in it.
1.1
Diffusion in a gas
We have seen that diffusion is essentially a series of hops with random directions, and that the typical distance
travelled increases only as the square root of time. We found this to be true in 1 dimension but it is also true
in 3 dimensions. In 3 dimensions we have a vector displacement r(t), which has a magnitude r(t). The mean
of the square of r increases linearly with time. Thus for a particle that was at the origin at time t = 0, at
some later time t
hr2 i ∝ t.
(20)
Now, dimensional analysis tells us that the proportionality constant between hr2 i and t has dimensions of
length squared divided by time. Essentially, the length in the diffusion constant is the size of the hops we
considered in the previous section, and the time is the time a single hop takes. The proportionality constant
defines the diffusion constant D. In fact it is 6 times D, so we have
p
hr2 i = 6Dt
hr2 i = (6Dt)1/2 .
(21)
The factor of 6 appears because the diffusion constant is defined so that the flux of molecules per unit area
j = −D∇c(r), where c(r) is concentration of molecules per unit volume at position r. This equation is called
Fick’s Law. It leads to the diffusion equation you saw in level two
∂c
∂2c
= D 2.
∂t
∂x
(22)
Note that D appears both when we consider a microscopic phenomenon, and a macroscopic phenomenon. The
microscopic phenomenon is the motion of a single molecule that is diffusing. The macroscopic phenomenon is
the flux of particles per second through some large area, which is due to the motion of many many molecules
not one. The same constant, D, controls both. This means that gases of molecules that diffuse rapidly also
have large fluxes j, when there is a gradient of the concentration ∇c(r).
1.1.1
Calculation of an estimate for D
In this section we will estimate the diffusion constant D in gases like the Earth’s atmosphere as it is this that
allows us to work out how far molecules can diffuse in a given time, which molecules diffuse rapidly and which
diffuse slowly etc.
5
λ
σ
v
volume = πσ 2 λ
Figure 4: Schematic diagram of a gas atom (left-hand side black circle) travelling a distance λ in a straight
line at a speed u, before colliding with another gas atom (right-hand side black circle). After the collision
direction of travel of the atom is very different from that before. It then moves off along this new direction,
shown by the dotted arrow and circle.
The diffusion constant D has dimensions of length squared over time. The length is the distance of a
single hop and the time is the time taken during a single hop. The length of a hop being the distance over
which a molecule moves in a straight line, which is the average distance between collisions. Collisions tend to
randomise the direction of travel of molecules so a diffusing molecule undergoes a random walk in which the
hops or steps are the distances between collisions. If the length of a typical hop is λ and the time per hop is
τ , then we have
D ≈ λ2 τ −1 .
(23)
For example, if a particle is making hops of length 1nm and it makes 1010 hops per second then the diffusion
constant D ≈ (10−9 )2 /10−10 = 10−8 m2 s−1 . Note that this only gives an approximation to D. Calculating
diffusion constants exactly can be done but is complicated. During this course we only consider simple
approximate ways to calculate diffusion constants.
So, to estimate the diffusion constant we need to estimate: 1) the length of the hops, λ, and 2) the time
per hop, τ . For molecules in a gas (like the Earth’s atmosphere) this is easy. A hop is the distance a molecule
travels in a straight line, before colliding with another molecule and so heading off in another direction. The
atoms behave just like billiard balls or ball bearings colliding with each other. Imagine a box full of tiny ball
bearings (the atoms) each travelling at the speed of sound. The atoms are a few tenths of a nm across and
the typical separation between the atoms is about ten times that, about 1nm. Atoms at room temperature
are moving with a typical speed, u, of around the speed of sound, a few hundred m s−1 .
Each atom will collide with any other atom it gets within a few tenths of a nm of. We call this distance
σ, it is the size of the atom, i.e., of the cloud of electrons that surrounds the nucleus. Thus, as it moves each
atoms sweeps out a cylindrical volume of radius σ. See Fig. 4. If the typical distance between collisions with
another atom is λ then it sweeps out a volume of πσ 2 λ before it collides with another atom. Once we have λ
we can find the time for a single hop, as it is just the hop length over the speed, τ = λ/u. Here u is a typical
speed, which is the root mean square of the velocity u = (hv 2 i)1/2 .
Now, by definition πσ 2 λ is the volume an atom sweeps out before it encounters and collides with one other
atom. But we know the average volume that contains one atom, it is just V /N , where V is the total volume
and N is the total number of molecules. Thus we have that
σ 2 λ ≈ V /N
or
6
λ≈
V
σ2N
(24)
where we dropped the factor of π as we are only calculating a rough order of magnitude estimate. The V /N
factor can be understood if we note that if we divided the box of volume V into N little cubes each of volume
V /N then on average each cube would contain 1 atom. These cubes have sides of length (V /N )1/3 . Then if we
consider two adjacent cubes each with 1 atom these two adjacent atoms will be of order (V /N )1/3 apart. Thus
the typical separation between atoms s = (V /N )1/3 . The hop length λ can then be written as λ ≈ s3 /σ 2 .
Having determined λ we need the time per hop, which is the time the atom takes to move a distance
λ. We call this time τ and it is given by τ = λ/u, for u the typical velocity of the atom. Now, from the
equipartition theorem in the previous section
1/2
u ≈ hv i
=
2
kT
m
1/2
.
(25)
Now that we know u we know τ = λ/u and we have everything. Thus the diffusion constant is approximately
D ≈ λ2 /τ = λ2 /(λ/u) = λu ≈
s3 u
.
σ2
(26)
In the Earth’s atmosphere there are approximately N/V = 1026 molecules per unit volume. Thus the typical
separation s = (V /N )1/3 = 2 nm. The molecules are roughly a fifth this size, σ = 0.4 nm. Also, for O2 at
room temperature v = (KT /m)1/2 = (4 × 10−21 /5 × 10−26 )1/2 ≈ 300 m s−1 . Thus the diffusion constant of
the molecules in the atmosphere
D≈
2 × 10−9
(0.4 ×
3
× 300
10−9 )2
≈ 10−5 m2 s−1 .
(27)
This estimate is close to the measured value for the diffusion constant for O2 in air at 20◦ C, which is 2 ×
10−5 m2 s−1 . So, in 1s an oxygen molecule diffuses a distance of approximately (6D × 1)1/2 ≈ (6 × 10−5 )1/2 ≈
10−2 m= 1 cm.
A molecule diffuses rapidly over distances as short as a cm. However, as the distance increases only as
the square root of time, the time required to move larger distances becomes quite long. For example, it takes
10, 000 s ≈ 3 hours to diffuse 1 m, and a day to diffuse 10 m. In practice, movement over distances of a metre
or more occurs not by diffusion but by convection and other mechanisms in which the air flows.
7
5
u(x) / kT
4
3
1/2
1/2
2
(< x 2 > ) = ( kT / φ )
1
0
-10
0
-5
x / nm
10
5
Figure 5: Schematic of a one-dimensional quadratic potential well, u = (1/2)φx2 , for a force constant φ =
0.1kT /nm. The approximate size of the root-mean square displacement from the minimum, (hx2 i)1/2 , is
shown.
2
Fluctuations and Response
There is a general relationship between how much a variable fluctuates due to thermal energy, and how much
the mean value of a variable changes when an external field is applied. In its most general form, the theorem
that relates the two won Lars Onsager the Nobel prize in chemistry in 1968. We will consider how it works
for a simple example. This is a (classical) particle moving along the x axis in a harmonic potential. So, the
energy of the particle as a function of x, u(x), is given by
1
u(x) = φx2
2
(28)
Here the spring constant is φ (we can’t call it k as we are using k for Boltzmann’s constant). With this energy
the Boltzmann weight for position x of the particle is exp[−φx2 /(2kT )]. The probability of the particle being
at position x, p(x), is proportional to this weight. So if we introduce the proportionality constant c, we have
p(x) = c exp[−u(x)/kT ] = c exp[−φx2 /(2kT )]
(29)
As the problem is symmetric around the origin the particle is as likely to be left of the origin as right of the
origin and the mean value of the position of the particle hxi = 0.
This is of course useless as a measure of how far thermal motion of the particle can take the particle away
from the origin. So we need the root of the mean of the square of x, (hx2 i)1/2 . This can be estimated as we
know that the particle will have about kT thermal energy at a temperature T . This is sufficient to get it a
distance x from the origin given by
1 2
φx = kT
(30)
2
If we approximate (hx2 i)1/2 by the value of x2 in this equation we have
1
φhx2 i ≈ kT
2
kT
hx i ≈
φ
2
2
1/2
(hx i)
≈
kT
φ
1/2
(31)
where we dropped the factor 1/2 after the first equation as the value we get is just an estimate. So the typical
distance from the origin varies with temperature as T 1/2 , and with the size of the force constant φ, as 1/φ1/2 .
8
So, now we have characterised the size of the fluctuations of the position x of the particle, they are of
order (kT /φ)1/2 . We will now look at how the mean value, hxi, varies if we apply an external force. With an
external force of strength f the energy as a function of position is now
1
u(x) = φx2 − f x
2
(32)
The contribution to the energy is −f x because a force pushing the particle to the right (f > 0) lowers the
energy to the right, at positive values of x. Thus the probability of the particle being at position x is now
proportional to the new Boltzmann weight
p(x) = c exp[−u(x)/kT ] = c exp −φx2 /(2kT ) + f x/kT
(33)
The mean value of x is then
hxi =
Z
∞
xp(x)dx = c
−∞
Z
∞
−∞
x exp −φx2 /(2kT ) + f x/kT
(34)
For non-zero f this will be non-zero.
Now, to see the relationship between the response of the mean position to an external force, and the
fluctuations we take the derivative of hxi with respect to f
Z ∞
Z ∞ x
∂hxi
∂
2
x exp −φx /(2kT ) + f x/kT dx = c
c
x exp −φx2 /(2kT ) + f x/kT dx
=
∂f
∂f
−∞ kT
Z ∞−∞
Z ∞
c
1
=
x2 exp −φx2 /(2kT ) + f x/kT dx =
x2 p(x)dx
kT −∞
kT −∞
1 2
∂hxi
=
hx i
(35)
∂f
kT
In words: the rate of change of the mean position, hxi, equals 1/kT times the square of the size of fluctuations
in position, hx2 i. The larger are the fluctuations in x, the larger is the change in the mean of x when a force
is applied. Another way of putting the same thing is to say that the size of the fluctuations, hx2 i, equals kT
times the rate of change of the mean when an external force is applied.
As we have estimated the size of the fluctuations we also know the rate of change of hxi with f
∂hxi
1 2
1
kT
1
=
hx i ≈
×
=
∂f
kT
kT
φ
φ
(36)
Finally, here we have only proved that there is a relationship between the fluctuations in the position of
a particle, and the rate of change of the mean position of the particle with the strength of an external force.
However, the relationship is in fact general. Consider any variable, call it a, that has a mean of zero without
an external force, but fluctuates. Then if an external force fa is applied, such that it makes a contribution to
the energy of −fa a, then we have
1 2
∂hai
=
ha i
(37)
∂fa
kT
9