Probing the extensive nature of entropy
N. Chetty1,2
1 Department
2 National
T. Salagaram1
of Physics, University of Pretoria (UP)
Institute for Theoretical Physics (NITheP)
CCP - Kobe, Japan, 14-18 October 2012
University of Pret
N. Chetty, T. Salagaram (UP, NITheP)
CCP 2012
1 / 29
Overview of computational physics research
Fundamental CPHY
Algorithmic PHY
Applied CPHY
* Large production codes
* Use high level languages
* Operate software (Black box)
* Short computational problems * e.g. Mathlab, Mathematica, EJS * e.g. large production codes
* Engaged in all aspects
* Good algorithmic understanding, * Superficial understanding of
of the computational project
but numerical methods are hidden algorithmic and programming details
⇒ Computational physicists
⇒ Algorithmic physicists
⇒ Computational technicians
We need to develop more fundamental computational physicists ⇒ marketable skills
1 Develop short computational problems for the physics classroom
Probing the extensive nature of entropy (main focus of this talk)
Undergraduate students
2 Develop paradigms for large production codes
e.g. Electronic structure problem (brief presentation of ideas)
Graduate students
N. Chetty, T. Salagaram (UP, NITheP)
University of Pret
CCP 2012
2 / 29
Part [1]: Short computational problems for the physics
classroom
University of Pret
N. Chetty, T. Salagaram (UP, NITheP)
CCP 2012
3 / 29
Probing the extensive nature of entropy
Entropy is measure of state of randomness of a system
Students still have conceptual difficulties understanding Entropy
We have developed a new algoritmn to compute entropy efficiently
Applicable to any single-particle spectrum
Am. J. Phys 79 (11), November 2011, p1127.
Microcanonical ensemble
Recursive method sN (e) → sN+1 (e)
Independent of the single-particle spectrum,
sN (e) → s∞ a constant as N → ∞ extensive nature
University of Pret
N. Chetty, T. Salagaram (UP, NITheP)
CCP 2012
4 / 29
New developments
Demonstrate for the first time the manner in which entropy
approaches its thermodynamic limit
(sN (e) − s∞ ) → N −α independent of the details of the single
particle spectrum
Derive thermodynamic properties for finite system sizes
Demonstrate entropic interference
Enhance the understanding of entropy
University of Pret
N. Chetty, T. Salagaram (UP, NITheP)
CCP 2012
5 / 29
Harmonic oscillator
Entropy per particle vs Energy per particle
30
N=5
N=10
Entropy per particle
25
20
15
10
5
0
0
20
N. Chetty, T. Salagaram (UP, NITheP)
40
60
Energy per particle
80
100
University of Pret
CCP 2012
6 / 29
Counting microstates: A Recursive Method for Ω(N, E)
N independent, distinguishable, noninteracting particles
Ω(N, E) = number of microstates accessible to the system with N
particles and total energy E
1 particle
N−1 particles
N particles with energy E
Ω(N, E) =
E
X
0
0
Ω(N − 1, E − E ) × Ω(1, E ).
University of Pret
(1)
0
E =0
N. Chetty, T. Salagaram (UP, NITheP)
CCP 2012
7 / 29
Inefficient algorithm
Brute force method
Ω(N, E) increases exponentially with N and E
Several million years to do a moderate number of particles
Numbers become too large → overflow errors
A small improvement by counting in terms of r = smallest number
addressable by the computer
We don’t need Ω(N, E), we need log Ω(N, E) !
University of Pret
N. Chetty, T. Salagaram (UP, NITheP)
CCP 2012
8 / 29
A Recursive Method for S(N, E)
S(N, E) = S(N−1, E) + kB ln
E
X
0
E =0
exp
!
0
0
S(N − 1, E − E ) − S(N − 1, E)
×Ω(1, E ).
kB
We were first to write down this recursive statement for entropy
Calculations with N = 2000 and E = 2 × 104 takes of order a day
to run.
University of Pret
N. Chetty, T. Salagaram (UP, NITheP)
CCP 2012
9 / 29
Model
N independent, distinguishable, noninteracting particles
Single particle energy spectrum: e = p, e = p2 , e = p3 ,..,
e = p2 + q 2 , etc.
Non-trivial structure (degeneracies)
Not analytically solvable, e.g. e(p, q) = p2 + q 4
Number of microstates accessible to the system: Ω(N, E)
Entropy: S(N, E) = kB ln Ω(N, E).
University of Pret
N. Chetty, T. Salagaram (UP, NITheP)
CCP 2012
10 / 29
Extensive behaviour
3.5
iv,v
Entropy per particle / kB
3.0
i,ii,iii
Model 2 (ε = p2+ q4)
2.5
Model 1 (SHO)
2.0
1.5
1.0
0.5
0.0
0
2
4
6
Energy per particle e
8
10
Model 1: Harmonic oscillator ε = p
(i) s∞ (e) = (1 + e) log(1 + e) − e log(e)
(ii) s∞ calculated in the canonical ensemble
(iii) s3000 calcuated using our recursive method
Model 2: ε = p2 + q 4
University of Pret
(iv) s∞ calculated in the canonical ensemble
(v) s3000 calcuated using our recursive method
N. Chetty, T. Salagaram (UP, NITheP)
CCP 2012
11 / 29
sN (e) → s∞ a constant as N → ∞
True for any single particle spectrum ⇒ extensive nature of
statistical entropy.
University of Pret
N. Chetty, T. Salagaram (UP, NITheP)
CCP 2012
12 / 29
Approach to thermodynamic limit
-0.002
e=10
-0.004
-0.006
[sN(e)-s∞(e)] / kB
-0.008
-0.010
-0.012
-0.014
-0.016
-0.018
-0.020
-0.022
0
500
1000
1500
2000
Number of particles N
2500
3000
N = 100 → N = 1000 results in a reduction in the deviation of
sN (e) from s∞ (e) by less than 1.5%.
University of Pret
s∞ (e) approached slowly from below → power-law behaviour.
N. Chetty, T. Salagaram (UP, NITheP)
CCP 2012
13 / 29
Power-law behaviour
-3.5
e=10
log|sN(e)-s∞(e)|
-4.0
-4.5
-5.0
-5.5
-6.0
-6.5
5.5
6.0
6.5
7.0
7.5
8.0
8.5
log N
Linear plot confirms power-law behaviour.
N. Chetty, T. Salagaram (UP, NITheP)
University of Pret
CCP 2012
14 / 29
Universal behaviour
1.00
ε=p
ε=p2
ε=p3
0.90
ε=p4
ε=p2+q4
α(e)
0.80
0.70
0.60
0.50
0
2
4
6
8
10
Energy per particle e
Universal exponent α(e) for entropy
N. Chetty, T. Salagaram (UP, NITheP)
University of Pret
CCP 2012
15 / 29
Finite system sizes
0.000
-0.004
-0.006
-0.008
-0.010
-2.0
-2.5
η(e)
[sN(e)-s∞(e)] / kB
-1.5
e=1
e=2
e=3
e=4
e=5
e=6
e=7
e=8
e=9
e=10
-0.002
-0.012
-3.0
-0.014
-3.5
-0.016
-0.018
-4.0
-0.020
-0.022
0.000
0.001
0.002
0.003
0.004
0.005
0.006
0.007
-4.5
N-α(e)
0
2
4
6
Energy per particle e
8
10
1st order correction to s(e):
sN (e) = s∞ (e) + η(e)N −α(e) as N → ∞
University of Pret
η(e) unique for each system - evaluated separately to calculate the
thermodynamic properties to order N −α .
N. Chetty, T. Salagaram (UP, NITheP)
CCP 2012
16 / 29
Temperature of a system of finite size
% difference between TN and T∞
1
eq.6
numerical data for T
0.8
0.6
0.4
0.2
0
0
200
400
600
800
1000
1200
1400
1600
1800
2000
Number of particles N
1
TN
=
1
T∞
N. Chetty, T. Salagaram (UP, NITheP)
+ [η 0 (e) − η(e)α0 (e) log N]N −α(e)
University of Pret
CCP 2012
17 / 29
Chemical potential of a system of finite size N
µ
T
=
N
µ
T
∞
+ −eη 0 (e) + η(e)[1 − α(e) + eα0 (e) log N] N −α(e)
University of Pret
N. Chetty, T. Salagaram (UP, NITheP)
CCP 2012
18 / 29
sN (e) vs e
Spin 1
0.7
N=250
N=500
N=750
N=1000
N=2000
N=3000
N=4000
N=5000
canonical
Entropy per particle / kB
0.6
0.5
0.4
0.3
0.2
1.2
N=250
N=500
N=750
N=1000
N=2000
N=3000
N=4000
N=5000
canonical
1
Entropy per particle / kB
Spin 1/2
0.8
0.6
0.4
0.2
0.1
0
0
-1
-0.5
0
energy per particle e
0.5
1
-1
-0.5
0
0.5
1
energy per particle e
University of Pret
N. Chetty, T. Salagaram (UP, NITheP)
CCP 2012
19 / 29
Spin 1/2
Spin 1
1.00
N=250
N=500
N=750
N=1000
N=2000
N=3000
N=4000
N=5000
Canonical
Energy per particle
0.50
1.0
0.5
0.00
0.0
-0.50
-0.5
-1.00
-10
N=250
N=500
N=750
N=1000
N=2000
N=3000
N=4000
N=5000
Canonical
-1.0
-5
0
Temperature
5
10
-4
-2
0
Temperature
2
4
Deviation from thermodynamic limit: less than 0.1% even for N = 250.
Similar behaviour observed for magnetization and heat capacity.
University of Pret
N. Chetty, T. Salagaram (UP, NITheP)
CCP 2012
20 / 29
Entropic interference
e.g ε = p2 + q 4
P
P −β(p2 )
P
2
4
4
z = p,q e−β(p +q ) =
× q e−β(q )
p e
2
4
2
4
q
p
p +q
(e)
(e) + s∞
(e) = s∞
s∞
=⇒ entropy decouples in the thermodyamic limit
However we don’t expect Ωp
2
Ωp (N, E)
and
2
Ωp (N, E)
2 +q 4
(N, E) to be simply related to
since this corresponds to the counting of
states!
2
4
2
4
2 +q 4
We conclude that sNp +q (e) = sNp (e) + sNq (e) + JNp
=⇒
2
4
JNp +q (e)
(e)
→ 0 as N → ∞
N. Chetty, T. Salagaram (UP, NITheP)
University of Pret
CCP 2012
21 / 29
% difference between sN(p2+q4) and (sN(p2)+sN(q4))
3.60
e=10
3.40
3.20
3.00
2.80
2.60
2.40
2.20
2.00
1.80
1.60
500
1000
1500
2000
2500
3000
Number of particles
Entropy is only additive in the thermodynamic limit
University of Pret
N. Chetty, T. Salagaram (UP, NITheP)
CCP 2012
22 / 29
Conclusions
We have demonstrated in a direct manner that entropy is an
extensive quantity independent of the details of the single particle
spectrum
We have provided the first order corrections for entropy for a finite
system of size N
This is the basis for deriving various thermodynamic quantities for
a finite system of size N
The convergence of entropy to its thermodynamic limit is
extremely slow (powerlaw behaviour)
For large values of N, the slopes of sN (e) versus e are almost
parallel which account for the very small deviations of the various
thermodynamic quantities from the canonical results
University of Pret
We have shown that entropy is only additive in the thermodynamic
N. Chetty, T. Salagaram (UP, NITheP)
CCP 2012
23 / 29
Part [2]: Paradigms for large production codes
Brief presention of ideas
University of Pret
N. Chetty, T. Salagaram (UP, NITheP)
CCP 2012
24 / 29
The Electronic Structure Problem
Atomic Pseudopotential
Crystal Structure
Guess input potential
Solve Kohn Sham equations
Output density
Selfconsistent?
YES
Calculate Total Energy, Forces, Stresses, etc
N. Chetty, T. Salagaram (UP, NITheP)
NO
University of Pret
CCP 2012
25 / 29
The Electronic Structure Problem
Pseudopotentials
Brillouin zone integrations
Calculation of the Fermi level
Iterative diagonalisation
Orthogonlisation of states
Calculation of Hellman-Feynman forces
Atomic relaxations
Quantum mechanical stresses
Molecular dynamics simulations
University of Pret
etc
N. Chetty, T. Salagaram (UP, NITheP)
CCP 2012
26 / 29
Paradigms
Identify a mini-problem, e.g. calculation of Fermi level
Develop a selfcontained computational exercise
The exercise need only "mimic" the technically advanced solution
−→ Paradigms for Large Production Codes
Enhances understanding of the large production code
Basis for publishable research in Computational Physics
Education
Source of exciting new problems at undergraduate level
N. Chetty, T. Salagaram (UP, NITheP)
University of Pret
CCP 2012
27 / 29
ASESMA
AFRICAN SCHOOL OF ELECTRONIC STRUCTURE METHODS
AND APPLICATIONS (ASESMA)
N. Chetty - University of Pretoria, South Africa
What is ASESMA?









Two week biennial School on Electronic Structure Methods and Applications
2010 School - Cape Town South Africa
2012 School - Eldoret, Kenya
Applications are now open for hosting 2014 School
About 40 participants chosen from more than 200 applications across Africa
Strive for stringent selection process and high academic standards; ICTP manages administration
Participants are advanced doctoral students or young faculty members
Women participants are especially encouraged to attend
World-class lecturers drawn from around the world; many pay their own way
Why is ASESMA needed?






Africa has an abundance of minerals; needs to develop a materials science culture
A materials science culture is a basis for establishing a materials industry
This has potential to impact on the economy
Computational materials science is an efficient means to jump-start research in materials science
Codes are readily available; scientists are more networked; no need for expensive infrastructure
Africa needs more scientifically literate graduates with computational skills which are transferable to
many other scientific endeavours and to mainstream commerce and industry
ASESMA2010 at the African School of Mathematical Sciences, Muizenberg, Cape Town
ASESMA Academic Programme









Density Functional Theory, methods, algorithms, codes
Main focus on Quantum Espresso Package developed at Democritos and ICTP
However, individual lecturers free to advance their own methods and software packages
Intense morning lectures + lengthy hands-on sessions on computers + evening lectures
Quizzes and tests
Overview talks + technical lectures + talks on research applications, e.g materials for energy
Participants work on small exercise problems during the first week
Groups of participants work on a substantial project for the second week
Project presentations are given on the final day with awards in various different categories
Prof Richard Martin (U. Illinois) hands a copy of his
book “Electronic Structure - Basic Theory and
Practical Methods” to the top performing 2010 student Steven Ndengu (Cameroon)
Role of Mentors and Tutors








Mentors are drawn from Africa and the rest of the world; postdoctoral fellows or advanced PhD students
Assist participants with the exercises
Help manage the projects
Encourage participants and share experiences of being a young scientist
Ensure that each participant can progress at a rate based on his/her own abilities and prior experience
Help identify potential top performers, and also those who might need more assistance
Maintain ties with participants well after the School
This is a novel aspect of ASESMA and very successful; strongly developed by ICMR
ASESMA2012 at the Chepkoilel University, Eldoret,
Kenya
On-going networking and collaborations
International Advisory Panel
 An electronic newsletter has been setup; a face book has been set up
 Website: http://www.asesma.org
 Money available for ASESMA participants to travel to conferences, workshops, based on an merit
 Money available for regional mini-workshops
 Many participants have gone on to pursue doctoral studies based on contacts made at the School
 Focus is to stem the brain drain; help develop a local vibrant intellectual community that does not feel
Richard M. Martin - IUPAP C20, Chair (University of Illinois and Stanford University, USA), O.O. Adewoye (President, African Materials Research Society), Mebarek Alouani (IPCMS, Strasbourg, France), George Amolo (Moi University, Kenya), Peter Borcherds IUPAP C20,
(Birmingham, UK), Roberto Car (Princeton University, USA), Richard Catlow (University College London,UK), XinGao Gong (Fudan University, Shanghai, China), Jim Gubernatis IUPAP C20, (Los Alamos National Laboratory, USA), Walter Kohn (Kavli Institute for Theoretical
Physics, Santa Barbara, USA), Anthony Leggett (University of Illinois, USA), Yu Lu (Institute of Physics, Chinese Academy of Sciences, China), Samuel Yeboah Mensah (University of Cape Coast, Ghana), Bernard M'Passi-Mabiala (Marien NGouabi University, Congo, Brazzaville), Shobhana Narasimhan (Jawaharlall Nehru Centre for Scientific Research, India), Chukwuemeka M I Okoye (University of Nigeria,
Nsukka, Nigeria), David Pettifor (Oxford University, UK), Kennedy Reed IUPAP C13, (Lawrence Livermore National Laboratory, USA),
Sandro Scandolo IUPAP C13, (International Centre for Theoretical Physics, Italy), Wole Soboyejo (Princeton University, USA), Nicola Spaldin (Director, Intl Center for Materials Research, University of California, Santa Barbara, USA), Stefano Baroni (Democritos and Sissa, Trieste, Italy), Paul Woafo, IUPAP C3, (University of Yaounde I, Cameroon and President of the Cameroon Physical Society), Nicola Marzari
(Oxford University, UK)
isolated
 On-going networking and collaborations ensures that ASESMA is not just a two week School but a concerted and a sustainable programme of intellectual development
Executive Board
Nithaya Chetty (Univ Pretoria), Richard Martin (Univ Illinois), Sandro Scandolo (ICTP), George Amolo (Chepkoilel Univ, Kenya)
University of Pret
N. Chetty, T. Salagaram (UP, NITheP)
CCP 2012
28 / 29
THANK YOU!
University of Pret
N. Chetty, T. Salagaram (UP, NITheP)
CCP 2012
29 / 29