Statistical Mechanics by Konstantin K. Likharev - HTML preview
Download the book in PDF, ePub, Kindle for a complete version.
SM: Statistical Mechanics
This ensemble serves as the basis for the formulation of the postulate which is most frequently called the microcanonical distribution (or, more adequately, “the main statistical postulate” or “the main statistical hypothesis”): in the thermodynamic equilibrium of a microcanonical ensemble, all its states have equal probabilities,
1
Micro-
W
const.
(2.20) canonical
m
M
distribution
Though in some constructs of statistical mechanics, this equality is derived from other axioms, which look more plausible to their authors, I believe that Eq. (20) may be taken as the starting point of the statistical physics, supported “just” by the compliance of all its corollaries with experimental observations.
Note that the postulate (20) is closely related to the macroscopic irreversibility of the systems that are microscopically virtually reversible (closed): if such a system was initially in a certain state, its time evolution with even minuscule interactions with the environment (which is necessary for reaching the thermodynamic equilibrium) eventually leads to the uniform distribution of its probability among all states with essentially the same energy. Each of these states is not “better” than the initial one; rather, in a macroscopic system, there are just so many of these states that the chance to find the system in the initial state is practically nil – again, think about the ink drop diffusion into a glass of water.9
Now let us find a suitable definition of entropy S of a microcanonical ensemble’s member – for now, in the thermodynamic equilibrium only. This was done in 1877 by another giant of statistical physics, Ludwig Eduard Boltzmann – on the basis of the prior work by James Clerk Maxwell on the kinetic theory of gases – see Sec. 3.1 below. In present-day terminology, since S is a measure of disorder, it should be related to the amount of information10 lost when the system went irreversibly from the full order to its current state – in equilibrium, to the full disorder, i.e. from one definite state to the microcanonical distribution (20). In an even more convenient formulation, this is the amount of information necessary to find the exact state of a certain system in a microcanonical ensemble.
In the information theory, the amount of information necessary to make a definite choice between two options with equal probabilities (Fig. 2a) is defined as
I (2) log 2 1.
(2.21)
2
(a)
(b)
Fig. 2.2. “Logarithmic trees” of binary decisions
for choosing between (a) M = 2, and (b) M = 4
1 bit
1 bit
opportunities with equal probabilities.
1 bit
9 Though I have to move on, let me note that the microcanonical distribution (20) is a very nontrivial postulate, and my advice to the reader is to find some time to give additional thought to this keystone of the whole building of statistical mechanics.
10 I will rely on the reader’s common sense and intuitive understanding of what information is, because even in the formal information theory, this notion is essentially postulated – see, e.g., the wonderfully clear short textbook by J. Pierce , An Introduction to Information Theory, Dover, 1980.
Chapter 2
Page 6 of 44
SM: Statistical Mechanics
This unit of information is called a bit. Now, if we need to make a choice between four equally probable opportunities, it can be made in two similar steps (Fig. 2b), each requiring one bit of information, so the total amount of information necessary for the choice is
I (4) 2 I (2) 2 log 4.
(2.22)
2
An obvious extension of this process to the choice between M = 2 m states gives I ( M ) mI (2) m log M .
(2.23)
2
This measure, if extended naturally to any integer M, is quite suitable for the definition of entropy at equilibrium, with the only difference that, following tradition, the binary logarithm is replaced with the natural one:11
S ln M.
(2.24a)
Using Eq. (20), we may recast this definition in its most frequently used form Entropy
1
in
S ln
ln W .
(2.24b)
m
equilibrium
Wm
(Again, please note that Eqs. (24) are valid in thermodynamic equilibrium only!) Note that Eq. (24) satisfies the major properties of the entropy discussed in thermodynamics.
First, it is a unique characteristic of the disorder. Indeed, according to Eq. (20), M (at fixed E) is the only possible measure characterizing the microcanonical distribution, and so is its unique function ln M.
This function also satisfies another thermodynamic requirement to the entropy, of being an extensive variable. Indeed, for several independent systems, the joint probability of a certain state is just a product of the partial probabilities, and hence, according to Eq. (24), their entropies just add up.
Now let us see whether Eqs. (20) and (24) are compatible with the 2nd law of thermodynamics.
For that, we need to generalize Eq. (24) for S to an arbitrary state of the system (generally, out of thermodynamic equilibrium), with an arbitrary set of state probabilities Wm. Let us first recognize that M
in Eq. (24) is just the number of possible ways to commit a particular system to a certain state m ( m = 1, 2,… M), in a statistical ensemble where each state is equally probable. Now let us consider a more general ensemble, still consisting of a large number N >> 1 of similar systems, but with a certain number Nm = WmN >> 1 of systems in each of M states, with the factors Wm not necessarily equal. In this case, the evident generalization of Eq. (24) is that the entropy SN of the whole ensemble is S ln M ( N , N ,..) ,
(2.25)
N
1
2
where M ( N 1, N 2,…) is the number of ways to commit a particular system to a certain state m while keeping all numbers Nm fixed. This number M ( N 1, N 2,…) is clearly equal to the number of ways to distribute N distinct balls between M different boxes, with the fixed number Nm of balls in each box, but 11 This is of course just the change of a constant factor: S( M) = ln M = ln2 log2 M = ln2 I( M) 0.693 I( M). A review of Chapter 1 shows that nothing in thermodynamics prevents us from choosing such a constant coefficient arbitrarily, with the corresponding change of the temperature scale – see Eq. (1.9). In particular, in the SI units, where Eq. (24b) becomes S = – k Bln Wm, one bit of information corresponds to the entropy change Δ S = k B ln2 ≈
0.693 k B 0.96510-23 J/K. (The formula “S = k log W” is engraved on L. Boltzmann’s tombstone in Vienna.) Chapter 2
Page 7 of 44
SM: Statistical Mechanics
in no particular order within it. Comparing this description with the definition of the so-called multinomial coefficients,12 we get
M
N!
M ( N , N
, ) NC
,
with N
.
(2.26)
1
2
Nm
N , N ,..., N
1
2
M
N ! N
N
!...
!
1
2
M
m1
To simplify the resulting expression for SN, we can use the famous Stirling formula, in its crudest, de Moivre’s form,13 whose accuracy is suitable for most purposes of statistical physics: ln( N!)
N(ln ).
1
N
N
(2.27)
When applied to our current problem, this formula gives the following average entropy per system,14
SN
1
M
1
M
S
ln( N!)
ln( N !)
N
N
N
N
m
ln 1
N
mln
m
1
N
N
N
m 1
m
m 1
(2.28)
M
N
N
m ln m ,
m 1
N
N
and since this result is only valid in the limit Nm anyway, we may use Eq. (2) to represent it as M
M
1
Entropy
S W ln W W ln
.
(2.29) out of
m
m
m
equilibrium
m1
m1
Wm
This extremely important result15 may be interpreted as the average of the entropy values given by Eq.
(24), weighed with specific probabilities Wm per the general formula (7).16
Now let us find what distribution of probabilities Wm provides the largest value of the entropy (29). The answer is almost evident from a good glance at Eq. (29). For example, if for a subgroup of M’
M states, the coefficients Wm are constant and equal to 1/ M’, so Wm = 0 for all other states, all M’ nonzero terms in the sum (29) are equal to each other, so
1
S M'
ln M' ln M' ,
(2.30)
M'
and the closer M’ to its maximum value M the larger S. Hence, the maximum of S is reached at the uniform distribution given by Eq. (24).
12 See, e.g., MA Eq. (2.3). Despite the intimidating name, Eq. (26) may be very simply derived. Indeed, N! is just the number of all possible permutations of N balls, i.e. of the ways to place them in certain positions – say, inside M boxes. Now to take into account that the particular order of the balls in each box is not important, that number should be divided by all numbers Nm! of possible permutations of balls within each box – that’s it!
13 See, e.g., MA Eq. (2.10).
14 Strictly speaking, I should use the notation S here. However, following the style accepted in thermodynamics, I will drop the averaging signs until we will really need them to avoid confusion. Again, this shorthand is not too bad because the relative fluctuations of entropy (as those of any macroscopic variable) are very small at N >> 1.
15 With the replacement of ln Wm with log2 Wm (i.e. division of both sides by ln2), Eq. (29) becomes the famous Shannon (or “Boltzmann-Shannon”) formula for the average information I per symbol in a long communication string using M different symbols, with probability Wm each.
16 In some textbooks, this interpretation is even accepted as the derivation of Eq. (29); however, it is evidently less rigorous than the one outlined above.
Chapter 2
Page 8 of 44
SM: Statistical Mechanics
In order to prove this important fact more strictly, let us find the maximum of the function given by Eq. (29). If its arguments W 1, W 2, … WM were completely independent, this could be done by finding the point (in the M-dimensional space of the coefficients Wm) where all partial derivatives S/ Wm equal zero. However, since the probabilities are constrained by condition (4), the differentiation has to be carried out more carefully, taking into account this interdependence:
S
S
W
m'
S W
( , W ,...)
.
(2.31)
1
2
W
W
W
W
m
m
m' m
m'
m
cond
At the maximum of the function S, such expressions should be equal to zero for all m. This condition yields S/ Wm = , where the so-called Lagrange multiplier is independent of m. Indeed, at such point Eq. (31) becomes
W
W
W
m'
m
m'
S( W , W ,...)
)
1
( 0
.
(2.32)
1
2
W
W
W
W
W
m
m' m
m
m
m m
'
cond
m
m
For our particular expression (29), the condition S/ Wm = yields
S
d
W ln W
W
(2.33)
m
m ln
1 .
m
W
dW
m
m
The last equality holds for all m (and hence the entropy reaches its maximum value) only if Wm is independent of m. Thus the entropy (29) indeed reaches its maximum value (24) at equilibrium.
To summarize, we see that the statistical definition (24) of entropy does fit all the requirements imposed on this variable by thermodynamics. In particular, we have been able to prove the 2nd law of thermodynamics using that definition together with the fundamental postulate (20).
Now let me discuss one possible point of discomfort with that definition: the values of M, and hence Wm, depend on the accepted energy interval Δ E of the microcanonical ensemble, for whose choice no exact guidance is offered. However, if the interval Δ E contains many states, M >> 1, as was assumed before, then with a very small relative error (vanishing in the limit M → ∞), M may be represented as M g( E) E
,
(2.34)
where g( E) is the density of states of the system:
d( E)
g( E)
,
(2.35)
dE
Σ( E) being the total number of states with energies below E. (Note that the average interval E between energy levels, mentioned at the beginning of this section, is just E/ M = 1/ g( E).) Plugging Eq. (34) into Eq. (24), we get
S ln M ln g( E) ln E
,
(2.36)
so the only effect of a particular choice of Δ E is an offset of the entropy by a constant, and in Chapter 1
we have seen that such a constant offset does not affect any measurable quantity. Of course, Eq. (34), and hence Eq. (36) are only precise in the limit when the density of states g( E) is so large that the range available for the appropriate choice of E,
Chapter 2
Page 9 of 44
SM: Statistical Mechanics
1
g ( E) E
E,
(2.37)
is sufficiently broad: g( E) E = E/ E >> 1.
In order to get some gut feeling of the functions g( E) and S( E) and the feasibility of the condition (37), and also to see whether the microcanonical distribution may be directly used for calculations of thermodynamic variables in particular systems, let us apply it to a microcanonical ensemble of many sets of N >> 1 independent, similar harmonic oscillators with frequency ω. (Please note that the requirement of a virtually fixed energy is applied, in this case, to the total energy EN of each set of oscillators, rather to energy E of a single oscillator – which may be virtually arbitrary though certainly much less than EN ~ NE >> E.) Basic quantum mechanics tells us17 that the eigenenergies of such an oscillator form a discrete, equidistant spectrum:
1
E
(2.38)
m
m
,
where m ,
0 ,
1 ,...
2
2
If ω is kept constant, the ground-state energy ω/2 does not contribute to any thermodynamic properties of the system,18 so for the sake of simplicity we may take that point as the energy origin, and replace Eq.
(38) with Em = m ω. Let us carry out an approximate analysis of the system for the case when its average energy per oscillator,
E
E
N
,
(2.39)
N
is much larger than the energy quantum ω.
For one oscillator, the number of states with energy 1 below a certain value E 1 >> ω is evidently Σ( E 1) ≈ E 1/ ω ( E 1/ ω)/1! (Fig. 3a). For two oscillators, all possible values of the total energy ( ε 1 + ε 2) below some level E 2 correspond to the points of a 2D square grid within the right triangle shown in Fig. 3b, giving Σ( E 2) ≈ (1/2)( E 2/ ω)2 ( E 2/ ω)2/2!. For three oscillators, the possible values of the total energy ( ε 1 + ε 2 + ε 3) correspond to those points of the 3D cubic grid, that fit inside the right pyramid shown in Fig. 3c, giving Σ( E 3) ≈ (1/3)[(1/2)( E 3/ ω)3] ( E 3/ ω)3/3!, etc.
ε 2
ε 2
(c)
(a)
(b)
E
E 3
2
E 1
ε
2 ω
1
0
…
ω 2
ω
ω
0
ε 1
ε 1
Σ( E 1) ω
E 3
0
ω 2 ω E 2
ε
3
E 3
Fig. 2.3. Calculating functions Σ( EN) for systems of (a) one, (b) two, and (c) three harmonic oscillators.
17 See, e.g., QM Secs. 2.9 and 5.4.
18 Let me hope that the reader knows that the ground-state energy is experimentally measurable – for example, using the famous Casimir effect – see, e.g., QM Sec. 9.1. (In Sec. 5.5 below I will briefly discuss another method of experimental observation of that energy.)
Chapter 2
Page 10 of 44
SM: Statistical Mechanics
An evident generalization of these formulas to arbitrary N gives the number of states19
1
N
E
( E )
N
.
(2.40)
N
N!
Differentiating this expression over the energy, we get
N 1
d ( E )
E
N
1
g( E )
N
(2.41)
N
dE
( N
N
,
)!
1
N
so
S ( E ) ln g( E ) const ln
(2.42)
N
N
N
( N
)!
1 ( N )
1 ln E N ln(
N
)
.
const
For N >> 1, we may ignore the difference between N and ( N – 1) in both instances, and use the Stirling formula (27) to simplify this result as
E
E
N
E
N
S ( E) const N ln
1
N ln
ln
.
(2.43)
N
N
(The second step is only valid at very high E/ ratios when the logarithm in Eq. (43) is substantially larger than 1.) Returning for a second to the density of states, we see that in the limit N → , it is exponentially large:
N
S
E
g( E )
N
e
,
(2.44)
N
so the conditions (37) may be indeed satisfied within a very broad range of Δ E.
Now we can use Eq. (43) to find all thermodynamic properties of the system, though only in the limit E >> . Indeed, according to thermodynamics, if the system’s volume and the number of particles in it are fixed, the derivative dS/ dE is nothing else than the reciprocal temperature in thermal equilibrium – see Eq. (1.9). In our current case, we imply that the harmonic oscillators are distinct, for example by their spatial positions. Hence, even if we can speak of some volume of the system, it is certainly fixed.20 Differentiating Eq. (43) over energy E, we get
Classical
1
dS
oscillator:
N
N
1
.
(2.45)
average
T
dE
E
E
N
N
energy
Reading this result backward, we see that the average energy E of a harmonic oscillator equals T (i.e.
k B T K is SI units). At this point, the first-time student of thermodynamics should be very much relieved to see that the counter-intuitive thermodynamic definition (1.9) of temperature does indeed correspond to what we all have known about this notion from our kindergarten physics courses.
The result (45) may be readily generalized. Indeed, in quantum mechanics, a harmonic oscillator with eigenfrequency may be described by the Hamiltonian operator
19 The coefficient 1/ N! in this formula has the geometrical meaning of the (hyper)volume of the N-dimensional right pyramid with unit sides.
20 For the same reason, the notion of pressure P in such a system is not clearly defined, and neither are any thermodynamic potentials but E and F.
Chapter 2
Page 11 of 44
SM: Statistical Mechanics
ˆ 2
p
ˆ2
q
ˆ
H
,
(2.46)
2m
2
where q is some generalized coordinate, p is the corresponding generalized momentum, m is the oscillator’s mass,21 and is its spring constant, so = (/m)1/2. Since in the thermodynamic equilibrium the density matrix is always diagonal in the basis of stationary states m (see Sec. 1 above), the quantum-mechanical averages of the kinetic and potential energies may be found from Eq. (7): 2
2
2
2
p
ˆ p
q
ˆ
q
W m
m ,
W m
m
(2.47)
m
,
2
m
m
m0
2m
2
m0
2
where Wm is the probability to occupy the m th energy level, while bra- and ket-vectors describe the stationary state corresponding to that level.22 However, both classical and quantum mechanics teach us that for any m, the bra-ket expressions under the sums in Eqs. (47), which represent the average kinetic and mechanical energies of the oscillator on its m th energy level, are equal to each other, and hence each of them is equal to Em/2. Hence, even though we do not know the exact probability distribution Wm yet (it will be calculated in Sec. 5 below), we may conclude that in the “classical limit” T >> , 2
2
p
q
T
.
(2.48) Equipartition
2m
2
2
theorem
Now let us consider a system with an arbitrary number of degrees of freedom, described by a more general Hamiltonian:23
ˆ 2
p
ˆ2
q
ˆ
ˆ
ˆ
H H ,
with
j
j
j
H
,
(2.49)
j
j
j
2m
2
j
with (generally, different) frequencies j = ( j/m j)1/2. Since the “modes” (effective harmonic oscillators) contributing to this Hamiltonian, are independent, the result (48) is valid for each of the modes. This is the famous equipartition theorem: at thermal equilibrium at temperature T >> j, the average energy of each so-called half-degree of freedom (which is defined as any variable, either pj or qj, giving a quadratic contribution to the system’s Hamiltonian), is equal to T/2.24 In particular, for each of three Cartesian component contributions to the kinetic energy of a free-moving particle, this theorem is valid for any temperature, because such components may be considered as 1D harmonic oscillators with vanishing potential energy, i.e. j = 0, so condition T >> j is fulfilled at any temperature.
21 I am using this fancy font for the mass to avoid any chance of its confusion with the state number.
22 Note again that while we have committed the energy EN of N oscillators to be fixed (to apply Eq. (36), valid only for a microcanonical ensemble at thermodynamic equilibrium), the single oscillator’s energy E in our analysis may be arbitrary – within the very broad limits << E EN ~ NT.
23 As a reminder, the Hamiltonian of any system whose classical Lagrangian function is an arbitrary quadratic form of its generalized coordinates and the corresponding generalized velocities may be brought to the form (49) by an appropriate choice of “normal coordinates” qj which are certain linear combinations of the original coordinates – see, e.g., CM Sec. 6.2.
24 This also means that in the classical limit, the heat capacity of a system is equal to one-half of the number of its half-degrees of freedom (in the SI units, multiplied by k B).
Chapter 2
Page 12 of 44
SM: Statistical Mechanics
I believe that this case study of harmonic oscillator systems was a fair illustration of both the strengths and the weaknesses of the microcanonical ensemble approach.25 On one hand, we could readily calculate virtually everything we wanted in the classical limit T >> , but calculations for an arbitrary T ~ , though possible, would be rather unpleasant because for that, all vertical steps of the function Σ( E N) have to be carefully counted. In Sec. 4 below, we will see that other statistical ensembles are much more convenient for such calculations.
Let me conclude this section with a short notice on deterministic classical systems with just a few degrees of freedom (and even simpler mathematical objects called “maps”) that may exhibit essentially disordered behavior, called deterministic chaos.26 Such chaotic system may be approximately characterized by an entropy defined similarly to Eq. (29), where Wm are the probabilities to find it in different small regions of phase space, at well-separated small time intervals. On the other hand, one can use an expression slightly more general than Eq. (29) to define the so-called Kolmogorov (or
“Kolmogorov-Sinai”) entropy K that characterizes the speed of loss of the information about the initial state of the system, and hence what is called the “chaos depth”. In the definition of K, the sum over m is replaced with the summation over all possible permutations { m} = m 0, m 1, …, mN-1 of small space regions, and Wm is replaced with W{ m}, the probability of finding the system in the corresponding regions m at time moment tm, with tm = m, in the limit 0, with N = const. For chaos in the simplest objects, 1D maps, K is equal to the Lyapunov exponent > 0.27 For systems of higher dimensionality, which are characterized by several Lyapunov exponents , the Kolmogorov entropy is equal to the phase-space average of the sum of all positive . These facts provide a much more practicable way of (typically, numerical) calculation of the Kolmogorov entropy than the direct use of its definition.28
2.3. Maxwell’s Demon, information, and computation
Before proceeding to other statistical distributions, I would like to make a detour to address one more popular concern about Eq. (24) – the direct relation between entropy and information. Some physicists are still uneasy with entropy being nothing else than the (deficit of) information, though, to the best of my knowledge, nobody has yet been able to suggest any experimentally verifiable difference between these two notions. Let me give one example of their direct relationship.29 Consider a cylinder containing just one molecule (considered as a point particle), and separated into two halves by a movable partition with a door that may be opened and closed at will, at no energy cost – see Fig. 4a. If the door is open and the system is in thermodynamic equilibrium, we do not know on which side of the partition the molecule is. Here the disorder, i.e. the entropy has the largest value, and there is no way to get, from a large ensemble of such systems in equilibrium, any useful mechanical energy.
25 The reader is strongly urged to solve Problem 2, whose task is to do a similar calculation for another key (“two-level”) physical system, and compare the results.
26 See, e.g., CM Chapter 9 and the literature therein.
27 For the definition of , see, e.g., CM Eq. (9.9).
28 For more discussion, see, e.g., either Sec. 6.2 of the monograph H. G. Schuster and W. Just, Deterministic Chaos, 4th ed., Wiley-VHS, 2005, or the monograph by Arnold and Avez, cited in Sec. 1.
29 This system is frequently called the Szilard engine, after L. Szilard who published its detailed theoretical discussion in 1929, but is essentially a straightforward extension of the thought experiment suggested by J.
Maxwell as early as 1867.
Chapter 2
Page 13 of 44
SM: Statistical Mechanics
(a)
(b)
(c)
v
F
Fig. 2.4. The Szilard engine: a cylinder with a single molecule and a movable partition: (a) before and (b) after closing the door, and (c) after opening the door at the end of the expansion stage.
Now, let us consider that we know (as instructed by, in Lord Kelvin’s formulation, an omniscient Maxwell’s Demon) on which side of the partition the molecule is currently located. Then we may close the door, trapping the molecule, so its repeated impacts on the partition create, on average, a pressure force F directed toward the empty part of the volume (in Fig. 4b, the right one). Now we can get from the molecule some mechanical work, say by allowing the force F to move the partition to the right, and picking up the resulting mechanical energy by some deterministic (zero-entropy) external mechanism.
After the partition has been moved to the right end of the volume, we can open the door again (Fig. 4c), equalizing the molecule’s average pressure on both sides of the partition, and then slowly move the partition back to the middle of the volume – without its resistance, i.e. without doing any substantial work. With the continuing help from Maxwell’s Demon, we can repeat the cycle again and again, and hence make the system perform unlimited mechanical work, fed “only” by the molecule’s thermal motion, and the information about its position – thus implementing the perpetual motion machine of the 2nd kind – see Sec. 1.6. The fact that such heat engines do not exist means that getting any new information, at a non-zero temperature (i.e. at a substantial thermal agitation of particles) has a non-zero energy cost.
In order to evaluate this cost, let us calculate the maximum work per cycle that can be made by the Szilard engine (Fig. 4), assuming that it is constantly in the thermal equilibrium with a heat bath of temperature T. Formula (21) tells us that the information supplied by the demon (on what exactly half of the volume contains the molecule) is exactly one bit, I (2) = 1. According to Eq. (24), this means that by getting this information we are changing the entropy of our system by
S ln 2 .
(2.50)
I
Now, it would be a mistake to plug this (negative) entropy change into Eq. (1.19). First, that relation is only valid for slow, reversible processes. Moreover (and more importantly), this equation, as well as its irreversible version (1.41), is only valid for a fixed statistical ensemble. The change SI does not belong to this category and may be formally described by the change of the statistical ensemble – from the one consisting of all similar systems (experiments) with an unknown location of the molecule to a new ensemble consisting of the systems with the molecule in its certain (in Fig. 4, left) half.30
Now let us consider a slow expansion of the “gas” after the door had been closed. At this stage, we do not need the Demon’s help any longer (i.e. the statistical ensemble may be fixed), and can indeed use the relation (1.19). At the assumed isothermal conditions ( T = const), this relation may be integrated 30 This procedure of the statistical ensemble re-definition is the central point of the connection between physics and information theory, and is crucial in particular for any (or rather any meaningful :-) discussion of measurements in quantum mechanics – see, e.g., QM Secs. 2.5 and 10.1.
Chapter 2
Page 14 of 44
SM: Statistical Mechanics
over the whole expansion process, getting Q = T S. At the final position shown in Fig. 4c, the system’s entropy should be the same as initially, i.e. before the door had been opened, because we again do not know where in the volume the molecule is. This means that the entropy was replenished, during the reversible expansion, from the heat bath, by S = – SI = +ln2, so Q = T S = T ln2. Since by the end of the whole cycle, the internal energy E of the system is the same as before, all this heat could have gone into the mechanical energy obtained during the expansion. Thus the maximum obtained work per cycle (i.e. for each obtained information bit) is T ln2 ( k B T Kln2 in the SI units), about 410-21 Joule at room temperature. This is exactly the energy cost of getting one bit of new information about a system at temperature T. The smallness of that amount on the everyday human scale has left the Szilard engine an academic theoretical exercise for almost a century. However, recently several such devices, of various physical nature, were implemented experimentally (with the Demon’s role played by an instrument measuring the position of the particle without a substantial effect on its motion), and the relation Q =
T ln2 was proved, with a gradually increasing precision.31
Actually, discussion of another issue closely related to Maxwell’s Demon, namely energy consumption at numerical calculations, was started earlier, in the 1960s. It was motivated by the exponential ( Moore’s-law) progress of the digital integrated circuits, which has led in particular, to a fast reduction of the energy E “spent” (turned into heat) per one binary logic operation. In the recent generations of semiconductor digital integrated circuits, the typical E is still above 10-17 J, i.e. still exceeds the room-temperature value of T ln2 410-21 J by several orders of magnitude.32 Still, some engineers believe that thermodynamics imposes this important lower limit on E and hence presents an insurmountable obstacle to the future progress of computation. Unfortunately, in the 2000s this delusion resulted in a substantial and unjustified shift of electron device research resources toward using “non-charge degrees of freedom” such as spin (as if they do not obey the general laws of statistical physics!), so the issue deserves at least a brief discussion.
Let me believe that the reader of these notes understands that, in contrast to naïve popular talk, computers do not create any new information; all they can do is reshape (“process”) the input information, losing most of it on the go. Indeed, any digital computation algorithm may be decomposed into simple, binary logical operations, each of them performed by a circuit called the logic gate. Some of these gates (e.g., the logical NOT performed by inverters, as well as memory READ and WRITE
operations) do not change the amount of information in the computer. On the other hand, such information- irreversible logic gates as two-input NAND (or NOR, or XOR, etc.) erase one bit at each operation, because they turn two input bits into one output bit – see Fig. 5a.
In 1961, Rolf Landauer argued that each logic operation should turn into heat at least energy Irreversible
computation:
E
T ln 2 k T ln 2 .
(2.51)
min
B K
energy cost
This result may be illustrated with the Szilard engine (Fig. 4), operated in a reversed cycle. At the first stage, with the partition’s door closed, it uses external mechanical work E = T ln2 to reduce the volume in that the molecule is confined, from V to V/2, pumping heat Q = E into the heat bath. To model a logically irreversible logic gate, let us now open the door in the partition, and thus lose one bit of 31 See, for example, A. Bérut et al. , Nature 483, 187 (2012); J. Koski et al., PNAS USA 111, 13786 (2014); Y. Jun et al., Phys. Rev. Lett. 113, 190601 (2014); J. Peterson et al., Proc. Roy. Soc. A 472, 20150813 (2016).
32 In practical computers, the effective E is even much higher (currently, above ~10-15 J) due to the high energy cost of moving data across a multi-component system, in particular between its logic and memory chips.
Chapter 2
Page 15 of 44
SM: Statistical Mechanics
information about the molecule’s position. Then we will never get the work T ln2 back, because moving the partition back to the right, with the door open, takes place at zero average pressure. Hence, Eq. (51) gives a fundamental limit for energy loss (per bit) at the logically irreversible computation.
(a)
(b)
A
A
A
F
Fig. 2.5. Simple examples
F
of (a) irreversible and (b)
potentially reversible logic
B
B
circuits. Each rectangle
B
denotes a circuit storing one
bit of information.
However, in 1973 Charles Bennett came up with convincing arguments that it is possible to avoid such energy loss by using only operations that are reversible not only physically, but also logically.33 For that, one has to avoid any loss of information, i.e. any erasure of intermediate results, for example in the way shown in Fig. 5b.34 At the end of all calculations, after the result has been copied into memory, the intermediate results may be “rolled back” through reversible gates to be eventually merged into a copy of input data, again without erasing a single bit. The minimal energy dissipation at such reversible calculation tends to zero as the operation speed is decreased, so the average energy loss per bit may be less than the perceived “fundamental thermodynamic limit” (51). The price to pay for this ultralow dissipation is a very high complexity of the hardware necessary for the storage of all intermediate results. However, using irreversible gates sparsely, it may be possible to reduce the complexity dramatically, so in the future such mostly reversible computation may be able to reduce energy consumption in practical digital electronics.35
Before we leave Maxwell’s Demon behind, let me use it to revisit, for one more time, the relation between the reversibility of the classical and quantum mechanics of Hamiltonian systems and the irreversibility possible in thermodynamics and statistical physics. In the thought experiment shown in Fig. 4, the laws of mechanics governing the motion of the molecule are reversible at all times. Still, at partition’s motion to the right, driven by molecular impacts, the entropy grows, because the molecule picks up the heat Q > 0, and hence the entropy S = Q/ T > 0, from the heat bath. The physical mechanism of this irreversible entropy (read: disorder) growth is the interaction of the molecule with uncontrollable components of the heat bath and the resulting loss of information about its motion.
Philosophically, such emergence of irreversibility in large systems is a strong argument against reductionism – a naïve belief that by knowing the exact laws of Nature at the lowest, most fundamental 33 C. Bennett, IBM J. Res. Devel. 17, 525 (1973); see also C. Bennett, Int. J. Theor. Phys. 21, 905 (1982).
34 For that, all gates have to be physically reversible, with no static power consumption. Such logic devices do exist, though they are still not very practicable – see, e.g., K. Likharev, Int. J. Theor. Phys. 21, 311 (1982).
(Another reason why I am citing, rather reluctantly, my own paper is that it also gave a constructive proof that the reversible computation may also beat the perceived “fundamental quantum limit”, E t > , where t is the time of the binary logic operation.)
35 Many currently explored schemes of quantum computing are also reversible – see, e.g., QM Sec. 8.5 and references therein.
Chapter 2
Page 16 of 44
Essential Graduate Physics
SM: Statistical Mechanics
level of its complexity, we can readily understand all phenomena on the higher levels of its organization.
In reality, the macroscopic irreversibility of large systems is a good example36 of a new law (in this case, the 2nd law of thermodynamics) that becomes relevant on a substantially new, higher level of complexity
– without defying the lower-level laws. Without such new laws, very little of the higher-level organization of Nature may be understood.
2.4. Canonical ensemble and the Gibbs distribution
As was shown in Sec. 2, the microcanonical distribution may be directly used for solving some important problems. However, its further development, also due to J. Gibbs, turns out to be much more convenient for calculations.
Let us consider a statistical ensemble of macroscopically similar systems, each in thermal equilibrium with a heat bath of the same temperature T (Fig. 6a). Such an ensemble is called canonical.
(a)
(b)
system
E
E
Σ
under study
dQ, dS
Em, T
E
HB = E – Em
Fig. 2.6. (a) A system in a heat
bath (i.e. a canonical ensemble’s
heat bath
E
member) and (b) the energy
m
E HB, T
spectrum of the composite system
0
(including the heat bath).
It is intuitively evident that if the heat bath is sufficiently large, any thermodynamic variables characterizing the system under study should not depend on the heat bath’s environment. In particular, we may assume that the heat bath is thermally insulated, so the total energy E of the composite system, consisting of the system of our interest plus the heat bath, does not change in time. For example, if the system under study is in a certain (say, m th ) quantum state, then the sum E E E
(2.52)
m
HB
is time-independent. Now let us partition the considered canonical ensemble of such systems into much smaller sub-ensembles, each being a microcanonical ensemble of composite systems whose total, time-independent energies E are the same – as was discussed in Sec. 2, within a certain small energy interval
E << E – see Fig. 6b. Due to the very large size of each heat bath in comparison with that of the system under study, the heat bath’s density of states g HB is very high, and E may be selected so 1
E
E E E ,
(2.53)
m
HB
g
m'
HB
where m and m’ are any states of the system of our interest.
36 Another famous example is Charles Darwin’s theory of biological evolution.
Chapter 2
Page 17 of 44
SM: Statistical Mechanics
According to the microcanonical distribution, within each of these microcanonical sub-ensembles, the probabilities to find the composite system in any state are equal. Still, the heat bath energies E HB = E – Em (Fig. 6b) of the members of this sub-ensemble may be different – due to the difference in Em. The probability W( Em) to find the system of our interest (within the selected sub-ensemble) in a state with energy Em is proportional to the number M of the corresponding heat baths in the sub-ensemble. As Fig. 6b shows, in this case we may write M = g HB( E HB) E. As a result, within the microcanonical sub-ensemble with the total energy E,
W M g ( E ) E g ( E E ) E .
(2.54)
m
HB
HB
HB
m
Let us simplify this expression further, using the Taylor expansion with respect to relatively small Em << E. However, here we should be careful. As we have seen in Sec. 2, the density of states of a large system is a nearly exponential function of energy, so if we applied the Taylor expansion directly to Eq. (54), the Taylor series would only converge for very small Em. A much broader applicability range may be obtained by taking the logarithms of both parts of Eq. (54) first: ln W const ln
g
E E
E
S
E E ,
(2.55)
m
(
)
HB
m
ln
const
(
)
HB
m
where the last step used Eq. (36) for the heat bath, and incorporated ln ( E) into the (inconsequential) constant. Now, we can Taylor-expand the (much more smooth) function of energy on the right-hand side of Eq. (55), and limit ourselves to the two leading terms of the series:
dS
ln W const
HB
S
E .
(2.56)
m
HB E
0
E 0 m
m
dE
m
HB
But according to Eq. (1.9), the derivative participating in this expression is nothing other than the reciprocal temperature of the heat bath, which (due to the large bath size) does not depend on whether Em is equal to zero or not. Since our system of interest is in the thermal equilibrium with the bath, this is also the temperature T of the system – see Eq. (1.8). Hence Eq. (56) is merely E
ln W
m
const
.
(2.57)
m
T
This equality describes a substantial decrease of Wm as Em is increased by ~ T, and hence our linear approximation (56) is virtually exact as soon as E HB is much larger than T – the condition that is rather easy to satisfy, because as we have seen in Sec. 2, the average energy per one degree of freedom of the system of the heat bath is also of the order of T, so its total energy is much larger than T because of its much larger size.
Now we should be careful again because so far, Eq. (57) was only derived for a sub-ensemble with a certain fixed E. However, since the second term on the right-hand side of Eq. (57) includes only Em and T, which are independent of E, this relation, perhaps with different constant terms, is valid for all sub-ensembles of the canonical ensemble, and hence for that ensemble as the whole. Hence for the total probability of finding our system of interest in a state with energy Em, in the canonical ensemble with temperature T, we can write
Em 1
Em
W const exp
(2.58) Gibbs
m
exp
.
distribution
T Z
T
Chapter 2
Page 18 of 44
SM: Statistical Mechanics
.
This is the famous Gibbs distribution,37 sometimes called the “canonical distribution”, which is arguably the summit of statistical physics,38 because it may be used for a straightforward (or at least conceptually straightforward :-) calculation of all statistical and thermodynamic variables of a vast range of systems. Its physical sense is very clear: the interaction with the heat bath “punishes” the system states (by the reduction of their probability) for having higher energies – on the scale T of its thermal agitation.
Now let us calculate the coefficient Z participating in Eq. (58). Requiring, per Eq. (4), the sum of all Wm to be equal to 1, we get
Em
Statistical
Z exp
,
(2.59)
sum
m
T
where the summation is formally extended to all quantum states of the system, though in practical calculations, the sum may be truncated to include only the states that are noticeably occupied. The apparently humble normalization coefficient Z turns out to be so important for applications that it has a special name – or actually, two names: either the statistical sum or the partition function of the system.
To appreciate the importance of Z, let us use the general expression (29) for entropy to calculate it for the particular case of the canonical ensemble, i.e. the Gibbs distribution (58) of the probabilities Wn: ln Z
E
E
m
1
m
S W ln W
E
(2.60)
m
m
exp exp
m
.
m
Z m
T ZT m
T
On the other hand, according to the general rule (7), the thermodynamic (i.e. ensemble-averaged) value E of the internal energy of the system is
1
Em
E W E
E
(2.61a)
m
m
exp
m
,
m
Z m
T
so the second term on the right-hand side of Eq. (60) is just E/T, while the first term equals ln Z, due to Eq. (59). (By the way, using the notion of reciprocal temperature 1/ T, with the account of Eq. (59), Eq. (61a) may be also rewritten as
(ln Z)
E from Z
E
.
(2.61b)
This formula is very convenient for calculations if our prime interest is the average internal energy E
rather than F or Wn.) With these substitutions, Eq. (60) yields a very simple relation between the statistical sum and the entropy of the system:
E
S
ln Z .
(2.62)
T
37 The temperature dependence of the type exp{–const/ T}, especially when showing up in rates of certain events, e.g., chemical reactions, is also frequently called the Arrhenius law – after chemist S. Arrhenius who has noticed this law in numerous experimental data. In all cases I am aware of, the Gibbs distribution is the underlying reason for the Arrhenius law. (We will see several examples of that later in this course.) 38 This is the opinion of many physicists, including Richard Feynman – who climbs on this “summit” already on the first page of his brilliant book Statistical Mechanics, CRC Press, 1998. (This is a collection of lectures on a few diverse, mostly advanced topics of statistical physics, rather than its systematic course, so it can hardly be used as the first textbook on the subject. However, I can highly recommend its first chapter to all my readers.) Chapter 2
Page 19 of 44
SM: Statistical Mechanics
Now using Eq. (1.33), we see that Eq. (62) gives a straightforward way to calculate the free energy F of the system from nothing other than its statistical sum (and temperature): F E TS T
ln Z.
(2.63) F from Z
The relations (61b) and (63) play a key role in the connection of statistics to thermodynamics, because they enable the calculation, from Z alone, of the thermodynamic potentials of the system in equilibrium, and hence of all other variables of interest, using the general thermodynamic relations – see especially the circular diagram shown in Fig. 1.6, and its discussion in Sec. 1.4. Let me only note that to calculate pressure P from the second of Eqs. (1.35), we would need to know the explicit dependence of F, and hence of the statistical sum Z on the system’s volume V. This would require the calculation, by appropriate methods of either classical or quantum mechanics, of the dependence of the eigenenergies Em on the volume. Numerous examples of such calculations will be given later in the course.
Before proceeding to such examples, let us notice that Eqs. (59) and (63) may be readily combined to give an elegant equality,
F
Em
exp exp
.
(2.64)
T
m
T
This formula, together with Eq. (59), enables us to rewrite the Gibbs distribution (58) in another form:
F Em
W exp
(2.65)
m
,
T
more convenient for some applications. In particular, this expression shows that since all probabilities Wm are below 1, F is always lower than the lowest energy level. Also, Eq. (65) clearly shows that the probabilities Wm do not depend on the energy reference, i. e. on an arbitrary constant added to all Em –
and hence to E and F.
2.5. Harmonic oscillator statistics
The last property may be immediately used in our first example of the Gibbs distribution application to a particular but very important system – the harmonic oscillator, for a much more general case than was done in Sec. 2, namely for an arbitrary relation between T and .39 Let us consider a canonical ensemble of similar oscillators, each in contact with a heat bath of temperature T. Selecting the ground-state energy /2 for the origin of E, the oscillator eigenenergies (38) become Em = m
(with m = 0, 1,…), so the Gibbs distribution (58) for probabilities of these states is 1
E
1
m
m
W
exp
exp
,
(2.66)
m
Z
T Z
T
with the following statistical sum:
m
m
Z
exp
,
where exp
.
1
(2.67)
m0
T m0
T
39 The task of making a very similar (and even simpler) calculation for another key class of quantum-mechanical objects, two-level systems, is left for the reader’s exercise.
Chapter 2
Page 20 of 44
SM: Statistical Mechanics
This is just the well-known infinite geometric progression (the “geometric series”),40 with the sum Harmonic
1
1
oscillator:
Z
,
(2.68)
Z
1
1
/ T
e
so Eq. (66) yields
Harmonic
W
(2.69)
oscillator:
m
1
/ T
e
m / T
e
.
Wm
Figure 7a shows Wm for several lower energy levels, as functions of temperature, or rather of the T/ ratio. The plots show that the probability of finding the oscillator in each particular state (except for the ground one, with m = 0) vanishes in both low- and high-temperature limits, and reaches its maximum value Wm ~ 0.3/ m at T ~ m, so the contribution m Wm of each excited level to the average oscillator energy E is always smaller than .
(a)
(b)
1
2
W 0
E
S
1
W 1
C
W
2
W 3
0.1
0
F
1
0.01
2
0.1
1
10
0
0.5
1
1.5
2
2.5
3
T /
T /
Fig. 2.7. Statistical and thermodynamic parameters of a harmonic oscillator, as functions of temperature.
This average energy may be calculated in either of two ways: either using Eq. (61a) directly:
E E W
T
m
T
e
m
e
(2.70)
m
m
1 /
/
,
m0
m0
or (simpler) using Eq. (61b), as
1
E
ln Z
ln1
exp ,
where .
(2.71)
T
Both methods give (of course) the same result,41
40 See, e.g., MA Eq. (2.8b).
Chapter 2
Page 21 of 44
SM: Statistical Mechanics
1
Harmonic
E E(, T )
,
(2.72) oscillator:
/ T
average
e
1
energy
which is valid for arbitrary temperature and plays a key role in many fundamental problems of physics.
The red line in Fig. 7b shows this result as a function of the normalized temperature. At relatively low temperatures, T << , the oscillator is predominantly in its lowest (ground) state, and its energy (on top of the constant zero-point energy /2, which was used in our calculation as the reference) is exponentially small: E exp{-/ T} << T, . On the other hand, in the high-temperature limit, the energy tends to T. This is exactly the result (a particular case of the equipartition theorem) that was obtained in Sec. 2 from the microcanonical distribution. Please note how much simpler is the calculation using the Gibbs distribution, even for an arbitrary ratio T/.
To complete the discussion of the thermodynamic properties of the harmonic oscillator, we can calculate its free energy using Eq. (63):
1
F T ln
T ln
/ T
1 e
.
(2.73)
Z
Now the entropy may be found from thermodynamics: either from the first of Eqs. (1.35), S = –(∂ F/∂ T) V, or (even more easily) from Eq. (1.33): S = ( E – F)/ T. Both relations give, of course, the same result:
1
S
ln
/ T
1 e
.
(2.74)
/ T
T e
1
Finally, since in the general case, the dependence of the oscillator properties (essentially, of ) on volume V is not specified, such variables as P, , G, W, and are not defined, and what remains is to calculate the average heat capacity C per one oscillator:
2
2
/
E
e
T
/ 2 T
C
.
(2.75)
T
T ( /
e
T )
1 2
sinh( / 2 T )
The calculated thermodynamic variables are plotted in Fig. 7b as functions of temperature. In the low-temperature limit ( T << ), they all tend to zero. On the other hand, in the high-temperature limit ( T >> ), F – T ln( T/) –, S ln( T/) +, and C 1 (in the SI units, C k B). Note that the last limit is the direct corollary of the equipartition theorem: each of the two “half-degrees of freedom” of the oscillator gives, in the classical limit, the same contribution C = ½ into its heat capacity.
Now let us use Eq. (69) to discuss the statistics of the quantum oscillator described by the Hamiltonian (46), in the coordinate representation. Again using the density matrix’s diagonality in thermodynamic equilibrium, we may use a relation similar to Eqs. (47) to calculate the probability density to find the oscillator at coordinate q:
(
w q)
W w ( q) W q
e
T
e m
T q ,
(2.76)
m
m
2
( )
m
m
1
/
2
/
( )
m
m0
m0
m0
41 It was first obtained in 1924 by S. Bose and is sometimes called the Bose distribution – a particular case of the Bose-Einstein distribution to be discussed in Sec. 8 below.
Chapter 2
Page 22 of 44
SM: Statistical Mechanics
where m( q) is the normalized eigenfunction of the m th stationary state of the oscillator. Since each
m( q) is proportional to the Hermite polynomial42 that requires at least m elementary functions for its representation, working out the sum in Eq. (76) is a bit tricky,43 but the final result is rather simple: w( q) is just a normalized Gaussian distribution (the “bell curve”),
1
2
q
(
w q)
,
(2.77)
2
exp
1/ 2
2
q
(
2 q
)
with q = 0, and
q 2 q
2
coth
.
(2.78)
2 m
T
2
Since the function coth tends to 1 at → , and diverges as 1/ at → 0, Eq. (78) shows that the width
q of the coordinate distribution is nearly constant (and equal to that, (/2 m)1/2, of the ground-state wavefunction 0) at T << , and grows as ( T/ m2)1/2 ( T/)1/2 at T/ → .
As a sanity check, we may use Eq. (78) to write the following expression,
2
q
/ ,
4
for T
,
U
coth
(2.79)
2
4
2 T
T
/2,
for
T ,
for the average potential energy of the oscillator. To comprehend this result, let us recall that Eq. (72) for the average full energy E was obtained by counting it from the ground state energy /2 of the oscillator. If we add this reference energy to that result, we get
Harmonic
oscillator:
E
coth .
(2.80)
E
e / T 1
2
2
T
2
We see that for arbitrary temperature, U = E/2, as was already discussed in Sec. 2. This means that the average kinetic energy, equal to E – U, is also the same:44
p 2
q 2
E
coth .
(2.81)
2 m
2
2
4
T
2
In the classical limit T >> , each of the energies tends to T/2, reproducing the equipartition theorem (48).
2.6. Two important applications
The results of the previous section, especially Eq. (72), have innumerable applications in physics and related disciplines, and here I have time for a brief discussion of only two of them.
(i) Blackbody radiation. Let us consider a free-space volume V limited by non-absorbing (i.e.
ideally reflecting) walls. Electrodynamics tells us45 that the electromagnetic field in such a “cavity” may be represented as a sum of modes with a time evolution similar to that of the usual harmonic oscillator.
42 See, e.g., QM Sec. 2.10.
43 The calculation may be found, e.g., in QM Sec. 7.2.
44 As a reminder: the equality of these two averages, for arbitrary temperatures, was proved already in Sec. 2.
Chapter 2
Page 23 of 44
Essential Graduate Physics
SM: Statistical Mechanics
If the volume V is large enough,46 the number of these modes within a small range dk of the wave vector magnitude k is
gV
3
gV
dN
d k
4 k 2
dk ,
(2.82)
3
3
(2 )
(2 )
where for electromagnetic waves, the degeneracy factor g is equal to 2, due to their two different independent (e.g., linear) polarizations of waves with the same wave vector k. With the linear, isotropic dispersion relation for waves in vacuum, k = / c, Eq. (82) yields 2
2
2
V
d
dN
4
V
d .
(2.83)
(2 )3
3
2 3
c
c
On the other hand, quantum mechanics says47 that the energy of such a “field oscillator” is quantized per Eq. (38), so at thermal equilibrium its average energy is described by Eq. (72). Plugging that result into Eq. (83), we see that the spectral density of the electromagnetic field’s energy, per unit volume, is
3
E dN
1
Planck’s
u()
.
(2.84) radiation
2 3
/ T
V d
c e
1
law
This is the famous Planck’s blackbody radiation law.48 To understand why its common name mentions radiation, let us consider a small planar part, of area dA, of a surface that completely absorbs electromagnetic waves incident from any direction. (Such “perfect black body” approximation may be closely approached using special experimental structures, especially in limited frequency intervals.) Figure 8 shows that if the arriving wave was planar, with the incidence angle , then the power d P() absorbed by the surface of small area dA, within a small frequency interval d, i.e. the energy incident at that area in unit time, would be equal to the radiation energy within the same frequency interval, contained inside an imaginary cylinder (shaded in Fig. 8) of height c, base area dA cos, and hence volume dV = c dA cos :
d P
( ) u
( )
d dV u
( )
d c dA
cos .
(2.85)
dA cos
dA
c
Fig. 2.8. Calculating the relation
between d P () and u() d.
45 See, e.g., EM Sec. 7.8.
46 In our current context, the volume should be much larger than ( c/ T)3, where c 3108 m/s is the speed of light. For the room temperature ( T k B300K 410-21 J), this lower bound is of the order of 10-16 m3.
47 See, e.g., QM Sec. 9.1.
48 Let me hope the reader knows that this law was first suggested in 1900 by Max Planck as an empirical fit for the experimental data on blackbody radiation, and this was the historic point at which the Planck constant (or rather h 2) was introduced – see, e.g., QM Sec. 1.1.
Chapter 2
Page 24 of 44
SM: Statistical Mechanics
Since the thermally-induced field is isotropic, i.e. propagates equally in all directions, this result should be averaged over all solid angles within the polar angle interval 0 /2: d P ()
1
d ()
1 / 2
2
c
d cu()
sin d d cos u() .
(2.86)
P
dAd
4
dAd
4
4
0
0
Hence Planck’s expression (84), multiplied by c/4, gives the power absorbed by such a “blackbody”
surface. But at thermal equilibrium, this absorption has to be exactly balanced by the surface’s own radiation, due to its non-zero temperature T.
I hope the reader is familiar with the main features of the Planck law (84), including its general shape (Fig. 9), with the low-frequency asymptote u() 2 (due to its historic significance, bearing the special name of the Rayleigh-Jeans law), the exponential drop at high frequencies (the Wien law), and the resulting maximum of the function u(), reached at the frequency max with
82
.
2
T ,
(2.87)
max
i.e. at the wavelength max = 2/ k max = 2 c/max 2.22 c/ T.
10
1
u( )1
u
0
Fig. 2.9. The frequency dependence of the
0.1
blackbody radiation density, normalized by
u 0 T 3/22 c 3, according to the Planck law
(red line) and the Rayleigh-Jeans law (blue
line).
0.010.1
1
10
/ T
Still, I cannot help mentioning a few important particular values: one corresponding to the visible light (max ~ 500 nm) for the Sun’s effective surface temperature T K 6,000 K, and another one corresponding to the mid-infrared range (max ~10 m) for the Earth’s surface temperature T K 300 K.
The balance of these two radiations, absorbed and emitted by the Earth, determines its surface temperature and hence has the key importance for all life on our planet. This is why it is at the front and center of the current climate change discussions. As one more example, the cosmic microwave background (CMB) radiation, closely following the Planck law with T K = 2.725 K (and hence having the maximum density at max 1.9 mm), and in particular its (very small) anisotropy, is a major source of data for modern cosmology.
Now let us calculate the total energy E of the blackbody radiation inside some volume V. It may be found from Eq. (84) by its integration over all frequencies: 49,50
49 The last step in Eq. (88) uses a table integral, equal to (4)(4) = (3!)(4/90) = 4/15 – see, e.g., MA Eq. (6.8b), with s = 4, and then MA Eqs. (6.7e), and (2.7b).
Chapter 2
Page 25 of 44
Essential Graduate Physics
SM: Statistical Mechanics
3
4
3
2
d
VT
d
4
E V u() d V
V
T
.
(2.88)
2 3
2
3 3
3 3
/
c e
T 1 c e 1
15 c
0
0
0
Using Eq. (86) to recast Eq. (88) into the total power radiated by a blackbody surface, we get the well-known Stefan (or “Stefan-Boltzmann”) law 51
2
d P
4
4
T T
,
(2.89a) Stefan
dA
60 3 2
K
c
law
where is the Stefan-Boltzmann constant
2
W
Stefan-
4
8
k 67
.
5
10
.
(2.89b) Boltzmann
3 2
B
2
4
60 c
m K
constant
By this point, the thoughtful reader should have an important concern ready: Eq. (84) and hence Eq. (88) are based on Eq. (72) for the average energy of each oscillator, referred to its ground-state energy /2. However, the radiation power should not depend on the energy origin; why have not we included the ground energy of each oscillator into the integration (88), as we have done in Eq. (80)? The answer is that usual radiation detectors only measure the difference between the power Pin of the incident radiation (say, that of a blackbody surface with temperature T) and their own back-radiation power Pout, corresponding to some effective temperature T d of the detector – see Fig. 10. But however low T d is, the temperature-independent contribution /2 of the ground-state energy to the back radiation is always there. Hence, the term /2 drops out from the balance, and cannot be detected – at least in this simple way. This is the reason why we had the right to ignore this contribution in Eq. (88) –
very fortunately, because it would lead to the integral’s divergence at its upper limit. However, let me repeat that the ground-state energy of the electromagnetic field oscillators is physically real and may be important – see Sec. 5.5 below.
d in
P
E
( , T )
d
2
T
Fig. 2.10. The power balance at
T d
the electromagnetic radiation
d out
P
E
( , T )
d
d
power measurement.
2
One more interesting result may be deduced from the free energy F of the electromagnetic radiation, which may be calculated by integration of Eq. (73) over all the modes, with the appropriate weight (83):
50 Note that the heat capacity CV ( E/ T) V, following from Eq. (88), is proportional to T 3 at any temperature, and hence does not obey the trend CV const at T . This is the result of the unlimited growth, with temperature, of the number of thermally-exited field oscillators with frequencies below T/.
51 Its functional part ( E T 4) was deduced in 1879 by Joseph Stefan from earlier experiments by John Tyndall.
Theoretically, it was proved in 1884 by L. Boltzmann, using a result derived earlier by Adolfo Bartoli from the Maxwell equations for the electromagnetic field – all well before Max Planck’s work.
Chapter 2
Page 26 of 44
SM: Statistical Mechanics
2
/ T
/ T dN
/ T
F T ln1 e
Tln1 e
d Tln1 e
V
d
. (2.90)
2 3
d
0
0
c
Representing 2 d as d(3)/3, we can readily work out this integral by parts, reducing it to a table integral similar to that in Eq. (88), and getting a surprisingly simple result: 2
E
4
F V
T
.
(2.91)
45 3 3
c
3
Now we can use the second of the general thermodynamic relations (1.35) to calculate the pressure exerted by the radiation on the walls of the containing volume V:52
F
2
4
E
P
T
.
(2.92a)
V
45 3
c 3
V
3
T
Rewritten in the form,
E
Photon gas:
PV
,
(2.92b)
PV vs. E
3
this result may be considered as the equation of state of the electromagnetic field, i.e. from the quantum-mechanical point of view, of the photon gas. Note that the equation of state (1.44) of the ideal classical gas may be represented in a similar form, but with a coefficient generally different from Eq. (92).
Indeed, according to the equipartition theorem, for an ideal gas of non-relativistic particles whose internal degrees of freedom are in a fixed (say, ground) state, the temperature-dependent energy is that of the three translational “half-degrees of freedom”, E = 3 N( T/2). Expressing from here the product NT
= (2 E/3), and plugging it into Eq. (1.44), we get a relation similar to Eq. (92), but with a twice larger factor before E. On the other hand, a relativistic treatment of the classical gas shows that Eq. (92) is valid for any gas in the ultra-relativistic limit, T >> mc 2, where m is the rest mass of the gas’ particle.
Evidently, photons (i.e. particles with m = 0) satisfy this condition at any energy.53
Finally, let me note that Eq. (92) allows for the following interesting interpretation. The last of Eqs. (1.60), being applied to Eq. (92), shows that in this particular case, the grand thermodynamic potential equals (– E/3), so according to Eq. (91), it is equal to F. But according to the definition of , i.e. the first of Eqs. (1.60), this means that the chemical potential of the electromagnetic field excitations (photons) vanishes:
F
0 .
(2.93)
N
In Sec. 8 below, we will see that the same result follows from the comparison of Eq. (72) and the general Bose-Einstein distribution for arbitrary bosons. So, from the statistical point of view, photons may be considered bosons with zero chemical potential.
(ii) Specific heat of solids. The heat capacity of solids is readily measurable, and in the early 1900s, its experimentally observed temperature dependence served as an important test for the then-52 This formula may be also derived from the expression for the forces exerted by the electromagnetic radiation on the walls (see, e.g. EM Sec. 9.8), but the above calculation is much simpler.
53 Note that according to Eqs. (1.44), (88), and (92), the difference between the equations of state of the photon gas and an ideal gas of non-relativistic particles, expressed in the more usual form P = P( V, T), is much more dramatic: P T 4 V 0 vs. P T 1 V-1.
Chapter 2
Page 27 of 44
SM: Statistical Mechanics
emerging quantum theories. However, the theoretical calculation of CV is not simple54 – even for insulators, whose specific heat at realistic temperatures is due to thermally-induced vibrations of their crystal lattice alone.55 Indeed, at relatively low frequencies, a solid may be treated as an elastic continuum. This continuum supports three different modes of mechanical waves with the same frequency , that all obey linear dispersion laws, = vk, but the velocity v = vl for one of these modes (the longitudinal sound) is higher than that ( vt) of two other modes (the transverse sound).56 At such frequencies, the wave mode density may be described by an evident generalization of Eq. (83): 1 1
2
dN V
2
4
d .
(2.94a)
( 3
2 ) v 3
v 3
l
t
For what follows, it is convenient to rewrite this relation in a form similar to Eq. (83): 1
/ 3
2
3 V
d
1 1
2
dN
4
,
with v
.
(2.94b)
3
3
3
3
(2 )
v
3
v
v
l
t
However, the basic wave theory shows57 that as the frequency of a sound wave in a periodic structure is increased so much that its half-wavelength / k approaches the crystal period d, the dispersion law ( k) becomes nonlinear before the frequency reaches its maximum at k = / d. To make things even more complex, 3D crystals are generally anisotropic, so the dispersion law is different in different directions of the wave propagation. As a result, the exact statistics of thermally excited sound waves, and hence the heat capacity of crystals, is rather complicated and specific for each particular crystal type.
In 1912, P. Debye suggested an approximate theory of the specific heat’s temperature dependence, which is in surprisingly good agreement with experiment for many insulators, including polycrystalline and amorphous materials. In his model, the linear ( acoustic) dispersion law = vk, with the effective sound velocity v defined by the second of Eqs. (94b), is assumed to be exact all the way up to some cutoff frequency D, the same for all three wave modes. This Debye frequency may be defined by the requirement that the total number of acoustic modes, calculated within this model from Eq. (94b),
D
3
1
3
V
2
D
N V
4 d
,
(2.95)
3
3
2 3
(2 ) v
2 v
0
is equal to the universal number N = 3 nV of the degrees of freedom (and hence of independent oscillation modes) in a 3D system of nV elastically coupled particles, where n is the atomic density of the crystal, i.e. the number of atoms per unit volume.58 For this model, Eq. (72) immediately yields the following expression for the average energy and specific heat (in thermal equilibrium at temperature T ):
D
1
3
E V
2
4 d nVT
3
D x
,
(2.96)
3
3
x T T
(2 ) v
/
0 e
T
/
D
1
54 Due to a rather low-temperature expansion of solids, the difference between their CV and CP is small.
55 In good conductors (e.g., metals), specific heat is contributed (and at low temperatures, dominated) by free electrons – see Sec. 3.3 below.
56 See, e.g., CM Sec. 7.7.
57 See, e.g., CM Sec. 6.3, in particular, Fig. 6.5 and its discussion.
58 See, e.g., CM Sec. 6.2.
Chapter 2
Page 28 of 44
SM: Statistical Mechanics
Debye
C
1
( )
V
E
dD x
law
c
3 D( x) x
,
(2.97)
V
nV
nV T
dx
V
x T
/ T
D
where T D D is called the Debye temperature,59 and
3 x 3
d
,
1
for
x ,
0
D( x)
3
x
e
(2.98)
x
x
0
1
4 / 5 3, for ,
is the Debye function. The red lines in Fig. 11 show the temperature dependence of the specific heat cV
(per particle) within the Debye model. At high temperatures, it approaches a constant value of three, corresponding to the energy E = 3 nVT, in agreement with the equipartition theorem for each of three degrees of freedom (i.e. six half-degrees of freedom) of each mode. (This value of cV is known as the Dulong-Petit law.) In the opposite limit of low temperatures, the specific heat is much smaller: 3
12 4
T
c
1,
(2.99)
V
5
T D
reflecting the reduction of the number of excited phonons with < T as the temperature is decreased.
3
10
1
2
c
c
0.1
V
V
0.01
1
1 1
0 3
0
1 1
0 4
0
0.5
1
1.5
0.01
0.1
1
T / T
T / T
D
D
Fig. 2.11. The specific heat as a function of temperature in the Debye (red lines) and Einstein (blue lines) models.
As a historic curiosity, P. Debye’s work followed one by A. Einstein, who had suggested (in 1907) a simpler model of crystal vibrations. In his model, all 3 nV independent oscillatory modes of nV
atoms of the crystal have approximately the same frequency, say E, and Eq. (72) immediately yields
E 3
E
nV
,
(2.100)
/ T
E
e
1
so the specific heat is functionally similar to Eq. (75):
59 In the SI units, the Debye temperature T D is of the order of a few hundred K for most simple solids (e.g., ~430
K for aluminum and ~340 K for copper), with somewhat lower values for crystals with heavy atoms (~105 K for lead), and reaches its highest value ~2200 K for diamond, with its relatively light atoms and very stiff lattice.
Chapter 2
Page 29 of 44
SM: Statistical Mechanics
2
1 E
/ 2 T
c
(2.101)
V
3
E
.
nV T
sinh( / 2 T )
V
E
This dependence cV( T) is shown with blue lines in Fig. 11 (assuming, for the sake of simplicity, that E = T D). At high temperatures, this result does satisfy the universal Dulong-Petit law ( cV = 3), but for T << T D, Einstein’s model predicts a much faster (exponential) drop of the specific heat as the temperature is reduced. (The difference between the Debye and Einstein models is not too spectacular on the linear scale, but in the log-log plot shown on the right panel of Fig. 11, it is rather dramatic.60) The Debye model is in much better agreement with experimental data for simple, monoatomic crystals, thus confirming the conceptual correctness of his wave-based approach.
Note, however, that when a genius such as Albert Einstein makes an error, there is usually some deep and important background under it. Indeed, crystals with the basic cell consisting of atoms of two or more types (such as NaCl, etc.), feature two or more separate branches of the dispersion law ( k) –
see, e.g., Fig. 12. While the lower, “acoustic” branch is virtually similar to those for monoatomic crystals and may be approximated by the Debye model, = vk, reasonably well, the upper (“optical”61) branch does not approach = 0 at any k. Moreover, for large values of the atomic mass ratio r, the optical branches are almost flat, with virtually k-independent frequencies 0, which correspond to simple oscillations of each light atom between its heavy neighbors. For thermal excitations of such oscillations, and their contribution to the specific heat, Einstein’s model (with E = 0) gives a very good approximation, so for such solids, the specific heat may be well described by a sum of the Debye and Einstein laws (97) and (101), with appropriate weights.
“optical” branch
( k)
(arbitrary units,
linear
scale)
Fig. 2.12. The dispersion relation for
“acoustic” branch
mechanical waves in a simple 1D model of a
solid, with similar interparticle distances d, but
alternating particle masses, plotted for a
particular mass ratio r = 5 – see CM Chapter 6.
0
0.5
1.0
kd /
2.7. Grand canonical ensemble and distribution
As we have seen, the Gibbs distribution is a very convenient way to calculate the statistical and thermodynamic properties of systems with a fixed number N of particles. However, for systems in which N may vary, another distribution is preferable for applications. Several examples of such situations (as 60 This is why there is the following general “rule of thumb” in quantitative sciences: if you plot your data on a linear rather than log scale, you better have a good excuse ready. (An example of a valid excuse: the variable you are plotting changes its sign within the range you want to exhibit.)
61 This term stems from the fact that at k 0, the mechanical waves corresponding to these branches have phase velocities v ph ( k)/ k that are much higher than that of the acoustic waves, and may approach the speed of light.
As a result, these waves can strongly interact with electromagnetic (practically, optical) waves of the same frequency, while acoustic waves cannot.
Chapter 2
Page 30 of 44
Essential Graduate Physics
SM: Statistical Mechanics
well as the basic thermodynamics of such systems) have already been discussed in Sec. 1.5. Perhaps even more importantly, statistical distributions for systems with variable N are also applicable to some ensembles of independent particles in certain single-particle states even if the number of the particles is fixed – see the next section.
With this motivation, let us consider what is called the grand canonical ensemble (Fig. 13). It is similar to the canonical ensemble discussed in Sec. 4 (see Fig. 6) in all aspects, besides that now the system under study and the heat bath (in this case, more often called the environment) may exchange not only heat but also particles. In this ensemble, all environments are in both the thermal and chemical equilibrium, with their temperatures T and chemical potentials the same for all members.
system
under study
dQ, dS
Em,N, T, μ
dN
environment
Fig. 2.13. A member of the grand canonical
T, μ
ensemble.
Let us assume that the system of interest is also in chemical and thermal equilibrium with its environment. Then using exactly the same arguments as in Sec. 4 (including the specification of microcanonical sub-ensembles with fixed E and N), we may generalize Eq. (55), taking into account that the entropy S env of the environment is now a function of not only its energy E env = E – Em,N, 62 but also of the number of particles N env = N – N, with E and N fixed: ln W
ln M ln g ( E E
, N N ) ln E
S
E E
N N
m, N
env
m, N
(
,
) const
env
m, N
S
S
(2.102)
env
env
S
E
N const.
env E , N
E , N
m, N
E ,
E
N
N
env
env
To simplify this relation, let us rewrite Eq. (1.52) in the following equivalent form: 1
P
dS
dE
dV
dN.
(2.103)
T
T
T
Hence, if the entropy S of a system is expressed as a function of E, V, and N, then
S
1
S
P
S
,
,
.
(2.104)
E
T
V
T
N
T
V , N
E, N
E, V
Applying the first one and the last one of these relations to the last form of Eq. (102), and using the equality of the temperatures T and the chemical potentials in the system under study and its environment, at equilibrium (as was discussed in Sec. 1.5), we get
62 The additional index in the new notation Em,N for the energy of the system of interest reflects the fact that its spectrum is generally dependent on the number N of particles in it.
Chapter 2
Page 31 of 44
SM: Statistical Mechanics
1
ln W
S ( E , N ) E
N const .
(2.105)
m, N
env
m,
T
N
T
Again, exactly as at the derivation of the Gibbs distribution in Sec. 4, we may argue that since Em, N, T, and do not depend on the choice of environment’s size, i.e. on E and N, the probability Wm,N for a system to have N particles and be in m th quantum state in the whole grand canonical ensemble should also obey Eq. (105). As a result, we get the so-called grand canonical distribution: 1
N
E
Grand
m N
N
Em N
,
1
W
exp
(2.106) canonical
m N
exp
exp
,
,
,
Z
T
Z
T
T
distribution
G
G
where, just as in the case of the Gibbs distribution (2.58), the constant Z G (most often called the grand statistical sum, but sometimes the “grand partition function ” ) should be determined from the probability normalization condition. However, now the summation of the probabilities Wm,N should be over all possible values of both m and N:
N
Em, N
Z
exp
.
(2.107) Grand
G
canonical
m, N
T
sum
The last multiplier in the last form of Eq. (106) is the same as in the Gibbs distribution, and its physical interpretation is similar: states are “punished” by lower probability for their excessively higher energy. The handwaving interpretation of the first multiplier, with its opposite sign, is different: in the absence of the energy-related penalty Em, N, the environment with an average particle energy > 0
“wants” to flood the system with more particles.
Now let us see how the grand canonical distribution may be used for calculations of measurable variables. First, using the general Eq. (29) to calculate the entropy from Eq. (106) (exactly like we did it for the canonical ensemble), we get the following expression,
E
N
S
W
ln W
ln Z ,
(2.108)
m, N
m, N
G
m, N
T
T
which is evidently a generalization of Eq. (62).63 We see that now the grand thermodynamic potential
(rather than the free energy F) may be expressed directly via the normalization coefficient ZG: 1
N
Em N
F N E TS N T ln
T ln exp
,
.
(2.109) from ZG
Z
T
G
m, N
Finally, solving the last equality for ZG, and plugging the result back into Eq. (106), we can rewrite the grand canonical distribution in the form
N
Em, N
W
exp
,
(2.110)
m, N
T
similar to Eq. (65) for the Gibbs distribution. Indeed, in the particular case when the number N of particles is fixed, N = N, so + N = + N F, Eq. (110) is reduced to Eq. (65).
63 The average number of particles N is exactly what was called N in thermodynamics (see Chapter 1), but I keep this explicit notation here to make a clear distinction between this average value of the variable, and its particular values participating in Eqs. (102)-(110).
Chapter 2
Page 32 of 44
SM: Statistical Mechanics
2.8. Systems of independent particles
Now let us apply the general statistical distributions discussed above to a simple but very important case when the system we are considering consists of many similar particles whose direct interactions are negligible. As a result, each particular energy value Em,N of such a system may be represented as a sum of energies εk of the particles, where the index k numbers single-particle states –
rather than those of the whole system as the index m does.
Let us start with the classical limit. In classical mechanics, the energy quantization effects are negligible, i.e. there is a formally infinite number of quantum states k within each finite energy interval.
However, it is convenient to keep, for the time being, the discrete-state language, with the understanding that the average number Nk of particles in each of these states, usually called the state occupancy, is very small. In this case, we may apply the Gibbs distribution to the canonical ensemble of single particles, and hence use it with the substitution Em → εk, so Eq. (58) becomes Boltzmann
k
N
c exp
(2.111)
distribution
k
,
1
T
where the constant c should be found from the normalization condition:
N .
1
(2.112)
k
k
This is the famous Boltzmann distribution.64 Despite its formal similarity to the Gibbs distribution (58), let me emphasize the conceptual difference between these two important formulas. The Gibbs distribution describes the probability to find the whole system on one of its states with energy Em, and it is always valid – more exactly, for any canonical ensemble of systems in thermodynamic equilibrium. On the other hand, the Boltzmann distribution describes the occupancy of an energy level of a single particle, and, as we will see in just a minute, is valid for quantum particles only in the classical limit Nk << 1, even if the particles do not interact directly.
The last fact may be surprising, because it may seem that as soon as particles of the system are independent, nothing prevents us from using the Gibbs distribution to derive Eq. (111), regardless of the value of Nk . This is indeed true if the particles are distinguishable, i.e. may be distinguished from each other – say by their definitely different spatial positions, or by the states of certain internal degrees of freedom (say, spin), or by any other “pencil mark”. However, it is an experimental fact that elementary particles of each particular type (say, electrons) are identical to each other, i.e. cannot be
“pencil-marked”.65 For such particles we have to be more careful: even if they do not interact directly, 64 The distribution was first suggested in 1877 by L. Boltzmann. For the particular case when is the kinetic energy of a free classical particle (and hence has a continuous spectrum), it is reduced to the Maxwell distribution (see Sec. 3.1 below), which was derived earlier – in 1860.
65 This fact invites a natural question: what particles are “elementary enough” for their identity? For example, protons and neutrons have an internal structure, in some sense consisting of quarks and gluons; can they be considered elementary? Next, if protons and neutrons are elementary, are atoms? molecules? What about really large molecules (such as proteins)? viruses? The general answer to these questions, given by quantum mechanics (or rather experiment :-), is that any particles/systems, no matter how large and complex they are, are identical if they not only have the same internal structure but also are exactly in the same internal quantum state – for example, in the ground state of all their internal degrees of freedom. Evidently, the more complex are the particles/systems, the harder it is to enforce this situation in experiment.
Chapter 2
Page 33 of 44
SM: Statistical Mechanics
there is still some indirect dependence in their behavior, which is especially evident for the so-called fermions (elementary particles with semi-integer spin): they obey the Pauli exclusion principle that forbids two identical particles to be in the same quantum state, even if they do not interact directly.66
Note that the term “the same quantum state” carries a heavy meaning load here. For example, if two particles are confined to stay at different spatial positions (say, reliably locked in different boxes), they are distinguishable even if they are internally identical. Thus the Pauli principle, as well as other particle identity effects such as the Bose-Einstein condensation to be discussed in the next chapter, are important only when identical particles may move in the same spatial region. To emphasize this fact, it is common to use, instead of “identical”, a more precise (though grammatically rather unpleasant) adjective indistinguishable.
In order to take these effects into account, let us examine the statistical properties of a system of many non-interacting but indistinguishable particles (at the first stage of calculation, either fermions or bosons) in equilibrium, applying the grand canonical distribution (109) to a very unusual grand canonical ensemble: a subset of particles in the same quantum state k (Fig. 14).
single-particle energy levels:
…
k
…
Fig. 2.14. The grand canonical
ensemble of particles in the same
1
quantum state with energy
k –
0
schematically.
particle #: 1 2 … j …
In this ensemble, the role of the environment may be played just by the set of particles in all other states k’ k, because due to infinitesimal interactions, the particles may gradually change their states. In the resulting equilibrium, the chemical potential and temperature T of the system should not depend on the state number k, though the grand thermodynamic potential k of the chosen particle subset may. Replacing N with Nk – the particular (not average!) number of particles in the selected k th state, and the particular energy value Em,N with kNk, we reduce the final form of Eq. (109) to
N k
N
N
k
k
k
k
T ln
T
(2.113)
k
exp