Understanding the Uniform Probability Distribution in Statistical Ensembles

[atyy]

Exactly that's my point. Whatever you do to make a better educated guess about probabilities than just making up something, to be sure you have to do experiments with sufficiently large ensembles. Of course, you use Bayes's formula for conditional probabilities, but that doesn't mean that you are a Bayesianist denying the frequentist interpretation of probabilities.

I'm a Frequentist, so I agree with you.

However, my attack (based on the non-uniqueness of the entropy) is only against objective Bayesians like Jaynes.

Since subjective Bayesians freely admit their subjectivity, they are immune to our criticisms. Furthermore, they can show derive Frequentist behaviour. And they can show that their position is coherent. So while I am not a subjective Bayesian, I don't believe their position can be criticized, except for being impractical.

 

[stevendaryl]

Since subjective Bayesians freely admit their subjectivity, they are immune to our criticisms. Furthermore, they can show derive Frequentist behaviour. And they can show that their position is coherent. So while I am not a subjective Bayesian, I don't believe their position can be criticized, except for being impractical.

Yes, that's what I think--from the point of view of (subjective) Bayesianism, frequentism is a heuristic, or shortcut, to simplify reasoning. It's way too much work to assign probabilities to every possible hypothesis, and compute how all those probabilities change in light of new information.

 

[Demystifier]

What's the "probablity in the Bayesian form"?

In the context of statistical mechanics, it is the assumption that probability of a given state is proportional to the phase-space volume of that state. In equilibrium it is assumed to be true even for a single particle at a single instant of time, implying that the frequentist probability cannot be applied. Since frequentist probability cannot be applied, what remains is the Bayesian probability.

 

[A. Neumaier]

the concept of an ensemble is an experimental concept

No; it is primarily a theoretical concept. It is properly introduced the first time in statistical mechanics - in the theory of equilibrium states. The experimental part of this theory is called thermodynamics and has no notion of an ensemble.

The experimental version of the ensemble concept is ''repetition of an experiment under comparable conditions''. Experimenters do not talk about ensembles but about statistical procedures.

that the ensemble is a virtual imagined ensemble, not a real ensemble, and this is what creates confusion.

This is how Gibbs, who coined the term, defined it. It is perfectly reasonable; why should there be confusion?
The ensemble describes a single glass of water (or any other chemical substance or mixture), and the ensemble is a fiction employed to justify intuitively the statistical machinery to the single glass of water. It was necessary in Gibbs' days where the idea that mathematics provides only models and that the same math can describe very different subject matters was not yet developed. it was clear to Gibbs that no actual repetition was relevant for the interpretation of the results.

Today we know that probability and statistics can be applied to very deterministic subject matters such as the (fully objective!) distribution of prime numbers or rounding errors in computer calculations. Therefore no notion of ensemble or repetition under similar condition must be in the background - so the name is just a historical leftover (just as the particle notion in quantum field theory). Just as the abstract notion of a vector is typically unrelated to arrows in 2- or 3-dimensional space but just means elements of anything satisfying the vector space axioms, so the abstract notion of an ensemble is often unrelated to betting games or repetition of experiments (these are just some specific applications) but can be anything that gives rise to expectations with the usual properties.

 

[Demystifier]

No; it is primarily a theoretical concept. It is properly introduced the first time in statistical mechanics - in the theory of equilibrium states. The experimental part of this theory is called thermodynamics and has no notion of an ensemble.

You and me are starting from different points. Your starting experimental point is thermodynamics, which you want to explain statistically. My starting experimental point is coin flipping, which I want to explain statistically. My starting point is much more basic then yours, so I think that my perspective is more fundamental then yours. Of course, you think that probability cannot be applied to a single flipping of a coin, while I think it can. Yesterday we established that we cannot agree on that basic thing, so we also cannot agree on all other things that follow from this basic one.

 

[A. Neumaier]

My starting point is much more basic then yours, so I think that my perspective is more fundamental then yours.

I completely disagree. Coin flipping is not even a subject of physics, except if you are interested in the motion of a coin rather than in head or tail. Thus it is strictly speaking off-topic here.

Coins and measurements are human artifacts, hence cannot be basic for an understanding of physics (the only concern in this forum). The objects that already existed in Nature before humans appeared (water, light, etc.) are far more basic.

 

[Demystifier]

I completely disagree. Coin flipping is not even a subject of physics, except if you are interested in the motion of a coin rather than in head or tail. Thus it is strictly speaking off-topic here.

Coins and measurements are human artifacts, hence cannot be basic for an understanding of physics (the only concern in this forum). The objects that already existed in Nature before humans appeared (water, light, etc.) are far more basic.

Then let as agree that we disagree.

 

[vanhees71]

I completely disagree. Coin flipping is not even a subject of physics, except if you are interested in the motion of a coin rather than in head or tail. Thus it is strictly speaking off-topic here.

Coins and measurements are human artifacts, hence cannot be basic for an understanding of physics (the only concern in this forum). The objects that already existed in Nature before humans appeared (water, light, etc.) are far more basic.

Measurements are at the heart of physics. Physics is all about measurements and what we can learn about how nature behaves in terms of quantitative observation of objective (i.e., repeatedly confirmable) phenomena. In this sense physics is a complete human artifact. Nature couldn't care less about our understanding of her in terms of the natural sciences!

 

[Demystifier]

Measurements are at the heart of physics. Physics is all about measurements and what we can learn about how nature behaves in terms of quantitative observation of objective (i.e., repeatedly confirmable) phenomena. In this sense physics is a complete human artifact. Nature couldn't care less about our understanding of her in terms of the natural sciences!

If you said that in the context of quantum foundations, I might not agree. But here, in the context of foundations of statistical mechanics, I agree. Even though I like to think that physics, in general, might be something more then a human tool for a description of nature, I don't have problems with admitting that statistical physics is not much more than that.

 

[A. Neumaier]

Measurements are at the heart of physics. Physics is all about measurements and what we can learn about how nature behaves in terms of quantitative observation of objective (i.e., repeatedly confirmable) phenomena. In this sense physics is a complete human artifact. Nature couldn't care less about our understanding of her in terms of the natural sciences!

Measurements are human artifacts used to check and perhaps to arrive at physical theories. But they are nothing fundamental - they don't figure in Newton's laws, Einstein's general relativity, or the standard model. Moreover, how things are measured changes considerable with time, while the fundamental physics is supposed to be time-invariant (though less completely known at earlier times). Otherwise we couldn't apply physics to the past or the far distance (where we only see radiation from the far past).

 

[vanhees71]

If you said that in the context of quantum foundations, I might not agree. But here, in the context of foundations of statistical mechanics, I agree. Even though I like to think that physics, in general, might be something more then a human tool for a description of nature, I don't have problems with admitting that statistical physics is not much more than that.

What makes quantum theory different from any other physical theory (in fact there's only one alternative, namely classical physics)? It's a quantitative description of (the objective aspects of) nature nothing more but also nothing less. It has a wide range of validity with the limitations yet unknown (except that there's no satisfactory quantum description of gravity). In this sense we consider it as the fundamental theory underlying all physics, but it's based on empirical evidence as any theory in physics and thus subject to changes whenever a reproducible contradiction between theory and experiment occurs!

 

[vanhees71]

Measurements are human artifacts used to check and perhaps to arrive at physical theories. But they are nothing fundamental - they don't figure in Newton's laws, Einstein's general relativity, or the standard model. Moreover, how things are measured changes considerable with time, while the fundamental physics is supposed to be time-invariant (though less completely known at earlier times). Otherwise we couldn't apply physics to the past or the far distance (where we only see radiation from the far past).

What else figures into any physical theory if not empirical experience? I don't know any physical theory that is successful in describing nature, which has no solid foundation in empirical evidence in form of quantitative observations/measurements. Of course, the technology has made and hopefully will still made a tremendous progress in just a few decades, and this also brings more to be observable and quantified (the most recent example are gravitational waves) and/or better and better accuracy. This in turn might force us to refine or even completely modify our contemporary theories and models. That's progress of science. Of course, a lot seems to be known already, and our theories are quite comprehensive (concerning about 4% of the energy-momentum content of our Universe ;-)), and the extrapolation of the locally discovered laws to even the evolution of the entire universe is pretty successful, but this doesn't mean that this is the end of physics. Who knows, what will be discovered with even better and more sensitive instruments in the future?

 

[Demystifier]

What makes quantum theory different from any other physical theory (in fact there's only one alternative, namely classical physics)? It's a quantitative description of (the objective aspects of) nature nothing more but also nothing less. It has a wide range of validity with the limitations yet unknown (except that there's no satisfactory quantum description of gravity). In this sense we consider it as the fundamental theory underlying all physics, but it's based on empirical evidence as any theory in physics and thus subject to changes whenever a reproducible contradiction between theory and experiment occurs!

Suppose that you live in the beginning of the 20'th century knowing nothing about modern QM. But you know very well pure classical mechanics (Newton, Lagrange, Hamilton), as well as works of Boltzmann and Gibbs on classical statistical mechanics. And suppose somebody tells you that pure classical mechanics tells us what really happens in Nature, while classical statistical mechanics only tells us what we can know about Nature in some circumstances involving many particles. What would you tell him?

 

[vanhees71]

It depends on the application. You cannot describe a gas of $10^{23}$ particles on all microscopic detail and thus you look at the relevant coarse-grained macroscopic observables, employing probability theory for what I ignore. The answer is not different to that given to a 21st-century physicist "knowing" that QT describe what "really happens in nature".

 

[Demystifier]

It depends on the application. You cannot describe a gas of $10^{23}$ particles on all microscopic detail and thus you look at the relevant coarse-grained macroscopic observables, employing probability theory for what I ignore. The answer is not different to that given to a 21st-century physicist "knowing" that QT describe what "really happens in nature".

Yes but - the hypothetical person from the beginning of the 20'th century further argues - we don't use any probability in pure classical mechanics, so nothing is ignored. Does it mean that pure classical mechanics, when it is applicable, tells us what really happens?

What's your answer? (You still know nothing about modern QM.)

 

[vanhees71]

I guess, I'd had argued that indeed within classical physics the probabilities are just an effective description of our ignorance due to the complexity of the many-body system. In fact it has been argued by Boltzmann, Gibbs et al at the time when they tried to establish the very idea of statistical physics. Of course, they had a very hard time to convince many of their colleagues about the effectiveness of their approach. The very existence of "atoms" was even highly suspicious to most of their contemporary physicist, while naturally they were pretty much accepted by chemists. E.g., Planck, as an expert on thermodynamics, didn't like the statistical approach at all, but got convinced later. The most important step for the acceptance of the atomistic structure of matter within physics has been perhaps Einstein's work on fluctuations, including is famous work on Brownian motion and, perhaps even more convincing, the critical opalescence and the quantitative determination of the Avogadro constant.

Now comes again some philosophical mambo-jambo, if you ask, whether physical theories tell us "what really happens". What does it mean, when you say something really happens. This can only be an opinion of any individual physicist but is not subject of science itself. I strongly believe that Nature doesn't care very much about us and our knowledge about what's going on. So I think it exists independently of us, and we never know, "what really happens", but we know not too badly what happens in a given situation due to the natural laws, which are descriptions of empirical quantitative observations of the objective part of our experience of what happens in nature or as it occurs to us, including a tremendous extention of our senses by technological aids.

 

[stevendaryl]

Yes but - the hypothetical person from the beginning of the 20'th century further argues - we don't use any probability in pure classical mechanics, so nothing is ignored. Does it mean that pure classical mechanics, when it is applicable, tells us what really happens?

What's your answer? (You still know nothing about modern QM.)

I'm not exactly sure what point you are making, but it does seem to me that there is a difference between classical and quantum physics in that classical physics was supposed to describe the way the world works, even if there are no scientists or observers or measurement devices around, while the usual interpretation of quantum mechanics, which is that it describes the probabilities of outcomes of measurements, is hard to make sense of in the absence of measurement devices.

 

[A. Neumaier]

the usual interpretation of quantum mechanics, which is that it describes the probabilities of outcomes of measurements, is hard to make sense of in the absence of measurement devices.

This just implies that the orthodox interpretations are much more limited that the true scope of quantum mechanics.
Quantum mechanics is known to apply to things everywhere and anytime in the world, including many situations where one can observe only very indirect consequences.

 

[stevendaryl]

This just implies that the orthodox interpretations are much more limited that the true scope of quantum mechanics.
Quantum mechanics is known to apply to things everywhere and anytime in the world, including many situations where one can observe only very indirect consequences.

I believe that, but there is a mismatch between that universal applicability and the way it is (usually) presented, which is in terms of probabilities for observables (or expectations for observables, in the density matrix formulation).

 

[A. Neumaier]

there is a mismatch between that universal applicability and the way it is (usually) presented, which is in terms of probabilities for observables (or expectations for observables, in the density matrix formulation).

The probability interpretation is questionable as a foundation, as it it always associated with the idea of frequent measurement (or even more anthropocentric ideas). But measurements are a comparably rare event in Nature (especially if we average over the duration of the existence of the universe).

The shut-up-and-calculate version of quantum mechanics is universally applied, always making use of the notion of expectation - typically without reference to measurements, and only sometimes using their interpretation in terms of probabilities (needed only for interpreting scattering experiments, where it has a rational basis in abundant statistics). Thus a good interpretation should only be based on expectation, not on probabilities.

Chapters 8 and 10 of my online book on quantum mechanics were designed explicitly to take this into account, resulting in a presentation without the mismatch that you mention. The basics were also discussed here on PF.

I got the idea from a book on classical probability by Peter Whittle, Probability via expectation (4th edition, 2000). From the preface to the third edition (starting with a reference to the first edition from 1970):

The particular novelty of the approach was that expectation was taken as the prime concept, and the concept of expectation axiomatized rather than that of a probability measure. [...] In re-examining the approach after this lapse of time I find it more persuasive than ever. [...] I would briefly list the advantages of the expectation approach as follows.

• (i) It permits a more economic and natural treatment at the elementary level.
• (ii) It opens an immediate door to applications, because the quantity of interest in many applications is just an expectation.
• (iii) Precisely for this last reason, one can discuss applications of genuine interest with very little preliminary development of theory. On the other hand, one also finds that a natural unrolling of ideas leads to the development of theory almost of itself.
• (iv) The approach is an intuitive one, in that people have a well-developed intuition for the concept of an average. Of course, what is found 'intuitive' depends on one's experience, but people with a background in the physical sciences have certainly taken readily to the approach. [...]
• (v) The treatment is the natural one at an advanced level. [...] The accepted concepts and techniques of weak convergence and of generalized processes are characterized wholly in terms of expectation.
• (vi) Much conventional presentation of probability theory is distorted by a preoccupation with measure-theoretic concepts which is in a sense premature and irrelevant. These concepts (or some equivalent of them) cannot be avoided indefinitely. However, in the expectation approach, they find their place at the natural stage.
• (vii) On the other hand, a concept which is notably and remarkably absent from conventional treatments is that of convexity. (Remarkable, because convexity is a probabilistic concept, and, in optimization theory, the necessary invocations of convexity and of probabilistic ideas are intimately related.) In the expectation approach convexity indeed emerges as an inevitable central concept.
• (viii) Finally, in the expectation approach, classical probability and the probability of quantum theory are seen to differ only in a modification of the axioms - a modification rich in consequences, but succinctly expressible.

The 4th edition treats quantum mechanics in the final Chapter 20. In particular, in Theorem 20.1.5, Whittle derives the Born rule as conditional probability, thus removing all weirdness from its interpretation. (Later, he characterizes the Schroedinger equation, unfortunately placing the $i$ systematically on the wrong side of the equation, so getting the dynamics backwards. But in spite of this small lapse, I can highly recommend the book!

 

[vanhees71]

This just implies that the orthodox interpretations are much more limited that the true scope of quantum mechanics.
Quantum mechanics is known to apply to things everywhere and anytime in the world, including many situations where one can observe only very indirect consequences.

Apply in which sense? We always look on ensembles or otherwise coarse grained observables (expectation values) and compare them with the predictions by quantum theory. So what else is there within QT than the probablities predicted by the formalism and their experimental tests via the usual statistical methods?

 

[A. Neumaier]

Apply in which sense? We always look on ensembles or otherwise coarse grained observables (expectation values) and compare them with the predictions by quantum theory. So what else is there within QT than the probabilities predicted by the formalism and their experimental tests via the usual statistical methods?

Apply in the sense that statistical mechanics applies to a single glass of water. One uses ensemble expectation values for the single quantum system [and, according to Gibbs, nonphysical, imagined repetitions to justify the ensemble language for the single use case] to assign a temperature and other things that can be measured.

Single, nonrepeated measurements of temperature, pressure and volume can be used to check the predictions of quantum mechanics in equilibrium. These measurements have nothing to do with any of the mock measurements of identically prepared systems discussed in the traditional interpretations of quantum mechanics.

 

[stevendaryl]

Thus a good interpretation should only be based on expectation, not on probabilities.

I don't see a big difference, in principle, between basing it on expectation and basing it on probabilities. What is the difference in principle between saying that observable $A$ has values $a_i$ with probability $p_i$ and saying observable $A$ has expectation value $\langle A \rangle$?

In classical statistical mechanics, one would say either that $A$ fluctuates unpredictably, but the average value is $\langle A \rangle$, or that $A$ has a definite, though unknown, value, and $\langle A \rangle$ represents the average over many systems that are macroscopically identical to the one of interest.

 

[A. Neumaier]

What is the difference in principle?

The difference is that in the second (expectation) case you don't need (and actually don't want!) a probability interpretation.

Nobody using statistical mechanics for applications employs the probability interpretation you propose. Instead, what is always (except when the subject matter is introduced) used is the interpretation given in an earlier PF discussion.

 

[A. Neumaier]

one would say either that A fluctuates unpredictably, but the average value is ⟨A⟩

The Hamiltonian $H$ is invariant in time, hence does not fluctuate at all. So which meaning do you ascribe to the internal energy $\langle H \rangle\>$ of a particular glass of water?

Note that this internal energy can be measured in the traditional sense of the notion - by computing it from single measurements of $P,V,T$ together with the equation of state of water (which can be derived in some approximation from classical statistical mechanics).

 

[A. Neumaier]

If you think that an abstract mathematical concept (such as that of expectation) must necessarily be interpreted in the way it arose in the application it was abstracted from then you would also have to interpret every wave function (vector in a Hilbert space) as a little arrow in ordinary space, since that is what the concept of a vector originally meant.

 

[stevendaryl]

The Hamiltonian $H$ is invariant in time, hence does not fluctuate at all. So which meaning do you ascribe to the internal energy $\langle H \rangle\>$ of a particular glass of water?

If the system of interest is coupled to a reservoir at a constant temperature, then the total energy of the system is not constant, since it can exchange energy with the reservoir. In the case of a glass of water, there is the possibility of an exchange of energy with the environment.

 

[A. Neumaier]

If the system of interest is coupled to a reservoir at a constant temperature, then the total energy of the system is not constant, since it can exchange energy with the reservoir. In the case of a glass of water, there is the possibility of an exchange of energy with the environment.

Put the water in a thermally isolated flask; then no energy is exchanged.

 

[stevendaryl]

If you think that an abstract mathematical concept (such as that of expectation) must necessarily be interpreted in the way it arose in the application it was abstracted from then you would also have to interpret every wave function (vector in a Hilbert space) as a little arrow in ordinary space, since that is what the concept of a vector originally meant.

I think you're fooling yourself if you think that going from probabilities to expectation values means that you understand things better. You can certainly work with things abstractly, in which case, you don't actually need to know what you're talking about. That's the beauty of the "shut up and calculate" interpretation of QM. But if you think that you are doing anything more than shut up and calculate, I think you're fooling yourself.

 

[stevendaryl]

Put the water in a thermally isolated flask; then no energy is exchanged.

That's why I had an either/or. Either the expectation value represents fluctuation in time, or it represents microscopic differences between macroscopically identical systems. (Or both)

 

[A. Neumaier]

I think you're fooling yourself.

I think you simply want to make a fool of me because you don't understand the nature of abstraction.

Calling something an expectation is simply a choice of words like calling something a vector. It conveys no other information than what is given in the definition of the concept. For expectation, the definition (given by Whittle) requires linearity, positivity, and continuity. Nothing else.

In applications where one has sufficiently many repetitions one may interpret the expectation as an average, just as in applications where a vector has three position coordinates you may interpret the vector as an arrow in ordinary space.

In applications where one has no repetitions one cannot interpret the expectation as an average, just as in applications where a vector represents a wave function $\psi(x)$ one cannot interpret the vector as an arrow in ordinary space.

It is as simple as that. Only the application determines the way how an abstract concept is to be interpreted in a concrete situation.

 

[A. Neumaier]

That's why I had an either/or. Either the expectation value represents fluctuation in time, or it represents microscopic differences between macroscopically identical systems. (Or both)

But neither applies in case of a single bottle of water when the bottle is thermally isolated.

 

[stevendaryl]

But neither applies in case of a single bottle of water when the bottle is thermally isolated.

Yes, it does. Given the macroscopic description of the bottle of water, in terms of total energy, mass, etc., there are many different microscopic states that are consistent with that macroscopic description.

 

[stevendaryl]

I think you simply want to make a fool of me because you don't understand the nature of abstraction

I don't want to make a fool of you, but I think that you are claiming insights that you don't actually have. You aren't doing anything different than "shut up and calculate".

 

[A. Neumaier]

Yes, it does. Given the macroscopic description of the bottle of water, in terms of total energy, mass, etc., there are many different microscopic states that are consistent with that macroscopic description.

But the measurement is done on the single system only. The others are just fictitious copies (as Gibbs told us) without any influence on the measured system. Your expectation would be an average over fictitious measurements, which makes no sense.