Why Is Quantum Mechanics So Difficult? - Comments

[atyy]

Can you explain the meaning of these enigmatic statements? Also in condensed matter physics the highest symmetry is reached at (asymptotic) high energies, where matter becomes an ideal gas of elementary particles (of quite probably yet unknown fundamental degrees of freedom), but that cannot be what the condensed-matter physicist wanted to express.

Probably best to hear him explain it himself, rather than my garbling it. He says it right at the start: http://online.kitp.ucsb.edu/online/adscmt_m09/xu/.

 

[Dr. Courtney]

Perhaps by philosophical bits you don't mean the same thing as most of us do? Or perhaps you are working in a field such as quantum optics where quantum bits are more important than in most other branches of quantum physics? To check this out, can you name a few philosophical bits which you have in mind?

I prefer to describe the "philosophical bits" by exclusion rather than enumeration. Is it a part of QM that is essential for understanding the periodic table or for computing the spectra of atoms or molecules? If not, I tend to regard it more on the philosophical side that likely need not receive much emphasis in the first two semesters of undergrad QM.

I don't mind a section in the book or some brief classroom discussions to set the stage for what may be learned in more detail later, but I would regard it as out of balance if more than a few percent of the points in a 2 semester undergrad sequence depended on the philosophical material.

 

[vanhees71]

Well, you are certainly right that quantum optics can be done without philosophy. Nevertheless, if you think about typical quantum-optics experiments such as those that involve violation of Bell inequalities, weak measurements, or delayed choice quantum erasers, philosophic questions occur more naturally than in other branches of quantum physics. Yes, you can resist thinking about philosophical aspects of such experiments if you have a strong character, but temptation is quite strong. Some experimentalists in that field even call it - experimental metaphysics.

But Bell's inequality is a perfect example of the opposite! By giving a clear definition of what's meant by a deterministic local hidden-variable theory he derived is famous inequality which is violated in quantum theory, and thus it became a question of science which could be empirically tested. So it's completely (and in my opinion only) understandable from science, and it's not even too complicated. It can be explained in QM1 easily, as are the experiments like the Aspect experiment with polarization-entangled photons. There's no philosophy.

Of course, Bell's work was strongly motivated by philosophical issues and all the hype about the EPR paper, but the breakthrough was that this work brought these vague philosophical questions into the realm of objectively testable observational facts about nature!

 

[Demystifier]

I prefer to describe the "philosophical bits" by exclusion rather than enumeration. Is it a part of QM that is essential for understanding the periodic table or for computing the spectra of atoms or molecules?

That's quite an unusual definition of philosophy.

 

[Demystifier]

But Bell's inequality is a perfect example of the opposite! By giving a clear definition of what's meant by a deterministic local hidden-variable theory he derived is famous inequality which is violated in quantum theory, and thus it became a question of science which could be empirically tested. So it's completely (and in my opinion only) understandable from science, and it's not even too complicated. It can be explained in QM1 easily, as are the experiments like the Aspect experiment with polarization-entangled photons. There's no philosophy.

Of course, Bell's work was strongly motivated by philosophical issues and all the hype about the EPR paper, but the breakthrough was that this work brought these vague philosophical questions into the realm of objectively testable observational facts about nature!

The right question is this. Without using a philosophic question as a motivation, can you explain why Bell inequalities are important and interesting?

 

[Dr. Courtney]

That's quite an unusual definition of philosophy.

I wouldn't try and force the mathematical rigor of a "definition" on more of a working description. But it is based on defining science as testable in repeatable experiments. Historically, the experiments testing the "less philosophical" parts came much earlier.

My point is that an awful lot of experimental results can be explained by the aspects of QM that were historically used for decades to describe experimental results without the aspects that have been more debated on the philosophical side but not as important historically in the early decades of QM.

You seem to be wanting something more fundamental, I am offering something more practical and pedagogical - what to emphasize in undergrad QM courses.

I've published a lot of papers in atomic physics and QM without much use of the philosophical side, and so have a lot of atomic, molecular, and optical physicists.

 

[atyy]

But Bell's inequality is a perfect example of the opposite! By giving a clear definition of what's meant by a deterministic local hidden-variable theory he derived is famous inequality which is violated in quantum theory, and thus it became a question of science which could be empirically tested. So it's completely (and in my opinion only) understandable from science, and it's not even too complicated. It can be explained in QM1 easily, as are the experiments like the Aspect experiment with polarization-entangled photons. There's no philosophy.

Of course, Bell's work was strongly motivated by philosophical issues and all the hype about the EPR paper, but the breakthrough was that this work brought these vague philosophical questions into the realm of objectively testable observational facts about nature!

Why should we even care about a local hidden-variable theory? That is philosophy, since hidden variables are motivated by reality. If you don't like philosophy, Bell's inequality is not about hidden variables.

 

[lavinia]

No. Using vectors, matrices and functions is the natural way of describing any (mathematical or physical) system with a large number of degrees of freedom. For example, nonlinear manifolds are represented in terms of vectors when doing actual computations.

The classical phase space for a particle in an external field is also a vector space $R^6$ (or $C^3$ if you combine position and momentum to a complex position $z=q+i\kappa p$ with a suitable constant $\kappa$). And, unlike in the quantum case, one can form linear combinations of classical states.

Thus the problem with quantum mechnaics cannot lie in the use of vectors and their linear combinations. In the quantum case you just have many more states than classically, which is no surprise since it describes systems form a more microscopic (i.e., much more detailed) point of view.

What one must get used to is not the superpositions but the meaning attached to a pure quantum state, since this meaning has no classical analogue.

However, for mixed states (and almost all states in Nature are mixed when properly modeled), quantum mechanics is very similar to classical mechanics in all respects, as you could see from my book. (Note that the math in my book is no more difficult than the math you know already, but the intuition conveyed with it is quite different from what you can get from a textbook.)

Thus the difficulty is not intrinsic to quantum mechanics. It is created artificially by following the historically earlier road of Schroedinger rather than the later statistical road of von Neumann.

I frankly think you are misinterpreting what I said. That this is more linguistic than anything else. But if not I suggest that you get in touch with Leonard Susskind and explain to him what he has been getting wrong all these years.

Superposition to me is linear combination in a complex vector space. Then I guess if you want to be super accurate you then have to normalize.

In classical phase space e.g. a smooth manifold with some non-trivial topology, I would love to know how to take linear combinations of points on the manifold.

 

[A. Neumaier]

classical phase space e.g. a smooth manifold with some non-trivial topology, I would love to know how to take linear combinations of points on the manifold.

Classical phase space has very often a trivial topology, so that it is $C^{3n}$ in a very natural coordinatization. Taking linear combinations is straightforward.

On the other hand, thinking about superpositions rather than states is not needed in most of quantum theory, as it is not needed in most of classical theory. The real actors are the density operators resp. density functions, which encode the states once the problems get somewhat realistic (i.e., include the dissipative effect of the environment resp. friction). One cannot do this with superpositions.

 

[vanhees71]

The right question is this. Without using a philosophic question as a motivation, can you explain why Bell inequalities are important and interesting?

I consider the question of how to understand the indeterminism of quantum theory for both part of physics and philosophy. It's a very fundamental question whether nature is deterministic or not and thus it's part of philosophy as well as the natural sciences. The merit of Bell's work, in my opinion, was to make it a clearly answerable question of the natural sciences.

 

[vanhees71]

Why should we even care about a local hidden-variable theory? That is philosophy, since hidden variables are motivated by reality. If you don't like philosophy, Bell's inequality is not about hidden variables.

I don't understand this statement. To test a theory Bell thought about a class of alternative theories, namely a deterministic local theories, derived a consequence (Bell's inequality) which is violated by QT. Thus you can test it with experiments in the lab (nowadays there are many of them, starting with the pioneering work by Aspect). That's science, not philosophy!

 

[Demystifier]

That's science, not philosophy!

Science and philosophy are not mutually exclusive. If a scientific method can answer a deep question interesting also to philosophers, then it's also philosophy. Science is defined by a certain objective method, but philosophy is not defined by negation of that method. Philosophy is defined by the type of questions it asks. The intersection between scientific method and philosophic questions is not empty.

 

[ChrisVer]

"same structure as a standard wave equation": hmm... really? It always looked more like the difussion equation.

 

[pliep2000]

Considering the amount of answers, the question: Why is quantum so difficult, seems pretty difficult itself.

If one focuses on teaching of qf, the idea that one should or should not start with the old quantum theory is not so evident.

In 1992 Fischler & Lichtfelt (http://dx.doi.org/10.1080/0950069980200905) concluded that teaching the Bohr-model was a bad idea.
In 2008 McKagan (http://dx.doi.org/10.1103/PhysRevSTPER.4.010103) concludes that the Bohr-model can be used, if it is done right (and mcKagan tells how).

 

[vanhees71]

From my own learning experience I'm with the first paper. For me it was quite difficult to "unlearn" the Bohr-Sommerfeld model, which is mathematically appealing and to a certain extent intuitive although it's totally inconsistent in itself since if the electron was moving in elliptic orbits around the nucleus (or even more accurately both moving around their center of mass), the atom should radiate and be instable; in the BS-model it's simply stated that there are "allowed orbits", where this doesn't happen, but it's still inconsistent with classical physics upon which the model rests.

 

[Simon Phoenix]

I have no idea what the 'best' way to teach QM is - and I oscillate between the historical approach and the formal. In one sense, since the maths can be quite easy (if you teach the basic principles and axioms using 2 state systems), it just becomes a matter of (initially) setting down the recipe and cooking and seeing what it tastes like at the end. After working out several meals in this fashion, so we become comfortable with just applying the recipe, one can then go back and try to figure out what the recipe actually means.

QM, in its traditional 'textbook' form is such a massive disconnect from all the physics we learn up to that point - that this might indeed be the best way to do it, for most.

But I also think there's merit in the historical approach too. Presenting the bare axioms, drawn from the magicians hat like some hapless white rabbit, and just working out the consequences has the potential to leave the student with the letters WTF indelibly tattooed on their forehead (my tattoo is fading a bit now).

I wanted to know 'why'. Why did those amazing physicists like Planck, Einstein, Bohr, Heisenberg, Schrödinger, Dirac (and so on and so on) feel it necessary to change things so radically? Why were they forced to adopt this (on the surface) really bizarre set of axioms that looked like nothing that had gone before? Whatever possessed them?

We get some insight into this by looking at the original papers. Einstein's 1905 paper on the photoelectric effect is, for me, a tour de force of theoretical physics. It's quite simply breath taking. A few years later (in 1909 I think, but not sure) he published a paper showing that the fluctuations in the black body spectrum are partly to due 'wave' fluctuations and partly due to 'particle' fluctuations - astounding stuff indeed - hitting the fundamental kookiness that was looming squarely on the head.

Do we get a better insight into QM from this? I don't know - but you certainly get a better sense of why QM was needed at all - and why the classical approach that had been so successful just fell apart. I think it's important to 'understand' QM by looking at how it differs from classical thinking. We can make the difference seem 'small' by investigating the formalism (let's call QM a C* algebra but with non-commuting variables, to give a somewhat oversimplified example) - but ultimately there has to be a significant physical difference - and understanding that difference and why it's there is (probably) crucial for really grokking things.

But is this the best way to present QM in the first place? Don't know - I suspect not.

 

[vanhees71]

Well, to answer your final demand, what's the significant physical difference between classical and quantum physics, I'd say it's the possibility of "entanglement". Einstein called it the inseparabality of far-distant parts of a quantum system, and that's a very precise description. The possibility of the corresponding correlations between parts of a quantum system is not understandable from the point of view of a deterministic local theory, as has been shown by Bell. I agree, however, with your final statement that to start with this is not the best way to introduce QT. For that you need a good understanding of the formalism. I also tend to a mixture of the historic approach but telling from the very beginning that one should not get too much involved in old and partially inaccurate accounts of the history, among which are Einsteins 1905 approach to the photoelectric effect, which in fact does not prove the necessity to quantize the electromagnetic field (see my Insight article on this "sin" of physics didactics), the Bohr-Sommerfeld model of atoms, and inconsistent pictures like the wave-particle duality, which are all clarified and avoided by modern QT.

 

[Simon Phoenix]

Well, to answer your final demand, what's the significant physical difference between classical and quantum physics, I'd say it's the possibility of "entanglement".

But even entanglement can be viewed as an application of superposition (or maybe path-indistinguishability) to 2 objects (or a tensor product Hilbert space). So is it 'entanglement' that's fundamental or superposition?

For me the essential difference boils down to the fact that in a classical phase space one can, in principle, distinguish between two arbitrarily close states (q,p) and (q',p') but in QM the very concept of distinguishability is tied into the notion of orthogonal states so that 2 non-orthogonal states can, with a certain probability, 'mimic' the other in a given experiment.

among which are Einsteins 1905 approach to the photoelectric effect, which in fact does not prove the necessity to quantize the electromagnetic field

Indeed - wasn't it Lamb (and Scully?) who showed this way back when, something like half a century ago?

But I think the 'photoelectric' part of that paper is, for me, perhaps the least wonderful bit :-)

 

[vanhees71]

I don't know, who first did the derivation I've posted in my insights article. Nowadays it's a standard exercise for 1st-order time-dependent perturbation theory.

https://www.physicsforums.com/insights/sins-physics-didactics/

 

[dextercioby]

In my college days, some while back that is, we first had a „physics of atoms and molecules” course which indeed put forth in the first 2 lectures the history behind QM but without going into too many details. The only detailed historical models were: the blackbody radiation and Rutherford's scattering theory. The 3rd lecture had the Schrödinger equation already underway and that is it. Then, the next year, the proper QM course had as 1st lecture: „Prehilbert and Hilbert Space. Orthonormal systems”. No pre 1925 content whatsoever. I was happy with this. Still am.

 

[vanhees71]

In high school we had a long unit about "Bohr-Sommerfeld atoms". After that the teacher told us that we have to unlearn it and introduced the wave-mechanical picture a la Schrödinger (with the correct probability distribution). That was hard to digest. I never understood, why she had not introduced the wave-mechanical picture from the very beginning. That would have spared us from the hard "unlearning process". In university we had an "Introduction to quantum mechanics" before the theory-course lecture (QM1). There also they started with a historical introduction, and again the Bohr-Sommerfeld model was taught in some detail (this time of course using what we had learned in the classical mechanics lecture, i.e., Hamilton and phasespace formulation). At the end of the lecture we had of course also covered Schrödinger wave mechanics. QM 1 was then the "true" representation free formalism a la Dirac, which was a revelation for me. All of a sudden one could understand the structure of the theory, the meaning of states, operators representing observables, their eigenvalues and eigenvectors. I think, it's better to go in a way as you described, i.e., a very brief discussion of only a very few lectures with history of the "old quantum theory", but not in much details but rather a description of the "development of ideas", which finally lead to modern quantum theory, which was invented independently by 3 groups (roughly): (a) Heisenberg (idea in his famous but imho quite enigmatic Helgoland paper), Born, Jordan (formalism), Pauli (applications, including the solution of the non-relativistic hydrogen atom) => "matrix mechanics"; (b) Schrödinger ("wave mechanics" including a lot of the standard methods and applications we still learn today in the more wave-mechanics oriented lectures/books; equivalence between wave and matrix mechanics); (c) Dirac ("transformation theory"; basically the representation free formalism we learn today still almost as Dirac formulated it as "bra-ket formalism"). Then there was of course the more mathematical branch starting from von Neumann, leading to the development of modern functional analysis. Already the book by Courant and Hilbert played an important role for Schrödinger in developing wave mechanics.

 

[Demystifier]

In high school we had a long unit about "Bohr-Sommerfeld atoms". After that the teacher told us that we have to unlearn it and introduced the wave-mechanical picture a la Schrödinger (with the correct probability distribution). That was hard to digest. I never understood, why she had not introduced the wave-mechanical picture from the very beginning.

If nothing else, it teaches future scientists that theoretical physics is about constructing models that in the future can be superseded by better models. Some physicists attach too strongly to theories that were state of the art when they were young. The pedagogy of learning theories soon to be rejected and replaced by better theories may prevent this.

 

[Collin237]

The line between these two is not continuous, at least, not as of now.

Reference https://www.physicsforums.com/insights/quantum-mechanics-difficult/

I think that's the heart of the problem. There's no such thing as physics "as of now" versus future physics. The laws of physics have existed long before us and will continue to exist long after. They don't change; only our understanding of them changes.

There must be a continuity between quantum and classical physics, because classical matter is made of quantum stuff. That we haven't discovered it is no excuse for decreeing it doesn't exist. The idea of "Von Neumann's split" is untestable, unwarranted, and hands QM to mystics on a platter.

QM is the produce of science, not faith; and it must be taught as such. A successful student of QM has the right and duty to question the answers just as much as they answer the questions, provided they stay within the QM toolkit and use it correctly.

 

[ZapperZ]

I think that's the heart of the problem. There's no such thing as physics "as of now" versus future physics. The laws of physics have existed long before us and will continue to exist long after. They don't change; only our understanding of them changes.

There must be a continuity between quantum and classical physics, because classical matter is made of quantum stuff. That we haven't discovered it is no excuse for decreeing it doesn't exist. The idea of "Von Neumann's split" is untestable, unwarranted, and hands QM to mystics on a platter.

QM is the produce of science, not faith; and it must be taught as such. A successful student of QM has the right and duty to question the answers just as much as they answer the questions, provided they stay within the QM toolkit and use it correctly.

You quoted it out of context. Here's the entire passage that are relevant to that statement:

We use the identical words such as particle, wave, spin, energy, position, momentum, etc… but in QM, they attain a very different nature. You can’t explain these using existing classical concepts. The line between these two is not continuous, at least, not as of now. How does one use classical idea of a “spin” to explain a spin 1/2 particle in which one only regains the identical symmetry only upon two complete revolutions? We simply have to accept that we use the same word, but to ONLY mean that it produces a magnetic moment. It has nothing to do with anything that’s spinning classically. We can’t build the understanding of the QM spin using existing classical spin that we have already understood.

Notice that I did not say that QM cannot merge to explain classical observations. That wasn't what is "discontinuous". Rather, it is the USAGE of the terms such as "spin, position, etc. You cannot simply bring over the classical concepts of such things and expect them to be the same in QM. I used the concept of of "spin" as an example. We use the same word, but that word has a different meaning in QM.

This is NOT the "Von Neumann split".

Zz.

 

[Dr. Courtney]

I think that's the heart of the problem. There's no such thing as physics "as of now" versus future physics. The laws of physics have existed long before us and will continue to exist long after. They don't change; only our understanding of them changes.

There must be a continuity between quantum and classical physics, because classical matter is made of quantum stuff. That we haven't discovered it is no excuse for decreeing it doesn't exist. The idea of "Von Neumann's split" is untestable, unwarranted, and hands QM to mystics on a platter.

QM is the produce of science, not faith; and it must be taught as such. A successful student of QM has the right and duty to question the answers just as much as they answer the questions, provided they stay within the QM toolkit and use it correctly.

Ahh, but figuring out how to arrive at classical physics from QM may well require pushing the boundaries or tweaking how the QM toolkit is used.

Our understanding of QM may need to change to address all the issues in the "classical limit."

 

[houlahound]

The mystical stuff excites the general public and sells nontechnical books.

 

[carllooper]

We still use the term "planet" to reference those objects in the sky called planets, even though the word "planet", historically speaking, means "wandering god". But rather than change the word we've just changed the meaning of the word. And it was the invention of the telescope that altered the meaning. After looking through a telescope, at the god Saturn, we discovered Saturn was ball shaped and had a ring around his belly. A similar change in meaning has occurred in the use of words within quantum physics. An early model of the atom had the electrons orbiting the nucleus in the same the way the moon orbits the Earth. But it soon became apparent, from observations, that this model was inadequate. A better model was elaborated. One can still find use of the word "orbit" in relation to electrons but it now has a different meaning.

So we can see from these anecdotes that it's not a question of working out how to reconstruct ancient ideas, such as wandering gods, from observations of planets, or how to reconstruct classical ideas, such as planetary orbits, from observations of electron interference patterns. It's how to do something akin to the reverse. How to construct new models (ideas) from the observables: ones that will be in agreement with the observables, rather than ones that would be in disagreement. And in conjunction with such is also how to construct new observations - the telescope being a good historical example of such.

 

[carllooper]

...

Dr Courtney -
Thanks for the link in your signature to FFT source code.

 

[bhobba]

Ahh, but figuring out how to arrive at classical physics from QM may well require pushing the boundaries or tweaking how the QM toolkit is used. Our understanding of QM may need to change to address all the issues in the "classical limit."

Not really.

Derive the PLA which is fairly simple as I have posted a number of times. You start out with <x'|x> then you insert a ton of ∫|xi><xi|dxi = 1 in the middle to get ∫...∫<x|x1><x1|...|xn><xn|x> dx1...dxn. Now <xi|xi+1> = ci e^iSi so rearranging you get
∫...∫c1...cn e^ i∑Si.

Focus in on ∑Si. Define Li = Si/Δti, Δti is the time between the xi along the jagged path they trace out. ∑ Si = ∑Li Δti. As Δti goes to zero the reasonable physical assumption is made that Li is well behaved and goes over to a continuum so you get ∫L dt.

Now Si depends on xi and Δxi. But for a path Δxi depends on the velocity vi = Δxi/Δti so its very reasonable to assume when it goes to the continuum L is a function of x and the velocity v.

In this way you see the origin of the Lagrangian. And by considering close paths we see most cancel and you are only left with the paths of stationary action.

Then get copy of Landau - Mechanics where all of Classical mechanics is derived from this alone - including the existence of mass and that its positive. Strange but true. Actually some other assumptions are also made, but its an interesting exercise first seeing what they are, and secondly their physical significance.

You want to understand classical mechanics - this is how to do it. Honours students with calculus BC behind them and a course in multivariable calculus in parallel could do it. In fact it would probably be beest teaching an integrated course with chemistry along the lines of:

Multivarible Calculus
Linear Algebra
Quantum Mechanics
Classical Mechanics
Chemistry taught as probability models like at Princetons integrated science.

Thanks
Bill

 

[Collin237]

The mystical stuff excites the general public and sells nontechnical books.

And conversely? That humble, honest, up-to-date explanations of what physicists have actually found out don't sell to the general public? More likely, those who could reach out this way aren't even trying!

 

[houlahound]

Didnt Hawking famously say - every equation he puts in a book takes X amount of sales off.

Paraphrased.

 

[Collin237]

Supposedly, it's absolutely impossible to put the equations of modern physics into plain English. I don't buy that. Has any group of physicists ever even attempted to "Gutenbergize" QM for the public?

 

[houlahound]

I believe that would be most uncompelling and ugly.

 

[Collin237]

Then get copy of Landau - Mechanics where all of Classical mechanics is derived from this alone

For something worthy of this honor, it would need to be fully consistent with the applicable regimes of Newton, Hamilton, Lagrange, Schrodinger, Heisenberg, Einstein (at least as far as SR), and also all the various types of material physics. And on top of all that, it also has to be profinite, so it can be put to practical use.

Are you saying that the path integral is up to that challenge?

 

[Collin237]

I believe that would be most uncompelling and ugly.

Sic transit gloria. You have a better idea?