Why Is Quantum Mechanics So Difficult?
Table of Contents
Quantum Mechanics Key points:
- Quantum mechanics (QM) is often perceived as difficult, especially by non-physicists.
- The difficulty lies in the conceptual foundation of QM, as it doesn’t connect well with classical understanding.
- Unlike other areas of physics, there’s no direct continuity between classical concepts and QM.
- While the conceptual understanding is challenging, the mathematical formulation of QM is familiar and follows from existing knowledge.
- Mathematical formalism is crucial in QM, as it provides a bridge between classical and quantum worlds.
- QM’s mathematical formalism is the foundation on which our understanding is built.
- Disagreements often arise in the interpretation of QM, but the source, mathematical formalism, remains consistent.
- The mathematical aspect of QM is a means of conveying ideas and principles accurately, akin to musical notes conveying music.
- In QM, mathematics is a form of communication that accurately describes our universe.
- Quantum mechanics doesn’t fully make “sense” without embracing its mathematical continuity.
QM’s formalism
Strangely enough, QM’s formalism isn’t any more difficult than other areas of physics. The mathematics of the “standard” QM isn’t any worse than, let’s say, electromagnetism. Yet, to many people, especially non-physicists, QM presents a very daunting effort to understand.
I strongly believe that it all comes down to how we understand things and how we expand our knowledge. Typically, when we teach students new things, what we do is build upon their existing understanding. We hope that a student already has a foundation of knowledge in certain areas, such as basic mathematics, etc. so that we can use that to teach them about forces, motion, energy, and other fun stuff in intro physics. Then, after they understand the basic ideas, we show them the same thing, but with more complications added to it.
The same thing occurs when we try to help a student doing a homework problem. We always try to ask what the student knows already, such as the basic principle being tested in that question. Does the student know where to start? What about the most general form of the equation that is relevant to the problem? Once we know a starting point, we then build on that to tackle that problem.
The common thread in both cases is that there exists a STARTING point as a reference foundation on which, other “new” stuff is built upon. We learn new and unknown subjects based on what we have already understood. This is something crucial to keep in mind because, in the study of QM, this part is missing! I am certain that for most non-physicists, this is the most common reason why QM is puzzling, and why quacks and other people who are trying to use QM in other areas such as “metaphysics” or mysticism, are using it in a completely hilarious fashion.
The Disconnect
There is a complete disconnect between our “existing” understanding of the universe based on classical understanding and QM. There is nothing about our understanding of classical mechanics that we can build on to understand QM. We use identical words such as particle, wave, spin, energy, position, momentum, etc… but in QM, they attain a very different nature. You can’t explain these using existing classical concepts. The line between these two is not continuous, at least, not as of now. How does one use the classical idea of a “spin” to explain a spin 1/2 particle in which one only regains the identical symmetry upon two complete revolutions? We simply have to accept that we use the same word but to ONLY mean that it produces a magnetic moment. It has nothing to do with anything that’s spinning classically. We can’t build the understanding of the QM spin using the existing classical spin that we have already understood.
Now interestingly enough, the MATHEMATICAL FORMULATION of QM is quite familiar! The time-dependent Schrodinger equation has the same structure as a standard wave equation. We call the energy operator the Hamiltonian not for nothing since it looks very familiar with the Hamiltonian approach to classical mechanics. The matrix formulation isn’t anything new. What this means is that while the conceptual foundation of QM is completely disconnected from our traditional conceptual understanding, the mathematical formulation of QM completely follows from our existing understanding! Mathematically, there is no discontinuity. We build the formalism of QM based on our existing understanding!
Mathematical formulation
This is why, in previous threads in PF, I disagree that we should teach students the concepts of QM FIRST, rather than the mathematical formulation straightaway. There is nothing to “build on” in terms of conceptual understanding. We end up telling the students what they are out of thin air. The postulates of QM did not come out of our classical understanding of our world. Instead, mathematical formalism is the only thing that saves us from dangling in mid-air. It is the only thing on which our existing understanding can be built.
What this implies is that, if one lacks the understanding of the mathematical formalism of QM, one hasn’t understood QM at all! One ends up with all these weird, unexplained, unfamiliar, and frankly, rather strange ideas on how the world works. These conceptual descriptions of QM may even appear “mystical”. It is not surprising that such connections are being made between QM and various forms of mysticism. One lacks any connection with the existing reality that one has understood. So somehow, since QM can do this, it seems as if it’s a license to simply invent stuff..
The mathematical formalism of QM is what defines the QM description. The “conceptual description” is secondary, and is only present because we desire some physical description based on what we already have classically. It is why people can disagree on the interpretation of QM, yet they all agree on the source, the mathematical formalism of QM.
QM as musical notes
This, however, does not mean that QM is nothing more than “just mathematics”. This is no more true than saying the musical notes on a sheet of paper are just scribbles. The notes are not the important object. Rather, it is the sound that it represents that’s the main point. The musical notes are simply a means to convey that point clearly and unambiguously. Similarly, the mathematics that is inherent in QM and in all of physics, is a means to convey an idea or principle. It is a form of communication, and so far it is the ONLY form of communication accurate and unambiguous enough to describe our universe. It reflects completely our understanding of phenomena. So a mathematical formulation isn’t “just math”.
You cannot use your existing understanding of the universe to try to understand the various concepts of QM. There is a discontinuity between the two. It is only via the mathematical continuity of the description can there is a smooth transition to build upon. Without this, QM will not make “sense”.
PhD Physics
Accelerator physics, photocathodes, field-enhancement. tunneling spectroscopy, superconductivity
Why should this be ugly?
Explaining equations in plain English does not mean there are no more equations. It means that the equations are written to be teaching tools themselves, rather than merely something to learn. I never thought it possible, but apparently they've been there for quite a while, hidden beneath a pile of Google trolls, and now the trolls have disappeared. (Obama's parting gift?) Here's an example:http://www.imperial.ac.uk/media/imperial-college/research-centres-and-groups/theoretical-physics/msc/dissertations/2012/Alec-Owens-Dissertation.pdfI suppose you think this math is ugly?
[QUOTE="houlahound, post: 5588490, member: 551046"]Define understand and how you know you understand something.”As far as explaining physics to the public, I'd define understanding as a form of economic utility. Math is the only reliable way to grapple with the "Big Questions". If people are to expend time, money, and effort to learn mathematical physics, they expect to see how that's done. If that's not what they see promoted, they'll take the low road and learn nothing or less.
Define understand and how you know you understand something.Personally I think the word understand is vague and rather useless.
[QUOTE="houlahound, post: 5588474, member: 551046"]Yes, learn the math and see the beauty in full.”There's nothing beautiful about having to observe everything projected onto a random choice among real-valued probabilities. But that's how the actual world works, and that's the mystery the public wants to understand.
[QUOTE="houlahound, post: 5588450, member: 551046"]Didnt Hawking famously say – every equation he puts in a book takes X amount of sales off.Paraphrased.”That's valid for popular science books. For textbooks on physics the opposite is true: The more formulae the author offers, the more steps are made explicit in a derivation, for the student the more simple it is to follow and understand the argument ;-).
Yes, learn the math and see the beauty in full.
[QUOTE="houlahound, post: 5588460, member: 551046"]I believe that would be most uncompelling and ugly.”Sic transit gloria. You have a better idea?
[QUOTE="bhobba, post: 5588378, member: 366323"]Then get copy of Landau – Mechanics where all of Classical mechanics is derived from this alone”For something worthy of this honor, it would need to be fully consistent with the applicable regimes of Newton, Hamilton, Lagrange, Schrodinger, Heisenberg, Einstein (at least as far as SR), and also all the various types of material physics. And on top of all that, it also has to be profinite, so it can be put to practical use.Are you saying that the path integral is up to that challenge?
I believe that would be most uncompelling and ugly.
Supposedly, it's absolutely impossible to put the equations of modern physics into plain English. I don't buy that. Has any group of physicists ever even attempted to "Gutenbergize" QM for the public?
Imo
[QUOTE="houlahound, post: 5588079, member: 551046"]The mystical stuff excites the general public and sells nontechnical books.”And conversely? That humble, honest, up-to-date explanations of what physicists have actually found out don't sell to the general public? More likely, those who could reach out this way aren't even trying!
[QUOTE="Dr. Courtney, post: 5588067, member: 117790"]Ahh, but figuring out how to arrive at classical physics from QM may well require pushing the boundaries or tweaking how the QM toolkit is used. Our understanding of QM may need to change to address all the issues in the "classical limit."”Not really.Derive the PLA which is fairly simple as I have posted a number of times. You start out with <x'|x> then you insert a ton of ∫|xi><xi|dxi = 1 in the middle to get ∫…∫<x|x1><x1|……|xn><xn|x> dx1…..dxn. Now <xi|xi+1> = ci e^iSi so rearranging you get∫…..∫c1….cn e^ i∑Si.Focus in on ∑Si. Define Li = Si/Δti, Δti is the time between the xi along the jagged path they trace out. ∑ Si = ∑Li Δti. As Δti goes to zero the reasonable physical assumption is made that Li is well behaved and goes over to a continuum so you get ∫L dt.Now Si depends on xi and Δxi. But for a path Δxi depends on the velocity vi = Δxi/Δti so its very reasonable to assume when it goes to the continuum L is a function of x and the velocity v.In this way you see the origin of the Lagrangian. And by considering close paths we see most cancel and you are only left with the paths of stationary action.Then get copy of Landau – Mechanics where all of Classical mechanics is derived from this alone – including the existence of mass and that its positive. Strange but true. Actually some other assumptions are also made, but its an interesting exercise first seeing what they are, and secondly their physical significance.You want to understand classical mechanics – this is how to do it. Honours students with calculus BC behind them and a course in multivariable calculus in parallel could do it. In fact it would probably be beest teaching an integrated course with chemistry along the lines of:Multivarible CalculusLinear AlgebraQuantum MechanicsClassical MechanicsChemistry taught as probability models like at Princetons integrated science.ThanksBill
I don't think your linguistic comment is useful nowadays. Adding polysemia (or simply erasing an old meaning to some words and replacing it with a new meaning) is not a scientific process anymore. New theories come with new words.
[QUOTE="Dr. Courtney, post: 5588067, member: 117790"] … “Dr Courtney – Thanks for the link in your signature to FFT source code.
We still use the term "planet" to reference those objects in the sky called planets, even though the word "planet", historically speaking, means "wandering god". But rather than change the word we've just changed the meaning of the word. And it was the invention of the telescope that altered the meaning. After looking through a telescope, at the god Saturn, we discovered Saturn was ball shaped and had a ring around his belly. A similar change in meaning has occurred in the use of words within quantum physics. An early model of the atom had the electrons orbiting the nucleus in the same the way the moon orbits the Earth. But it soon became apparent, from observations, that this model was inadequate. A better model was elaborated. One can still find use of the word "orbit" in relation to electrons but it has a different meaning.So we can see from this anecdotes that it's not a question of working out how to reconstruct wandering gods from planets, or moons orbiting the Earth from electron interference patterns. It's how to do the reverse.
The mystical stuff excites the general public and sells nontechnical books.
[QUOTE="Collin237, post: 5588003, member: 577701"]I think that's the heart of the problem. There's no such thing as physics "as of now" versus future physics. The laws of physics have existed long before us and will continue to exist long after. They don't change; only our understanding of them changes.There must be a continuity between quantum and classical physics, because classical matter is made of quantum stuff. That we haven't discovered it is no excuse for decreeing it doesn't exist. The idea of "Von Neumann's split" is untestable, unwarranted, and hands QM to mystics on a platter.QM is the produce of science, not faith; and it must be taught as such. A successful student of QM has the right and duty to question the answers just as much as they answer the questions, provided they stay within the QM toolkit and use it correctly.”Ahh, but figuring out how to arrive at classical physics from QM may well require pushing the boundaries or tweaking how the QM toolkit is used.Our understanding of QM may need to change to address all the issues in the "classical limit."
[QUOTE="Collin237, post: 5588003, member: 577701"]I think that's the heart of the problem. There's no such thing as physics "as of now" versus future physics. The laws of physics have existed long before us and will continue to exist long after. They don't change; only our understanding of them changes.There must be a continuity between quantum and classical physics, because classical matter is made of quantum stuff. That we haven't discovered it is no excuse for decreeing it doesn't exist. The idea of "Von Neumann's split" is untestable, unwarranted, and hands QM to mystics on a platter.QM is the produce of science, not faith; and it must be taught as such. A successful student of QM has the right and duty to question the answers just as much as they answer the questions, provided they stay within the QM toolkit and use it correctly.”You quoted it out of context. Here's the entire passage that are relevant to that statement:[quote]We use the identical words such as particle, wave, spin, energy, position, momentum, etc… but in QM, they attain a very different nature. You can’t explain these using existing classical concepts. The line between these two is not continuous, at least, not as of now. How does one use classical idea of a “spin” to explain a spin 1/2 particle in which one only regains the identical symmetry only upon two complete revolutions? We simply have to accept that we use the same word, but to ONLY mean that it produces a magnetic moment. It has nothing to do with anything that’s spinning classically. We can’t build the understanding of the QM spin using existing classical spin that we have already understood.[/quote]Notice that I did not say that QM cannot merge to explain classical observations. That wasn't what is "discontinuous". Rather, it is the USAGE of the terms such as "spin, position, etc. You cannot simply bring over the classical concepts of such things and expect them to be the same in QM. I used the concept of of "spin" as an example. We use the same word, but that word has a different meaning in QM.This is NOT the "Von Neumann split".Zz.
“The line between these two is not continuous, at least, not as of now.Reference https://www.physicsforums.com/insights/quantum-mechanics-difficult/“I think that's the heart of the problem. There's no such thing as physics "as of now" versus future physics. The laws of physics have existed long before us and will continue to exist long after. They don't change; only our understanding of them changes.There must be a continuity between quantum and classical physics, because classical matter is made of quantum stuff. That we haven't discovered it is no excuse for decreeing it doesn't exist. The idea of "Von Neumann's split" is untestable, unwarranted, and hands QM to mystics on a platter.QM is the produce of science, not faith; and it must be taught as such. A successful student of QM has the right and duty to question the answers just as much as they answer the questions, provided they stay within the QM toolkit and use it correctly.
[QUOTE="vanhees71, post: 5563803, member: 260864"]In highschool we had a long unit about "Bohr-Sommerfeld atoms". After that the teacher told us that we have to unlearn it and introduced the wave-mechanical picture a la Schrödinger (with the correct probability distribution). That was hard to digest. I never understood, why she had not introduced the wave-mechanical picture from the very beginning.”If nothing else, it teaches future scientists that theoretical physics is about constructing models that in the future can be superseded by better models. Some physicists attach too strongly to theories that were state of the art when they were young. The pedagogy of learning theories soon to be rejected and replaced by better theories may prevent this.
In highschool we had a long unit about "Bohr-Sommerfeld atoms". After that the teacher told us that we have to unlearn it and introduced the wave-mechanical picture a la Schrödinger (with the correct probability distribution). That was hard to digest. I never understood, why she had not introduced the wave-mechanical picture from the very beginning. That would have spared us from the hard "unlearning process". In university we had an "Introduction to quantum mechanics" before the theory-course lecture (QM1). There also they started with a historical introduction, and again the Bohr-Sommerfeld model was taught in some detail (this time of course using what we had learnt in the classical mechanics lecture, i.e., Hamilton and phasespace formulation). At the end of the lecture we had of course also covered Schrödinger wave mechanics. QM 1 was then the "true" representation free formalism a la Dirac, which was a revelation for me. All of a sudden one could understand the structure of the theory, the meaning of states, operators representing observables, their eigenvalues and eigenvectors. I think, it's better to go in a way as you described, i.e., a very brief discussion of only a very few lectures with history of the "old quantum theory", but not in much details but rather a description of the "development of ideas", which finally lead to modern quantum theory, which was invented independently by 3 groups (roughly): (a) Heisenberg (idea in his famous but imho quite enigmatic Helgoland paper), Born, Jordan (formalism), Pauli (applications, including the solution of the non-relativistic hydrogen atom) => "matrix mechanics"; (b) Schrödinger ("wave mechanics" including a lot of the standard methods and applications we still learn today in the more wave-mechanics oriented lectures/books; equivalence between wave and matrix mechanics); (c) Dirac ("transformation theory"; basically the representation free formalism we learn today still almost as Dirac formulated it as "bra-ket formalism"). Then there was of course the more mathematical branch starting from von Neumann, leading to the development of modern functional analysis. Already the book by Courant and Hilbert played an important role for Schrödinger in developing wave mechanics.
In my college days, some while back that is, we first had a „physics of atoms and molecules” course which indeed put forth in the first 2 lectures the history behind QM but without going into too many details. The only detailed historical models were: the blackbody radiation and Rutherford's scattering theory. The 3rd lecture had the Schrödinger equation already underway and that is it. Then, the next year, the proper QM course had as 1st lecture: „Prehilbert and Hilbert Space. Orthonormal systems”. No pre 1925 content whatsoever. I was happy with this. Still am.
I don't know, who first did the derivation I've posted in my insights article. Nowadays it's a standard exercise for 1st-order time-dependent perturbation theory.https://www.physicsforums.com/insights/sins-physics-didactics/
[QUOTE="vanhees71, post: 5563680, member: 260864"]Well, to answer your final demand, what's the significant physical difference between classical and quantum physics, I'd say it's the possibility of "entanglement".”But even entanglement can be viewed as an application of superposition (or maybe path-indistinguishability) to 2 objects (or a tensor product Hilbert space). So is it 'entanglement' that's fundamental or superposition?For me the essential difference boils down to the fact that in a classical phase space one can, in principle, distinguish between two arbitrarily close states (q,p) and (q',p') but in QM the very concept of distinguishability is tied into the notion of orthogonal states so that 2 non-orthogonal states can, with a certain probability, 'mimic' the other in a given experiment.[QUOTE="vanhees71, post: 5563680, member: 260864"]among which are Einsteins 1905 approach to the photoelectric effect, which in fact does not prove the necessity to quantize the electromagnetic field”Indeed – wasn't it Lamb (and Scully?) who showed this way back when, something like half a century ago?But I think the 'photoelectric' part of that paper is, for me, perhaps the least wonderful bit :-)
Well, to answer your final demand, what's the significant physical difference between classical and quantum physics, I'd say it's the possibility of "entanglement". Einstein called it the inseparabality of far-distant parts of a quantum system, and that's a very precise description. The possibility of the corresponding correlations between parts of a quantum system is not understandable from the point of view of a deterministic local theory, as has been shown by Bell. I agree, however, with your final statement that to start with this is not the best way to introduce QT. For that you need a good understanding of the formalism. I also tend to a mixture of the historic approach but telling from the very beginning that one should not get too much involved in old and partially inaccurate accounts of the history, among which are Einsteins 1905 approach to the photoelectric effect, which in fact does not prove the necessity to quantize the electromagnetic field (see my Insight article on this "sin" of physics didactics), the Bohr-Sommerfeld model of atoms, and inconsistent pictures like the wave-particle duality, which are all clarified and avoided by modern QT.
I have no idea what the 'best' way to teach QM is – and I oscillate between the historical approach and the formal. In one sense, since the maths can be quite easy (if you teach the basic principles and axioms using 2 state systems), it just becomes a matter of (initially) setting down the recipe and cooking and seeing what it tastes like at the end. After working out several meals in this fashion, so we become comfortable with just applying the recipe, one can then go back and try to figure out what the recipe actually means.QM, in its traditional 'textbook' form is such a massive disconnect from all the physics we learn up to that point – that this might indeed be the best way to do it, for most.But I also think there's merit in the historical approach too. Presenting the bare axioms, drawn from the magicians hat like some hapless white rabbit, and just working out the consequences has the potential to leave the student with the letters WTF indelibly tattooed on their forehead (my tattoo is fading a bit now).I wanted to know 'why'. Why did those amazing physicists like Planck, Einstein, Bohr, Heisenberg, Schrödinger, Dirac (and so on and so on) feel it necessary to change things so radically? Why were they forced to adopt this (on the surface) really bizarre set of axioms that looked like nothing that had gone before? Whatever possessed them?We get some insight into this by looking at the original papers. Einstein's 1905 paper on the photoelectric effect is, for me, a tour de force of theoretical physics. It's quite simply breath taking. A few years later (in 1909 I think, but not sure) he published a paper showing that the fluctuations in the black body spectrum are partly to due 'wave' fluctuations and partly due to 'particle' fluctuations – astounding stuff indeed – hitting the fundamental kookiness that was looming squarely on the head.Do we get a better insight into QM from this? I don't know – but you certainly get a better sense of why QM was needed at all – and why the classical approach that had been so successful just fell apart. I think it's important to 'understand' QM by looking at how it differs from classical thinking. We can make the difference seem 'small' by investigating the formalism (let's call QM a C* algebra but with non-commuting variables, to give a somewhat oversimplified example) – but ultimately there has to be a significant physical difference – and it's understanding that difference and why it's there is (probably) crucial for really grokking things.But is this the best way to present QM in the first place? Don't know – I suspect not.