What are generalized probability functions?

[TimH]

Okay I posted a question a few days ago about Luders Rule but didn't get any responses. I've studied this stuff in Hughes (The Structure and Interpretation of Quantum Mechanics) a bit more so I can ask a slightly different question. Hughes says you can create a "generalized probability function" that can apply to situations where a classical probability function can't. In particular, they apply to the "orthoalgebras" that describe quantum observables (where some observables are incompatible). When these generalized probability functions are expressed using density operators many wonderful things happen. In particular composite systems (like the 2-slit experiment) are represented by a new pure state, rather than a mixture of states. I.e. the 2-slit experiment with both slits open is a totally different pure state from the state with only one slit open. He then says that when you use the time-dependent Schrodinger equation on this pure state, you get "diffraction of the state function" which explains the interference pattern on the screen: its due to evolution of the state function as the state evolves in moving from the slit to the screen. This seems really neat, but I don't understand what the "generalized probability function" brings to the density operator/projector/Hilbert space formalism that makes all this possible.

Can anybody explain what these wonderful generalized probability functions are that can do all this? Thanks.

 

[Peter Morgan]

RIG Hughes' book is good, but it comes from a time when there was a recurrence of great hopes for quantum logic, which I think have to some extent faded -- although there are certainly good mathematicians still working on orthoalgebras and the rest, and a wider interest could return.

You have to come to your own way to be familiar with this mathematics. For myself, I interpret both quantum logic and Hilbert space methods as ways of generating probability measures appropriate to modeling different experimental contexts of preparation and measurement apparatuses. "Generalized probability distributions" from my somewhat empiricist (properly post-empiricist) point of view is too grand a name for the mathematics, but for those who take the mathematics somewhat realistically it is not.
The only thing to do is to read other books and papers on such methods, as many as you can obtain, until you find someone whose sensibilities makes sparks for you. For myself, I've always liked Paul Busch's approach to the application of algebraic methods to quantum mechanics. His "Operational Quantum Physics", Springer, 1995 (with Grabowski and Lahti) is quite beautifully done mathematics (but probably two steps up in mathematical sophistication from Hughes, so it may not be much use to you if Hughes is already a stretch). One way to proceed would be to scan a few papers by Busch on ArXiv.org. You could do worse than look at papers on ArXiv by people who he cites approvingly in his papers. There is a strong tradition of developing algebraic approaches in Eastern Europe.
If anyone more steeped in algebraic approaches checks in here, perhaps they could also offer names of people whose work has sparked for them.

 

[TimH]

Thanks so much for your post. Hughes is my first book on QM and I didn't realize the generalized probability approach was so well developed (and as you suggest, now out of fashion). It just seems a little like a rabbit pulled out of a hat as he presents it, as conditional probabilities using Luders rule. I'm not going to belabor the subject-- I'll just move on and finish the book. Can you suggest a good current text on QM that develops the math (I'm leaning towards Griffiths just because its so popular). Thanks.

 

[f95toli]

A generalized probability distribution/function is essentially just the QM "equivalent" (meaning it is used in almost the same way) to "classical" probability functions/distributions. A good example is the Wigner distribution which is what you get if you try to written down a distribution function for momentum and position (i.e. the probability to find a particle with some momentum p and position x) in QM.
In some cases the Wigner distribution really behaves almost as a classical distribution but it can also do very "non-classical" things such as take on negative values.
There are also a number of other representations such as the P representation.

These are all widely used in quantum optics. So if you read about them you can find more information in e.g. Walls&Milburn's "Quantum Optics" or a similar book.

 

[Peter Morgan]

I'm so biased it's horrible. The single book that has had most influence on me, which I've spent about the last ten years constantly referring to, is Rudolph Haag's "Local Quantum Physics", Springer, 1996. Don't try this second, unless RIG Hughes doesn't stretch you mathematically. Also, think twice about going the LQP way, which is a minority pastime for mathematicians; it's arguably a dead-end at the same time as being arguably the best way to think about quantum field theory.

A book that I found enlightening in a slightly alternative way is Greenstein & Zajonc, "The Quantum Challenge: Modern Research on the Foundations of Quantum"; there's the "Operational Quantum Physics" of my previous post; there's Max Jammer, "The conceptual development of quantum mechanics"; ... .

The post that's just come up, suggesting that you look at quantum optics literature, is a very good idea indeed! Probably better than anything I've suggested above, although it will depend on your sensibilities. Also, the Wigner function is indeed spoken of as a generalized probability distribution, because it will sometimes be negative-valued (not very classical, that). I was thinking in terms of Hughes's book, which doesn't introduce Wigner functions. For anyone who has access to an academic library, I would recommend Leon Cohen's "Time-Frequency Distributions-A Review", Proceedings of the IEEE, Vol. 77, 1989. Must look at Wall&Milburn myself, quantum optics is not an area I'm very familiar with.

The way to understand this stuff, however, is to sign up at the ArXiv for the daily e-mailing of new papers on quant-ph, and trawl through the abstracts of all the papers that are posted there. Download any paper that has an abstract that looks "interesting" to you; read the first couple of pages, don't worry that you don't understand much -- absorption by familiarity is a wonderful thing if you're willing to go at it for a length of time -- make a note of some of the references and move on to the next paper. I do hep-th as well, even though to me it's pure torture, and I'm thinking tremulously of adopting math-ph. Doing this is a way of life, not a book.

 

[TimH]

Thanks for the great idea of using ArXiv. I'll investigate this. I'm curious about "absorption by familiarity." I've always been drawn to doing this. As a high-school student I would read what I could of the Astrophysical Journal. But then another part of me says "Be slow, methodical, get a textbook" (e.g. Griffiths). Absorption of papers is certainly more engaging/exciting. Trudging through a textbook, however, let's me feel I'm doing "foundational work" in developing my understanding. Would appreciate your take on these two competing impulses.

 

[Hurkyl]

Absorption of papers is certainly more engaging/exciting. Trudging through a textbook, however, let's me feel I'm doing "foundational work" in developing my understanding. Would appreciate your take on these two competing impulses.

Discover what you want to learn, and then learn it.

 

[Peter Morgan]

Do both.

DO BOTH! Follow your nose. When you find a paper that really speaks to you, Follow the References. Progress by absorption is not recommended for graduate students, however, because it's too slow.

It has occurred to me that Hughes was a very good choice for a read about QM, if you can handle the math. Certainly worth a brief return to it in a year or two as well. Who suggested it? Ask for more from them.

Textbooks have the significant failing of generally not covering the wild edges very well. This is where the ArXiv excels! For example, Stochastic Electrodynamics is a fringe activity that I would say should not be pursued by anyone (type Stochastic Electrodynamics into Amazon, ArXiv, Wikipedia, sadly the online "Stanford Encyclopedia of Philosophy" doesn't have anything on it, but it's a place you should refer to often for interesting reads -- read everything on QM there and learn much, follow references from there and learn lots more), but it should be looked at seriously as a strange different world, which let's you see the conventional view through different glasses. SED occurs to me because there is a paper by Luis de la Pena on the ArXiv this morning, in quant-ph, "0712.2023v1" (paste that into the search window on ArXiv.org; definitely not good for a first look at SED, but the references reflect the SED worldview; reference [19] happens to be the Physicist's basic reference for Wigner functions, not as good as the reference I gave in my previous post, in my opinion, but a must-read to understand how Physicists understand Wigner functions).
With enough different views -- add in de Broglie-Bohm, for example (which textbooks ignored totally for years, although it now has a presence in some) which is definitely not recommended as a pursuit, by me at least, but I recommend it whole-heartedly as a different view -- you can come to some sort of synthesis, the relations between the different views can gel, and it all can look not so weird. Hopefully someone eventually will write a clear exposition of QM/QFT, so we don't have to do it this way, but there's nothing anywhere near that good yet.
Beware, however, for the details matter.

No-one can tell you which wild idea will grab you hard and swing you into uncharted territories from which you may never come back. I'm pretty much certain, myself, that SED taken too seriously would be a bad idea, for example. Even worse might be my own approach to quantum theory, through my home-page. If you do come back, I hope your travel stories excite your grandchildren.


[Sorry for the convoluted bracket structure in the long paragraph!]

 

[TimH]

Great post! Thanks so much. As to Hughes, I used Amazon and the book stood out as exceptional according to online reviewers. I would like, ideally, to only read exceptional books (life's too short for less...:) ). I've just been to ArXiv and copied a few more basic articles on entanglement, which, with my philosophy background, really grabs me as a subject. Did you say you can get an email every day? I'll check that out. I really appreciate all the advice. I'm a 46 year-old Harvard educated stay-at-home dad (done so I have time to self-study). While I have two graduate degrees in the humanities, charting the best path to learning science is a challenge, on top of the science itself.

 

[Peter Morgan]

Great post! Thanks so much. As to Hughes, I used Amazon and the book stood out as exceptional according to online reviewers. I would like, ideally, to only read exceptional books (life's too short for less...:) ). I've just been to ArXiv and copied a few more basic articles on entanglement, which, with my philosophy background, really grabs me as a subject. Did you say you can get an email every day? I'll check that out. I really appreciate all the advice. I'm 46 and, while I have two graduate degrees in the humanities, charting the best path to learning science is a challenge, on top of the science itself.

There's a help page for daily e-mails, http://arxiv.org/help/subscribe".

Good books better than bad books!

On Philosophy, I think there's a fine line between doing too much and doing too little metaphysics. IMO, People sometimes end up a long way from that line. Entanglement has people all over the place on the metaphysics. Personally, I think entanglement is a sideshow, but then most others think that quantum field theory, which I think is the main event for achieving an understanding of quantum theory, is a sideshow for that purpose. Time to disagree gracefully, perhaps?

 

[TimH]

Well, since I know nothing about QFT, no disagreement...! I've just been learning about entanglement for a few weeks. What I can't figure out is how much of it is a real, mind-bending fact, and how much of it is bad philosophical agitation. Specifically, do you have to have a very narrow idea of quantum properties to end up with nonlocality? I.e. if I don't think of particles as "having" properties "when I don't measure them" do the correlations still force me to conclude that one measurement influences the other? Hughes keeps arguing that, as he says "properties are the unicorns of quantum physics," and that "none of us are honest enough to catch one."

I'm not really interested in philosophy anyway (as my intellectual mentor Wittgenstein said, "Philosophical problems arise when language goes on holiday"). Actually I'm really interested in the concepts of space and light in physics and whether they need to be modified. It bugs me that physics seems to take place in what I call "the Big Room." Does QFT discuss/explain space and why it exists, or is it a given?

PS. Will your "Bell inequalities and incompatible measurements" help me on any of this?

 

[Peter Morgan]

We're getting off the topic, right? Is there a conversation if no-one else hears?

I didn't know that Hughes says "properties are the unicorns of quantum physics," and that "none of us are honest enough to catch one." These may be useful for me! I think he's absolutely right. I would put it more strongly, there are no particles in a simple sense, so there are no properties. Suppose we talk about a hill -- where does the hill end and the next hill start? What part of the mantle under the hill is part of the hill? We can say that the hill weighs 10^10 tonnes in a rough pragmatic way, if we tell everyone approximately where the boundaries of the hill are, but a precise, to 100 decimal places or to infinite real number accuracy, not so much. So there can be hills in geography, but their detailed properties depend on where we draw the lines. The analogy of hills with not-well-defined-particles in quantum theory is of course fragile.

Almost all quantum mechanics and the hidden variable literature talk about particles and their properties. Consequence big mess. Almost all the most successful Physics, since Faraday and including quantum fields, is about fields. So where are classical field models in the literature? Almost nowhere. Can interesting classical field models be constructed that can compete with quantum mechanics and quantum field theory? Maaaayybee?

I'm intending to post a new topic introducing my approach either on "Quantum Physics" or on "Beyond the Standard Model", not quite sure which, nor whether it will be tomorrow. I'd be glad of one response from you, if you can rise to it, since I suspect it will otherwise be ignored as too esoteric (which it may be).

 

[Peter Morgan]

Sorry, I missed your big question in my enthusiasm to tell you about my big question.

Quantum Field Theory takes Minkowski space-time as a given. QFT in CST, the big boys version, takes a curved space-time as a given. Quantum Gravity is for those who want to apply the mathematical formalism of quantum theory to space-time geometry before they understand quantum theory; there are some variations in which the given is not a 4-dimensional space-time. The more unlike 4-dimensional space-time whatever is given is, the more trouble there is making 4-dimensional appear naturally from it.

Since Mathematics is about given this, then this other, if we introduce something new as a simpler given to "explain" space-time, our children will have to "explain" what we introduced. Still useful to ask what new given we might use, but to me it's more a technical matter of creativity than a big deal. Can we disagree now? The biggest technical arguments arise when we have to go conceptually sideways, for example when it turned out that Phlogiston was not, after all, such a great idea (noting, however, that most of the Phlogiston people were intellectually not at fault, and it seemed more or less better than other available theories for maybe half a century).

Also, about the 'PS. Will your "Bell inequalities and incompatible measurements" help me on any of this?' Looking at it for the first time in a couple of years, section 2 is more-or-less what is in the HTML page on Bell inequalities, section 3 is very hand-waving, certainly not journal publishable. So it's a bad book, I'd say. I think the Vaxjo Conference proceedings is probably better as approximately what I think nowadays at a mathematically more-or-less elementary level.

I'm verrry interested, by the way, in anything you might have to say about my HTML Bell inequalities page, http://pantheon.yale.edu/~pwm22/Bell-inequalities.html".
It's somewhat heterodox, though, so as always it's definitely only a small piece of jigsaw.

 

[TimH]

Peter, its really interesting talking to a "real live theoretical physicist" (woo-hoo!). The Hughes talk of unicorns is on page 217. I will of course look at your Bell inequalities page, though I'm a beginner so who knows: I was just making a Venn diagram of the inequality to try to see what its saying.

You should definitely post your ideas. This would be an explanation of a continuous random field?

 

[Peter Morgan]

Not live!

No, not live. I'm a Research affiliate in the Yale Physics department mostly because my wife is a Professor in Classics, although I do have a very little track record to my credit -- four papers doesn't get you tenure anywhere, certainly not at Yale. I don't teach at Yale.

An explanation of a continuous random field is in "Bell inequalities for random fields", in Appendix B, but looking at it now I see that it's pretty technical.

A random field is almost trivial, it's an indexed set of random variables (that's to say, if you know about random variables, a random field is easy, Wikipedia perhaps?). Two random variables is enough. A continuous random field effectively makes the index set be the points of space-time, so that we could talk about random variables $$\phi(x)$$ at every point of a (Minkowski) space-time, very similarly to a quantum field, but there are technical (and also notational) matters that make it mathematically much better to make the index set be a linear space of functions, which are called "test functions". It is possible to talk of a continuous random field as a "random variable valued distribution", which is fairly directly comparable to the "operator valued distribution" that a quantum field is sometimes called. That continuous random fields are conceptually quite close to quantum fields is a large part of why there is a small hope of casting some light on quantum mechanics. With honorable exceptions, almost all attempts to understand quantum mechanics have been through the nonrelativistic theory, quantum field theory has been felt to be too complicated for any understanding of it to be possible until quantum mechanics has already been understood. Big mistake, I believe.

Incidentally, Yale just e-mailed that a few of their courses have just been made freely available, at http://open.yale.edu/courses/". Shankar's textbook on QM is much used, I believe.

 

[TimH]

Thank you for the link to Shankar. Now when I'm not reading Hughes during the day I can watch Shankar at night! After I finish Hughes I've been debating working my way through Halliday and Resnick to lay a foundation before moving on to a QM textbook. It seems like it would take awhile to do this, but it might be a good discipline to get experience doing physics problems. I just finished working my way through a Vector Analysis textbook so I could instead review EM next (which I studied at BU awhile back). Do you think it would pay to go slow and do a broad overview of basic physics before EM/QM? I have to admit survey-type courses never got me very excited.

I would like to be able to say something intelligent about a continuous random field, but I'm still looking at the wikipedia page for "random variable"... give me maybe ten years...