Quantum mechanics and randomness

[SeventhSigma]

Hello, all --

I feel as if I've only really been exposed to classical physics thus far, and I want to educate myself a bit further.

What really puzzles me the most is the concept of randomness, which I've always attributed to an inability to measure all variables involved with the outcome. The more information we know about a system's properties/variables at a given point in time, the more we know about that system at a given point in a future based on various laws of physics. I'm familiar with the Heisenberg Uncertainty Principle but I feel like it's an "older" view. My current understanding is that trying to find the position of something is like searching for a blimp in a dark room -- by the time you find it, you've already touched/moved it. But I suspect the Principle is something far more fundamental than "interfering" via observation.

I do not understand how QM is thought to somehow be "truly random" at some level or if there are simply other variables we haven't found yet. I know that the "hidden variables" argument is rendered very unlikely by other rules, but I have never really seen the full argument. Simply speaking, I want to know what is *known* in QM and what is still debatable/up to speculation.

Apologies if my questions make no sense. Simply put, I'm here to learn and would love some guidance in the right direction. I want to erase any misconceptions I have and replace them with more up-to-date ones.

 

[arkajad]

"I want to know what is *known* in QM and what is still debatable/up to speculation."

Well, if you will read the papers in arxiv.org, then you will see that there is no one idea that is not debatable. What is not debatable is the necessity of getting a good agreement with the experiment, at least as good as the standard textbook QM allows us to get.

But even the standard textbook quantum mechanics is being rendered with different philosophies - depending on the particular author.

 

[SeventhSigma]

So what of the "hidden variables" theory?

My initial response to the notion that QM is fundamentally random was that there were possibly other variables influencing the outcomes that we are somehow not noticing/able to measure. What is the general response to this?

 

[arkajad]

As an intro I suggest reading http://arxiv.org/abs/1001.2758" . You may find it controversial, nevertheless it represents an active research area.

 

[ThomasT]

SeventhSigma, welcome -- type 'randomness' into a thread search and you'll get lots (I got 248 different threads) of discussions in lots of different forums at PF.

So what of the "hidden variables" theory?

My initial response to the notion that QM is fundamentally random was that there were possibly other variables influencing the outcomes that we are somehow not noticing/able to measure. What is the general response to this?

That's everybody's initial response. But it's a question that's complicated by semantics and logic. Again, a thread search on 'hidden variable theories' or 'local realistic theories' or something similar, should result in more than enough discussions to satisfy your curiosity. You can also consult the Stanford Encyclopedia of Philosophy, Wikipedia, etc. I don't think anybody in the qm forum wants to get into a protracted discussion of this stuff anytime in the foreseeable future.

 

[SeventhSigma]

I've done a lot of reading but I am still unconvinced. I feel like the general consensus is "We can't explain the randomness yet -- but local variables are essentially impossible, and you'd have to bend over backwards to explain the randomness with non-local variables. It's possible, but currently without any evidence."

What confuses me is how people seem to say that the randomness is fundamentally uncaused/random (an ontologically positive statement). Is it more accurate to say that "So far, QM appears random"?

 

[DrChinese]

I've done a lot of reading but I am still unconvinced. I feel like the general consensus is "We can't explain the randomness yet -- but local variables are essentially impossible, and you'd have to bend over backwards to explain the randomness with non-local variables. It's possible, but currently without any evidence."

What confuses me is how people seem to say that the randomness is fundamentally uncaused/random (an ontologically positive statement). Is it more accurate to say that "So far, QM appears random"?

OK: So far, nature appears random.

We don't know the underlying mechanisms at work. Maybe there is a causal explanation. But if there is, it does not lie in the past light cone: it must exist in the future, the present (non-local) or in other worlds.

 

[SeventhSigma]

Thanks for the reply!

How might an "underlying" mechanism be fundamentally non-local? Maybe my definition of "local" is off. I assume "local" means that if we have a closed system, we can say every variable involved is "local" to that system, whereas "non-local" is really just another way of saying "it's a variable local to your experiment but it's interfering in a way from afar that makes one think it is non-local, but it's outside the system," where a non-local variable that has influence is technically "local."

 

[arkajad]

"But if there is, it does not lie in the past light cone: it must exist in the future, the present (non-local) or in other worlds. "

Plus complexity.

 

[apeiron]

I do not understand how QM is thought to somehow be "truly random" at some level or if there are simply other variables we haven't found yet. I know that the "hidden variables" argument is rendered very unlikely by other rules, but I have never really seen the full argument.

You are asking an interesting question. And there is always almost constantly a thread going on this, so you could just browse the archives.

Anyway, you are taking a classical view of probability modelling. One that has some key hidden assumptions.

The basic idea is that you have a bag of numbered balls and you chose blindly. So an ensemble of microstates exists - they are a crisp or concrete reality - but the chosing process, the observation or act of measurement, is random.

Now already, you can see here that the "randomness" is not really fundamental to the balls, the microstates, but is saying something fundamental about the selection process. A blind choice is an undetermined one, as opposed to a "determinstic" choice where we peek, have a rummage, and pick out a ball with some particular number.

But QM goes further. It shows via evidence for nonlocality that something more than this is going on. Because the balls in the bag are mysteriously entangled in ways that chosing one somehow does something to the state of the others.

If you are indeed chosing balls blindly, and in your mind randomly, then you won't even notice this. It is only if you decided you want to sneak a peek, chose one ball deterministically and "know" that all the other balls remain in the bag, unchanged, that you would run into problems.

There isn't a general agreement about how to model this kind of "randomness", but I like approaches that are based on true indeterminancy - where the initial state of things is vague (a state of simple unformed potential) rather than crisp (definite, actual, already in existence).

So now, we may have a bag. But there is nothing yet inside it apart from a general propensity to generate "numbered ballishness". And reaching inside becomes an act of global constraint that produces some actual ball from this state of indeterminate potential - simultaneously ruling out the many other outcomes that might have been the case.

Of course, this is a qualitative description, not a quantitative one. Probability models based on ensembles of crisp microstates have a highly developed mathematical description that allows us to actually calculate stuff. This other approach based on vagueness, on true indeterminancy, is more philosophical. Though there is work being done that could make it properly mathematical one day.

But anyway, the first question to ask here is "where is the randomness"? In standard probability modelling, the microstates just are. They exist crisply. Their state has already been "determined". (Someone made the balls, numbered them so they could be told apart, and stuffed them in a bag). So at the local level, we are not modelling any random action.

The randomness lies instead at the global level, in the hand of the chooser. It describes the process - was it a blind or a deliberate choice? And if a blind choice, then certain probablistic patterns will emerge with high certainty.

QM must be more than this because the balls no longer behave themselves. They seem to change state for reasons we do not appear to control. Their individual, local, state is not crisply determinate but now in some respects undetermined.

So some kind of larger model will now be needed to see all the action, all the causality.

 

[SeventhSigma]

I'm not following. Even if we reach into the bag, there is still a choice involved. How do we know the choosing process is truly random? How do I know the balls are random versus the choosing process being random? How do I know that the states of the balls change only when I look at them versus not looking at them?

If I peek at quantum balls and then choose, is the result the same as peeking/choosing macro-level balls? How is it different when I don't peek? Where is the difference in the type of randomness?

 

[arkajad]

Quantum randomness may be not so random after all. Do we know for sure? How sure?

 

[ThomasT]

I'm not following. Even if we reach into the bag, there is still a choice involved. How do we know the choosing process is truly random? How do I know the balls are random versus the choosing process being random? How do I know that the states of the balls change only when I look at them versus not looking at them?

If I peek at quantum balls and then choose, is the result the same as peeking/choosing macro-level balls? How is it different when I don't peek? Where is the difference in the type of randomness?

SeventhSigma, I'm supposing that you're having difficulty seeing the difference between quantum randomness and classical randomness. Ok, there's no essential difference, even while there is some meaning and truth wrt statements that quantum phenomena are truly random.

The attribution of essential randomness to certain experimental phenomena simply results from their being quantum experimental phenomena -- because this is the deepest physical theory. Unfortunately, the deepest physical theory isn't really a deep physical theory. It's a theory of quantum experimental phenomena. That is, it's a theory that has to do with material and instrumental preparations, and the behavior of detectors. Since the behavior of detectors is random, that is it can't be precisely predicted, and since qm, the theory of these detectors, is the deepest theory there is, then this behavior is deemed truly random. But truly random wrt qm and qm phenomena doesn't mean anything other/different than random in the ordinary sense. That is, it doesn't mean anything other than unpredictable. Just as, say, superdeterminism doesn't mean anything other/different than determinism.

So, the bottom line is that quantum experimental phenomena are random in the same sense that a fair deal of cards or a roll of fair dice are random. They're unpredictable. That's all that can be said about it, because that's all that's known about it.

To reiterate, quantum experimental phenomena are called truly random simply because qm quantitatively accounts for things that classical mechanics doesn't and is therefore considered a more fundamental theory. But, as has been mentioned, qm isn't a theory of fundamental reality. It's a very precise probabilistic accounting of instrumental behavior. No more, no less. The correspondence that the various mathematical models incorporated within the general qm algorithm have with an underlying reality is unknown. The correspondence that the Schroedinger equation has with an underlying reality is unknown. The bottom line is that the qm formalism cannot, generally, precisely predict quantum experimental phenomena. But it's the deepest theory. So we call these phenomena truly random to differentiate them from the results of, say, dice rolls. But, really, there's no difference regarding their randomness. Random means unpredictable. Random is random. Randomness is evidence of our ignorance, assuming that reality is evolving lawfully -- and if we want to assume that it isn't evolving lawfully, then what need, what use, is there for fundamental science?

I should add that I will defer to whatever apeiron and arkajad have to say on this, as they are much more qualified to comment -- even though I think that what I said was essentially correct (but I'm certainly open to corrections, etc.).

 

[arkajad]

Random means unpredictable. Random is random.

But what is unpredictable today can be predictable tomorrow. So, perhaps, random is not necessarily random?

 

[wuliheron]

But what is unpredictable today can be predictable tomorrow. So, perhaps, random is not necessarily random?

Perhaps, but since random and orderly define each other it is pretty meaningless to say everything is random or orderly. I hear the same thing about energy all the time. People say everything is made of energy, but mass and energy define each other so it is a pretty meaningless and useless statement. The only way to make such a statement meaningful and useful is to place it in a specific context and life, the universe, and everything is not a specific context.

 

[ThomasT]

But what is unpredictable today can be predictable tomorrow. So, perhaps, random is not necessarily random?

Random means unpredictable. If you can predict an event today that you couldn't yesterday, then today that event isn't random, while yesterday it was.

I don't know what you want the word, random, to mean. Please tell me.

Would you agree that the attribution of "truly random" to quantum phenomena has to do with those phenomena not being quantitatively accounted for via classical mechanics? But quantum phenomena are accounted for, approximately/probabilistically via qm. The reason we call qm phenomena "truly random" is because qm is fundamental to (ie., it approximately quantitatively accounts for certain experimental phenomena more accurately than) classical mechanics. This is the only reason. So, I submit to you that the term "truly random" has no essentially different referents than the term "random" wrt dice rolls or card draws or any other classically "random" stuff. And that popularisations that present quantum phenomena as being "truly random", as differentiated from just "random", are basically bullsh*t.

Then again, reconsidering your statement, I think I have to agree with you, more or less.

 

[arkajad]

@ThomasT
What I know is that quantum randomness can be modeled by pseudo-random number generators used by our software. The only difference is the speed of generation. So I do not see any reason whatsoever for calling quantum randomness "truly random". Moreover nowadays we know that what we thought not so long ago to be beyond any control (like radioactive decay rates), can be, possibly, influenced by unknown factors. So, I do not exclude the possibility that some important discovery about what we call "quantum randomness" is in the air.

 

[wuliheron]

@ThomasT
What I know is that quantum randomness can be modeled by pseudo-random number generators used by our software. The only difference is the speed of generation. So I do not see any reason whatsoever for calling quantum randomness "truly random". Moreover nowadays we know that what we thought not so long ago to be beyond any control (like radioactive decay rates), can be, possibly, influenced by unknown factors. So, I do not exclude the possibility that some important discovery about what we call "quantum randomness" is in the air.

The difference is that you can predict one and not the other, ergo, we call one truly random and the other pseudo-random. It doesn't have to mean anything metaphysical, merely that at this time no one can predict the outcome. If you can't predict it, then for all practical purposes it is truly random. This is precisely the kind of specific context I was talking about that is necessary for the words random and orderly to have any demonstrable meaning.

 

[arkajad]

The difference is that you can predict one and not the other.

You can't predict the outcome of a complex deterministic machine when it it goes through a certain number of iterations. There is absolutely nothing particular about quantum randomness that would distinguish it, by objective tests, form sufficiently complex deterministic schemes. What is amazing in quantum theory is its apparent nonlocality of computation, not its "randomness".

 

[ThomasT]

@ThomasT
What I know is that quantum randomness can be modeled by pseudo-random number generators used by our software. The only difference is the speed of generation. So I do not see any reason whatsoever for calling quantum randomness "truly random". Moreover nowadays we know that what we thought not so long ago to be beyond any control (like radioactive decay rates), can be, possibly, influenced by unknown factors. So, I do not exclude the possibility that some important discovery about what we call "quantum randomness" is in the air.

Thanks arkajad. I think I can learn a lot from you. I hope you keep posting here as your time permits.

 

[ThomasT]

The difference is that you can predict one and not the other, ergo, we call one truly random and the other pseudo-random. It doesn't have to mean anything metaphysical, merely that at this time no one can predict the outcome. If you can't predict it, then for all practical purposes it is truly random. This is precisely the kind of specific context I was talking about that is necessary for the words random and orderly to have any demonstrable meaning.

Thanks for your contributions wuliheron. I think I can learn from you also.

 

[ThomasT]

What is amazing in quantum theory is its apparent nonlocality of computation, not its "randomness".

Ok, I don't think that this is so amazing. It's based on empirically established optical principles for the most part. The problem is that a local mechanistic model (using the same qualitative conceptualizations that the qm stuff is based on) doesn't seem to be possible.

 

[arkajad]

The problem is that a local mechanistic model (using the same qualitative conceptualizations that the qm stuff is based on) doesn't seem to be possible.

I would say that the problem is that a non-local mechanistic model is possible. So, I am asking, why are we so attached to local models? Is it just our inertia of thinking and nothing more? If non-local works - then why don't we use it without any hesitation whatsoever? Why don't we go full speed an efficient non-local physics?

 

[ThomasT]

I would say that the problem is that a non-local mechanistic model is possible. So, I am asking, why are we so attached to local models? Is it just our inertia of thinking and nothing more? If non-local works - then why don't we use it without any hesitation whatsoever? Why don't we go full speed an efficient non-local physics?

A nonlocal model is possible, and extant (dBB). But it isn't a mechanistic model. It's as nonrealistic as standard qm is.

Why are we attached to local models? Because the physical evidence suggests that our universe is evolving in accordance with the principle of local causality.

 

[SeventhSigma]

The problem is that I seem to hear that "quantum" randomness has nothing to do with our ignorance of the variables involved (hence Bell's Theorem and the impossibility of local hidden variables), but this may be incorrect on some level, perhaps?

At any rate, I feel like on a classical scale, even if we have a really complex deterministic process, we can always take those processes and their properties/attributes and figure out some probability/expected value from them and be able to predict the outcomes with some degree of certainty (even if it's very low). But if we are able to accurately measure those processes and simulate them, we may even be able to predict the outcomes of a complex machine with even higher certainty.

I suspect that quantum mechanics differs from this because we don't have any other variables to measure when it comes to explaining quantum randomness. I also suspect there is a difference in the speed of convergence to the expected value of a distribution on average?

 

[arkajad]

@Thomas
I do not necessarily agree that physical evidence suggests anything by itself. It suggests what we want it to suggest. The same evidence can suggest different things to different people - depending on their knowledge and understanding.
I agree that so far dBB is not yet realistic, but do we care about a given model being realistic or not? I think what we should really care about is the predictive and explanatory power.

@SeventhSigma
"But if we are able to accurately measure those processes and simulate them, we may even be able to predict the outcomes of a complex machine with even higher certainty."

It does not quite work this way. Because "even higher certainty" may well mean "essentially a complete uncertainty". We have only that much of a computing power under our our control in our macroscopic world. There may well be some ultimate limits. So what can we hope for? I think we can hope for discovering new ways of thinking about what "understanding Nature" may mean. If, suppose, Nature is nonlocal - perhaps we should learn how to think in these categories too?

 

[SeventhSigma]

Some certainty is better than no certainty at all, though, is my point -- if there's any certainty, it isn't fully random if we can predict/explain some of the variance. I am assuming we can't explain any of the variance on a quantum scale.

 

[nismaratwork]

@Thomas
I do not necessarily agree that physical evidence suggests anything by itself. It suggests what we want it to suggest. The same evidence can suggest different things to different people - depending on their knowledge and understanding.
I agree that so far dBB is not yet realistic, but do we care about a given model being realistic or not? I think what we should really care about is the predictive and explanatory power.

@SeventhSigma
"But if we are able to accurately measure those processes and simulate them, we may even be able to predict the outcomes of a complex machine with even higher certainty."

It does not quite work this way. Because "even higher certainty" may well mean "essentially a complete uncertainty". We have only that much of a computing power under our our control in our macroscopic world. There may well be some ultimate limits. So what can we hope for? I think we can hope for discovering new ways of thinking about what "understanding Nature" may mean. If, suppose, Nature is nonlocal - perhaps we should learn how to think in these categories too?

Does dBB make any predictions that have not already been made by QM, which are or can be verified?

 

[arkajad]

Does dBB make any predictions that have not already been made by QM, which are or can be verified?

dBB does not make any predictions, but I know some predictions based on dBB and on other models. For instance for tunneling times. These predictions may be at present hard to test, but they are different for different models. When testing will be available one would have to go back to the theory in order to see how the predictions are sensitive to various modeling assumptions about the experimental details.

 

[ThomasT]

@Thomas
I do not necessarily agree that physical evidence suggests anything by itself. It suggests what we want it to suggest.

I don't know what you mean by this.

The same evidence can suggest different things to different people - depending on their knowledge and understanding.

I don't know what you mean by this either. I mean, I don't know what you're getting at.

I agree that so far dBB is not yet realistic, but do we care about a given model being realistic or not? I think what we should really care about is the predictive and explanatory power.

dBB has less predictive power than standard qm. And since it's explicit nonlocality is contrary to what current theoretical and experimental physics suggests about reality, it occupies a fringe niche position wrt theoretical physics, as it should.

 

[arkajad]

dBB has less predictive power than standard qm.

Which should not be a surprise taking into account how much has been invested in the standard qm and how little in the alternatives. You will say that people invest in what is good and do not invest in what is not so good? Well, look what happens to many real investors - after a crash they say (if they do not commit suicides): Oh, I should have invested in something else...

 

[ThomasT]

The problem is that I seem to hear that "quantum" randomness has nothing to do with our ignorance of the variables involved ...

Then you're hearing it incorrectly. Quantum randomness has everything to do with physicists' ignorance of the variables involved. Why do you think that qm is a probability theory? It's because physicists don't know what's happening in the reality that underlies instrumental behavior. They're probing this underlying reality in a more or less random, haphazard way. And then they're making some inferences regarding these probings, and fornalizing some of them, and then doing more, more or less random haphazard, probings. It isn't an exact science. It's probabilities based on the behavior of detection devices. That's all there is.

 

[arkajad]

@ThomasT
But it is also possible that even knowing all the variables involved a given process can still be considered as random - simply because of its complexity - you will never have the computing power to calculate the result within a given error margin. Still you may be able to use probability theory in such a case.

 

[ThomasT]

Which should not be a surprise taking into account how much has been invested in the standard qm and how little in the alternatives. You will say that people invest in what is good and do not invest in what is not so good? Well, look what happens to many real investors - after a crash they say (if they do not commit suicides): Oh, I should have invested in something else...

Well, the best minds in physics have decided to endorse standard qm, which means that they endorse the Born-Copenhagen interpretation. And from what I know of qm, this is the most sensible way to think about it.

 

[ThomasT]

@ThomasT
But it is also possible that even knowing all the variables involved a given process can still be considered as random - simply because of its complexity - you will never have the computing power to calculate the result within a given error margin. Still you may be able to use probability theory in such a case.

Well I can't argue against this.