How Can Improbability and Infinitesimal Probabilities Exist in Real Life Events?

[FactChecker]

Even allowing for that, there is a difference between an indefinitely small probability and an outright impossibility. I'm really criticizing the incorrect use of language. I think it's logically not acceptable to say that something is both non-zero and zero.

It's more than a language problem. Suppose a point on the [0,1] line segment is randomly selected from a uniform distribution. A point WAS selected, yet there was a zero probability of that point having been selected. It was possible with an infinitely small probability. That is a difficulty of probability, not of the language. On the other hand, there are examples of truly impossible things, like obtaining a 10 from the roll of a single normal die.

 

[sysprog]

It's more than a language problem. Suppose a point on the [0,1] line segment is randomly selected from a uniform distribution. A point WAS selected, yet there was a zero probability of that point having been selected. It was possible with an infinitely small probability. That is a difficulty of probability, not of the language. On the other hand, there are examples of truly impossible things, like obtaining a 10 from the roll of a single normal die.

I firmly reject the complacent use of incorrect language as an expedient in the matter.

 

[WWGD]

I firmly reject the complacent use of incorrect language as an expedient in the matter.

Still, however imperfect, term overloading is arguably better than other alternatives. If you were to use absolutely precise and unambiguous terminology it would be essentially impossible to understand when you spoke and people would be upset. (Over) simplification and a certain level of ambiguity seem like necessary evils and used in most, if not all technical areas.

 

[FactChecker]

I firmly reject the complacent use of incorrect language as an expedient in the matter.

Sorry, I think I read too much into your prior post. Certainly, when this specific subject is being discussed, some clear language using different terms is practical and helpful.

That being said, I think that this subject is not usually an issue and using different terms in general would just be confusing and unnecessary.

 

[WWGD]

Dont get me wrong @sysprog , it is ambiguous and confusing but it is too difficult to be meticulously precise about very technical topics and most of the time it will not happen and the best I can rhink of doing is asking for clarification.

 

[PeroK]

It's more than a language problem. Suppose a point on the [0,1] line segment is randomly selected from a uniform distribution. A point WAS selected, yet there was a zero probability of that point having been selected. It was possible with an infinitely small probability. That is a difficulty of probability, not of the language. On the other hand, there are examples of truly impossible things, like obtaining a 10 from the roll of a single normal die.

It's only possible to select from a finite set with a uniform distribution; or from a coutable set with a non uniform distribution. It's not possible to devise an algorithm that could select from an include set - in the sense that the algorithm has an uncountable number of possible outputs.

PS you could have a random variable uniformly distributed on an uncountable set. But that is something else entirely. It's the same difference as defining an infinite sine function and supposing you have physically drawn an infinite sine function.

 

[Stephen Tashi]

It's only possible to select from a finite set with a uniform distribution; or from a coutable set with a non uniform distribution.

You could modify that statement to involve only the mathematical properties of probability distributions and stay within the domain of probability theory. Once you begin to speak of the possibility of taking random samples, you are wandering outside the scope of probability theory.

To repeat, the theory of probability says nothing about the possibility or impossibility of selecting random values from a distribution. The discussion of whether algorithms exist to do this falls under the heading of theories of computability or some other field of science or mathematics.

In particular, the question of whether algorithms exist that can take random samples is a narrower question than whether physical processes exist that do this. For example, if the time for an atom to decay is actually given by an exponential distribution then Nature can can sample from a continuous distribution, even if human beings can only measure the time of decay with finite precision. Whether it is possible for Nature to do this is a topic in physics. It is not covered by probability theory.

 

[FactChecker]

It's only possible to select from a finite set with a uniform distribution; or from a coutable set with a non uniform distribution. It's not possible to devise an algorithm that could select from an include set - in the sense that the algorithm has an uncountable number of possible outputs.

I am uncomfortable with that statement. It strikes me as confusing a human inability to define a process with the claim that no such thing exists. Sort of like claiming that there is no such thing as the area of a circle because there is no way to square a circle.

 

[PeroK]

I am uncomfortable with that statement. It strikes me as confusing a human inability to define a process with the claim that no such thing exists. Sort of like claiming that there is no such thing as the area of a circle because there is no way to square a circle.

A circle and the area of a circle are well defined mathematically. And loosely one can draw a circle. But, you shouldn't confuse the two.

The problem with your paradox that the impossible can happen is that it confuses real and mathematical processes.

 

[FactChecker]

The problem with your paradox that the impossible can happen is that it confuses real and mathematical processes.

I don't agree that I am confusing real and mathematical processes. I am trying to stay within the confines of mathematical definitions (not processes). I would leave the process of selection undefined and assume that any real number that exists can be selected somehow (maybe by a "god-like" process). I think this is an important difference from one which says that only a countable set can be selected from.

I admit that your position has a great advantage if one states that a selection must be done by some definable process. That does seem reasonable. Is there some body of work that addresses this issue, which you are basing your position on? I admit that I have never looked into it.

 

[PeroK]

I don't agree that I am confusing real and mathematical processes. I am trying to stay within the confines of mathematical definitions (not processes). I would leave the process of selection undefined and assume that any real number that exists can be selected somehow (maybe by a "god-like" process). I think this is an important difference from one which says that only a countable set can be selected from.

I admit that your position has a great advantage if one states that a selection must be done by some definable process. That does seem reasonable. Is there some body of work that addresses this issue, which you are basing your position on? I admit that I have never looked into it.

The related issue is that only a countable subset of the real numbers are computable. So, the real numbers generally cannot be selected and processed at all!

There's plenty of reference material on that.

The issue that one cannot have a uniform selection process on the natural numbers is well known. There must be reference on that.

You could look for something on the $[0,1]$ paradox. I don't remember what I found last time.

I'll have a look when I get the chance.

 

[FactChecker]

So, the real numbers generally cannot be selected and processed at all!

Is this somehow a denial of the Axiom of Choice?

 

[PeroK]

Is this somehow a denial of the Axiom of Choice?

No. But highlights the difference between sets of numbers you can study using mathematics and numbers that you can select, describe and process.

Look up "computable" numbers.

 

[FactChecker]

@PeroK , Suppose I define a selection process as follows:
I let you define a selection process on the [0,1] line segment that I have no knowledge of or influence on. Let $P$ denote the countable set of possible results of your process and $I$ denote the remainder of [0,1] of numbers that are impossible to select using your process. $P$ has measure zero and $I$ has measure 1. If I apply the Axiom of Choice to claim a chosen value $c$ from $I$, I must say that it had probability zero, even though it was selected.

 

[FactChecker]

Look up "computable" numbers.

This may be the crux of the matter. "computable" implies a finite, terminating algorithm. I like to think of probability of selection as including infinite, "god-like", selection processes. The limitations of humans to compute a number are not always applicable. But I am afraid that I am taking this into a philosophical turn that is not appropriate in this forum. I will look at the subject material that you suggested.

 

[PeroK]

@PeroK , Suppose I define a selection process as follows:
I let you define a selection process on the [0,1] line segment that I have no knowledge of or influence on. Let $P$ denote the countable set of possible results of your process and $I$ denote the remainder of [0,1] of numbers that are impossible to select using your process. $P$ has measure zero and $I$ has measure 1. If I apply the Axiom of Choice to claim a chosen value $c$ from $I$, I must say that it had probability zero, even though it was selected.

Let me describe the issue as follows. You have a real number lottery. Everyone gets to choose their own real number, say, and put it in a sealed envelope. You choose the winning number by whatever process you like. But, you must publish an actual number.

You are not allowed to say you picked "some" number $c$, but you don't know what it is. Nor can you describe it in any way.

Then you are limited to the computable numbers.

It's nothing to do with the axiom of choice.

 

[Quasimodo]

I think the answer to the OP's original question : "how improbable is impossible?" depends on the size of the sample space of the experiment used to derive the probabilities.

I think everyone here would agree, at first sight, that the probability of a random number generator ( producing 0-9 digits one at a time ) to output an infinite string of all 0's is 0 itself ( an impossible outcome. )
And yet there is a sample space where this probability is 1.

 

[PeroK]

I think the answer to the OP's original question : "how improbable is impossible?" depends on the size of the sample space of the experiment used to derive the probabilities.

I think everyone here would agree, at first sight, that the probability of a random number generator ( producing 0-9 digits one at a time ) to output an infinite string of all 0's is 0 itself ( an impossible outcome. )
And yet there is a sample space where this probability is 1.

A random number generator can only ever produce a finite sequence of digits.

 

[Quasimodo]

A random number generator can only ever produce a finite sequence of digits.

Let us please not argue for argument's sake, and accept that there is a true random number generator somewhere producing one digit 0-9 at a time forever, ok?

 

[PeroK]

Let us please not argue for argument's sake, and accept that there is a true random number generator somewhere producing one digit 0-9 at a time forever, ok?

It can produce numbers for an indefinite period, if you like, but it never produces an infinite sequence.

To get an infinite sequence you have to appeal directly to mathematics.

Let $s_n$ be an infinite sequence of digits, where each digit is uniformly distributed on $0-9$, is perfectly valid.

Saying that such a sequence could come from a random number generator is a confusion of mathematical and computational ideas.

 

[FactChecker]

A random number generator can only ever produce a finite sequence of digits.

Unless the first digit takes 1/2 sec, the second digit takes 1/4 sec, the third digit takes 1/8 digit, etc. I think that your logic and objections are based on physical constraints that are not applicable in all the theoretical and conceptual situations that probabilities can reasonably be applied to.

 

[Quasimodo]

Let snsns_n be an infinite sequence of digits, where each digit is uniformly distributed on 0−90−90-9, is perfectly valid.

ἔστω:
so be it, if you like!

 

[PeroK]

Unless the first digit takes 1/2 sec, the second digit takes 1/4 sec, the third digit takes 1/8 digit, etc. I think that your logic and objections are based on physical constraints that are not applicable in all the theoretical and conceptual situations that probabilities can reasonably be applied to.

It's a good point. Then we see precisely the reason that the "impossible" has happened.

1) we postulate a random number generator according to your specification.

2) it generates an infinite sequence in one second.

3) the probability that that precise sequence would be generated is zero.

4) the impossible has happened.

But, we have postulated a physically impossible random number generator. So, no mystery and no paradox. An impossible machine has done the impossible!

 

[FactChecker]

But, we have postulated a physically impossible random number generator. So, no mystery and no paradox. An impossible machine has done the impossible!

Impossible physically or impossible conceptually? In the real world, it is not possible to have an absolutely fair coin, so should we stop talking about the probabilities of a fair coin?

 

[Quasimodo]

Please read my post carefully!

I said, that we can show that there exists a sample space where this probability is 1 and NOT 0!

The proof relies on limits at infinity, so my previous example is realistically viable but if you want to argue trivialities with me, I might as well leave this conversation...

 

[PeroK]

Impossible physically or impossible conceptually? In the real world, it is not possible to have an absolutely fair coin, so should we stop talking about the probabilities of a fair coin?

No. But we have to be careful what we conclude. A real coin can be associated with a fair coin in a number of contexts. This is part of the mathematical modelling process.

 

[PeroK]

@FactChecker Let me boil down our debate as follows. First, I'm going to say:

Let $x_0 \in [0,1]$.

I've chosen one arbitrary real number.

You believe I have done something impossible. I don't believe I have done something impossible; I believe I've done something mathematical.

And, if by doing mathematics we are all doing the impossible all the time and that is part of your definition of impossible, then I guess there's no argument.

 

[PeroK]

Please read my post carefully!

I said, that we can show that there exists a sample space where this probability is 1 and NOT 0!

The proof relies on limits at infinity, so my previous example is realistically viable but if you want to argue trivialities with me, I might as well leave this conversation...

Limits are, quite explicitly, abstract mathematical constructions. And, the history of mathematics shows how important it is to have a rigorous mathematical definition. You can't mix up limits with real physical processes.

 

[Quasimodo]

Limits are, quite explicitly, abstract mathematical constructions. And, the history of mathematics shows how important it is to have a rigorous mathematical definition. You can't mix up limits with real physical processes.

Let k be the size of a sample probability space, how big we will soon find out.

Let a true random number generator produce a string of digits size n, each digit takes 1 second to be generated, etc., how big this n would be, we'll find out soon.

Is this ok with you? Shall we proceed?

 

[PeroK]

Let k be the size of a sample probability space, how big we will soon find out.

Let a true random number generator produce a string of digits size n, each digit takes 1 second to be generated, etc., how big this n would be, we'll find out soon.

Is this ok with you? Shall we proceed?

Don't let me stop you!

 

[FactChecker]

No. But we have to be careful what we conclude. A real coin can be associated with a fair coin in a number of contexts. This is part of the mathematical modelling process.

I'll buy that. I think that I understand where each of our positions is appropriate -- the physical versus the conceptual (including physically impossible). With that in mind, I see your point and will look at "computable" some more. Even if I allow myself to include the physically impossible, I still have the problem of distinguishing the mathematical concepts of "nearly impossible" from "logically impossible". That is what I was trying to address at the beginning.

 

[sysprog]

I'll buy that. I think that I understand where each of our positions is appropriate -- the physical versus the conceptual (including physically impossible). With that in mind, I see your point and will look at "computable" some more. Even if I allow myself to include the physically impossible, I still have the problem of distinguishing the mathematical concepts of "nearly impossible" from "logically impossible". That is what I was trying to address at the beginning.

Some good insights on those concerns and on related matters can be found here: https://terrytao.wordpress.com/2015/09/29/275a-notes-0-foundations-of-probability-theory/

 

[Twodogs]

And events with a probability of exactly one, what then? In a deterministic universe the state of the universe at one moment follows directly from the state of the universe at any prior time with a probability of exactly one. Anything less and the wheels come off. Is that the case?

 

[FactChecker]

No. One must distinguish between a logical certainty and a probability of one. They are not the same.
Suppose a number is selected randomly on the line segment [0,1]. The probability that the number is irrational is 1 because the subset of irrational numbers has a probability measure of 1. The rational numbers are countable and the rational subset has a probability measure of 0. If the number selected turns out to be rational, the consequences are not that "the wheels come off".

 

[sysprog]

And events with a probability of exactly one, what then? In a deterministic universe the state of the universe at one moment follows directly from the state of the universe at any prior time with a probability of exactly one. Anything less and the wheels come off. Is that the case?

From the link I posted in #67:

By default, mathematical reasoning is understood to take place in a deterministic mathematical universe. In such a universe, any given mathematical statement

(that is to say, a sentence with no free variables) is either true or false, with no intermediate truth value available. Similarly, any deterministic variable

can take on only one specific value at a time.

However, for a variety of reasons, both within pure mathematics and in the applications of mathematics to other disciplines, it is often desirable to have a rigorous mathematical framework in which one can discuss non-deterministic statements and variables – that is to say, statements which are not always true or always false, but in some intermediate state, or variables that do not take one particular value or another with definite certainty, but are again in some intermediate state. In probability theory, which is by far the most widely adopted mathematical framework to formally capture the concept of non-determinism, non-deterministic statements are referred to as events, and non-deterministic variables are referred to as random variables. In the standard foundations of probability theory, as laid out by Kolmogorov, we can then model these events and random variables by introducing a sample space (which will be given the structure of a probability space) to capture all the ambient sources of randomness; events are then modeled as measurable subsets of this sample space, and random variables are modeled as measurable functions on this sample space.

Prof. Tao explains difficult things with clarity, but he can't thereby make them not difficult.