Does a Probability of 1 Really mean a Dead Cert

[HallsofIvy]

This is an interesting question as if it is a Probability then the range should go from "Near zero" to One and not from Zero to One.

What reason do you have to say that a probability cannot be 0?

This is where the Quants made the error and then came up with the "Hundred Year Storm" as an excuse. Ito's Calculus is really Physics; (ballistic trajectory) and shouldn't be used in Finance. (just my opinion)

 

[Tamroes]

To say that something "has no chance" is an idiomatic way of speaking; but if it is a probability then there is always a chance; or it would be a certainty. Probability curves all contain Exponential (e) in some form and this never goes to zero;

 

[Tamroes]

Is a probability of 1 is a cert bet?.

I would say that a probability of 1 is a good bet.

 

[D H]

To say that something "has no chance" is an idiomatic way of speaking; but if it is a probability then there is always a chance; or it would be a certainty. Probability curves all contain Exponential (e) in some form and this never goes to zero;

Nonsense, on both accounts. The probability that roll of a standard die will show seven is zero. The probability mass function for the roll of a die does not contain an exponential in any form. The probability of drawing a two from U(0,1) is zero. The probability density function for a uniform random variable does not contain an exponential in any form. One final example: The probability that the distance from the origin to a point randomly from some distribution is negative is zero. Distance is positive semidefinite.

Note that the probability of drawing 0.5, exactly, from U(0,1) is also zero. A probability of zero does not necessarily indicate an impossible event.

 

[bhobba]

Note that the probability of drawing 0.5, exactly, from U(0,1) is also zero. A probability of zero does not necessarily indicate an impossible event.

Yea that's true - but how would you check you drew .5 - since that would imply an infinite measurement precision. I believe these are the inherently difficult issues with probabilities associated with a continuum. But one can still model such a thing (ie that say a particle is at exactly .5) by the Dirac measure - but its only a model - physically its not possible.

Thanks
Bill

 

[D H]

Bill, you should have posted your question in one of the science sections if you wanted to talk about the use of probability in the sciences. Since you instead posted it in the mathematics section, the responses are (or should be) in terms of mathematics. Mathematics doesn't care about whether it's physically impossible to measure something to infinite precision.

 

[Stephen Tashi]

Mathematics doesn't care about whether it's physically impossible to measure something to infinite precision.

And, to my mind at least, the formal mathematics of probability theory doesn't assert that you can actually take random samples (be they from discrete or continuous random variables). Give a distribution of a random variable X, there are definitions that define the properties of other random variables, such as the joint distribution of "n independent random samples from X". But according to that terminology the random variable "n independent random samples from X" exists only in the sense that a certain distribution ( product measure) can be show to exist.

The informal applied mathematics of probabiltiy theory speaks of random samples as if they are measurements than can actually be made. However, I don't think the formal math of probabilty theory does anything except assert that given a certain function, we can define other types of functions.

 

[Jano L.]

Yea that's true - but how would you check you drew .5 - since that would imply an infinite measurement precision. I believe these are the inherently difficult issues with probabilities associated with a continuum. But one can still model such a thing (ie that say a particle is at exactly .5) by the Dirac measure - but its only a model - physically its not possible.

Bill, you should have posted your question in one of the science sections if you wanted to talk about the use of probability in the sciences. Since you instead posted it in the mathematics section, the responses are (or should be) in terms of mathematics. Mathematics doesn't care about whether it's physically impossible to measure something to infinite precision.

I think Bill is right in that experimentally we cannot measure continuous variables (distance) with infinite accuracy. We usually obtain rational numbers with decimal expansions of few significant digits. Then the number of distinct possible results is finite and zero probability does not arise often.

D H is also right that this has no bearing on the mathematical meaning of probability.

But the crux of the matter does not depend on whether we use continuous or discrete sets. Consider rolling a die. If for some reason we have probabilities 0, 1/5, 1/5, 1/5, 1/5, 1/5 for rolling 6,5,4,3,2,1 , it does not follow that 6 cannot be rolled. It only implies that the number of rolls where 6 was rolled and the number of all rolls has expected ratio with limit 0 as the total number of rolls increases indefinitely:

$$
Prob(6) = 0~~~ \text{implies} ~~~ \lim_{N\rightarrow\infty} \frac{N_6(N)}{N} = 0
$$

This allows for the possibility that $N_6(N)$ is non-zero non-decreasing function of $N$. For example, a sequence of results that has $N_6 = \text{Trunc}(\ln N)$ represents possible sequence and obeys the above expectation, i.e. the frequency of 6 in the sequence is 0.

 

[Stephen Tashi]

It only implies that the number of rolls where 6 was rolled and the number of all rolls has expected ratio with limit 0 as the total number of rolls increases indefinitely:

$$
Prob(6) = 0~~~ \text{implies} ~~~ \lim_{N\rightarrow\infty} \frac{N_6(N)}{N} = 0
$$

If you are using the law of large numbers to show a result about probability then you must be specific about what you mean by $\lim_{N\rightarrow\infty} \frac{N_6(N)}{N}$. If you mean the ordinary limit of a function, you are not correct. The law of large numbers involves the limit of a probability of the fraction of occurences, not the limit of the fraction of occurences - i.e. not the limit of the fraction of occurences without mentioning anything about a probability of it.

 

[Jano L.]

You are right, the implication is incorrect. What I meant is that there are infinite sequences that contain arbitrary number of values "6" and are obviously consistent with probabilities (0, 1/5, 1/5, 1/5, 1/5, 1/5) for 6, 5, 4, 3, 2, 1.

 

[atyy]

So if Jano L is right, does this mean that deterministic equations reformulated probabilistically are not necessarily physically equivalent? For example, I (loosely) usually think of Liouville's theorem as a "probabilistic" version of deterministic equations.

I guess he is saying that x=2 is not the same as P(x)=δ(x-2). But I think something like that happens in Liouville's[/PLAIN] theorem where there is a measure on phase space, so it could have implications purely classically too, which may be easier to think about.

Thinking from Liouville's theorem, it seems there is some analogue of x=2 → P(x)=δ(x-2), but maybe it isn't possible to go in the reverse P(x)=δ(x-2) → x=2 ?

 

[Tamroes]

re the probability question I see that I should have the term "Ceterius Paribus" which means "use your brains" or "all things being equal". You don't need probability curve for tossing dice but I would have thought that everyone knows that you can't get a 7 if you only have one dice unless you have found some new type of dice.

 

[Jano L.]

Still, the Nature brings some surprises time to time. Check this out:

https://en.wikisource.org/wiki/Heim...n_Between_the_Kings.2C_and_Their_Game_at_Dice.

 

[bhobba]

Thinking from Liouville's theorem, it seems there is some analogue of x=2 → P(x)=δ(x-2), but maybe it isn't possible to go in the reverse P(x)=δ(x-2) → x=2 ?

.
Hmmmm. What does the Dirac measure at 2 mean? If you take any interval that doesn't contain 2 the probability is 0 it's in that interval (here 'it' means whatever you are modelling). If you take any interval containing 2 the probability is 1 its in that interval. The only conclusion is, its at 2. I can't quite follow how any other conclusion is possible.

Thanks
Bill

 

[bhobba]

If for some reason we have probabilities 0, 1/5, 1/5, 1/5, 1/5, 1/5 for rolling 6,5,4,3,2,1 , it does not follow that 6 cannot be rolled. It only implies that the number of rolls where 6 was rolled and the number of all rolls has expected ratio with limit 0 as the total number of rolls increases indefinitely:

Once you apply the Kolmogorov axioms then the interpretation is a probability of 0 means it never occurs.

To see this in your dice example you can remove the outcome 1 from the event space and the axioms still apply. This means we can model the situation without that event even in the event space so obviously it can never occur.

Its the same with a probability of 1. If something has probability 1 in your event space all other events have probability 0. You can remove or add any events to the event space you like (other than the event with probability 1 of course) and the axioms still apply. You can remove all events (except the one with probability 1) and the axioms apply. An event space with only one element has but one possible outcome - that element.

I think reasonable considerations like the above mean probability 0 never occurs and probability 1 must occur when you apply the axioms if you are to be able to apply them consistently.

Thanks
Bill

 

[bhobba]

re the probability question I see that I should have the term "Ceterius Paribus" which means "use your brains" or "all things being equal". You don't need probability curve for tossing dice but I would have thought that everyone knows that you can't get a 7 if you only have one dice unless you have found some new type of dice.

See my post above.

I think it follows from the requirement to be able to apply the Kolmogorov axioms consistently. One can add any event with probability 0 or remove any event with probability 0, and the axioms still apply. I think the only reasonable conclusion, considering it can be modeled just as well without events of probability 0, is they can never occur.

Thanks
Bill

 

[Jano L.]

One can add any event with probability 0 or remove any event with probability 0, and the axioms still apply. I think the only reasonable conclusion, considering it can be modeled just as well without events of probability 0, is they can never occur.

Bill, you keep referring to Kolmogorov axioms. I do not see how they could possibly support your claim. Removal of an event with probability 0 may not change other probabilities, but this does not mean the event cannot happen. It just happens rarely enough so that its probability is zero.

Generally, I do not think you cannot use probability theory to decide what can and what cannot happen (what is the mathematical space of events one should consider). That kind of decision seems to be prior to the application of the probability considerations. In the probability calculations I know, the space of possible events is always given or assumed, and only then one thinks about how to assign or calculate probabilities.

 

[bhobba]

Bill, you keep referring to Kolmogorov axioms. I do not see how they could possibly support your claim. Removal of an event with probability 0 may not change other probabilities, but this does not mean the event cannot happen. It just happens rarely enough so that its probability is zero.

The reason I keep coming back to the Kolmogorov axioms is they are the foundational axioms of modern probability theory. The frequentest interpretation and Bayesian approach are either axiomatically equivalent to it or follow from the axioms.

When you apply the Kolmogorov axioms to an event space you assign some probability to each element of the event space. I can't see how that is in any doubt. Now suppose any element has probability zero. Remove it. The event space still obeys the Kolomogorov axioms. But that event now can never occur, be selected, or whatever, because it is no longer in the space. To me the only conclusion, if applying the axioms is to consistent, is that that event never occurs. The similar argument applies to probability 1 - you can remove all the other events leaving only the event with probability 1. It is the only event - it must occur.]

There are subtleties with continuous event spaces that you need things like a Dirac measure to overcome but I think the principles are still the same - you could probably come up with some kind of limit argument to justify it.

If you can't see it - well this sort of thing occurs every now and then - one person can't see what is obvious to another and basically you agree to disagree.

Thanks
Bill

 

[pbuk]

$P(x) = 0$ means that however many trials there are, $x$ will not happen. It doesn't say anything about whether $x$ cannot happen, simply that it does not happen.

But I don't see how you can equate this with the statement $x$ does happen, just very rarely (unless you mean infinitely rarely).

 

[bhobba]

$P(x) = 0$ means that however many trials there are, $x$ will not happen. It doesn't say anything about whether $x$ cannot happen, simply that it does not happen.

Every probability book I have ever read uses the Kolmogorov axioms to justify other appoaches like the frequentest approach through the law of large numbers derived from those axioms eg Feller's classic. So you really need to start from those axioms. And if you do that then you find, as I have pointed out, removing an event with probability 0 means the event space still obeys those axioms - but it can never occur. To me being able to apply those axioms consistently means it must never occur - otherwise you would not be able to remove it and the axioms still apply. I really can't see any way out of it.

Thanks
Bill

 

[HallsofIvy]

$P(x) = 0$ means that however many trials there are, $x$ will not happen. It doesn't say anything about whether $x$ cannot happen, simply that it does not happen.

But I don't see how you can equate this with the statement $x$ does happen, just very rarely (unless you mean infinitely rarely).

Neither of those statements is true. Take, for example, the experiment of choosing a number from the interval [0, 1] with uniform probability (probability that the number chosen is in sub-interval [a, b] is b- a). Then the probablity that any specific number is chosen is 0, but on every trial some number is chosen.

 

[atyy]

The reason I keep coming back to the Kolmogorov axioms is they are the foundational axioms of modern probability theory. The frequentest interpretation and Bayesian approach are either axiomatically equivalent to it or follow from the axioms.

When you apply the Kolmogorov axioms to an event space you assign some probability to each element of the event space. I can't see how that is in any doubt. Now suppose any element has probability zero. Remove it. The event space still obeys the Kolomogorov axioms. But that event now can never occur, be selected, or whatever, because it is no longer in the space. To me the only conclusion, if applying the axioms is to consistent, is that that event never occurs. The similar argument applies to probability 1 - you can remove all the other events leaving only the event with probability 1. It is the only event - it must occur.]

There are subtleties with continuous event spaces that you need things like a Dirac measure to overcome but I think the principles are still the same - you could probably come up with some kind of limit argument to justify it.

If you can't see it - well this sort of thing occurs every now and then - one person can't see what is obvious to another and basically you agree to disagree.

Thanks
Bill

But if the event with probability 0 occurred, then you could not remove it from the space. You only show that the axioms are consistent with the probability 0 event not occurring, but you don't show that it is inconsistent with the event occurring.

For example, in the Bayesian interpretation, the event can occur, it's just that your subjective belief is consistent and wrong.

 

[Jano L.]

I just found there is an idea similar to what Bill is claiming, namely that events with probability close to 0 should be treated as "morally impossible" (Bernoulli), which now seems to be advocated under the name "Cournot principle":

http://www.probabilityandfinance.com/articles/15.pdf

I think it does make sense as a practical rule of a thumb, especially if you consider that people do not really believe that a brick will fall on their head when they go along the street, or that they will ever throw six sixes in one roll.

Yet the small probability of such events (1/46656 for the dice) does not make them impossible event in the true meaning of the word "impossible". In fact, six sixes is as likely as any other physical result, because they are all equally probable.

 

[Stephen Tashi]

The q

Every probability book I have ever read uses the Kolmogorov axioms to justify other appoaches like the frequentest approach through the law of large numbers derived from those axioms eg Feller's classic. So you really need to start from those axioms. And if you do that then you find, as I have pointed out, removing an event with probability 0 means the event space still obeys those axioms - but it can never occur. To me being able to apply those axioms consistently means it must never occur - otherwise you would not be able to remove it and the axioms still apply. I really can't see any way out of it.

I'll harp repetitively on assertions in my previous posts.

There is nothing in the Kolmorogov axioms for proability (as they are given in modern measure theory texts) about events actually "happening" or "not happening". (And, as a technical matter you cannot abitrarily remove an event from a measure space because a measure has to be defined on a "sigma algebra of sets". If you remove an event, you must redefine the sigma algebra. If you are successful, this puts you in a different measure space.)

Certainly any probability book should mention applications of probability theory that use the concept of probabiliy in situations where the observed frequency of something happening or not happening is the thing of interest. However, there is nothing in the axioms of probability theory that asserts that you can or cannot make such observations. The use of the term "event" for a set in does not (in mathematics) imply the set is a measurement that can actually be made.

This thread is very much like discussions of such weighty topics as "Are dy and dx numbers?" or "Does .9999... = 1?". It involves a variety of personal opinions about mathematics, some presented in a non-mathematical way. I dont' object to such threads, but I do object to claims that personal opinions are backed by standard mathematical definitions. If we want to talk about the axioms of probability theory, we should talk about the axioms as they are presented formally, not about the presentations made in elementary texts that rely on studen't's inituitions about thinking of an "event" as something that actually happens or doesn't.

 

[bhobba]

There is nothing in the Kolmorogov axioms for proability (as they are given in modern measure theory texts) about events actually "happening" or "not happening".

I find I am forced to agree with that.

What I am talking about is when the axioms are presented in books like Feller's examples to cement their meaning are given, as well as the development presented in the text. Those invariably concern the event space having events that occur or something similar. Once that is done then my argument applies.

Although I have never seen an example of an event space that can't be interpreted this way I do not doubt they exist.

Out of curiosity though can anyone think of an example?

Thanks
Bill

 

[Stephen Tashi]

Once that is done then my argument applies.

According your agument,if we are drawing numbers from a uniform distribution on [0,1], I can remove the possibility of drawing $\pi/4$ since it has probability 0. If I proceed to remove all events that have zero probability, there won't be any events left.

I'm not saying this mathematically invalidates your conclusion in a practical sense because the axioms of probability theory don't say whether you can or cannot take random samples from a uniform distribution in the first place.

I think any practical implementation of sampling amounts to sampling from a discrete random variable with a finite number of values. I don't think it is controversial to say that events with probability 1 in such probability spaces always happen if the space correctly models the events. (For example, if a coin has a possibility of landing on its edge and your probability space only includes the events "heads" and "tails" then you have a wrong model. )

When we consider spaces with infinite outcomes (such as space of infinite sequences of coin tosses or random draws from a uniformly distributed random variable) then whether events with probability 1 always happen cannot be tested by practical methods. (It's an interesting question whether Nature herself can take samples from such distributions.)

There are certain questions in mathematics that are "undecided". But to be any kind of question, it must be precise. For example if a statement about all groups is "undecided" then the statement is precise enough that you can look at some particular group and see if the statement is true or false about it. The question of whether whether an event with probability 1 is a "dead certainty" is not an undecided question in probability theory. It isn't even a question at all! It is not precise enough, within the terminology of probability theory, to have a specific meaning.

There is a theory called "possiblity theory" that uses the terminology of "possible" and "necessary" events. I don't know if people have worked on combining it with probability theory.

 

[atyy]

I don't think it is controversial to say that events with probability 1 in such probability spaces always happen if the space correctly models the events. (For example, if a coin has a possibility of landing on its edge and your probability space only includes the events "heads" and "tails" then you have a wrong model.

I think Jano L's argument is that we should say it the other way: if an event is certain, it is correctly modeled as having probability 1. (But not that if an event has probability 1, then it is certain.)

For the case of an event that is modeled as having probability zero, but occurred once, maybe the practical way to decide whether to reject the model is some "traditional way" like the chi-squared test? I haven't tried it out yet.

 

[bhobba]

According your agument,if we are drawing numbers from a uniform distribution on [0,1], I can remove the possibility of drawing $\pi/4$ since it has probability 0. If I proceed to remove all events that have zero probability, there won't be any events left.

Indeed continuous event spaces are problematical - I think a rigorous development along the lines of my argument would need some kind of limit procedures and the introduction of distribution theory - here I mean distributions in the Schwartz sense ie the Dirac Delta Function etc.

Thanks
Bill

 

[atyy]

I attempted to run a chi-squared test with

Category A, observed = 1, expected = 0
Category B, observed =10000000000, expected = 10000000000

I got as the result:
The chi-square test is not possible when any of the expected values are zero.

Which supports the interpretation that if an event with probability 0 occurs, then the model is inadequate (but not because it fails the hypothesis testing).