Confidence as used in Probability

[AngleWyrm]

TL;DR Summary

Examination of Confidence as used in probability, and ways to optimize it's selection

Example problem
A casino offers you a gamble with a 1% chance of winning a try. How many tries will it take to win at least once?

Solution
For this example, I chose 95% confidence, a willingness to be wrong once in twenty:
https://www.notion.so/How-many-tries-will-it-take-e3c37e2cf39a4ec99ee77b7409078807

But what about that decision to choose 95% confidence?
It was an arbitrary choice, governed by nothing more than a general rule of thumb. Can we do better?

Here's what I've figured out so far
Confidence is a linear variable, a Real in the range 0-1.
It can be expressed in this example problem as confidence = -0.0099 × tries + 0.9999

If I choose confidence = 0.5, then I'm making a prediction of the future designed to be wrong half the time
If I choose confidence = 1, then it would take infinite tries
So the range of admissible values for confidence appears to be (0.5, ..., 1) excluding end points

 

[Dale]

You could construct a confidence interval with less than 50% confidence.

 

[Stephen Tashi]

Here's what I've figured out so far
Confidence is a linear variable, a Real in the range 0-1.
It can be expressed in this example problem as confidence = -0.0099 × tries + 0.9999

If "tries" is an huge integer then your definition of confidence will produce a negative number.

 

[AngleWyrm]

Let's take a look at that, because I may have made a tragic error. There are two quantities and a formula that uses them.

First quantity is confidence
confidence = 0.95
chanceToBeWrong = 1 - confidence

Second quantity is chance of success each try
chanceOfSuccess = 0.01
chanceOfFailure = 1 - chanceOfSuccess

Formula to calculate number of tries using the above two quantities
tries = log(chanceToBeWrong) / log(chanceOfFailure)
= approximately 298 tries

 

[Dale]

chance of success each try
...
Formula to calculate number of tries

Can you explain what you mean by "success" and "try" and "number of tries"?

But again, you can have confidence intervals with less than 50% confidence.

 

[AngleWyrm]

The casino's offer of a gamble is a Bernoulli trial, the flip of a coin before the coin has been flipped. In this case a very unfair coin. The two possible outcomes are called success and failure to describe preference, such as winning a gamble. Each try is a movement from before we know the outcome to afterward.

Number of Tries and the Amazing Shrinking Failure Rate

One toss of a coin doesn't have any effect on the next, they are independent.

But as we continue to add more flips to an experiment, the number of possible outcomes explodes. With one coin there were two outcomes, with two there are four outcomes, with three there are eight, and so on.

The result of this is that any particular outcome out of the set of all possible outcomes becomes less likely to be the one we see.

But again, you can have confidence intervals with less than 50% confidence.

Yes, the range of confidence goes from 0% to 100%

But the merit of choosing to be wrong more than half the time in a process with only two possible outcomes would need some justification.

 

[Stephen Tashi]

Some people use the word "confidence" as a synonym for "probability". Discussion about statistics and probability tend to be vague and confusing enough without being made worse by the use of such synonyms. (For example, it's clearer to say "probability" when talking about probability rather than "chance", or "confidence" etc. As another example, It's clearer to say "standard deviation" rather than "uncertainty".)

In statistics, the standard use of the term "confidence" refers to a probability in a particular situation. The situation is that there is a random variable with some probability distribution. Within the possible outcomes of the random variable we define an interval of values. The probability that the random variable falls within that interval is the "confidence" associated with that interval. The interval is called a "confidence interval".

In the example you linked, the random variable is the number of tries to get the first "ace". The interval has the form [1,n]. The goal of the article seems to be to find the value n such that there is a .95 probability of the random variable taking a value in [1,n].

 

[Dale]

But as we continue to add more flips to an experiment, the number of possible outcomes explodes. With one coin there were two outcomes, with two there are four outcomes, with three there are eight, and so on.

Ok, but in what way is the number of tries a function of the chance to be wrong.

 

[AngleWyrm]

Ok, but in what way is the number of tries a function of the chance to be wrong.

Let's start with a model from the Mirror Universe.

In the Lab-O-Doom I conduct three trial runs of an experiment. I've got a bag of jelly beans and there's a yummy green one in there. There are also four other jelly beans in the bag.

And because I'm wearing a white smock and steam-punk eye goggles, I put the jelly bean back in the bag after each try, aka sampling with replacement. There are 2^3 = 8 possible outcomes that could appear on my clipboard of experimental results. Which one will it be?

Risk
The risk of a misadventure wherein I never did get that green jelly bean.

The probability of getting outcome #8 is just like the rest, the product of it's component parts
4/5 x 4/5 x 4/5 = (4/5)^3 = 64/125 chances = 0.512 probability = 51%

But it has the neat property of being easily represented in the compact form of (4/5)^3
risk = failure ^ tries

And from there we can re-arrange the formula to get the original, albeit stated with different nouns:
tries = log(risk) / log(failure)

These three values form a relationship so that any two define the third:

risk = failure ^ tries
tries = log(risk) / log(failure)
failure = risk^ (1/tries)

 

[Dale]

Instead of a cutesy story with irrelevant and distracting details could you just state explicitly and clearly in what way is the number of tries a function of the chance to be wrong.

It would also help if you would define your variables in clear standard statistical terms. For example, by “tries” do you mean “a number, $N$, of Bernoulli trials”. And by “chance to be wrong” do you mean “$q=1-p$ where $p$ is the probability of ‘success’ in a single Bernoulli trial”.

If so, then clearly $N$ and $q$ are independent and I don’t see how $N$ is a function of $q$.

 

[AngleWyrm]

I'm sensing hostility and denial; would you like to bargain? Because I hear the next step is depression.

by “chance to be wrong” do you mean “q=1-p where p is the probability of ‘success’ in a single Bernoulli trial”.

First quantity is confidence
confidence = 0.95
chanceToBeWrong = 1 - confidence

Second quantity is chance of success each try
chanceOfSuccess = 0.01
chanceOfFailure = 1 - chanceOfSuccess

 

[Dale]

I'm sensing hostility and denial; would you like to bargain? Because I hear the next step is depression.

No, I would like a clear statement from you answering the questions I have asked. The next step is not depression, it is a thread closure.

 

[Mark44]

I'm sensing hostility and denial; would you like to bargain?

Adding to @Dale's comment, what you're seeing is not hostility and denial. Instead, the request is for clarity, which can start with using the standard terms of probability and statistics.

 

[AngleWyrm]

could you just state explicitly and clearly in what way is the number of tries a function of the chance to be wrong.

Risk
The risk of a misadventure wherein I never did get that green jelly bean.

The probability of getting outcome #8 is just like the rest, the product of it's component parts
4/5 x 4/5 x 4/5 = (4/5)^3 = 64/125 chances = 0.512 probability = 51%

But it has the neat property of being easily represented in the compact form of (4/5)^3
risk = failure ^ tries

RE: "standard terms"
Variables are placeholders; they are nouns but they are not proper nouns.
They are always a mapping of the form a → b, such as "let Ω be the set of outcomes"
Use of these forums is not predicated on a specific naming convention.

 

[Dale]

They are always a mapping of the form a → b, such as "let Ω be the set of outcomes"

Yes, this is what I am asking for. Unfortunately since you are unwilling to be explicit about that mapping it is impossible for anyone else to understand what you are talking about. We cannot read your mind to get that mapping.

If you choose to write clearly so that there can be a productive conversation then you are welcome to start a new thread. This one is closed.