Help with the statistics of Upper Limits?

In summary, the conversation discusses the effect of adding systematic uncertainties on the expected upper limits to the signal strength in particle physics analyses. The likelihood model used considers observed events, expected background and signal events, nuisance parameters, and a prior distribution for the signal strength. By varying the background and signal uncertainties, researchers try to determine the signal strength that matches the observed events. However, adding nuisance parameters can lead to a broader distribution of the signal strength and potentially higher limits.
  • #1
ChrisVer
Gold Member
3,378
465
This could as well go to the statistics, but I am looking at it from particle physics point of view...
Why adding systematic uncertainties worsen the expected upper limits to the signal strength?
I am trying to find where the flaw enters in the following logic:

0. The model most analyses use is the following likelihood:
[itex]L( N_{obs} | b(\theta ) + \mu s(\theta ) ) = P(N_{obs} |b(\theta ) + \mu s(\theta ) ) U(\mu) \Pi_i Gaus(\theta_i | 0,1)[/itex]
Where [itex]N_{obs}[/itex] is the observed events, [itex]b/s[/itex] are the background/signal expected events, [itex]\theta_i[/itex] are the different nuisance parameters and [itex]\mu[/itex] is called the signal strength. In a Bayesian approach, one has to also to feed in a prior distribution for the signal strength parameter, which is the [itex]U(\mu)[/itex]- let's consider it Uniform. [itex]P(x|n)[/itex] is the poisson probability to get x observed given the expectation of n, and [itex]Gaus[/itex] is a way to represent the variation of the nuisance parameters (given you have symmetric errors).

1. In order for one to get the expected limits, they would set [itex]N_{obs}=N_{exp}=b[/itex].

2. Once they do it, they can start varying the background+signal uncertainties, [itex]\theta_{stat}[/itex] (+[itex]\theta_{sys}[/itex]) [these uncertainties don't affect the signal and background in the same way]

3. On the varied result, they would try to figure out what is the [itex]\mu[/itex] so that the [itex]b'+\mu s' = N_{obs}[/itex]

4. Doing that several times, you get a distribution for [itex]\mu[/itex] (after you marginalize over the uncertainties) which is called the posterior pdf...

5. From μ-distribution get the 95-quantile point.

Now for some reason, adding nuisance parameters (such as [itex]\theta_{sys}[/itex] on top of the statistical), moves the [itex]\mu_{.95}[/itex] higher.
Is that because the uncertainties are not the same for bkg/signal?
Intuitively I can see how eg by subtracting the background from the observed result with higher uncertainties is going to give a more unclear picture of how much signal you allow in the game... but I don't see where this fits in the above logic.
 
Physics news on Phys.org
  • #2
Nuisance parameters make your µ distribution broader. Ignoring asymmetries in the Poisson distribution, they should not shift the expected µ (which should be zero if your method is sound), you just get another uncertainty that gets added in quadrature.
 

FAQ: Help with the statistics of Upper Limits?

What are upper limits in statistics?

Upper limits in statistics refer to the maximum possible value of a variable or data point in a dataset. They are used to set boundaries for the data and determine the range of values that are considered acceptable or significant.

How are upper limits determined?

Upper limits are determined by analyzing the distribution of the data and identifying the highest data point or value. This can be done through various statistical techniques such as calculating the mean, median, or mode of the data.

Why are upper limits important in statistical analysis?

Upper limits are important in statistical analysis because they help to identify outliers or extreme values in the data. They also provide a way to set boundaries and determine the significance of data points within a dataset.

How are upper limits used in hypothesis testing?

In hypothesis testing, upper limits are used as a reference point to determine the probability of obtaining a certain result by chance. They can also be used to set the significance level for a hypothesis test, which helps to determine whether the results are statistically significant or not.

Can upper limits be adjusted or changed?

Yes, upper limits can be adjusted or changed depending on the specific needs of the analysis. For example, in some cases, researchers may choose to use a more conservative upper limit to reduce the risk of false positives. However, it is important to justify any changes made to upper limits and document them carefully.

Similar threads

Replies
4
Views
1K
Replies
49
Views
10K
Replies
11
Views
2K
Replies
12
Views
2K
Replies
1
Views
2K
Replies
2
Views
2K
Replies
8
Views
2K
Back
Top