Cumulative distribution function to probability density function how

[O.J.]

Hello!
I'm taking a mathematics course in probability and stochastic processes and we started covering the CDF (cumulative distribution function) which i understand perfectly and then the PDF (probability density function). The PDF was defined to be the derivative of the CDF. Now, the CDF is defined in terms of cumulative probabilities multiplied by a unit step function. Differentiating that we get the same probabilities multiplied by the imulse (or delta) function. Our professor stated that the sketching of the PDF will produce vertical lines at the end points of each random variable whose magnitude equals the jump between the previous and the current random variable probabilities. However, I am finding it hard to understand that mathematically as the probabilities will be multiplied by the impulse function (say Imp (X - Xi) whose value at Xi is infinite by definition. How is it that we get a finite value at these points when Imp(X-Xi) is supposed to be infinite? can someone please elaborate?

 

[mathman]

You need to get some understanding of how delta functions work. For the case of prob. distributions, the point is if the prob. of a specific number is >0, there is a jump in the distribution function at that point. To be precise mathematically, the derivative at that point does not exist. The delta function, mutiplied by the size of the jump, is used to handle this case.

 

[O.J.]

I understand how the delta function works individually, but when coupled with the CDF and PDF I cease to. I do not see how MATHEMATICALLY multiplying the individual probabilities by the impulse function WHICH IS INFINITE at Xi produces a line of finite length. :S

 

[O.J.]

Someone care shed some light on this?

 

[mathman]

I understand how the delta function works individually, but when coupled with the CDF and PDF I cease to. I do not see how MATHEMATICALLY multiplying the individual probabilities by the impulse function WHICH IS INFINITE at Xi produces a line of finite length. :S

The finite jump (line of finite length) is the weighted integral of the delta function at that point.

 

[O.J.]

well, what is a WEIGHTED integral?
moreover, the definition of the CDF doent mentiond anything about a weighted integral, it simply differentiates the unit step function which yields the impulse function.

 

[mathman]

The integral of a delta function is 1. Since the jump is usually <1, the delta function has to weighted (multiplied) by the size of the jump.

 

[O.J.]

This doesn't add up. Please understand me here, we are multiplying the individual probabilities by the IMPULSE FUNCTION itself not its integral. Thats what the mathematical derivation says (ie, differentiating the unit step function --> impulse function) and the impulse function is infinite at To. So?

 

[mathman]

I think your problem is understanding the distinction between cumulative probability function (which is a probability) and probability density function (which is NOT a probability). The delta function is used as part of the description of the density when there is a jump in the cumulative probability, i.e. the probability of that particular value is > 0.

 

[O.J.]

Ok, so here is what I have read:
The PDF is defined as being the DERIVATIVE of the CDF.
The CDF is usually defined as Fx = p(Xi) u(X-Xi). Differentiating that:
p(Xi) imp (X-Xi). But imp is infinite, so the value should be infinite..? shouldn't it? sorry for being so thick headed but I really am confused here.

 

[John Creighto]

Ok, so here is what I have read:
The PDF is defined as being the DERIVATIVE of the CDF.
The CDF is usually defined as Fx = p(Xi) u(X-Xi). Differentiating that:
p(Xi) imp (X-Xi). But imp is infinite, so the value should be infinite..? shouldn't it? sorry for being so thick headed but I really am confused here.

Why do you want to assign a value to the impulse function. It is only really defined in terms of an integral anyway.

 

[O.J.]

... The integral of which is the UNIT STEP function. So when you DIFFERENTIATE the CDF (which is basially a probability multiplied by the unit step) you get the IMPULSE times the probability. Am I being clear? or is my explanation vague?

 

[mathman]

... The integral of which is the UNIT STEP function. So when you DIFFERENTIATE the CDF (which is basially a probability multiplied by the unit step) you get the IMPULSE times the probability. Am I being clear? or is my explanation vague?

The statement is essentially correct. I think your problem has to do with understanding the role of the delta function, not with probability.

 

[O.J.]

what is its its role?

 

[jacophile]

Although the delta function has infinite value, the area under it is 1, yeh?
It is defined as having width $$\epsilon$$ and height $$\frac{1}{\epsilon}$$ in the limit that $$\epsilon$$ aproaches 0.

When you multiply it by f(x), given that the delta function is centred on x, the result is a delta function with area identicaly equal to f(x). So when you integrate the product you get the area under the curve which is...

Is this what's bugging you?

 

[O.J.]

I know that the area under a delta function centred at a specific value is 1 which is why the definition of the CDF is bugging me. This is because that definition defines the CDF as the derivative of the PDF resulting in the differentiation of the unit step function (which is already present in the PDF) giving us the IMPULSE function which is infinite. Then, the professor comes and displays the graph of the CDF as FINITE line jumps which is mathematically incorrect because of the presence of the delta function.

 

[mathman]

Then, the professor comes and displays the graph of the CDF as FINITE line jumps which is mathematically incorrect because of the presence of the delta function.

Why do think it is incorrect? The CDF has finite jumps, the delta function shows up in its derivative.

 

[John Creighto]

Then, the professor comes and displays the graph of the CDF as FINITE line jumps which is mathematically incorrect because of the presence of the delta function.

The integral of the delta function is finite so why shouldn't the jumps be finite.

 

[jacophile]

Then, the professor comes and displays the graph of the CDF as FINITE line jumps which is mathematically incorrect because of the presence of the delta function.

Well the integral of the PDF is the area enclosed between it and the x-axis. So if you accept that the a delta function centered on x has finite area 1 f(x) then why don't you accept that the integral increases by a finite step of f(x)?

Also, just to restate mathman's earlier comment, the value of the PDF at x is not P(X=x) it is the probability density at X=x which is infinite.

Heuristically speaking, this is because the sample space is descrete, not continuous. So the value of the pdf is zero for all points not equal to the allowable, discrete values and infinite for the zero-width intervals representing the discrete allowed values.

I mean if you want to be more rigorous and mathematically correct you need to talk about the limit as epsilon->0 where epsilon is the width of the delta function etc as I mentioned earlier. You also need to realize that the step function is the Heaviside step function which is an integral defined in terms of the delta function.

 

[jacophile]

This is because that definition defines the CDF as the derivative of the PDF

Oh, and by the way, the CDF is the integral of the PDF not the derivative...

 

[O.J.]

sorry about that. I forgot that the PDF is the derivative of the CDF (opposite to what I stated). So the PDF has the delta function in it (which is 0 everywhere and infinite at 0 or Xo). But my confusion came when my professor plotted its graph as a series of finite length lines representing the jumps. How can it be finite when every probability is being multiplied by an infinity at that specific point?

 

[O.J.]

bump.

 

[John Creighto]

sorry about that. I forgot that the PDF is the derivative of the CDF (opposite to what I stated). So the PDF has the delta function in it (which is 0 everywhere and infinite at 0 or Xo). But my confusion came when my professor plotted its graph as a series of finite length lines representing the jumps. How can it be finite when every probability is being multiplied by an infinity at that specific point?

The delta function is used for probability density not probability. The probability of x being between a and b is the integral of the probability density from a to b. We know the integral of the delta function is finite.

 

[O.J.]

I agree, it is not the probability that worries me. It is the graph of the PDF that is shown as a series of FINITE jumps. This is how our professor sketched it (for the example of rolling a die):

| | |
| | |
| | |
-----------------------------

the length of each jump is 1/6 and they are six lines i didnt draw them all. how can each line have a finite length when at each X (1,...6) the delta function is ON and its value is infinite.

 

[John Creighto]

I agree, it is not the probability that worries me. It is the graph of the PDF that is shown as a series of FINITE jumps. This is how our professor sketched it (for the example of rolling a die):

| | |
| | |
| | |
-----------------------------

the length of each jump is 1/6 and they are six lines i didnt draw them all. how can each line have a finite length when at each X (1,...6) the delta function is ON and its value is infinite.

It’s just a drawing. Your proof just drew the height of the line to represent the area of the delta function. It is not a graph of the delta function. Don’t read too much into it. I’ve seen a delta function represented as a Gaussian, a rect function or just an arrow. A line is just another visualization.