Sum of Identically Distributed Independent Random Variables

[ObliviousSage]

Homework Statement

The random variables X₁ and X₂ are independent and identically distributed with common density f_X(x) = e^-x for x>0. Determine the distribution function for the random variable Y given by Y = X₁ + X₂.

Homework Equations

Not sure. Question is from Ch4 of the book, and convolutions for computing distributions of sums of random variables aren't introduced until Ch8.

Answer in the back of the book is F_Y(y) = 1-e^-y(1+y) for y>0.

The Attempt at a Solution

Tried every convolution formula I could find (I found a bunch of different variations, and I'm not sure which ones are equivalent or correct): integral from zero to positive infinity of f(t)F(y-t)dt, F(t)f(y-t)dt, F(t)F(y-t)dt, and f(t)f(y-t)dt. All of them end up with a $$\int$$e^te^-t, which gives me an infinity term.

Am I solving the integral wrong or not setting it up right or something else?

 

[Tangent87]

I don't know how your book is explaining convolutions, but this book does it very well (scroll down to page 291 and page 292 for the exponential distribution example):

http://www.dartmouth.edu/~chance/teaching_aids/books_articles/probability_book/Chapter7.pdf

That has everything you need.

 

[vela]

You don't need to know the concept of convolution to solve this problem.

You can find F_Y(y) using

$$F_Y(y) = P(Y\leq y) = \iint\limits_{Y\leq y} f(x_1,x_2)\,dx_1\,dx_2$$

First, you should be able to tell us what the joint probability density f(x₁,x₂) is equal to. Next, sketch the region that corresponds to 0≤Y≤y in the X₁X₂ plane to figure out the limits on the double integral. Then integrate to get the answer.

 

[ObliviousSage]

I guess the problem, then is that I don't understand how to get the joint density. Like I said, the back of the book has the answer, I just can't see how they get it. When I differentiate the joint distribution given, I get f(y) = ye^-y and I'm not sure where that comes from.

I don't see anything in this chapter of the book explaining how to get the joint density (though that doesn't mean it's not in there). I know my professor didn't cover it in his lectures, and he only loosely follows the book. He also doesn't look too closely at the problems he assigns out of the book to make sure they cover something he's explained.

Can anyone point me at a resource where I can read up on how to do this?

 

[vela]

The key is that the two random variables are independent. What does that mean mathematically?

 

[ObliviousSage]

I know that when two variables are independent, the product of their marginal densities is the joint density. Surely that can't be all I need to use in this case?

For one thing, that gives me a joint density of e^-y instead of the ye^-y it looks like I'm supposed to be getting. For another, while it works nicely in the case of Y = X₁ + X₂, I can think of a lot of densities and ways to combine X₁ and X₂ that would keep the product of their densities from being expressed in terms of Y.

Their independence also means their conditional densities are equal to their marginal densities (essentially another way of expressing the previous info, since the conditional density is just the joint divided by marginal). The arithmetic expectation of Y should equal the sums of the arithmetic expectations of the two Xs; useful for confirming when I've got the right answer, but not helpful for finding it. Ditto for their variance, and from there you can get skew.

The moment generating function of Y should be a product of the moment generating function of the Xs, which I suppose could help. X is an exponential distribution with $$\lambda$$ = 1. Thus it should have a moment generating function of M_x(t) = 1/(1-t). The product of two such moment generating functions is the moment generating function of Y: M_Y(t) = [1/(1-t)]². From what I can see on Wikipedia, that could be the moment generating function for a gamma distribution. That doesn't quite fit with my book's description of the gamma distribution, though (though it DOES fit with the book's tendency to use special distributions from later chapters in the chapters on general theory).

 

[vela]

That's the joint density in terms of the X's, but you need the distribution in terms of Y. Right now, you're assuming dy=dx₁dx₂, but that's not true. That's why you need to integrate to find the cdf for Y. When you differentiate that, you'll get the pdf for Y.

 

[ObliviousSage]

Hmm. It's been 10 years since I took calculus; I can do basic integrals and differentiation, but figuring out how to set up some of this more complex stuff is something I probably forgot how to do within 6 months of learning it.

What you're saying is that the joint density of X₁ and X₂ is indeed e^x₁+x₂, and I need to integrate that somehow to get Y's density? Or I need to integrate it somehow to get Y's distribution directly, which could then be differentiated to get Y's density if the problem asked for it?

Can you point me to somewhere online where I can read about this, preferably with at least one example and possibly some images? I know you're trying to prod me into figuring out the right answer on my own (I do the same when I'm tutoring people on subjects that actually make sense to me ), but I don't know that I'm going to be able to figure it out from just your hints.

I think I can make the gamma distribution shortcut work, since it's due in about 12 hours and based on your post times I don't know that you'll respond before then. I'd still like to know how I was actually supposed to do it, though, and I know I'll have to figure that out for myself since my professor never works the homework problems for us.

 

[ObliviousSage]

Looked at some double integrals of e^x1+x2, since I think what you're saying is that to get the distribution, I need to integrate the X's joint density with respect to both x₁ and x₂, with some term including y (and, in the case of the first x I integrate with respect to, possibly the other x) in the limits of integration in at least one case.

Thanks to the back of the book, I know I'm supposed to get -e^y(1+y)+C.

My first thought was to first integrate with respect to x₁ and set the limits of integration to 0 to y-x₂ but that gets me a sinh(y)-cosh(y) factor that won't disappear in the second integration, so that's probably out.

If I set the limits of the first integration to 0 to y, I get e^-x(e^-y-1), and I don't think there's any limits I can plug in that would get me what I'm supposed to get, so I'm pretty sure that's not it.

I'm pretty sure the lower limit of integration in both integrals needs to be zero (since X only takes values greater than or equal to zero), but I'm having trouble finding a workable upper limit. The fact that the joint density integrated with respect to one of the two Xs gives sinh and cosh terms isn't helping.

 

[vela]

Here, take a look at these lecture notes from Caltech, specifically page 4.

http://ee162.caltech.edu/notes/lect8.pdf

 

[ObliviousSage]

You don't need to know the concept of convolution to solve this problem.

You can find F_Y(y) using

$$F_Y(y) = P(Y\leq y) = \iint\limits_{Y\leq y} f(x_1,x_2)\,dx_1\,dx_2$$

First, you should be able to tell us what the joint probability density f(x₁,x₂) is equal to. Next, sketch the region that corresponds to 0≤Y≤y in the X₁X₂ plane to figure out the limits on the double integral. Then integrate to get the answer.

The bolded isn't making sense to me. It seems like you would need a 3rd dimension for Y (or y?) in addition to the X₁X₂ plane. While I can sort of visualize it, and I think I know which region I need, I'm not at all sure how to describe it mathematically in terms of X₁ and X₂.

 

[vela]

Looked at some double integrals of e^x1+x2, since I think what you're saying is that to get the distribution, I need to integrate the X's joint density with respect to both x₁ and x₂, with some term including y (and, in the case of the first x I integrate with respect to, possibly the other x) in the limits of integration in at least one case.

Thanks to the back of the book, I know I'm supposed to get -e^y(1+y)+C.

My first thought was to first integrate with respect to x₁ and set the limits of integration to 0 to y-x₂ but that gets me a sinh(y)-cosh(y) factor that won't disappear in the second integration, so that's probably out.

If I set the limits of the first integration to 0 to y, I get e^-x(e^-y-1), and I don't think there's any limits I can plug in that would get me what I'm supposed to get, so I'm pretty sure that's not it.

I'm pretty sure the lower limit of integration in both integrals needs to be zero (since X only takes values greater than or equal to zero), but I'm having trouble finding a workable upper limit. The fact that the joint density integrated with respect to one of the two Xs gives sinh and cosh terms isn't helping.

It sounds like you have the right idea, but you're just having problems with the execution.

 

[vela]

The bolded isn't making sense to me. It seems like you would need a 3rd dimension for Y (or y?) in addition to the X₁X₂ plane. While I can sort of visualize it, and I think I know which region I need, I'm not at all sure how to describe it mathematically in terms of X₁ and X₂.

I think you have it right already. Your limits for x₁ are from 0 to y-x₂. What are your limits for x₂, from 0 to what?

I just suggested drawing the sketch because it usually helps people see what the limits are, but you seem to have reasoned them out already. The picture is: in the x₁x₂-plane, the condition x₁+x₂<=y is represented by a half plane with the boundary being the line x₁+x₂=y. Like you said in your other post, both X's are positive, so you're confining yourself to the first quadrant. As long as y>0, you'll have a triangular region of integration, and you can see what the limits should be on the sketch.

 

[ObliviousSage]

I think you have it right already. Your limits for x₁ are from 0 to y-x₂. What are your limits for x₂, from 0 to what?

I just suggested drawing the sketch because it usually helps people see what the limits are, but you seem to have reasoned them out already. The picture is: in the x₁x₂-plane, the condition x₁+x₂<=y is represented by a half plane with the boundary being the line x₁+x₂=y. Like you said in your other post, both X's are positive, so you're confining yourself to the first quadrant. As long as y>0, you'll have a triangular region of integration, and you can see what the limits should be on the sketch.

Yeah, just worked it out again, integrating first with x₁ going from 0 to y-x₂ then with x₂ going from 0 to infinity; didn't come out right so apparently that's not it, but I feel like I'm getting closer, and I figured out what I was doing wrong that was getting me those stupid hyperbolic trig functions.

 

[vela]

If x₁+x₂<y, x₂ can't go to infinity for any finite value of y, right?

 

[ObliviousSage]

If x₁+x₂<y, x₂ can't go to infinity for any finite value of y, right?

So, integrate with respect to x₁ first, from 0 to y-x₂, then integrate with respect to x₂, from 0 to y (since it can't possibly exceed y, as x₁ will never be negative)?

Edit: Hmm, tried that, got just 1-e^-y. Still not sure how to get the ye^-y in the 1-ye^-y-e^-y I know I'm supposed to be getting...

 

[vela]

Those are the correct limits.

What do you have after integrating with respect to x₁?

 

[Stephen Tashi]

So, integrate with respect to x₁ first, from 0 to y-x₂, then integrate with respect to x₂, from 0 to y (since it can't possibly exceed y, as x₁ will never be negative)?

I vote yes.

Still not sure how to get the ye^-y

If you are doing it correctly you are integrating one term that is $e^{-y}$ with respect to $x_2$. That term is constant with respect to $x_2$, so it's antiderivative is $x_2 e^-{y}$. After that step, the limits of integration put $y$ back in for $x_2$.

 

[ObliviousSage]

OK, let's see if we can find where my rusty calculus skills betrayed me.

(going to avoid using latex to put in the integral since it seems to come out weird when I preview it)

joint density = e^-x1e^-x2

we want: integral(0<=X₂<=y) of e^-x2 times ( integral(0<=X₁<=y-x₂) of e^-x1dx₁ ) times dx₂

that resolves to: integral(0<=X₂<=y) of e^-x2 times ( -e^-t )^t=y-x2_t=0 times dx₂

which resolves to: integral(0<=X₂<=y) of e^-x2(1-e^ye^-x2)dx₂

we can distribute that out to: ( integral(0<=X₂<=y) of e^-x2dx₂ ) - ( e^y * integral(0<=X₂<=y) of e^-x2e^-x2dx₂ )

after we integrate we get: ( -e^-t )^t=y_t=0 - e^y( (-1/2)e^-2t )^t=y_t=0

which gives: 1-e^-y + (1/2)e^y(1-e^-2y)

a little quick distribution turns that into: 1 - e^-y + (1/2)(e^y-e^-y)

which finally gives us: 1 + (1/2)e^y - (3/2)e^-y

...not what we're supposed to get.

 

[vela]

You made a sign error when you plugged in the upper limit for x₁.

 

[ObliviousSage]

Thwarted by basic arithmetic! Even more depressing than being thwarted by real calculus...

Yeah, I see how it's supposed to work now. Will grind it out tomorrow AM, as it's well past time for me to escape into unconsciousness.

Thanks for all the help vela, and you as well Stephen!

 

[Stephen Tashi]

(going to avoid using latex to put in the integral since it seems to come out weird when I preview it)

Everybody has that problem! There's a bug with the forum's LaTex. You have to do the preview and then tell your browser to reload the page. You must do a similar process when you edit a post.