Probability density function of Z=X+Y and Z=XY

[docnet]

Homework Statement

Let $X$ and $Y$ be independent random variables with probability densities $f_X(x)$ and $f_Y(x)$ respectively. Find the probability density function for the random variables $Z_1=X+Y$ and $Z_2=XY$ in terms of $f_X(x)$ and $f_Y(x).$

Relevant Equations

.

I assume since $X$ and $Y$ are independent and their probability density functions are functions of $x$
$$f_{Z_1}=\frac{f_X(x)+f_Y(x)}{2}$$
and
$$f_{Z_2}=\frac{f_X(x)\cdot f_Y(x)}{\int_{\mathbb{R}}(f_X(x)\cdot f_Y(x))dx}.$$
The divisions occur because it must be that
$$\int_{\mathbb{R}}f_Z(x)dx=1.$$
Is this a correct assumption to make? I saw examples of similar problems where double integrals were involved, but I am not sure if they apply here.

 

[FactChecker]

I assume since $X$ and $Y$ are independent and their probability density functions are functions of $x$
$$f_{Z_1}=\frac{f_X(x)+f_Y(x)}{2}$$

You are approaching this wrong. You should not say that the values of $x$ and $y$ are identical. For any given value of $Z_1 = z$, there are a lot of combinations of $x$ and $y$ that give $z=x+y$. You will have to integrate all those combinations.

 

[Delta2]

I am not sure if @FactChecker is completely right but he certainly points to an issue that worth further discussion.
For the moment lets say that X,Y are positive integer (including zero as possible value) variables and Z=X+Y. Then for example what is $P(Z=2)$? It seems to me that $$P(Z=2)=P(X=1,Y=1)+P(X=2,Y=0)+P(X=0,Y=2)$$

 

[Delta2]

I can tell you what I get in terms of discrete variables for the case of $Z=X+Y$:

$$P_Z(z)=\sum_{n} P_X(n)P_Y(z-n)+\sum_{n}P_X(z-n)P_Y(n)-P_X(\frac{z}{2})P_Y(\frac{z}{2})$$. Try to generalize this in the case of continuous variables and probability density functions.

 

[FactChecker]

I can tell you what I get in terms of discrete variables for the case of $Z=X+Y$:

$$P_Z(z)=\sum_{n} P_X(n)P_Y(z-n)+\sum_{n}P_X(z-n)P_Y(n)-P_X(\frac{z}{2})P_Y(\frac{z}{2})$$. Try to generalize this in the case of continuous variables and probability density functions.

I think that your two summations are identical, just in a different order. And I do not understand the last subtracted term.
IMO, all that is needed is your first summation.

 

[Delta2]

Well I think you are right here :D. I thought i double count only when n=z/2, but i double counted everything...

I ll leave the post as it is cause I think the OP might get some extra insight from that post and your correction.

 

[docnet]

Thank you for catching me on my mistake! I did some reading on Wikipedia and found the answers. Although, I could not find a derivation for the density function of $Z=X+Y$.

Per Hogg, the probability distribution of $Z=X+Y$ is given by the convolution
$$f_{Z_1}=\int^\infty_{-\infty}f_X(x)f_Y(z-x)dx.$$

The probability distribution of $Z=XY$ is derived by Springer et al., from the cumulative distribution function of $Z_2$.
\begin{align*}
F_{Z_2} &=\mathbb{P}(Z\leq z)\\
&=\mathbb{P}(XY\leq z)\\
&=\mathbb{P}(XY\leq z, X\geq 0)+\mathbb{P}(XY\leq z, X\leq 0)\\
&=\mathbb{P}(Y\leq z/X,X\geq 0)+\mathbb{P}(Y\geq z/X,X\leq 0)\\
&=\int^{\infty}_0f_X(x)\int^{z/x}_{-\infty}f_Y(y)dydx+\int^{0}_{-\infty}f_X(x)\int^{\infty}_{z/x}f_Y(y)dydx.
\end{align*}
Both sides are operated with $\frac{d}{dz}$.
\begin{align*}
f_{Z_2}(z)&=\int^{\infty}_0f_X(x)f_Y(z/x)\frac{1}{x}dx+\int^{0}_{-\infty}f_X(x)f_Y(z/x)\frac{1}{x}dx\\
&=\int_{-\infty}^{\infty}f_X(x)f_Y(z/x)\frac{1}{|x|}dx.
\end{align*}

$Z=XY$
$Z=X+Y$

 

[vela]

Thank you for catching me on my mistake! I did some reading on Wikipedia and found the answers. Although, I could not find a derivation for the density function of Z=X+Y.

You could follow the technique used in Springer to find $P(Z \le z)$.

In the $(X,Y)$ plane, integrate $f_X f_Y$ over the region $Z = X+Y \le z$ to find $P(Z \le z)$, and then differentiate the result with respect to $z$.

 

[FactChecker]

I could not find a derivation for the density function of $Z=X+Y$.

Per Hogg, the probability distribution of $Z=X+Y$ is given by the convolution
$$f_{Z_1}=\int^\infty_{-\infty}f_X(x)f_Y(z-x)dx.$$

I'm not sure that there is anything to derive except to see that it is what you want because setting $y=z-x$ gives you $z=x+y$. This is something that you may see often.

 

[WWGD]

IIRC, we use convolution or MGFs to compute the pdf of independent RVs too.

 

[Delta2]

Using infinitesimal logic to prove the Hogg result:
By definition for a continuous random variable W, it is $$P(w<W<w+dw)=f(w)dw (1)$$

Consider $$Z=X+Y$$

Then $$f_Z(z)dz=^{(1)}P(z<Z<z+dz)=P(z<X+Y<z+dz)=$$
$$=P(-\infty<X<+\infty , z-X<Y<z-X+dz)=$$
$$=\int_{-\infty}^{\infty}f_X(x)f_Y(z-x)dx dz(2)$$ and hence (dividing both sides of (2) with $dz$ $$f_Z(z)=\int_{-\infty}^{+\infty}f_X(x)f_Y(z-x)dx$$

I know the last equality step in (2) might not be so clear, hold on while I try to explain it.

Also the last equality in (2) is not a double integral it is rather $$g(z)dz$$ where $$g(z)=\int_{-\infty}^{+\infty} f_X(x)f_Y(z-x)dx$$

 

[Delta2]

I have to write a separate post about the last step in (2) of #11 cause the explanation is not so simple.

We can write the "event" A that $$A=-\infty<X<+\infty,(z-X)<Y<(z-X)+dz$$ as $$A=\bigcup_{x,dx}(x<X<x+dx,z-X<Y<(z-X)+dz)$$

hence $$P(A)=\sum_{x,dx} P(x<X<x+dx)P(z-X<Y<z-X+dz)=\sum_{x,dx}f_X(x)dxf_Y(z-x)dz$$ where by (1) of post #11 we have that $$P(x<X<x+dx)=f_X(x)dx$$ and $$P(z-X<Y<z-X+dz)=f_Y(z-x)dz (*)$$.

I ll leave it as a vague qualitative intuitive step that the $$\sum_{x,dx}$$ is the same as $$\int_{-\infty}^{+\infty} (...) dx$$

(*) this needs some explanation too, I ll try to do my best later.

 

[Delta2]

You could follow the technique used in Springer to find $P(Z \le z)$.

In the $(X,Y)$ plane, integrate $f_X f_Y$ over the region $Z = X+Y \le z$ to find $P(Z \le z)$, and then differentiate the result with respect to $z$.

In the Springer derivation in post #7 I find the last step equality (of the first paragraph, where we pass from the probabilities P to integrals) a bit vague, and also a bit vague your suggestion here that we have to integrate (double integral or single integral?) the product (why the product???) $f_Xf_Y$

 

[docnet]

Though I appreciate your eagerness to help, I thought I am the one who is supposed to solve the problem .
It is okay, but if I wrote down Delta2's solution, I don't think the TA's would believe it was my own work.Here is what I was able to find.The probability distribution of $Z=X+Y$ is derived from $F_{Z_1}$.
\begin{align*}
F_{Z_1}&=\mathbb{P}(Z\leq z)\\
&=\mathbb{P}(X+Y\leq z)\\
&=\int^\infty_{-\infty}f_X(x)\Big[\int^{z-x}_{-\infty}f_Y(y)dy\Big]dx.
\end{align*}
\begin{align*}
f_{Z_1}(z)&=\frac{d}{dz}F_{Z_1}\\
&=\int^\infty_{-\infty}f_X(x)\Big[\frac{d}{dz}\int^{z-x}_{-\infty}f_Y(y)dy\Big]dx\\
&=\int^\infty_{-\infty}f_X(x)f_Y(z-x)dx.
\end{align*}

Link to source

 

[docnet]

In the Springer derivation in post #7 I find the last step equality (of the first paragraph, where we pass from the probabilities P to integrals) a bit vague, and also a bit vague your suggestion here that we have to integrate (double integral or single integral?) the product (why the product???) $f_Xf_Y$

I am wondering that as well. I wonder if we are evaluating the product because $\mathbb{P}(A\cap B)=\mathbb{P}(A)\mathbb{P}(B)$ given they are independent variables.

 

[vela]

I am wondering that as well. I wonder if we are evaluating the product because $\mathbb{P}(A\cap B)=\mathbb{P}(A)\mathbb{P}(B)$ given they are independent variables.

Exactly

 

[Delta2]

I am wondering that as well. I wonder if we are evaluating the product because $\mathbb{P}(A\cap B)=\mathbb{P}(A)\mathbb{P}(B)$ given they are independent variables.

Ehm then when we pass from probabilities to integrals , the integrals shouldn't be convoluted, but something like $$\int f_X(x)dx\int f_Y(z-x)dx$$. I think my posts give an insight on why we have convolution, though I understand that I am not very rigorous in posts #11,#12

 

[pasmith]

Ehm then when we pass from probabilities to integrals , the integrals shouldn't be convoluted, but something like $$\int f_X(x)dx\int f_Y(z-x)dx$$. I think my posts give an insight on why we have convolution, though I understand that I am not very rigorous in posts #11,#12

You are calculating the area, weighted by the joint PDF $f(x,y)$, of a region of $\mathbb{R}^2$. That requires a double integral. Since in this case we are dealing with functions of two independent random variables, the joint PDF is the product of the individual PDFs: $$\begin{split} P((X \in [x,x+\delta x)) \wedge (Y \in [y,y+\delta y))) &= P(X \in [x,x+\delta x))P(Y \in [y,y+\delta y))\\ &= \left(\int_x^{x+\delta x} f_X(x)\,dx\right)\left( \int_y^{y+\delta y} f_Y(y)\,dy\right) \\ &= \int_x^{x +\delta x} \int_y^{y + \delta y} f_X(x)f_Y(y)\,dx\,dy. \end{split}$$ Then $P(X + Y \leq z)$ is found by integrating this joint PDF over the region $x + y \leq z$ which leads to the change of variable $x = v, y = u - v$ so that $$P(X + Y \leq z) = \int_{-\infty}^z \left(\int_{-\infty}^\infty f_X(v)f_Y(u - v)\,dv\right)\,du.$$ This depends on $z$ only through the upper limit of the $u$ integral, so $$f_{X + Y}(z) = \int_{-\infty}^\infty f_X(v)f_Y(z - v)\,dv.$$

 

[Delta2]

@pasmith you are silently using Fubini's Theorem (While I do not) (and the Fundamental theorem of Calculus but ok cant blame you for that e hehe) but not sure if fubini's applies for the Pdf's. But if it does apply you are for sure more rigorous than me.

Well done.

 

[pasmith]

@pasmith you are silently using Fubini's Theorem (While I do not) (and the Fundamental theorem of Calculus but ok cant blame you for that e hehe) but not sure if fubini's applies for the Pdf's. But if it does apply you are for sure more rigorous than me.

Fubini's theorem does appeaar to apply: https://en.wikipedia.org/wiki/Fubini's_theorem#For_complete_measures.