Proof that covariance matrix is positive semidefinite

[AlanTuring]

Hello,

i am having a hard time understanding the proof that a covariance matrix is "positive semidefinite" ...

i found a numbe of different proofs on the web, but they are all far too complicated / and/ or not enogh detailed for me.

View attachment 3290

Such as in the last anser of the link :
probability - What is the proof that covariance matrices are always semi-definite? - Mathematics Stack Exchange

(last answer, in particular i don't understad how they can passe from an expression
E{u T (x−x ¯ )(x−x ¯ ) T u}
to
E{s^2 }

... from where does thes s^2 "magically" appear ?

;)

 

[Opalg]

Hello,

i am having a hard time understanding the proof that a covariance matrix is "positive semidefinite" ...

i found a numbe of different proofs on the web, but they are all far too complicated / and/ or not enogh detailed for me.

View attachment 3290

Such as in the last anser of the link :
probability - What is the proof that covariance matrices are always semi-definite? - Mathematics Stack Exchange

(last answer, in particular i don't understad how they can passe from an expression
E{u T (x−x ¯ )(x−x ¯ ) T u}
to
E{s^2 }

... from where does thes s^2 "magically" appear ?

;)

Hi Machupicchu, and welcome to MHB!

Three basic facts about vectors and matrices: (1) if $w$ is a column vector then $w^{\mathsf{T}}w \geqslant0$; (2) for matrices $A,B$ with product $AB$, the transpose of the product is the product of the transposes in reverse order, in other words $(AB)^{\mathsf{T}} = B^{\mathsf{T}}A^{\mathsf{T}}$; (3) taking the transpose twice gets you back where you started from, $(A^{\mathsf{T}})^{\mathsf{T}} = A$.

You want to show that $v^{\mathsf{T}}Cv\geqslant0$, where \(\displaystyle C = \frac1{n-1}\sum_{i=1}^n(\mathbf{x}_i - \mathbf{\mu}) (\mathbf{x}_i - \mathbf{\mu})^{\mathsf{T}}\). Since \(\displaystyle v^{\mathsf{T}}Cv = \frac1{n-1}\sum_{i=1}^nv^{\mathsf{T}}(\mathbf{x}_i - \mathbf{\mu}) (\mathbf{x}_i - \mathbf{\mu})^{\mathsf{T}}v\), it will be enough to show that $v^{\mathsf{T}}(\mathbf{x}_i - \mathbf{\mu}) (\mathbf{x}_i - \mathbf{\mu})^{\mathsf{T}}v$ (for each $i$). But by those basic facts above, $v^{\mathsf{T}}(\mathbf{x}_i - \mathbf{\mu}) (\mathbf{x}_i - \mathbf{\mu})^{\mathsf{T}}v = \bigl((\mathbf{x}_i - \mathbf{\mu})^{\mathsf{T}}v\bigr)^{\mathsf{T}} \bigl((\mathbf{x}_i - \mathbf{\mu})^{\mathsf{T}}v\bigr) \geqslant0$.