How to Prove This Identity in the Classical Normal Linear Regression Model?

[Usagi]

The context of the following identity is in the Classical Normal Linear Regression Model, ie, $\boldsymbol{y} = \boldsymbol{X}\boldsymbol{\beta}+ \boldsymbol{u}$ where $\boldsymbol{u}$ is a $n \times 1$ matrix and $u_i \sim iid.N(0, \sigma^2)$ for $i = 1, 2, \cdots, n$

Show that $(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta})'(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta}) = (\boldsymbol{y}-\boldsymbol{X}\boldsymbol{b})'(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{b})+(\boldsymbol{\beta}-\boldsymbol{b})'\boldsymbol{X}'\boldsymbol{X}(\boldsymbol{\beta}-\boldsymbol{b}) \ \cdots (1)$

where:

$\boldsymbol{y}$ is a $n \times 1$ matrix

$\boldsymbol{X}$ is a $n \times k$ matrix

$\boldsymbol{\beta}$ is a $k \times 1$ matrix

$\boldsymbol{b}$ is a $k \times 1$ matrix

$rank(\boldsymbol{X}) = k$

$\boldsymbol{b} = (\boldsymbol{X}'\boldsymbol{X})^{-1}\boldsymbol{X}'\boldsymbol{y}$ ----------

**Question:** How do I algebraically manipulate the LHS of $(1)$ to become the RHS?

 

[jvicens]

it is important to understand the mathematical concepts and principles behind any equations or models that we use. In this case, we are dealing with the Classical Normal Linear Regression Model, where we have a dependent variable $\boldsymbol{y}$ that is linearly related to a set of independent variables represented by the matrix $\boldsymbol{X}$. The error term $\boldsymbol{u}$ represents the random variation in the dependent variable and is assumed to follow a normal distribution.

To algebraically manipulate the LHS of $(1)$ to become the RHS, we can use the properties of matrix multiplication and transpose. First, let's expand the LHS of $(1)$:

$(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta})'(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta}) = \boldsymbol{y}'\boldsymbol{y} - \boldsymbol{y}'\boldsymbol{X}\boldsymbol{\beta} - \boldsymbol{\beta}'\boldsymbol{X}'\boldsymbol{y} + \boldsymbol{\beta}'\boldsymbol{X}'\boldsymbol{X}\boldsymbol{\beta}$

Next, let's expand the RHS of $(1)$:

$(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{b})'(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{b})+(\boldsymbol{\beta}-\boldsymbol{b})'\boldsymbol{X}'\boldsymbol{X}(\boldsymbol{\beta}-\boldsymbol{b})$

$= \boldsymbol{y}'\boldsymbol{y} - \boldsymbol{y}'\boldsymbol{X}\boldsymbol{b} - \boldsymbol{b}'\boldsymbol{X}'\boldsymbol{y} + \boldsymbol{b}'\boldsymbol{X}'\boldsymbol{X}\boldsymbol{b} + \boldsymbol{\beta}'\boldsymbol{X}'\boldsymbol{X}\boldsymbol{\beta} - \boldsymbol{\beta}'\boldsymbol{X}'\boldsymbol{X}\boldsymbol{b} - \boldsymbol{b}'\boldsymbol{X}'\boldsymbol{X}\boldsymbol{\beta} + \boldsymbol{b}'\boldsymbol{X}'\boldsymbol{X