Variance of ith deleted residual

[dragonoid122]

From linear model, $y = X\beta + \epsilon$, if $\hat{\sigma}^2 = \frac{|y - Xb|^2}{n-p}$ is the variance of error and $\hat{\sigma}_{-i}^2 = \frac{|y_{-i} - X_{-i}b_{-i}|^2}{n-p-1}$ is the estimate of the error variance σ obtained by fitting all the
observations except the i-th. Show that $\hat{\sigma}_{-i}^2 = \frac{(n - p)\hat{\sigma}^2 - e_i^2/(1-H_{i,i})}{n-p-1}$ where $e_i = y_i - \hat{y_i}$ and $H_{i,i} = x_i(X^TX)^{-1}x_i^T$ is the hat matrix.

ANy hints will be help ful.

 

[Valkarie]

Solution:Let $\hat{\beta}_{-i} = (X_{-i}^TX_{-i})^{-1}X_{-i}^Ty_{-i}$ be the estimate of the regression coefficients obtained by fitting all the observations except the $i^{th}$. Then, we have $\hat{\sigma}_{-i}^2 = \frac{|y_{-i} - X_{-i}\hat{\beta}_{-i}|^2}{n-p-1}$Substituting $y_i$ for $y_{-i}$ and $X_i$ for $X_{-i}$ in the above equation, we get$\hat{\sigma}_{-i}^2 = \frac{|y_i - X_i\hat{\beta}_{-i}|^2}{n-p-1}$Using the relationship $\hat{\beta}_{-i} = \hat{\beta} + (X^TX)^{-1}x_i(e_i - x_i^T\hat{\beta})$, we get$\hat{\sigma}_{-i}^2 = \frac{|y_i - X_i(\hat{\beta} + (X^TX)^{-1}x_i(e_i - x_i^T\hat{\beta}))|^2}{n-p-1}$Simplifying,$\hat{\sigma}_{-i}^2 = \frac{|e_i - x_i^T\hat{\beta} - x_i^T(X^TX)^{-1}x_i(e_i - x_i^T\hat{\beta})|^2}{n-p-1}$Rearranging,$\hat{\sigma}_{-i}^2 = \frac{(n - p)\hat{\sigma}^2 - e_i^2/(1-H_{i,i})}{n-p-1}$