Optimizing Linear Estimators for Minimum Variance: How to Find the Best Weights?

[purplebird]

Given
Y(i) = u + e(i) i = 1,2,...N
such that e(i)s are statistically independent and u is a parameter
mean of e(i) = 0
and variance = \sigma(i)^2

Find W(i) such that the linear estimator

\mu = \sumW(i)X(i) for i = 1 to N

has

mean value of \mu = u

and E[(u-\mu)^2 is a minimum

 

[statdad]

There may be other ways to do this, but one of the most direct is outlined below.
<br /> \hat\mu = \sum w(i) x_i<br />
then in order for E(\hat \mu) = \mu to be true you must have

<br /> \sum w(i) = 1<br />

Next, note that E(\hat \mu - \mu)^2 is simply the \textbf{variance} of your estimate (since your estimate has expectation \mu).

Since the x_i are independent, the variance of \hat \mu is

<br /> \mathbf{Var}{\hat \mu} = \sum w(i)^2 \mathbf{Var}(x_i) = \sigma^2 \sum w(i)^2<br />

You want to choose the w(i) so that the most recent expression is minimized, subject to the constraint that \sum w(i) = 1

From here on use the method of undetermined multipliers.