Linear regression and maximum likelihood estimates

In summary: Once you have all three MLEs, you can plug them back into the original equation to get the maximum likelihood estimate of the parameters a, b, and σ for the given data. In summary, to find the MLE of a, b, and σ for the given data, you will need to take the joint distribution of the random variables Yi, differentiate it with respect to each parameter, and set the resulting equations equal to zero. Solving this system of equations will give you the MLE for each parameter, which can then be used to estimate the parameters for the given data.
  • #1
stukbv
118
0

Homework Statement



Suppose that data (x1,y1),(x2,y2),.?.,(xn,yn) is modeled with xi being non random and Yi being observed values of random variables Y1,Y2,...Yn which are given by
Yi = a + b(xi-xbar) + σεi
Where a, b, σ are unknown parameters and εi are independent random variables each having the Gaussian distribution with mean 0 variance 1. xbar = 1/n * Ʃxi

Find the maximum likelihood estimate of a, b and σ



2. The attempt at a solution
Firstly I know I need to find the joint distribution of the random variables Yi
Yi is a linear combination of Gaussian random variables so Yi has a normal distribution too
E[Yi] = E[a + b(xi-xbar) + σεi] = a + b(xi-xbar)
Var[Yi] = var [a + b(xi-xbar) + σεi] = var[σεi] = σ2
All Yi's are mutually independent so the joint distribution of all the Yi's is just the product of n normally distributed random variables with means and variances as shown above.

So now we want MLE of a so we take logs of the joint distribution and differentiate wrt to a and set this equal to zero, we then rearrange to get the a= xbar is the MLE for a. Is this correct, if so I can go on to do the rest, if not why not?
Thanks
 
Physics news on Phys.org
  • #2
for your help!

Your approach is correct for finding the MLE of a. To find the MLE of b and σ, you will need to take partial derivatives with respect to b and σ and set them equal to zero. This will give you a system of equations that you can solve to find the MLE of b and σ.
 

FAQ: Linear regression and maximum likelihood estimates

What is linear regression and how is it used in scientific research?

Linear regression is a statistical method used to model the relationship between a dependent variable and one or more independent variables. It is commonly used in scientific research to analyze and predict the relationship between variables and to identify patterns and trends.

What is the maximum likelihood estimation method and why is it used in linear regression?

The maximum likelihood estimation method is a statistical approach used to estimate the parameters of a statistical model. It is used in linear regression to find the values of the model's coefficients that maximize the likelihood of the observed data. This method is preferred because it produces the most accurate and efficient estimates.

How does linear regression differ from other regression methods?

Linear regression differs from other regression methods in that it assumes a linear relationship between the dependent and independent variables. Other regression methods, such as logistic regression, allow for non-linear relationships and can handle categorical variables.

What are the assumptions of linear regression?

The main assumptions of linear regression include linearity, independence of errors, homoscedasticity (equal variance of errors), and normality of errors. These assumptions should be checked before using linear regression to ensure the validity of the results.

How can the accuracy of linear regression models be evaluated?

The accuracy of linear regression models can be evaluated by looking at measures such as the R-squared value, which indicates the percentage of variance in the dependent variable that is explained by the independent variables. Other measures, such as root mean square error and mean absolute error, can also be used to assess the model's predictive power. Additionally, cross-validation techniques can be used to test the model's performance on new data.

Similar threads

Back
Top