- #1
fog37
- 1,569
- 108
- TL;DR Summary
- Difference between nonlinear least squares vs ordinary least squares
hello,
I understand that the method of ordinary least squares (OLS) is about finding the coefficients that minimize the sum where is the statistical model chosen to fit the data. Beside OLS, there clearly other coefficient estimation methods (MLE, etc.)
In general, OLS is fair game when the model is "linear with respect with the parameters" (linear regression, polynomial regression, etc.): any model that is the sum of several terms with each term being the product of the estimated coefficient and whatever variable: where are like the basis functions. For example, is linear and the basis functions are the three functions ...
Of course, the OLS approach is valid as long as specific assumptions on the residuals are met. Additionally, after taking the first derivative and setting them to zero, we are able to arrive to nice analytical formulas for the coefficients.
That said, what is the issue with using OLS when is a nonlinear model? I know that sometimes we "convert" a nonlinear model so that it assume the form of a linear model. That strategy then allows us to use OLS on the new model based on the transformed variables...That is a useful hack.
But I have reading about "nonlinear least squares". Isn't it the same approach as OLS but when the model is nonlinear where we directly plug the nonlinear model in ? We may not end up with analytical estimators and have to solve for the coefficients using some numerical method...But I don't see an issue apply OLS to nonlinear models...
Thank you.
I understand that the method of ordinary least squares (OLS) is about finding the coefficients that minimize the sum
In general, OLS is fair game when the model
Of course, the OLS approach is valid as long as specific assumptions on the residuals are met. Additionally, after taking the first derivative and setting them to zero, we are able to arrive to nice analytical formulas for the coefficients.
That said, what is the issue with using OLS when
But I have reading about "nonlinear least squares". Isn't it the same approach as OLS but when the model is nonlinear where we directly plug the nonlinear model
Thank you.