Linear regression Definition and 118 Threads

In statistics, linear regression is a linear approach to modelling the relationship between a scalar response and one or more explanatory variables (also known as dependent and independent variables). The case of one explanatory variable is called simple linear regression; for more than one, the process is called multiple linear regression. This term is distinct from multivariate linear regression, where multiple correlated dependent variables are predicted, rather than a single scalar variable.In linear regression, the relationships are modeled using linear predictor functions whose unknown model parameters are estimated from the data. Such models are called linear models. Most commonly, the conditional mean of the response given the values of the explanatory variables (or predictors) is assumed to be an affine function of those values; less commonly, the conditional median or some other quantile is used. Like all forms of regression analysis, linear regression focuses on the conditional probability distribution of the response given the values of the predictors, rather than on the joint probability distribution of all of these variables, which is the domain of multivariate analysis.
Linear regression was the first type of regression analysis to be studied rigorously, and to be used extensively in practical applications. This is because models which depend linearly on their unknown parameters are easier to fit than models which are non-linearly related to their parameters and because the statistical properties of the resulting estimators are easier to determine.
Linear regression has many practical uses. Most applications fall into one of the following two broad categories:

If the goal is prediction, forecasting, or error reduction, linear regression can be used to fit a predictive model to an observed data set of values of the response and explanatory variables. After developing such a model, if additional values of the explanatory variables are collected without an accompanying response value, the fitted model can be used to make a prediction of the response.
If the goal is to explain variation in the response variable that can be attributed to variation in the explanatory variables, linear regression analysis can be applied to quantify the strength of the relationship between the response and the explanatory variables, and in particular to determine whether some explanatory variables may have no linear relationship with the response at all, or to identify which subsets of explanatory variables may contain redundant information about the response.Linear regression models are often fitted using the least squares approach, but they may also be fitted in other ways, such as by minimizing the "lack of fit" in some other norm (as with least absolute deviations regression), or by minimizing a penalized version of the least squares cost function as in ridge regression (L2-norm penalty) and lasso (L1-norm penalty). Conversely, the least squares approach can be used to fit models that are not linear models. Thus, although the terms "least squares" and "linear model" are closely linked, they are not synonymous.

View More On Wikipedia.org
  1. I

    MHB Statistsics Mathematics Problem: Linear Regression

    Find the equation of the regression line for the given data. then construct A SCATTER PLOT of the data and draw the regression line. (each pair of variables has a significant correlation.) then use the regression equation to predict the value of y for each of the given x- values, if meaningful...
  2. 9

    Statistics: given total sum of squares, find R²

    Homework Statement Given: Σ(xi - x̄)² = 500 Σ(yi - ybar)² = 800 (total sum of squares, SST)) Σ(ŷ - ybar)² = 400 (total sum of estimators, SSE) Σ(xi - x̄)²(yi) = 200 Σ(xi - x̄)²(εi) = 0 n = 1000 s² = 4 Find (or explain why you cannot find): β1 β0 variance of β R² Homework Equations [/B]...
  3. R

    Linear regression, sources for this wikipedia link

    I really like the derivations here http://en.wikipedia.org/wiki/Proofs_involving_ordinary_least_squares Could some one recommend a good book for them. I'm tired of googling these equations every time I want to use them. Thanks!
  4. gfd43tg

    Linear regression with least squares for quadratic function

    Homework Statement We want to determine the coefficients of a polynomial of the form: ##p(x)=c_{1}x^2 +c_{2}x+c_{3}##The polynomial ##p(x)## must satisfy the constraint ##p(1)=1##. We would also like ##p(x)## to satisfy the following 4 constraints: ##p(−1)=5## ##p(0)=−1## ##p(2)=6##...
  5. C

    Linear Regression Error on Excel

    This is for an experimental physics homework,I am using the latest version of MS Excel. I have a set of data, I perform linear regression on them and it gives me a line y=ax + b. Given that both a and b have physical significance I would like to know how could I know the uncertainty...
  6. X

    When y is negative in linear regression?

    I am using linear regression to predict 'y' based on 8 variables. With my example, most the Betas that I got are negative. So, y, the value to predict, is negative. To my data, y is a time in seconds, so I think it shouldn't be negative. I my example in python, and I want to know if y...
  7. P

    Multiple Linear Regression - Hypothesis Testing

    Homework Statement I'm looking through some example problems that my professor posted and this bit doesn't make sense How do you come up with the values underlined? Homework Equations The Attempt at a Solution Upon researching it, I find that you should use α/2 for both...
  8. M

    Estimating measurement error using error from linear regression

    Sorry if I'm in the wrong subforum. This is a rather simple and straightforward question, I hope. I'm doing a measurement that requires me to do a linear regression on data points to get a value of the slope. The slope is the value of the actual property that I am measuring. Assuming...
  9. X

    Linear regression with the same X value

    In a linear regression with 1 independent variable, if X is always the same (let's call I am unlucky), but Y present different values for the same X, I still can find the coefficient of the straight line equation?
  10. X

    Find the error in a linear regression

    Hi, I am trying to understand how I find the error in linear regression, and what to do with it. I am using linear regression to predict the time of execution based on the size of the input and the number of tasks used in the computer to get the result. 1 - In a linear regression, I calculate...
  11. X

    How Do Linear Regression and R-Squared Differ?

    I am trying to understand how linear regression and R-squared differ. 1 - Can anyone give me an example of use of linear regression and R-squared? 2 - They have some relation between them? E.g., they are useful for each other? 3 - What are the dangers when analysing the linear regression...
  12. J

    Least squares estimator and distribution (simple linear regression)

    Homework Statement Under the simple linear regression model Y= A + Bx + e, where A is the intercept (a known concept), B is the slope parameter (unknown) and e is a random error term satisfying the normality assumption. If (X1,Y1)...(Xn,Yn) are the n data points observed, find the least squares...
  13. D

    Linear regression. How to calculate this problem

    Homework Statement Why does excel give me this: http://postimg.org/image/68b9z1lqt/ And various online calulators (for example http://www.alcula.com/calculators/statistics/linear-regression/), and my own calulations give me this: http://postimg.org/image/kpljm4awx/ Homework...
  14. Z

    Coding Variables for linear regression

    Homework Statement I have to design an experiment with 3 factors. One factor has to be quantitative with at least 3 levels. One Qualitative with at least 3 levels. And the last one can be either quant/qual with at least 2 levels. My question is in regards to coding the variables. For example...
  15. Z

    Multiple linear Regression Expreiment

    Homework Statement Hi, I need to create an experiment for my regression class and I would appreciate some ideas which would allow me to generate the data with minimal resources (preferably something on the computer where I can get data instantly) The main criteria for the experiment...
  16. N

    Linear Regression in Polar Space

    I have posted this question before but I don't think I was clear on what i was trying to do exactly. I am trying to simulate a set of muon detecting drift tubes in 2d space. I have 2 sets of detector tubes (shown as black circles in the image), a particle trajectory goes through all tubes...
  17. N

    Linear regression to radii of multiple circles

    Hi, I am trying to simulate muon paths through drift tubes and I have ran into a problem performing a linear regression. I have generated simulated muon trajectories in 2 dimensions and they passes through my simulated drift tubes represented as black circles with a '+' in the center. As the...
  18. O

    Linear Regression: LineFit Method Explained

    I wasn't sure where to put this question. Can anyone tell me what method LineFit uses to perform linear regression with error in both coordinates? Thank you.
  19. iVenky

    Are Equations for Linear Regression Right?

    I read about "Linear regression" and I want to make sure that what I read is right Just tell if these equations are right- Slope of line of regression for y on x is given by m=\frac{E(XY)-E(X)E(Y)}{E(X^{2})-[E(X)]^{2}} \\ m=\frac{Cov(XY)}{Var(X)} \\ m=\frac{ρσ_{x}σ_{y}}{σ_{x}^{2}} \\...
  20. S

    Statistics question: error of slope in linear regression from r

    A text says that if you calculate the linear regression of data points and you get the equation y=mx+b with an r2 value, the error in the slope is given by: δm/m=2(1-r) No explanation was given. Could someone please explain this formula? Thanks!
  21. C

    Effortlessly Linearize y(x)= a(1-e-bx) with Expert Help

    Hello! how I linearize this function? y(x)= a(1-e-bx) a and b are constants
  22. S

    Linear Regression β: Estimating η with MLEs

    βHomework Statement Data y1,y2...yn are modeled as observations of random variables Y1,..Yn given by Yi = α + β(xi-xbar) + σεi Where α , β and σ are unknown parameters x1,x2...xn are known constants and xbar is (1/n)Ʃxi and εi's are independent random variables each with the...
  23. T

    Physics Experiment Linear Regression Issue

    Homework Statement Hello, I have done a laboratorial experiment (electron diffraction) and I've been doing the analysis of the obtained data. I have plotted the data obtained experimentally, and the slope of the obtained linear regression should give a certain value. What's happening...
  24. J

    Question about propagation error and linear regression?

    I have couple questions about this and I was hoping someone with some stats knowledge could clarify. First, when people report numbers such as 10 plus or minus 5, what does the 5 mean? Is it the standard deviation or the confidence interval or the variance? What is the relationship between...
  25. D

    Intermediate Physics Lab Analysis, Uncertainty and Linear Regression

    Homework Statement "You are asked to do an experiment where you will need to use a rotating blade to measure the wind speed. You measure the number of rotations of the blade at 10 different wind speeds, 10 times each and will make a linear fit to determine the wind speed as a function of...
  26. S

    Linear regression and maximum likelihood estimates

    Homework Statement Suppose that data (x1,y1),(x2,y2),.?.,(xn,yn) is modeled with xi being non random and Yi being observed values of random variables Y1,Y2,...Yn which are given by Yi = a + b(xi-xbar) + σεi Where a, b, σ are unknown parameters and εi are independent random variables each...
  27. B

    Linear Regression: Pros and cons of Normal vs. simplified methods?

    I'm currently looking at a linear regression handout from Uni and there are two methods to calculate the equation. The Normal one is to find a and b for y=a+bx, the equations for a and b are given in the handout but I'll assume you're familiar with them. The simplified one is using y = Bx +...
  28. W

    Linear Regression of estimated measures / outliers

    Hi all, I would like to understand the theory for determining outliers in the following scenario. Let's say I am to fit a linear model to the data of house size v. sale price for a particular location. And let's say I have a fairly good linear relationship, as house size increases, so does...
  29. C

    Linear regression and bivariate normal, is there a relationship?

    Hi everyone, This is not a homework question. I just want to understand an aspect of linear regression better. The book "Applied Linear Models" by Kutchner et al, states that a linear regression model is of the form Y_i = B_0 + B_1 X_i + \epsilon_i where Y_i is the value of the...
  30. S

    Analytical linear regression: is it possible?

    I've been told that their exists no perfect mathematical method of obtaining a line of best fit from a population of data. This doesn't make a whole lot of sense to me, so I have made an attempt at doing such (see google docs link)...
  31. A

    Find k in y=kx: linear regression or just average of (y/x)?

    Dear all, let's say I want to know the elasticity constant of a spring (k), so I measure several times different values for the force applied to the spring, F, and the displacement of the spring, x. So, for N measures, I have xi and Fi and their uncertainties. Now, I'm really not an expert of...
  32. H

    Multiple Linear Regression Analysis

    Hi, I've asked this question on another forum, but no response until now. Maybe I will have a little bit of luck here. So .. I have a problem. I have a set of 8 parameter and I use this parameters in order to compute a measure (I vary each parameter with a step of 50%). I would like to know...
  33. F

    Linear regression, error in both variables

    Hi y'all, wondering if you could help me with this. I have a data set with a linear relationship between the independent and dependent variables. Both the depended and independent variables have error due to measurement and this error is not constant. For example, {x1, x2, x3, x4, x5}...
  34. T

    Linear regression with asymmetric error bars

    I've been trying to figure out how to do a linear regression on data with asymmetric x and y error bars (different for each data point). Any help would be much appreciated.
  35. T

    Stats: Simple Linear Regression

    Homework Statement [PLAIN]http://img822.imageshack.us/img822/4421/statsii.jpg The Attempt at a Solution Done parts (a) and (b). How do I do parts (c) and (d)? Is the simple linear regression model just Y_i=\beta_0+\beta_1 X_i + \varepsilon_i where \varepsilon_i \stackrel...
  36. brainpushups

    Uncertainty in the parameters A and B for a linear regression

    I'm working through John Taylor's An Introduction to Error Analysis and so far this is the only problem I haven't been able to solve. I was hoping someone could lend me some insight. The problem asks you to use error propagation to verify that the uncertainties in A and B for a line of the...
  37. T

    How Do You Derive the Least Squares Solution in Linear Regression?

    In the least square linear regression, say we have y=Xb+e (y,b,e are vector and X is matrix, y is observations, b is coefficient, e is error term) so we need to minimize e'e=(y-Xb)'(y-Xb)=y'y-y'Xb-b'X'y+b'X'Xb we can take the derivative of e'e to b, and we can get the answer is 0-2X'y+2X'Xb...
  38. E

    Multiple linear regression + QQplots problem Includes pics

    I want to do multiple linear regression, but one of the requirements is the residuals to be normally distributed, and I can check that with QQplots but then the QQ plot shows it is about 95% of data fit into the normal line, but 5% is way off! can I still proceed ?*or do I have to find a way...
  39. C

    Multiple Linear Regression (2 factors, 1 output)

    Hello all; I'm doing work for my job, and I've forgotten my statistics =(. I first want to know if what I'm trying to do is possible. I want to create a linear regression of the form Y = a * x1 + b * x2 + c. http://imgur.com/Q4vGP" As you can see, there is space that is grayed...
  40. majormuss

    Finding Y1 on the TI-83 Plus Calculator for Linear Regression

    Homework Statement Where can I find Y(subscript: 1) on the TI-83 plus calculator? I was working on Linear Regressions in the book and I came across a question that requires me to input Y1 on the screen but I can't find the 'y'.. how can I find it? Homework Equations The Attempt at...
  41. B

    Linear regression and high correlation problems

    Hi guys, I have data of 20 peoples height, weight, calorie intake and skinfold thickness. I have carried out a regression of calorie on height, on weight and on height and weight. I have done the same thing for skinfold thickness. I then used R to work out the summary of results. each...
  42. T

    Testing Hypotheses in Multivariate Linear Regression Using SAS?

    I have a model y= beta0 + beta1 x1 + beta2 x2 + eps, eps~N(0,1). How to test hypothesis beta1=0 ? Is the same test for beta2=0?
  43. maverick280857

    Coefficient of Determination in case of repeat points, in linear regression

    Hello, In simple linear regression (or even in multiple linear regression) how does one prove that the coefficient of determination, given by R^2 = \frac{SS_{Reg}}{SS_{Total}} = 1-\frac{SS_{Res}}{SS_{Total}}= 1-\frac{\sum_{i=1}^{n}(y_i-\hat{y}_i)^2}{\sum_{i=1}^{n}(y_i-\overline{y})^2} is...
  44. L

    Propagating Measurement Uncertainty into a Linear Regression Model

    I am trying to figure out how to combine uncertainty (in x and y) into the standard error of the best fit line from the linear regression for that dataset. I am plotting units of concentration (x) versus del t/height (y) to get a value for the flux (which is the slope) I understand how...
  45. S

    Non Linear Regression Initial Guesses

    Hi: This is my first post and I'm not sure if this is the right forum. Please redirect if necessary. I'm new to nonlinear regression, but from what I've read I realize that making "good" initial guesses for the model parameters is very important, otherwise a "best fit" may not result...
  46. N

    F90/C Weighted Linear Regression Code

    Hi Guys, Can anyone recommend a code, preferably in fortran 90/77, or possibly in C, which provides a weighted linear regression of a 3 column file? Natski
  47. M

    How Does Firm Age Influence Growth When Evaluated at Mean Values?

    Homework Statement Evaluate the partial effect of age of a firm on growth.(evaluated at the means) Homework Equations Growth=\beta0+\beta1age+\beta2age^2+\beta3size*age+\beta4plant*age The Attempt at a Solution We're supposed to do something like this...
  48. K

    Sum of the residuals in multiple linear regression

    In my textbook, the following results are proved in the context of SIMPLE linear regression: ∑e_i = 0 ∑(e_i)(Y_i hat)= 0 I tried to modify the proofs to mutliple linear regression, but I am unable to do so, so I am puzzled... Are these results also true in MULTIPLE linear regression...
  49. K

    Regression SS in multiple linear regression

    In MULTIPLE linear regression, is it still true that the regression sum of squares is equal to ∑ (Y_i hat -Y bar)^2 ? My textbook defines regression SS in the chapters for simple linear regression as ∑ (Y_i hat -Y bar)^2, and then in the chapters for multiple linear regression, the...
  50. K

    Multiple linear regression: partial F-test

    "Suppose that in a MULTIPLE linear regression analysis, it is of interest to compare a model with 3 independent variables to a model with the same response varaible and these same 3 independent variables plus 2 additional independent variables. As more predictors are added to the model, the...
Back
Top