You are here:

Probability & Statistics/How to do OLS when y is not linear?

Advertisement


Question
Hi Clyde,

So far, all the questions I have done on Ordinary Least Squares (OLS) regression have assumed that the population can be modelled as:

y_i = a + bx_i + e_i

where a and b are constants and e_i is the error term with mean 0.

Now I am wondering how I would do OLS if y is not a linear function of x_i? For example:

y_i = a + sin(x_i) + e_i

a can be estimated as usual but it is 'b' that I do not know how to estimate using OLS. Do I differentiate the sum of the squared error terms with respect to sin (x)? I tried doing that I got some really complicated expression so I decided to seek help before going further.

Could you please provide to me some worked out examples on how to perform OLS estimate in such cases?

Answer
It might be useful if you included more parameters in your regression:

y_i = a + b sin(c x_i + d) + e_i

Regardless of your choice, you can compute the values of a, b, c, d or any other parameters that minimize the sum-of-squares of the e_i. The methodology is the same, but you will have more complicated numbers / expressions. There's no way to avoid that, since you are using the sine function. You can use a variety of numerical and symbolic methods to find the optimum values of (a,b,c,d). You haven't really explained what level of work you can do so I don't know how to give you any more information. If you are at the level of introductory statistics, the calculus or numeral analysis required to do this type of regression will be unknown to you.

For a population you should also consider an exponential or logistic model, which will give you solutions that are neither linear nor sinusoidal, like:

y_i = a e^(r x_i) + e_i

or

y_i = a / ( b + e^(r x_i) ) + e_i

There is a general methodology for finding this even without knowing that it is sine or any other function: https://en.wikipedia.org/wiki/Non-linear_least_squares

However that is more advanced than what you seem to be asking.

Probability & Statistics

All Answers


Answers by Expert:


Ask Experts

Volunteer


Clyde Oliver

Expertise

I can answer all questions up to, and including, graduate level mathematics. I do not have expertise in statistics (I can answer questions about the mathematical foundations of statistics). I am very much proficient in probability. I am not inclined to answer questions that appear to be homework, nor questions that are not meaningful or advanced in any way.

Experience

I am a PhD educated mathematician working in research at a major university.

Organizations
AMS

Publications
Various research journals of mathematics. Various talks & presentations (some short, some long), about either interesting classical material or about research work.

Education/Credentials
BA mathematics & physics, PhD mathematics from a top 20 US school.

Awards and Honors
Various honors related to grades, various fellowships & scholarships, awards for contributions to mathematics and education at my schools, etc.

Past/Present Clients
In the past, and as my career progresses, I have worked and continue to work as an educator and mentor to students of varying age levels, skill levels, and educational levels.

©2016 About.com. All rights reserved.