{"id":677,"date":"2018-06-17T06:11:52","date_gmt":"2018-06-17T06:11:52","guid":{"rendered":"http:\/\/muthu.co\/?p=677"},"modified":"2021-05-24T03:37:52","modified_gmt":"2021-05-24T03:37:52","slug":"maths-behind-polynomial-regression","status":"publish","type":"post","link":"http:\/\/write.muthu.co\/maths-behind-polynomial-regression\/","title":{"rendered":"Maths behind Polynomial regression"},"content":{"rendered":"\n

Polynomial regression is a process of finding a polynomial function that takes the form f<\/i>( x<\/i> ) = c<\/i>0<\/sub> + c<\/i>1 <\/sub>x<\/i> + c<\/i>2<\/sub> x<\/i>2<\/sup> \u22ef c<\/i>n<\/i><\/sub> <\/i>x<\/i>n<\/i><\/sup> where n<\/i> is the degree of the polynomial and c<\/i> is a set of coefficients. Through polynomial regression we try to find an nth degree polynomial function which is the closest approximation of our data points. Below is a sample random dataset which has been regressed upto 3 degree and plotted on a graph. The blue dots represent our data set and the lines represent our polynomial functions of different degrees. The higher the degree, closer is the approximation but higher doesn’t always mean right which we will discuss in later articles.<\/p>\n\n\n\n

y_train=[[100],[110],[124],[142],[159],[161],[170],[173],[179],[180],[217],[228],[230],[284],[300],[330],[360],[392],[414],[435],[451],[476],[499],[515],[543],[564]]<\/pre>\n\n\n\n
\"\"<\/a><\/figure><\/div>\n\n\n\n
To understand the math behind the above analysis lets start by constructing a line function that passes through two points: (1, 1) and (6, 9). The general formula of the equation of line:<\/p>\n\n\n\n
\"\"<\/a><\/figure><\/div>\n\n\n\n
\"\"<\/a><\/figure><\/div>\n\n\n\n
Our job here is to find the two unknowns m<\/em> and b<\/em> in the above line equation. By putting our sample points in the above line equation, we get:
we can solve for m and b using substitution method:<\/p>\n\n\n\n
\"\"<\/a><\/figure><\/div>\n\n\n\n
Now substituting m<\/em> and b<\/em> in our line equation, we get the below formula:<\/p>\n\n\n\n
\"\"<\/a><\/figure><\/div>\n\n\n\n
The above algebraic method works best for small number and one degree polynomials but for higher degree polynomials we will be using matrices and linear algebra. But before we dive into the math of higher degree functions lets work out the same equation we derived from our previous example using matrices.<\/p>\n\n\n\n
Our data points in a matrix will look like:<\/p>\n\n\n\n
\"\"<\/a><\/figure><\/div>\n\n\n\n
Solving it using linear algebra:<\/p>\n\n\n\n
\"\"<\/a><\/figure><\/div>\n\n\n\n
As you can see, we get the same equation of line using matrix and linear algebra. The above method works great when we have 2 points because we know we can draw a straight line passing through them but we cannot assure the same when we have more than 2 data points. In my article on linear regression<\/a> I used the below formula to find the coefficients of linear equation y = Bx + A. <\/em><\/p>\n\n\n\n
\"\"<\/a><\/figure><\/div>\n\n\n\n
How we derive the above equation will give us the general formula for a polynomial function of any degree.<\/p>\n\n\n\n
In any regression analysis we try to find an equation of line which is a close approximation of the actual training data points. We use a method called the “Sum of Least squares”<\/a> to derive our equation. Basically what do is find all possible line equations for a given set of data points and select the one which has the least squared sum of errors or residuals – Residuals or errors being the difference between the predicted value and the actual value.<\/p>\n\n\n\n
\"\"<\/a><\/figure><\/div>\n\n\n\n
Least squares has been beautifully explained in the below video:<\/p>\n\n\n\n
\n