Regression and Correlation of Data
Summary
Procedures for regression:
1. Assume a regression equation.
2. If the equation is simple linear form, use least
squares method to determine the coefficients.
If not, convert it to the form linear in coefficients and
then use least squares method.
Or use Excel functions such as Solver, Trendline or
Linest.
3. Evaluate the regression by statistical analysis
Regression and Correlation of Data
Summary
Simple Linear Regression:
yˆ i  a  bxi
Method of Least Squares: a and b are determined by
minimizing the sum of the squares of errors (SSE),
deviation, residuals or difference between the data set
and the straight line that approximate it.
n
n
n
i 1
i 1
i 1
SSE   ei2   [ y i  yˆ i ] 2   [ y i  (a  bxi )] 2

( SSE )  0
a

( SSE )  0
b
Regression and Correlation of Data
Summary
1  n  n

xi y i    xi   y i 

n  i 1  i 1 
i 1
n
b
1

xi    xi 

n  i 1 
i 1
n
n
a
y
i 1
n
2
2
n
i
 b xi
i 1
n
Centroidal point: ( x, y )
 y  bx
Regression and Correlation of Data
Summary
Procedures for regression:
1. Assume a regression equation.
2. If the equation is simple linear form, use least
squares method to determine the coefficients.
If not, convert it to the form linear in coefficients and
then use least squares method.
Or use Excel functions such as Solver, Trendline or
Linest.
3. Evaluate the regression by statistical analysis
Regression and Correlation of Data
Summary
Other forms linear in coefficients: Forms transformable to linear
in coefficients: e.g.
e.g.
1
 a  bx
y
3
ln y  a  bx y  a  bx
y  ae bx
ln y  ln a  bx
Y  A  BX
1 a
 b
y x
x
y
a  bx
Use least square method to determine the coefficients.
Convert the equation to the form containing the original variables.
Regression and Correlation of Data
Summary
Procedures for regression:
1. Assume a regression equation.
2. If the equation is simple linear form, use least
squares method to determine the coefficients.
If not, convert it to the form linear in coefficients and
then use least squares method.
Or use Excel functions such as Solver, Trendline or
Linest.
3. Evaluate the regression by statistical analysis
Regression and Correlation of Data
Summary
Statistical analysis of the regression:
Sum of the squares of errors (SSE),
n
SSE   [ y i  (a  bxi )] 2
i 1
SSE  S yy  bS xy
n
n
n
i 1
i 1
i 1
S yy   ( y i  y ) 2 , S xy   ( xi  x)( y i  y ), S xx   ( xi  x) 2
Estimated variance of the
points from the line:
s
2
y x
SSE

n2
Estimated standard deviation or
standard error of the points from the line:
sy x 
SSE
n2
The degrees of freedom=n data points – the number of estimated coefficients
Regression and Correlation of Data
Summary
Correlation Coefficient:
r 
S xy
S xx S yy
r =1: the points (xi,yi) are in a perfect straight line and the slope
of that line is positive;
r =-1: the points (xi,yi) are in a perfect straight line and the
slope of that line is negative;
r close to +1 or -1: X and Y follow a linear relation affected by
random errors.
r=0: there is no systematic linear relation between X and Y.
Regression and Correlation of Data
Summary
Coefficient of determination:
n
r2 
2
ˆ
(
y

y
)
 i
i 1
n
2
(
y

y
)
 i
i 1
n
2
ˆ
(
y

y
)
 i
 S xy
r 2  i n1

 S S
2
 ( yi  y)  xx yy
2

 for the simple linear form


i 1
The coefficient of determination is the fraction of the sum of
squares of deviations in the y-direction from y is explained
by the linear relation between y and x given by regression.
Regression and Correlation of Data
Summary
Statistical analysis of the regression:
Residuals (also used for graphical checks):
ei  yi  yˆ i
Percentage Error:
ei
yi  yˆ i
*100% 
*100%
yi
yi
Absolute Percentage Error:
yi  yˆ i
abs(
*100%)
yi