Least Square Method
Curve Fitting
Linear Least Square Method
y  m1 x  m2
y n  m1 xn  m2  Rn
y n2  m12 x n2  2m1 x n y n  2m2 y n  2m1 m2 x n  Rn2
 y
N
n 1
2
n

 m12 x n2  2m1 x n y n  2m2 y n  2m1 m2 x n   Rn2
n 1
N
  Rn2
n 1
m1
N
N
N
n 1
n 1
n 1
 2m1  x n2  2 x n y n  2m2  x n  0
N
  Rn2
n 1
m2
N
N
N
n 1
n 1
 2 m 2 N  2 y n  2 x n  0
m1 
m2 
N
N
N
n 1
n 1
n 1
2
N  xn y n   xn  y n
N


2
N  x n    xn 
n 1
 n 1 
N
N
N
N
N
n 1
n 1
n 1
n 1
2
2
x
 n  y n   xn y n  xn
N


2
N  x n    xn 
n 1
 n 1 
N
y  m1 x  m2
ในกรณีที่มคี า่ พารามิเตอร์
m จำนวน M
ตัว
M
y   xi mi
i 1
y  Xm ε
N
2
R
 n  ε ε  y Xm
T
T
y Xm
n 1
 y
N
n 1
2
n

N
 m12 x n2  2m1 x n y n  2m2 y n  2m1 m2 x n   Rn2
n 1
N
 R 2
n 1
m
 2y Xm X  0
T
XT y Xm  0
X y X Xm  0
T
T


m  X X
T
1
X y
T
Example 1 on page 362
k  p1T  p2
k  [141 138 119 115]T
T  [390 500 1000 1500]T
k  p1T  p2
 p1 
 [T 1] 
 p2 
p  X' X
1
X' y
% Linear1.m
k =[141 138 119
115]';
T = [390 500 1000
1500]';
X = [T,[1 1 1 1]'];
y = k;
p =inv(X'*X)*X'*y
Non Linear System of Equation
f1  x1 , x2 ,..., xn   0
f 2  x1 , x2 ,..., xn   0

f n  x1 , x2 ,..., xn   0
f k ,i 1  f k ,i  x1,i 1  x1,i 
 ...  xn ,i 1  xn ,i 
f k ,i
xn
f k ,i
x1
0
 x2,i 1  x2,i 
f k ,i
x2
f k ,i
 f k ,i  x1,i
 x1,i 1
x1
f k ,i
 f1,i
 x
 1
 f 2,i
Z   x2

 f
 n ,i
 x1
 sym
x1
 x2 , i
 x2,i 1
f1,i
x2
f 2,i
x2

f n ,i
x2
...
...

...
f k ,i
x2
f k ,i
x2
 ...  xn ,i
 ...  xn ,i 1
f1,i 

xn 
f 2,i 
xn 

 
f n ,i 
xn 
f k ,i
xn
f k ,i
xn
= Jacobian Matrix
x i   x1,i x2,i ...
x i 1T  x1,i 1 x2,i 1
T
Fi    f1,i f 2,i ...
T
xn ,i 
... xn ,i 1 
f n ,i 
 Fi   Zxi   Zxi 1
T
xi 1  Z
1


  Fi   Zxi 
T
Example
2
8
1
2 
4 2


2  x1 
x2  x1  x2   1  x1  x2   0
27 
3
3 
3 9

2
10 
1
1 
5 2

2  x1 
x2  x1  2 x2   1  x1  x2   0
27 
3
3 
3 9

 f1,i f1,i 
 x


x
2 
Z    1
 f 2,i f 2 ,i 


 x1 x2 
f1,i
8
2 1
2 
4 2
 4


 2  x1 
x2   x1  x2  1  x1  x2 
x1
27  9
3
3
3 
3 9
f1,i
8
4 1
2 
4 2
 16
x1  x2   1  x1  x2 
 2  x1 
x2  
x2
27  27
3
3
3 
3 9
f 2,i
10  4
2
1
1 
5 2
 2  x1 
x2    x1  2 x2   1  x1  x2 
x1
27  9
3
3
3 
3 9
f 2,i
10  20
2
1
1 
5 2
x1  2 x2   1  x1  x2 
 2  x1 
x2  
x2
27  27
3
3
3 
3 9
clear; c1 = 1; c2 = 1; %First Guess
NONLINEAR1.m
for i=1:10
X = [c1;c2];
f1 = 2*(4/3-2/9*c1-8/27*c2)*(c1+c2)+(1-1/3*c1-2/3*c2)^2;
f2 = 2*(5/3-2/9*c1-10/27*c2)*(c1+2*c2)+(1+1/3*c1+1/3*c2)^2;
F = [f1;f2];
df1c1 = 2*(4/3-2/9*c1-8/27*c2)- 4/9*(c1+c2)- 2/3*(1-1/3*c1-2/3*c2);
df1c2 = 2*(4/3-2/9*c1-8/27*c2)- 16/27*(c1+c2)- 4/3*(1-1/3*c1-2/3*c2);
df2c1 = 2*(5/3-2/9*c1-10/27*c2)- 4/9*(c1+2*c2)+ 2/3*(1+1/3*c1+1/3*c2);
df2c2 = 4*(5/3-2/9*c1-10/27*c2)- 20/27*(c1+2*c2)+ 2/3*(1+1/3*c1+1/3*c2);
Z = [df1c1,df1c2;df2c1,df2c2];
W = -F+Z*X;
Xnew = inv(Z)*W;
c1 =Xnew(1);c2= Xnew(2);
fprintf('
end
%g
%g
%g\n',i,c1,c2);
Non Linear Least Square
y  Xm ε
N
 R  ε ε  y Xm y Xm
n 1
N
 R 2
n 1
T
2
n
m
T
 2y Xm
 X 
 m


T
T
X
0
m
y Xm  0
Numerically
Expand {y} by Taylor
 Xm *
y  Xm * 
m m *  other

 m 
 Xm *
H




m


y Xm *  Hm m *
HT y Xm *  HT Hm m *
m  m * H H H y Xm *
T
1
T
Example : Stress Relaxation
y = a1 exp -a2t   a3 exp -a4t   a5
Time Force
.000001,
3.083900
.016670,
1.647600
.033330,
1.176900
.050000,
.935450
.066670,
.784570
.083330,
.681970
.100000,
.609550
Example : Stress Relaxation
y = a1 exp -a2t   a3 exp -a4t   a5
m  m * H H H y Xm *
T
1
T
 Xm *
H




m


 f i f i f i f i f i 
H


a

a

a

a

a
2
3
4
5 
 1
 f1,i f1,i 
 x


x
2 
Z    1
 f 2,i f 2 ,i 


 x1 x2 
y = a1 exp -a2t   a3 exp -a4t   a5
 f i f i f i f i f i 
H


a

a

a

a

a
2
3
4
5 
 1
f i
 exp -a 2t 
a1
f i
=  a1t exp -a 2t 
a2
f i
= exp -a 4t 
a3
f i
=  a3t exp -a4t 
a4
Stess.m
f i
=1
a5
% Parameters Estimation
% Using Stress Relaxation Data
% Five parameters
clear;
clc;
Data= ...
[ 1, .000001, 3.083900
2, .016670, 1.647600
3, .033330, 1.176900
4, .050000,
.935450
5, .066670,
.784570
6, .083330,
.681970
7, .100000,
.609550
8, .116670,
.555230
9, .133330,
.506950
10, .150000,
.470740
20, .316670,
.289690
30, .483330,
.223300
40, .650000,
.181050
50, .816670,
.156910
60, .983330,
.138810
70, 1.150000,
.126740
80, 1.316700,
.114670
90, 1.483300,
.108630
100, 1.650000,
.102600
110, 1.816700,
.090530
120, 1.983300,
.084490
130, 2.150000,
.084490
140, 2.316700,
.078460
150, 2.483300,
.078460
160, 2.650000,
.078460
170, 2.816700,
.072420
180, 2.983300,
.072420
190, 3.150000,
.066390
200, 3.316700,
.066390
220, 3.650000,
.060350
240, 3.983300,
.060350
260, 4.316700,
.054320
280, 4.650000,
.060350
300, 4.983300,
.048280
350, 5.816700,
.048280
400, 6.650000,
.048280
500, 8.316700,
.042250
600, 9.983300,
.036210];
clf;
t = Data(:,2);
y = Data(:,3)*1000/224.8089;
y = y/y(1);
plot(t,y,'ro');
legend('Data','Fit');
xlabel('Time');
ylabel('Force');
title ('Non Linear Regression');
axis([min(t) round(max(t)) 0 1]);
hold on;
Plothandle = plot(t,y,'-','EraseMode','xor');
a =[10;10;10;1;1];
% First Guess of three parameters
da =1;
i=0;
while da>1e-20
ym = a(1)*exp(-a(2)*t)+a(3)*exp(-a(4)*t)+a(5);
dyda1 = exp(-a(2)*t);
dyda2 = -a(1)*t.*exp(-a(2)*t);
dyda3 = exp(-a(4)*t);
dyda4 = -a(3)*t.*exp(-a(4)*t);
dyda5 = 1+zeros(size(t));
Jacob = [dyda1,dyda2,dyda3,dyda4,dyda5];
anew = a +0.1*inv(Jacob'*Jacob)*Jacob'*(y-ym);
da = norm(anew-a);
a = anew;
i=i+1;
fprintf( 'Iter = %g
Par =
%g
%g
%g
%g\n',i,a(1),a(2),a(3),a(4),a(5));
set (Plothandle,'ydata',ym);
drawnow;
hold off;
end
%g
ybar
=
mean(y);
ybar).^2);
Syy
=
sum((y-ybar).^2); R2
fprintf( ' Parameter =
%g
%g
%g\n',a(1),a(2),a(3),a(4),a(5));
fprintf( ' R_square =
%g \n',R2);
SSr
=
=
%g
SSr/Syy;
%g
sum((ym-