Gauss-Seidel: sufficient
condition for convergence:
| a ii | 
using relaxation: x inew  x inew  1   x iold
n
| a
j 1; j  i
ij
|
Polynomial least squares regression analysis:
y  a 0  a1 x  a 2 x 2  ..
f ( x1 ) j
y/ x
n
i 1
i 1
Sr
n2
Quadratic Regression:
{D}  [ Z j ]A E
f ( x1 ) j 


a1
a m 
 y1  f ( x1 ) 
f ( x 2 ) j
f ( x 2 ) j 




a1
a m  {D}   y2  f ( x2 ) 









 yn  f ( xn )
f ( x n ) j
f ( x n ) j 


a1
a m 
n
S r   ei2   ( y i  a 0  a1 xi ) 2
2x2 matrix (only a 0 , a1 ): linear,
3x3 matrix: quadratic, etc.
Nonlinear Regression:
 f ( x1 ) j

 a 0
 f ( x 2 ) j

[ Z j ]   a 0


 f ( x n ) j

 a 0
Error Analysis for linear regression:
Standard error of the estimate:
s 
 n

{E}  e1, e2, ,en 
  xi
{A}  a a , a , ,a 
 x i2

T
T
0,
1
2
m
x x
x x
x x
i
2
i
3
i
2
i
3
i
4
i
 a 0    y i 
  

  a1     x i y i  ; where
 a 2   x i2 y i 




n

i 1
[Z j ]T [Z j ]A  [Z j ]T D
n
Lagrange interpolating polynomials: f n ( x )   Li ( x ) f ( x i ) ; where Li ( x ) 
n

x  xj
xi  x j
( x  x1)( x  x 2 )
( x  x 0 )( x  x 2)
( x  x 0)( x  x1)
f 2 ( x) 
f ( x 0) 
f ( x1) 
f ( x 2)
( x 0  x1)( x 0  x 2)
( x1  x 0 )( x1  x 2)
( x 2  x 0)( x 2  x1)
Cubic spline interpolation: define f i (x ) on interval xi 1  x  xi as follows:
i 0
f i ( x) 
f '' ( xi 1 )
f '' ( x i )
( xi  x ) 3 
( x  xi 1 ) 3
6( xi  xi 1 )
6( xi  xi 1 )
j  0, j i
Solve: ( x i  x i 1 ) f '' ( x i 1 )  2( x i 1  x i 1 ) f '' ( x i )
6
 f ( xi 1 )  f ( xi )
 ( x i 1  x i ) f '' ( x i 1 ) 
x i 1  x i
6
 f ( xi 1 )  f ( xi )

x i  x i 1
 f ( xi 1 ) f '' ( xi 1 )( xi  xi 1 ) 


( x i  x )
6
 x i  xi 1

''
 f ( xi )
f ( xi )( xi  xi 1 ) 


( x  xi 1 )
6
 x i  xi 1

Natural (or free) conditions:
Clamped conditions: f 1' ( x 0 ) and f n' ( x n ) given, and
f '' ( x0 )  0 ; f '' ( xn )  0
x1  x 0 ''
x  x 0 ''
f ( x1 )  f ( x 0 )
f ( x0 )  1
f ( x1 ) 
 f 1' ( x 0 )
Periodic conditions:
3
6
x1  x 0
f '' ( x0 )  f '' ( xn ) ; xn1  xn  ( x1  x0 ) ; x n  x n 1 ''
x  x n 1 ''
f ( x n )  f ( x n 1 )
f ( x n 1 )  n
f ( x n )  f n' ( x n ) 
''
''
f ( xn1 )  f ( x1 ) and f ( xn 1 )  f ( x1 )
6
3
x n  x n 1
GJG 14042003
Root finding: False Position:
f " ( xi )
2
(
x

x
)
i

1
i
Taylor
f ( xu )( xl  xu )
2!
x

x

Series:
r
u
f " ' ( xi )
f ( n 1) ( )
f ( xl )  f ( xu )

( xi 1  xi ) 3   
( xi 1  xi ) n 1
3!
( n  1)!
Root finding: Newton-Raphson:
Secant:
Modified Newton-Raphson:
f ( xi ) f ' ( xi )
f ( xi )
f ( xi )( xi 1  xi )
xi 1  xi 
xi 1  xi 
xi 1  xi 
[ f ' ( xi )] 2  f ( xi ) f " ( xi )
f ' ( xi )
f ( xi 1 )  f ( xi )
System of linear algebraic equations: [A] {X} = {B}. Set [L] [U] = [A] with [L] and [U] as follows:
LU (Doolittle) decomposition:
Crout decomposition:
 1 0 0
u11 u12 u13 
l11 0 0 
1 u12 u13 






L  l 21 1 0 ; U    0 u22 u23 
L  l 21 l 22 0  ; U   0 1 u23 
0 u33 
1 
l 31 l 32 1
 0
l 31 l 32 l 33 
0 0
f ( xi 1 )  f ( xi )  f ' ( xi )( xi 1  xi ) 
Numerical integration:

b
a
f ( x )dx . left-to-right: trapezoidal, Simpson’s 1/3 and Simpson’s 3/8 rules
3h
h
h
 f ( x 0 )  f ( x1 ) 
 f ( x 0 )  3 f ( x1 )  3 f ( x 2 )  f ( x3 ) 
I   f ( x 0 )  4 f ( x1 )  f ( x 2 ) 
I
2
3
8
where I1,1 = trapezoidal rule with h1=b-a, I2,1 = trapezoidal rule with
4 k 1 I j 1,k 1  I j ,k 1
Romberg: I j ,k 
h2=h1/2, I3,1 = trapezoidal rule with h3=h2/2, etc.
4 k 1  1
I
Gauss quadrature (Gauss-Legendre polynomials):
where c i and x i are given as follows:

b
a
1
g ( x)dx   f ( x)dx  c 0 f ( x 0 )  c1 f ( x1 )  ...  c n 1 f ( x n 1 )
1
n  2 : c0  1; x0  1 / 3; c1  1; x1   x0
n  3 : c0  5 / 9; x0   0.6; c1  8 / 9; x1  0; c2  c0 ; x2   x0
n  4 : c0  0.347855; x0  0.86113631; c1  0.652145; x1  0.33998104; c 2  c1 ; x 2   x1 ; c3  c0 ; x3   x0
Numerical differentiation using finite divided difference formulas:
Forward: f ( xi )   f ( xi 1 )  f ( xi ) / h  O(h) ; f ( xi )   f ( xi  2 )  4 f ( xi 1 )  3 f ( xi )  /( 2h)  O(h 2 )
f ( xi )   f ( xi  2 )  2 f ( xi 1 )  f ( xi )  / h 2  O(h) ; f ( xi )   f ( xi  3 )  4 f ( xi  2 )  5 f ( xi 1 )  2 f ( xi )  / h 2  O(h 2 )
Backward: f ( xi )   f ( xi )  f ( xi 1 ) / h  O(h) ; f ( xi )  3 f ( xi )  4 f ( xi 1 )  f ( xi  2 )  /( 2h)  O(h2 )
f ( xi )   f ( xi )  2 f ( xi 1 )  f ( xi  2 )  / h 2  O(h) ; f ( xi )  2 f ( xi )  5 f ( xi 1 )  4 f ( xi  2 )  f ( xi 3 )  / h 2  O(h 2 )
Centered: f ( xi )   f ( xi 1 )  f ( xi 1 )  /( 2h)  O(h 2 )
f ( xi )   f ( xi  2 )  8 f ( xi 1 )  8 f ( xi 1 )  f ( xi  2 )  /(12h)  O(h 4 )
f ( xi )   f ( xi 1 )  2 f ( xi )  f ( xi 1 )  / h 2  O(h 2 )
f ( xi )   f ( xi  2 )  16 f ( xi 1 )  30 f ( xi )  16 f ( xi 1 )  f ( xi  2 )  /(12h 2 )  O(h 4 )
f ( xi )   f ( xi  2 )  2 f ( xi 1 )  2 f ( xi 1 )  f ( xi  2 )  /( 2h 3 )  O(h 2 )
System of nonlinear equations
f ( x)  0
 f1 ( x1 , x2 ,  xn ) 
 f ( x , x , x ) 
n 
f ( x)   2 1 2




 f n ( x1 , x2 ,  xn ) 
f ( xˆ ) 
 f1 ( xˆ ) f1 ( xˆ )
 1
 x
x2
xn 
1


f 2 ( xˆ ) 
 f 2 ( xˆ ) f 2 ( xˆ )

x2
xn 
Df ( xˆ )   x1

 





 f n ( xˆ ) f n ( xˆ )  f n ( xˆ ) 
 x1
x2
xn 
Newton’s algorithm: input f(x) and accuracy
i=0;
set initial point xi; evaluate f(xi);
while ( f ( x i )   ){ evaluate Df(xi);
Df ( xi ) xi 1   f ( xi )  Df ( xi ) xi ; i=i+1;}
i+1
output x
Solving ODEs (initial value problems): y   f ( x, y ) ; y 0 given: One-step methods:
Euler: y i 1  y i  f ( x i , y i )h
Runge Kutta 2 (RK2) or Midpoint: y i 1  y i  k 2 h , where k1  f ( x i , y i ) and k 2  f ( xi  h / 2, y i  k1 h / 2)
RK4: y i 1  y i  (k1  2k 2  2k 3  k 4 )h / 6 , where k1  f ( x i , y i ) ; k 2  f ( xi  h / 2, y i  k1 h / 2) ;
k 3  f ( xi  h / 2, y i  k 2 h / 2) and k 4  f ( xi  h, y i  k 3 h)
Solving ODEs (boundary value problems): y   f ( x, y, y ) ; in general: y ( n)  f ( x, y, y ,.., y ( n 1) ) ; y 0 , y n given
Finite difference method: replace all derivative terms by centered divided difference expressions of O(h 2 )
Solving linear, second-order PDEs (boundary value problems): Elliptic, Parabolic and Hyperbolic equations
Finite difference method: replace all partial derivative terms by centered difference expressions of O(h 2 )
GJG 14042003