Gaussian Elimination
Chemical Engineering Majors
Author(s): Autar Kaw
http://numericalmethods.eng.usf.edu
Transforming Numerical Methods Education for STEM
Undergraduates
Naïve Gauss Elimination
http://numericalmethods.eng.usf.edu
Naïve Gaussian Elimination
A method to solve simultaneous linear
equations of the form [A][X]=[C]
Two steps
1. Forward Elimination
2. Back Substitution
Forward Elimination
The goal of forward elimination is to transform the
coefficient matrix into an upper triangular matrix
 25 5 1  x1  106.8 
 64 8 1  x   177.2 

  2 

144 12 1  x3  279.2
5
1   x1   106.8 
25
 0  4.8  1.56  x    96.21

  2 

0
0.7   x3   0.735 
 0
Forward Elimination
A set of n equations and n unknowns
a11x1  a12 x2  a13 x3  ...  a1n xn  b1
a21x1  a22 x2  a23 x3  ...  a2 n xn  b2
.
.
.
.
.
.
an1x1  an 2 x2  an3 x3  ...  ann xn  bn
(n-1) steps of forward elimination
Forward Elimination
Step 1
For Equation 2, divide Equation 1 by a11 and
multiply by a21 .
 a21 
 a (a11x1  a12 x2  a13 x3  ...  a1n xn  b1 )
 11 
a21
a21
a21
a21x1 
a12 x2  ... 
a1n xn 
b1
a11
a11
a11
Forward Elimination
Subtract the result from Equation 2.
a21x1  a22 x2  a23 x3  ...  a2 n xn  b2
a21
a21
a21
− a21x1  a a12 x2  ...  a a1n xn  a b1
11
11
11
_________________________________________________


a21 
a21 
a21
 a22 
a12  x2  ...   a2 n 
a1n  xn  b2 
b1
a11 
a11 
a11


or
a x  ...  a x  b
'
22 2
'
2n n
'
2
Forward Elimination
Repeat this procedure for the remaining
equations to reduce the set of equations as
a11x1  a12 x2  a13 x3  ...  a1n xn  b1
'
'
a22
x2  a23
x3  ...  a2' n xn  b2'
'
'
a32
x2  a33
x3  ...  a3' n xn  b3'
.
.
.
.
.
.
.
.
.
'
an' 2 x2  an' 3 x3  ...  ann
xn  bn'
End of Step 1
Forward Elimination
Step 2
Repeat the same procedure for the 3rd term of
Equation 3.
a11x1  a12 x2  a13 x3  ...  a1n xn  b1
'
'
a22
x2  a23
x3  ...  a2' n xn  b2'
"
a33
x3  ...  a3" n xn  b3"
.
.
.
.
.
.
"
an" 3 x3  ...  ann
xn  bn"
End of Step 2
Forward Elimination
At the end of (n-1) Forward Elimination steps, the
system of equations will look like
a11 x1  a12 x2  a13 x3  ...  a1n xn  b1
'
'
a22
x2  a23
x3  ...  a2' n xn  b2'
"
a33
x3  ...  a3"n xn  b3"
.
.
.
.
.
.
n 1 
 n 1
ann
xn  bn
End of Step (n-1)
Matrix Form at End of Forward
Elimination
a11 a12
 0 a'
22

0
0




 0
0
a1n   x1   b1 
'
' 




 a 2 n x2
b2
  

 a"3n   x3    b3" 
  


      
(n1 )
  xn  bn(n-1 ) 
0 ann
a13 
a
'
23
"
33
a

0
Back Substitution
Solve each equation starting from the last equation
5
1   x1   106.8 
25
 0  4.8  1.56  x    96.21

  2 

0
0.7   x3   0.735 
 0
Example of a system of 3 equations
Back Substitution Starting Eqns
a11 x1  a12 x2  a13 x3  ...  a1n xn  b1
'
'
a22
x2  a23
x3  ...  a2' n xn  b2'
"
a33
x3  ...  an" xn  b3"
.
.
.
 n 1
.
.
.
n 1 
ann xn  bn
Back Substitution
Start with the last equation because it has only one unknown
( n 1)
n
( n 1)
nn
b
xn 
a
Back Substitution
( n 1)
n
( n 1)
nn
b
xn 
a
xi 
bii 1  ai,ii11 xi 1  ai,ii12 xi  2  ...  ai,in1 xn
i 1
aii
i 1
xi 
bi
  aiji 1 x j
n
j i 1
i 1
ii
a
for i  n  1,...,1
for i  n  1,...,1
THE END
http://numericalmethods.eng.usf.edu
Naïve Gauss Elimination
Example
http://numericalmethods.eng.usf.edu
Example: Liquid-Liquid Extraction
A liquid-liquid extraction process conducted in the Electrochemical
Materials Laboratory involved the extraction of nickel from the aqueous
phase into an organic phase. A typical set of experimental data from the
laboratory is given below:
Ni aqueous phase, a (g/l)
2
2.5
3
Ni organic phase, g (g/l)
8.57
10
12
Assuming g is the amount of Ni in organic phase and a is the amount of
Ni in the aqueous phase, the quadratic interpolant that estimates g is
given by
g  x1a 2  x2 a  x3 , 2  a  3
Example: Liquid-Liquid Extraction
The solution for the unknowns x1, x2, and x3 is given by
2 1  x1  8.57
 4
6.25 2.5 1  x    10 

  2 

 9
3 1  x3   12 
Find the values of x1, x2, and x3 using Naïve Gauss Elimination.
Estimate the amount of nickel in organic phase when 2.3 g/l is in
the aqueous phase using quadratic interpolation.
Number of Steps of Forward
Elimination
Number of steps of forward elimination is
(n1)(31)2
Example: Liquid-Liquid Extraction
Solution
Forward Elimination: Step 1
 6.25 
Row 2  
 Row1
 4 
Yields
2
1   x1   8.57 
4
0  0.625  0.5625  x    3.3906

  2 

9
3
1   x3   12 
Example: Liquid-Liquid Extraction
Forward Elimination: Step 1
9
Row3    Row1
4
Yields
2
1   x1   8.57 
4
0  0.625  0.5625  x    3.3906

  2 

0  1.5
 1.25   x3   7.2825
Example: Liquid-Liquid Extraction
Forward Elimination: Step 2
  1.5 
Row3  
 Row 2
  0.625 
Yields
2
1   x1   8.57 
4
0  0.625  0.5625  x    3.3906

  2 

0
0
0.1   x3   0.855 
This is now ready for Back Substitution
Example: Liquid-Liquid Extraction
Back Substitution: Solve for x3 using the third equation
0.1x3  0.855
0.855
x3 
0.1
 8.55
Example: Liquid-Liquid Extraction
Back Substitution: Solve for x2 using the second equation
(0.625) x2  (0.5625) x3  3.3906
 3.3906  (0.5625) x3
x2 
 0.625
 3.3906  (0.5625)  8.55

 0.625
 2.27
Example: Liquid-Liquid Extraction
Back Substitution: Solve for x1 using the first equation
4 x1  2 x2  x3  8.57
8.57  2 x2  x3
x1 
4
8.57  2  (2.27)  8.55

4
 1.14
Example: Liquid-Liquid Extraction
The solution vector is
 x1   1.140 
 x    2.27
 2 

 x3   8.55 
The polynomial that passes through the three
data points is then
g a   x1 (a)  x2 (a)  x3
2
 1.15a   2.27 a  8.55
2
Where g is the amount of nickel in the organic
phase and a is the amount of in the aqueous phase.
Example: Liquid-Liquid Extraction
When 2.3 g/l is in the aqueous phase, using quadratic
interpolation, the estimated amount of nickel in the organic
phase is
g 2.3  1.14  (2.3)  (2.27)  (2.3)  8.55
2
 9.3596 g/l
THE END
http://numericalmethods.eng.usf.edu
Naïve Gauss Elimination
Pitfalls
http://numericalmethods.eng.usf.edu
Pitfall#1. Division by zero
10 x2  7 x3  3
6 x1  2 x2  3x3  11
5 x1  x2  5 x3  9
0 10  7  x1   3 
6 2





3 x2  11

   
5  1 5   x3   9 
Is division by zero an issue here?
12 x1  10 x2  7 x3  15
6 x1  5 x2  3x3  14
5 x1  x2  5 x3  9
12 10  7  x1  15
6 5
3   x2   14

   
 5  1 5   x3   9 
Is division by zero an issue here?
YES
12 x1  10 x2  7 x3  15
6 x1  5 x2  3x3  14
24 x1  x2  5 x3  28
12 10  7  x1  15
6 5
3   x2   14 

   
24  1 5   x3  28
12 10  7  x1   15 
0
0
6.5  x2   6.5

   
12  21 19   x3   2
Division by zero is a possibility at any step
of forward elimination
Pitfall#2. Large Round-off Errors
15
10  x1   45 
 20
 3  2.249 7   x   1.751

  2 

1
3   x3   9 
 5
Exact Solution
 x1  1
 x   1
 2  
 x3  1
Pitfall#2. Large Round-off Errors
15
10  x1   45 
 20
 3  2.249 7   x   1.751

  2 

1
3   x3   9 
 5
Solve it on a computer using
6 significant digits with chopping
 x1   0.9625 
 x    1.05 
 2 

 x3  0.999995
Pitfall#2. Large Round-off Errors
15
10  x1   45 
 20
 3  2.249 7   x   1.751

  2 

1
3   x3   9 
 5
Solve it on a computer using
5 significant digits with chopping
 x1   0.625 
 x    1.5 
 2 

 x3  0.99995
Is there a way to reduce the round off error?
Avoiding Pitfalls
Increase the number of significant digits
• Decreases round-off error
• Does not avoid division by zero
Avoiding Pitfalls
Gaussian Elimination with Partial Pivoting
• Avoids division by zero
• Reduces round off error
THE END
http://numericalmethods.eng.usf.edu
Gauss Elimination with
Partial Pivoting
http://numericalmethods.eng.usf.edu
Pitfalls of Naïve Gauss Elimination
• Possible division by zero
• Large round-off errors
Avoiding Pitfalls
Increase the number of significant digits
•
Decreases round-off error
• Does not avoid division by zero
Avoiding Pitfalls
Gaussian Elimination with Partial Pivoting
• Avoids division by zero
• Reduces round off error
What is Different About Partial
Pivoting?
At the beginning of the kth step of forward elimination,
find the maximum of
akk , ak 1,k ,................, ank
If the maximum of the values is a pk
in the p
th
row, k  p  n, then switch rows p and k.
Matrix Form at Beginning of 2nd
Step of Forward Elimination
a11 a12
 0 a'
22

'
 0 a32




 0 a'n 2
a13

'
23
'
33



an' 3
an' 4
a
a

a1n   x1   b1 
' 
' 


a 2 n x2
b2
   
a3' n   x3    b3' 
   
      
'
  xn  bn' 
ann
Example (2nd step of FE)
6 14 5.1 3.7 6   x1   5 
0  7 6
1
2   x2    6

   
4
12 1 11  x3    8 
0

   
9
23 6
8   x4   9 
0
0  17 12 11 43  x5   3 
Which two rows would you switch?
Example (2nd step of FE)
6 14 5.1 3.7 6   x1   5 
0  17 12 11 43  x   3 

 2   
4
12 1 11  x3    8 
0

   
9
23 6
8   x4   9 
0
0  7 6
1
2   x5    6
Switched Rows
Gaussian Elimination
with Partial Pivoting
A method to solve simultaneous linear
equations of the form [A][X]=[C]
Two steps
1. Forward Elimination
2. Back Substitution
Forward Elimination
Same as naïve Gauss elimination method
except that we switch rows before each
of the (n-1) steps of forward elimination.
Example: Matrix Form at Beginning
of 2nd Step of Forward Elimination
a11 a12
 0 a'
22

'
 0 a32




 0 a'n 2
a13

a'23

'
33



a
a
'
n3
a
'
n4
a1n   x1   b1 
a'2 n   x2  b2' 
   
'
'
a3n   x3    b3 
   
      
'
'




ann   xn  bn 
Matrix Form at End of Forward
Elimination
a11 a12
 0 a'
22

0
0




 0
0
a1n   x1   b1 
 a'2 n   x2   b2' 
  

"
"
 a3n   x3    b3 
  


      
(n1 )
(n-1 )




0 ann   xn  bn 
a13 
a'23
"
33
a

0
Back Substitution Starting Eqns
a11 x1  a12 x2  a13 x3  ...  a1n xn  b1
'
'
a22
x2  a23
x3  ...  a2' n xn  b2'
"
a33
x3  ...  an" xn  b3"
.
.
.
 n 1
.
.
.
n 1 
ann xn  bn
Back Substitution
( n 1)
n
( n 1)
nn
b
xn 
a
i 1
xi 
bi
n
i 1
  aij x j
j i 1
i 1
ii
a
for i  n  1,...,1
THE END
http://numericalmethods.eng.usf.edu
Gauss Elimination with
Partial Pivoting
Example
http://numericalmethods.eng.usf.edu
Example 2
Solve the following set of equations
by Gaussian elimination with partial
pivoting
5 1  a1  106.8 
 25
 64





8 1 a 2  177.2

   

144 12 1  a3  279.2
Example 2 Cont.
 25 5 1  a1  106.8 
 25 5 1  106.8 
 64 8 1 a   177.2   

64
8
1

177
.
2
2

   



144 12 1  a3  279.2
144 12 1  279.2
1. Forward Elimination
2. Back Substitution
Forward Elimination
Number of Steps of Forward
Elimination
Number of steps of forward elimination is
(n1)=(31)=2
Forward Elimination: Step 1
• Examine absolute values of first column, first row
and below.
25 , 64 , 144
• Largest absolute value is 144 and exists in row 3.
• Switch row 1 and row 3.
 25 5 1  106.8 
144 12 1  279.2
 64 8 1  177.2    64 8 1  177.2 




144 12 1  279.2
 25 5 1  106.8 
Forward Elimination: Step 1 (cont.)
144 12 1  279.2
 64 8 1  177.2 


 25 5 1  106.8 
144
Divide Equation 1 by 144 and
64
 0.4444 .
multiply it by 64,
144
12 1  279.2 0.4444  63.99 5.333 0.4444  124.1
.
Subtract the result from
Equation 2
64
 63.99
0
8
1  177.2
5.333 0.4444 
124.1
2.667 0.5556  53.10
1
 279.2
Substitute new equation for 144 12
 0 2.667 0.5556  53.10
Equation 2


 25
5
1
 106.8 
Forward Elimination: Step 1 (cont.)
12
1
 279.2 Divide Equation 1 by 144 and
144
 0 2.667 0.5556  53.10
25

 multiply it by 25,
 0.1736.
144
 25
5
1
 106.8 
144
12 1  279.2 0.1736  25.00 2.083 0.1736  48.47
.
Subtract the result from
Equation 3
Substitute new equation for
Equation 3
25
 25
0
5
1  106.8
2.917
0.8264  58.33
2.083 0.1736  48.47
12
1
 279.2
144
 0 2.667 0.5556  53.10


 0 2.917 0.8264  58.33
Forward Elimination: Step 2
• Examine absolute values of second column, second row
and below.
2.667 , 2.917
• Largest absolute value is 2.917 and exists in row 3.
• Switch row 2 and row 3.
1
 279.2
1
 279.2
144 12
144 12
 0 2.667 0.5556  53.10   0 2.917 0.8264  58.33




 0 2.917 0.8264  58.33
 0 2.667 0.5556  53.10
Forward Elimination: Step 2 (cont.)
12
1
 279.2
144
 0 2.917 0.8264  58.33


 0 2.667 0.5556  53.10
0
Divide Equation 2 by 2.917 and
multiply it by 2.667,
2.667
 0.9143.
2.917
2.917 0.8264  58.33 0.9143  0 2.667 0.7556  53.33
Subtract the result from
Equation 3
0
 0
0
Substitute new equation for
Equation 3
12
1
 279.2 
144
 0 2.917 0.8264  58.33 


 0
0
 0.2   0.23
.
2.667 0.5556 
2.667 0.7556 
0
53.10
53.33
 0.2   0.23
Back Substitution
Back Substitution
1
 279.2 
1 
144 12
144 12
 0 2.917 0.8264  58.33    0 2.917 0.8264




 0
 0
0
 0.2   0.23
0
 0.2 
Solving for a3
 0.2a3  0.23
 0.23
a3 
 0.2
 1.15
 a1   279.2 
a    58.33 
 2 

 a3   0.23
Back Substitution (cont.)
1 
144 12
 0 2.917 0.8264


0
 0.2 
 0
 a1   279.2 
a    58.33 
 2 

 a3   0.23
Solving for a2
2.917 a2  0.8264a3  58.33
58.33  0.8264a3
a2 
2.917
58.33  0.8264  1.15

2.917
 19.67
Back Substitution (cont.)
1 
144 12
 0 2.917 0.8264


0
 0.2 
 0
 a1   279.2 
a    58.33 
 2 

 a3   0.23
Solving for a1
144a1  12a2  a3  279.2
279.2  12a2  a3
a1 
144
279.2  12 19.67  1.15

144
 0.2917
Gaussian Elimination with Partial
Pivoting Solution
 25 5 1  a1  106.8 
 64 8 1 a   177.2 

  2 

144 12 1  a3  279.2
 a1  0.2917 
a    19.67 
 2 

 a3   1.15 
Gauss Elimination with
Partial Pivoting
Another Example
http://numericalmethods.eng.usf.edu
Partial Pivoting: Example
Consider the system of equations
10 x1  7 x2  7
 3x1  2.099 x2  6 x3  3.901
5 x1  x2  5 x3  6
In matrix form
 7 0  x1 
 7 
 10
3.901
 3 2.099 6  x 


  2 = 
 6 
 5
 1 5  x 3 
Solve using Gaussian Elimination with Partial Pivoting using five
significant digits with chopping
Partial Pivoting: Example
Forward Elimination: Step 1
Examining the values of the first column
|10|, |-3|, and |5| or 10, 3, and 5
The largest absolute value is 10, which means, to
follow the rules of Partial Pivoting, we switch
row1 with row1.
Performing Forward Elimination
 7 0  x1   7 
 10
 3 2.099 6  x   3.901

 2  

 5
 1 5  x3   6 

7
0  x1   7 
10
 0  0.001 6  x   6.001

 2  

 0
2.5
5  x3   2.5 
Partial Pivoting: Example
Forward Elimination: Step 2
Examining the values of the first column
|-0.001| and |2.5| or 0.0001 and 2.5
The largest absolute value is 2.5, so row 2 is
switched with row 3
Performing the row swap
7
0  x1   7 
10
 0  0.001 6  x   6.001

 2  

 0
2.5
5  x3   2.5 

7
0  x1   7 
10
0
  x    2.5 
2
.
5
5

 2  

 0  0.001 6  x3  6.001
Partial Pivoting: Example
Forward Elimination: Step 2
Performing the Forward Elimination results in:
0   x1   7 
10  7
 0 2.5
  x    2.5 
5

 2  

 0 0 6.002  x3  6.002
Partial Pivoting: Example
Back Substitution
Solving the equations through back substitution
0   x1   7 
10  7
 0 2.5
  x    2.5 
5

 2  

 0 0 6.002  x3  6.002
6.002
x3 
1
6.002
2.5  5 x3
x2 
 1
2.5
7  7 x 2  0 x3
x1 
0
10
Partial Pivoting: Example
Compare the calculated and exact solution
The fact that they are equal is coincidence, but it
does illustrate the advantage of Partial Pivoting
 x1   0 
X  calculated   x2    1
 x3   1 
X  exact
 x1   0 
  x 2    1
 x3   1 
THE END
http://numericalmethods.eng.usf.edu
Determinant of a Square Matrix
Using Naïve Gauss Elimination
Example
http://numericalmethods.eng.usf.edu
Theorem of Determinants
If a multiple of one row of [A]nxn is added or
subtracted to another row of [A]nxn to result in
[B]nxn then det(A)=det(B)
Theorem of Determinants
The determinant of an upper triangular matrix
[A]nxn is given by
det A   a11  a22  ...  aii  ...  ann
n
  a ii
i 1
Forward Elimination of a
Square Matrix
Using forward elimination to transform [A]nxn to an
upper triangular matrix, [U]nxn.
Ann  U  nn
det  A  det U 
Example
Using naïve Gaussian elimination find the
determinant of the following square
matrix.
5 1
 25
 64

8
1


144 12 1
Forward Elimination
Forward Elimination: Step 1
 25 5 1
 64 8 1


144 12 1
25
Divide Equation 1 by 25 and
64
 2.56 .
multiply it by 64,
25
5 1 2.56  64 12.8 2.56
.
Subtract the result from
Equation 2
Substitute new equation for
Equation 2
64
 64
0
8
12.8
1
2.56
 4.8  1.56
5
1 
 25
 0  4.8  1.56


144 12
1 
Forward Elimination: Step 1 (cont.)
5
1 
 25
 0  4.8  1.56 Divide Equation 1 by 25 and

 multiply it by 144, 144  5.76 .
144 12
1 
25
25 5 1 5.76  144 28.8 5.76
.
Subtract the result from
Equation 3
Substitute new equation for
Equation 3
144 12
 144 28.8
0  16.8
1
5.76
 4.76
5
1 
25
 0  4.8  1.56 


 0  16.8  4.76
Forward Elimination: Step 2
5
1 
25
 0  4.8  1.56 


 0  16.8  4.76
0
Divide Equation 2 by −4.8
and multiply it by −16.8,
 16.8
 3.5 .
 4. 8
 4.8  1.56  3.5  0  16.8  5.46
.
Subtract the result from
Equation 3
Substitute new equation for
Equation 3
0
 0
0
 16.8  4.76
 16.8  5.46
0
0 .7 
5
1 
25
 0  4.8  1.56


 0
0
0.7 
Finding the Determinant
After forward elimination
5
1 
 25 5 1
25
 64 8 1   0  4.8  1.56




0
0.7 
144 12 1
 0
.
det A   u11  u22  u33
 25   4.8 0.7
 84.00
Summary
-Forward Elimination
-Back Substitution
-Pitfalls
-Improvements
-Partial Pivoting
-Determinant of a Matrix
Additional Resources
For all resources on this topic such as digital audiovisual
lectures, primers, textbook chapters, multiple-choice tests,
worksheets in MATLAB, MATHEMATICA, MathCad and
MAPLE, blogs, related physical problems, please visit
http://numericalmethods.eng.usf.edu/topics/gaussian_elimi
nation.html
THE END
http://numericalmethods.eng.usf.edu