Proof that Gaussian Elimination with Partial Pivoting Works Our main result is: Theorem 1. For any n × n matrix A Gaussian elimination with partial pivoting (GEPP) succeeds in constructing a unit lower triangular matrix L, an upper triangular matrix U and a permutation matrix P such that P A = LU. (1) A is singular if and only if a diagonal entry of U is zero. To prove Theorem 1 we will first prove: Theorem 2. Suppose at the k th step in Gaussian elimination without pivoting (GENP), for each k, 1 ≤ k ≤ n − 1, that the current diagonal entry ukk 6= 0. Then GENP succeeds in constructing a unit lower triangular matrix L and an upper triangular matrix U such that A = LU. We also prove: Remark 1. For k = 1, . . . , n, let Akk be the k × k upper left corner (the k by k leading principal submatrix) of A. Then GENP succeeds if Akk is nonsingular for k = 1, . . . , n. Proof. (Theorem 2) We will use induction. The induction hypothesis is that at step k of GENP there exists matrices Lk and Uk such that A = Lk Uk where Lk is the identity matrix except the first k − 1 columns below the diagonal are nonzero and Uk is upper triangular in the first k − 1 columns but the remaining columns have no special structure. Since IA = A the induction hypothesis is true for k = 1. Assume that the hypothesis is true for k. We need to show that the hypothesis is true for k + 1. Consider the structure of A = Lk Uk at step k: A = Lk Uk = 1 `21 .. . `k1 .. . . .. ` i1 . .. ∗ .. . .. .. . ··· . `k,k−1 1 ··· ∗ .. . 0 .. . ··· ··· ··· ··· `i,k−1 .. . ∗ 1 0 0 0 .. .. . . 0 0 .. . .. . ··· ··· 1 .. . ··· .. . 0 u11 0 .. . 0 .. . . .. . .. . .. 1 0 ··· .. . .. . ··· ··· ··· ··· u1k .. . uk−1,k−1 0 uk−1,k ukk ··· ··· 0 .. . ∗ .. . ··· 0 .. . uik .. . ··· 0 unk ··· ··· ··· ··· ··· ··· ··· ··· ··· u1j .. . ··· uk−1,j ukj .. . .. . ··· ··· uij .. . unj ··· ··· ··· ··· ··· ··· ∗ .. . ∗ ∗ ∗ ∗ ∗ ∗ ∗ (2) Now for i = k, . . . , n, j = k, . . . , n consider component aij of A in A = Lk Uk . aij is formed by u1j .. multiplying the ith row of Lk by the j th column of Uk . Let c = (`i1 , . . . , `i,k−1 ) and d = . . uk−1,j By the induction hypothesis, A = Lk Uk , we conclude from (2) that aij = c • d + uij , for i = k, . . . , n, j = k, . . . , n. 1 (3) When we apply Gaussian elimination we update Lk by adding a column of multipliers below the diagonal in column k of the current L. Therefore, in Lk+1 the first k − 1 columns are identical to the first k − 1 columns of Lk , but `ik = uik /ukk , i = k + 1, . . . , n. (4) Since ukk 6= 0, the multipliers are defined. We also update the lower right hand corner of the current U by making column k zero below the diagonal and subtracting multiples of row k to the rows below row k . Therefore the elements uij , i = k + 1, . . . , n, j = k, . . . , n (the lower right corner of U) are changed. Let these changed elements be represented by u bij , i = k + 1, . . . , n, j = k, . . . , n. Then u bik = 0, i = k + 1, . . . , n. Also, since in elimination we subtract the multiple `ik of row k from row i, for i > k, then u bij = uij − `ik ukj for i = k + 1, . . . , n, j = k + 1, . . . , n. (5) b = Lk+1 Uk+1 = Let A ∗ . .. `21 0 . ··· ··· · · · .. .. .. .. .. . . . . u u · · · u · · · ∗ k−1,k−1 k−1,k k−1,j `k1 · · · `k,k−1 1 0 0 u · · · u · · · ∗ kk kj .. .. . ··· ∗ ∗ 1 ··· 0 0 ∗ u bk+1,j · · · ∗ . . . .. .. .. .. .. .. .. . .. . ··· . . 0 ··· . . . . ··· ∗ . . .. .. . ` . 1 ··· . 0 ∗ u bij ··· ∗ i1 · · · `i,k−1 `ik 0 . . . .. .. .. .. .. .. .. .. .. . .. . . ··· . . . ··· . ··· . . . . ··· ∗ ∗ ··· ∗ ∗ 0 ··· ··· 0 1 0 ··· 0 0 ∗ u bnj ··· ∗ (6) b = A. From To prove that the induction hypothesis is true at step k + 1 we need to show that A the partial triangular structures in (2) and (6) the computations for aij and b aij are exactly the same for i = 1, . . . , k or for j = 1, . . . , k − 1 and therefore b aij and aij are identical for these values of i and j. For i = k + 1, . . . , n, from equation (6), if we multiply row i of Lk+1 by column k of Uk+1 , it follows that 1 b aik = c • d + `ik ukk = c • d + u11 ··· .. . .. . ··· ··· u1k .. . ··· uik ukk = c • d + uik = aik . ukk u1j .. . ··· (7) The last equality in (7) follows from (3). For i = k + 1, . . . , n and j = k + 1, . . . , n, also it follows by Equation (6), multiplying row i of Lk+1 by column j of Uk+1 , that b aij = c • d + `ik ukj + u bij = c • d + `ik ukj + (uij − `ik ukj ) = c • d + uij = aij . (8) b = A, In the second equality we have used (5) and in the last equality we have used (3). This shows that A proving the induction step and the theorem. Proof. (Theorem 1) Before beginning the proof we need a bit of background on elementary permutation matrices. Pki is an elementary permutation matrix if Pki A interchanges or switches rows k and i of A. Pki Pki A = A since, if we switch rows k and i of A and then switch them again, the original matrix is −1 recovered. Therefore, Pki Pki = I (and therefore Pki = Pki ). Pki is the identity matrix, except rows k 0 0 0 1 0 1 0 0 and i of I are switched. As an example, if n = 4 then P1,4 = 0 0 1 0 . Because of this structure, 1 0 0 0 if Pki is applied on the right then APki switches columns k and i of A. 2 To prove (1) we will use induction. The induction hypothesis is that at step k of GEPP there exists matrices Lk , Uk and Pk such that Pk A = Lk Uk where Pk is a permutation matrix, Lk is the identity matrix except the first k − 1 columns below the diagonal are nonzero and Uk is upper triangular in the first k − 1 columns but the remaining columns have no special structure. Since IA = IA the induction hypothesis is true for k = 1. Assume that the hypothesis is true for k. We need to show that the hypothesis is true for k + 1. Note that Lk and Uk in Pk A = Lk Uk have the same structure as Lk and Uk in Equation (2). Consider the special case that column k of Uk is entirely zero on or below the diagonal entry ukk in column k. Then Uk is upper triangular in its first k columns. Therefore we can let Pk+1 = Pk , Lk+1 = Lk and Uk+1 = Uk . Since, in this case, Pk A = Lk Uk = Pk+1 A = Lk+1 Uk+1 , the induction hypothesis is true for k + 1. Now consider the more common case that column k of Uk is not entirely zero on or below the diagonal entry ukk in column k. Let uik 6= 0 for i ≥ k be a largest magnitude entry on or below ukk in column k (the proof also works if we choose any non-zero uik ). Since Pki Pki = I and by our induction hypothesis Pk A = Lk Uk = Lk IUk = Lk Pki Pki Uk . Therefore Pk A = 1 u11 · · · ··· u1k · · · u1j ··· .. .. .. .. `21 0 . . ··· . ··· . ··· .. .. .. .. .. .. . . . . . uk−1,k−1 . · · · uk−1,j · · · `k1 · · · `k,k−1 1 0 ··· 0 u · ·· ukj ··· kk .. .. .. . Pki Pki . ··· ∗ 0 1 ··· 0 ∗ ··· . ··· . . . . . . . . .. .. 0 .. .. .. .. .. .. ··· ··· ··· ··· . . .. ` .. 1 ··· 0 uik · · · uij ··· i1 · · · `i,k−1 0 0 . . . . . .. .. .. .. .. . . . . .. . . ··· . . . ··· . ··· . . ··· . ··· ∗ ··· ∗ 0 0 ··· ··· 0 1 0 ··· 0 unk ··· unj ··· uk−1,k−1 0 u1k .. . .. . uik 0 .. . ∗ .. . ··· 0 .. . ukk .. . ··· 0 unk ··· (9) ∗ .. . ∗ ∗ ∗ ∗ ∗ ∗ ∗ We now have Pk A = (Lk Pki ) (Pki Uk ) = 1 `21 .. . `k1 .. . .. . . .. `i1 . .. . .. ∗ .. . .. . ··· ··· ··· ··· ··· ··· ··· ··· .. . `k,k−1 ∗ .. . .. . `i,k−1 .. . .. . ∗ 0 0 .. . ··· 1 .. . 0 ··· 1 0 . 0 .. .. .. . . 0 0 ··· .. 0 1 . 0 ··· ··· ··· ··· 1 0 .. . 0 .. . .. .. . ··· . ··· ··· 1 .. . .. ··· 0 . u11 0 .. . 0 .. . .. . .. . .. . 0 ··· .. . .. . ··· ··· ··· ··· ··· ··· ··· ··· ··· ··· ··· ··· ··· ··· u1j .. . ··· uk−1,j uij .. . .. . ··· ··· ukj .. . ··· unj 1 (10) Introducing Pki Pki has switched two rows of Uk but, fortunately, this has not changed the structure of the right hand matrix. However, introducing Pki Pki has given the left hand matrix in (10) the wrong structure – it is no longer lower triangular. There is an cure for this incorrect structure. If we multiply Pki by Pk A = (Lk Pki ) (Pki Uk ) then Pki Pk A = (Pki Lk Pki ) (Pki Uk ). In this equation Pki (Lk Pki ) will 3 ··· ··· ··· ··· ··· ∗ .. . ∗ ∗ ∗ ∗ ∗ ∗ ∗ switch rows k and i of the left hand matrix in (10). Switching these rows is just the operation that we need to recover the desired lower triangular structure. Therefore, if we let Pk+1 = Pki Pk we now have Pk+1 A = (Pki Pk )A = Pki (Pk A) = Pki (Lk Uk ) = (Pki Lk Pki ) (Pki Uk ) = 1 `21 .. . `i1 .. . . .. ` k1 . .. ∗ .. . .. .. . ··· . `i,k−1 1 ··· ∗ .. . 0 .. . ··· ··· ··· ··· `k,k−1 .. . ∗ 1 0 0 0 .. .. . . 0 0 .. . .. . ··· ··· 1 .. . ··· .. 0 . u11 0 .. . 0 .. . . .. . .. . .. 0 1 ··· .. . .. . ··· ··· ··· uk−1,k−1 0 u1k .. . .. . uik 0 .. . ∗ .. . ··· 0 .. . ukk .. . ··· 0 unk ··· ··· ··· ··· ··· ··· ··· ··· ··· ··· ··· ··· u1j .. . ··· uk−1,j uij .. . .. . ··· ··· ukj .. . ··· unj ··· ··· ··· ··· ··· ∗ .. . ∗ ∗ ∗ ∗ ∗ ∗ ∗ (11) Note that in (11) rows i and k of Uk have been switched and, as a consequence of the trick to recover the lower triangular structure of the left hand matrix, the first k − 1 entries of rows i and k of Lk have also been switched. Now the matrices in (11) have the same structure as the matrices in (2) and, by our row interchanges, the diagonal entry, uik , of the right hand matrix is not zero. Therefore we can apply elimination to the matrices in equation (11), exactly as we did as in the proof of Theorem 2. After the elimination step we will have Pk+1 A = Lk+1 Uk+1 where Lk+1 and Uk+1 have the proper structure: Lk+1 is the identity except for the strict lower triangle in the first k columns and the first k columns of Uk+1 are upper triangular. This proves the induction step. Therefore (1) is true. Finally, note by properties of determinants that (1) implies det(P ) det(A) = det(L) det(U ). The determinant of a permutation matrix is either +1 or -1 and the determinant of a triangular matrix is the product of the diagonal entries. Since the diagonal entries of L are all one it follows that det(A) = ±u11 u22 . . . unn . Therefore det(A) is zero if and only if some uii is zero. Since A is singular if and only if det(A) = 0, Theorem 1 follows. Proof (Remark 1) Suppose GENP succeeds up to the start of step k for some k ≤ n. Then (2) is true. Let Lkk be the k by k leading principal submatrix of Lk and let Ukk be the k by k leading principal submatrix of Uk . By the partial triangular structure of Lk and Uk in (2), Akk = Lkk Ukk . Therefore det(Akk ) = det(Lkk ) det(Ukk ) = u11 u22 · · · ukk . (12) The last equality follows since the determinant of a triangular matrix is the product of the diagonal entries of the matrix and since the diagonal entries of Lkk are one. Since det(Akk ) 6= 0 then by (12) ukk 6= 0 (and uii 6= 0 for i = 1, . . . , k). By Theorem 2 GENP succeeds. 4
© Copyright 2026 Paperzz