Math 552
Scientific Computing II
Spring 2010
SOLUTIONS: Homework Set 2
1. Write a MATLAB m-file (function) called mylusolve.m that takes the output a and p from
your mylu.m m-file, along with a righthand side b, and computes the solution of Ax = b
by performing the forward and backward substitution steps. The first line of your code
mylusolve.m should be as follows:
function x = mylusolve(a,p,b)
(your code here!)
Execute the following commands and turn in a copy of mylusolve.txt:
>>
>>
>>
>>
>>
>>
>>
diary mylusolve.txt
type mylusolve.m
a = rand(4); b = sum(a,2);
[a,p] = mylu(a);
x = mylusolve(a,p,b)
norm(x-[1 1 1 1]’,’inf’)
diary off
ANS: Here is mylusolve.txt:
type mylusolve.m
function x = mylusolve(a,p,b)
%
n = length(p);
x = zeros(n,1); z = zeros(n,1);
% forward sub to solve Lz=b
z(1) = b(p(1));
for i = 2:n
s = 0;
for j = 1:i-1
s = s+a(p(i),j)*z(j);
end
z(i) = b(p(i))-s;
end
% backsub to solve Ux=z
x(n) = z(n)/a(p(n),n);
for i = n-1:-1:1
s = 0;
for j = i+1:n
s = s+a(p(i),j)*x(j);
end
x(i) = (z(i)-s)/a(p(i),i);
end
a = rand(4); b = sum(a,2);
[a,p] = mylu(a);
x = mylusolve(a,p,b)
x =
1.0000
1.0000
1.0000
1.0000
norm(x-[1 1 1 1]’,’inf’)
ans =
8.8818e-16
diary off
2. The famous Hilbert matrices are given by Hij = 1/(i + j − 1). The n × n Hilbert matrix
Hn is easily produced in MATLAB using hilb(n). Assume the true solution of Hn x = b
for a given n is x = [1, . . . , 1]T . Hence the righthand side b is simply the row sums of Hn ,
and b is easily computed in MATLAB using b=sum(hilb(n)’)’. Use your codes mylu.m
and mylusolve.m to solve the system Hn x = b for n = 5, 10, 15, 20. For each n, using the
∞ − norm, compute the relative error and the relative residual. Discuss what is happening
here. You may find it useful to look at the cond command in MATLAB.
ANS: Here is a code to do the computations and the output:
n = [5 10 15 20]’;
rel_err = zeros(4,1);
rel_resid = zeros(4,1);
cond_H = zeros(4,1);
for i = 1:length(n)
H = hilb(n(i));
% generate matrix
b = sum(H,2);
% sum rows for RHS b
[H,p] = mylu(H);
% solve system
x = mylusolve(H,p,b);
rel_err(i) = max(abs(ones(n(i),1)-x)); % note ||x||_inf = 1
H = hilb(n(i));
% we need the original H
rel_resid(i) = max(abs(b-H*x))/max(abs(b));
cond_H(i) = cond(H,’inf’);
end
disp(’
disp(’
disp([n
n
rel_err
rel_resid
cond ’)
-------------------------------------------------’)
rel_err rel_resid cond_H]);
Warning: Matrix is close to singular or badly scaled.
Results may be inaccurate. RCOND = 9.144066e-19.
Warning: Matrix is close to singular or badly scaled.
Results may be inaccurate. RCOND = 1.291654e-19.
n
rel_err
rel_resid
cond
------------------------------------------------5.0000e+00
5.5467e-13
1.9449e-16
9.4366e+05
1.0000e+01
1.3982e-04
1.5162e-16
3.5353e+13
1.5000e+01
2.4345e+01
4.0150e-16
1.0975e+18
2.0000e+01
4.7309e+01
3.7031e-16
9.2117e+18
So we see that while for each n the relative residual is on the order of machine precision, the
relative error is growing. This is directly attributable to the growth in the condition number
of the Hilbert matrices. In fact, I’ve included the warnings that MATLAB displayed for
n = 15, 20 to note that it is having trouble accurately computing Hn−1 .
3. This probem emphasizes the difficulty of computing interpolating polynomials in standard
form. The matrices which arise are called Vandermonde matrices, and can be quite illconditioned. Consider the polynomial of degree n that interpolates a set of data {Fi } at
the points {xi } in the form
Pn (x) =
n
X
ai xi
i=0
and has the property that
Pn (xi ) = Fi .
(a) Write down the matrix system (for a general n) that must be solved to compute the
ai ’s.
(b) Compute the coefficients for the polynomial that interpolates the function
F (x) =
n
X
xi
i=0
at the n + 1 points xi = 1 + ni for i = 0, . . . , n (i.e. Fi = F (xi )). Do this for
n = 7, 9, 11, 13. The MATLAB code for n = 7 is
>>
>>
>>
>>
>>
>>
format long e
n=7;
x=(1+(0:1/n:1))’;
V=fliplr(vander(x));
F = sum(V,2);
a = V\F;
You should use the help in MATLAB to make sure you understand what each of
these commands. Note that fliplr is used so that V corresponds to the the matrix
from part (a), as well as the definition of V given in class last term.
(c) Compute the ∞-norm condition number of each Vandermode matrix V in part (b).
Use MATLAB’s cond command for this.
(d) Compute the relative error in the ∞-norm of the computed coefficients ai . The true
answer is given by the nth degree polynomial with ai = 1 for i = 0, . . . , n. Discuss
the results in lieu of the condition numbers from (c).
ANS: (a) For each i we have an equation,
one must solve is given by

1 x0
x20 · · ·
 1 x1
x21 · · ·

 ..
..
..
 .
.
.

 1 xn−1 x2n−1 · · ·
1 xn
x2n · · ·
and the (n + 1) × (n + 1) system V a = F that
xn0
xn1
..
.
xnn−1
xnn

a0
a1
a2
..
.






an
For parts (b)-(d) here is a MATLAB code and ouput:
CODE:


 
 
 
=
 
 
F0
F1
F2
..
.
Fn




.


n = [7 9 11 13]’;
rel_err = zeros(4,1);
cond_V = zeros(4,1);
format long e
for i = 1:length(n)
x = (1+(0:1/n(i):1))’;
% interpolation nodes
V = fliplr(vander(x));;
% generate Vandermonde matrix
cond_V(i) = cond(V,’inf’); % compute condition number
F = sum(V,2);
% sum rows for RHS b
a = V\F;
% solve system
disp([’n = ’, num2str(n(i))]), disp([’a = ’]), disp(a) disp(’ ’)
rel_err(i) = max(abs(ones(n(i)+1,1)-a)); % note ||a_true||_inf = 1
end
disp(’
disp(’
disp([n
n
rel_err
cond
’)
--------------------------------------------------------------------’)
rel_err cond_V]);
OUTPUT:
n = 7
a =
9.999999995647673e-01
1.000000002074155e+00
9.999999958136149e-01
1.000000004640408e+00
9.999999969480911e-01
1.000000001191393e+00
9.999999997442813e-01
1.000000000023290e+00
n = 9
a =
1.000000163540988e+00
9.999989784432664e-01
1.000002815820528e+00
9.999955041820527e-01
1.000004582815625e+00
9.999969066011437e-01
1.000001382893095e+00
9.999996051174539e-01
1.000000065365618e+00
9.999999952202283e-01
n = 11
a =
1.000021007657227e+00
9.998395822077357e-01
1.000553854312332e+00
9.988586628504313e-01
1.001559913058271e+00
9.985151562201965e-01
1.001004538974869e+00
9.995169442902384e-01
1.000161821998705e+00
9.999640301683905e-01
1.000004775098920e+00
9.999997131626831e-01
n = 13
a =
9.914092903599439e-01
1.078480422008933e+00
6.704572440779317e-01
1.842170357231789e+00
-4.614436788171800e-01
2.818571685511569e+00
-6.695045903943608e-01
2.144875334185679e+00
4.135212097342278e-01
1.221659184732196e+00
9.399191271078082e-01
1.011059923579160e+00
9.987606385808471e-01
1.000063852101455e+00
n
rel_err
cond
-------------------------------------------------------------------7.000000000000000e+00
4.640407613720754e-09
8.980828455113095e+08
9.000000000000000e+00
4.582815625431280e-06
5.035429847646414e+11
1.100000000000000e+01
1.559913058270634e-03
3.061787776747201e+14
1.300000000000000e+01
1.818571685511569e+00
1.938482966085804e+17
So for part (b), we see as n gets larger the computed solution begins to diverge from
the exact solution, which is a vector of all ones. For part (c), looking at the output just
above, in particular the infinity-norm condition number of the corresponding Vandermonde
matrix, we can see that it grows rapidly, and is already O(1017 ) for n = 13, indicating
that these systems are indeed highly ill-conditioned. For (d), looking again at the table
above, we see that the difference in the magnitude of the relative error and condition
number is about, again, O(1017 ). That is, there is a direct correlation, as expected, in the
relative error and the condition number. When the latter is large we generally start to
lose accuracy in the computed solution.
4. Consider the matrix, right side vector, and two approximate solutions,
1.2969 0.8648
0.8642
0
0.9911
A=
, b=
, x1 =
, x2 =
.
0.2161 0.1441
0.1440
1
−0.4870
(a) Show that x = [2, −2]T is the exact solution of Ax = b.
(b) Compute the error and residual vectors for x1 and x2 .
(c) Use MATLAB to find ||A||∞ , ||A−1 ||∞ , and κ∞ (A).
(d) In class we proved a theorem relating the condition number of A, the relative error,
and the relative residual. Check this result for the two approximate solutions x1 and
x2 (using the ∞ − norm).
ANS: For (a), a direct calculation gives
1.2969 0.8648
2
2 ∗ 1.2969 − 2 ∗ 0.8648
0.8642
Ax =
=
=
=b
0.2161 0.1441
−2
2 ∗ 0.2161 − 2 ∗ 0.1441
0.1440
Using MATLAB, for (b) we have
>> A = [1.2969 0.8648; 0.2161 0.1441]; b = [0.8642 0.1440]’;
>> x1 = [0 1]’; x2 = [0.9911 -0.4870]’; x = [2 2]’;
>> e1 = x-x1, r1 = b-A*x1
e1 =
2
1
r1 =
-0.000600000000000045
-0.000100000000000017
>> e2 = x-x2, r2 = b-A*x2
e2 =
1.0089
2.487
r2 =
1.00000001612699e-08
-9.99999999473644e-09
For (c), again using MATLAB
>> A_inf = norm(A,’inf’), Ainv_inf = norm(inv(A),’inf’), A_condinf = cond(A,’inf’)
A_inf =
2.1617
Ainv_inf =
151300000.022015
A_condinf =
327065210.047589
So while the condition number of A is small, the condition number of A−1 is quite large,
resulting in kAkkA−1 k = κ(A) ≈ 3.27 × 108 .
For (d), we proved in class that for any norm and corresponding induced matrix norm
kek
krk
krk
≤ κ(A)
= kAkkA−1 k
.
kxk
kbk
kbk
To see that this holds here in the infinity norm,
>> norm(e1,’inf’)/norm(x,’inf’) , A_condinf*norm(r1,’inf’)/norm(b,’inf’)
ans =
1
ans =
227076.054187188
>> norm(e2,’inf’)/norm(x,’inf’) , A_condinf*norm(r2,’inf’)/norm(b,’inf’)
ans =
1.2435
ans =
3.78460096415374
which indeed it does. Note that the relative residual for x1 is much larger than that for
x2, while the error in x2 is larger.
5. (Matrix norms and Spectral radius)
(a) Consider the matrix,


2 −3 1
A =  −4 1 2  .
5
0 1
Compute kAk∞ and find a vector x such that kAk∞ = kAxk∞ /kxk∞ .
(b) Find an example of a 2 × 2 matrix A such that kAk∞ = 1 but ρ(A) = 0. Explain
why this shows that the spectral radius ρ(A) is not a matrix norm.
(c) Given an n × n matrix A, prove that the 1-norm of A, kAk1 , equals the maximum
absolute column sum of A, namely:
kAk1 = max kAxk1 = max
1≤j≤n
kxk1 =1
n
X
|aij |.
i=1
ANS:
(a) Since kAk∞ is the maximum absolute column sum of A, it is easy to see that kAk∞ =
11. Letting x = [1 −1 1]T , we see that kxk∞ = 1 and kAxk∞ /kxk∞ = k[2 4 5]T k∞ /1 = 11.
0 0
. Then it is easily seen that kAk∞ = 1 but ρ(A) = 0. This follows
(b) Let A =
1 0
since the infinity norm of A is just the maximum absolute column sum, and for a lower
triangular matrix the eigenvalues lie on the diagonal. Since they are both 0, the spectral
radius is 0. This show that ρ(A) is not a matrix norm, for we have found a non-zero
matrix for which ρ(A) = 0. This violates one of the properties that any matrix norm must
satisfy, namely, kAk = 0 ⇐⇒ A = 0.
(c) For any n-vector x 6= 0 we have
kAxk1 =
Pn
i=1 |
Pn
j=1 aij xj |
≤
Pn Pn
=
Pn
j=1 |aij ||xj |
i=1
j=1 |xj |
Pn
≤ (max1≤j≤n
i=1 |aij |
Pn
i=1 |aij |) kxk1
Dividing by kxk1 6= 0 shows that
kAxk1
max
≤
x6=0 kxk1
max
1≤j≤n
n
X
!
|aij | .
i=1
To show that there is some x 6= 0 for which equality is achieved, let k be such that
max
1≤j≤n
n
X
|aij | =
n
X
i=1
|aik |,
i=1
then if ek is the k-th unit vector and ak the k-th colum of A,
kAek k1 = kak k1 =
n
X
i=1
|aik | = kAk1 ,
and we have thus proven that
kAk1 = max
1≤j≤n
n
X
i=1
|aij | = maximum absolute row sum of A.