Matrix Norms
Lemma: Given any vector norm ⋅ and any non-zero vector x ,
1
x is a unit vector. That is
x
1
x = 1.
x
Theorem 1.: Given vector norms ⋅ α and ⋅ γ defined for m − and n − dimensional real vector
spaces, respectively, , define the real values function on the space of m × n real matrices,
N ( A) = max{ Ax }
x γ =1
α
for all m × n real matrices A . This function is a norm.
Proof:
Since ⋅ γ is a vector norm, we have that for any x , Ax
α
≥ 0 . Thus
N ( A) = max{ Ax } ≥ 0 .
x γ =1
α
If A is the zero matrix, then for any x , Ax
α
= 0 , so max{ Ax } = 0 . If A is
x γ =1
α
not the zero matrix, then it must have a non-zero element - call it ai , j . Letting e j
be the j th coordinate vector, then with e j =
Ae j
α
> 0 , and e j
γ
1
e j , we have Ae j ≠ 0 so
ej γ
= 1 so N ( A) = max{ Ax } ≥ Ae j
x γ =1
α
a,
N ( aA) = max{ aAx }
x γ =1
α
= max{| a | Ax }
x γ =1
α
=| a | max{ Ax }
x γ =1
α
=| a | N ( A).
Lastly,
N ( A + B ) = max{ ( A + B ) x }
x γ =1
α
= max{ Ax + Bx }
x γ =1
α
≤ max{ Ax + Bx }
x γ =1
α
α
≤ max{ Ax } + max{ Ax }
x γ =1
α
≤ N ( A) + N ( B ).
Thus, N is a norm. ▪
x γ =1
α
α
> 0. For any real value
Theorem 2 gives an equivalent definition of the function N .
Theorem 2.: The function N defined in Theorem 1 satisfies,
Ax
α
N ( A) = sup{
}.
xγ
x ≠0
for all m × n real matrices A .
Proof: For any γ − norm unit vector x ,
Ax
α
Ax =
α
xγ
≤ sup{
.
Ax
x
x ≠0
α
}
γ
Since this holds for any γ − norm unit vector x ,
Ax
α
N ( A) = max{ Ax } ≤ sup{
}
α
x γ =1
xγ
x ≠0
Also for any γ − norm unit vector x ,
Ax α ≤ max{ Ax } = N ( A).
x γ =1
α
We have
N ( A) ≤ sup{
x ≠0
Ax
x
α
} ≤ N ( A) ,
γ
so the theorem is established. ▪
Given vector norms ⋅ α and ⋅ γ defined for m − and n − dimensional real vector spaces,
and matrix norm ⋅ β defined for m × n real matrices, we say the matrix norm is consistent
with the vector norms if for all m × n real matrices A and all real n − vectors x , the real
m − vector Ax satisfies
Ax α ≤ A ⋅ β x γ .
Theorem 3 establishes that the norm defined in Theorem 1 is the minimal consistent matrix norm. As such, we term it the matrix norm subordinate to the vector norms ⋅ α and ⋅ γ .
Theorem 3.: Given vector norms ⋅ α and ⋅ γ defined for m − and n − dimensional real vector
spaces, and matrix norm ⋅ β defined for m × n real matrices consistent with the vector norms,
N ( A) ≤ A β ,
for all m × n real matrices A .
Proof:
For any non-zero, real n − vectors x , Ax α ≤ A ⋅ β x γ , so
Ax α
x
≤ A⋅ β
γ
The right hand side is independent of x , so
Ax
α
sup{
} ≤ A⋅ β ,
xγ
x ≠0
but according to Theorem 2, N ( A) = sup{
x ≠0
Ax
x
α
}. ▪
γ
As an example, we show that with both vector norms taken as the ∞ norm (on m − and
n − dimensional real vector spaces, respectively) the subordinate vector norm is equal to
the maximal rows sum of absolute values.
Theorem 4: Let A be an m × n real matrix. Then
n
max{∑ | ai , j |} = max{ Ax ∞ } .
i =1,...,m
x ∞ =1
j =1
Thus, the matrix norm subordinate to the ⋅ ∞ vector norm is
n
A ∞ = max{∑ | ai , j |} .
i =1,...,m
j =1
Proof:
Consider any ∞ norm unit vector x . Since each of its components satisfies
| x j |≤ 1 , we have for i = 1,..., m
n
n
j =1
j =1
∑ ai, j x j ≤ ∑ | ai , j x j |
n
≤ ∑ | ai , j || x j |
j =1
n
≤ ∑ | ai , j | .
j =1
Thus by taking the maximum over i,
n
n
| ∑ ai , j x j | ≤ max ∑ | ai , j |,
i =1,...,m
j =1
j =1
holds for any ∞ norm unit vector x , thus
n
Ax
∞
n
= max | ∑ ai , j x j | ≤ max ∑ | ai , j |,
i =1,...,m
i =1,...,m
j =1
j =1
and by taking the maximum over all such vectors, we have
n
max{ Ax ∞ } ≤ max{∑ | ai , j |} .
x ∞ =1
i =1,...,m
j =1
For a fixed matrix A , let k be the index of a row with maximal row sum - that is,
n
∑| a
j =1
n
k, j
| = max{∑ | ai , j |} .
i =1,...,m
j =1
Now define a vector x as
 1 if ak , j ≥ 0
.
xj = 
 −1 if ak , j < 0
Notice that for j = 1,..., n , ak , j x j =| ak , j x j |=| ak , j | , so
n
n
max ∑ | ai , j |= ∑ | ak , j |
i =1,...,m
j =1
j =1
n
= ∑ ak , j x j
j =1
n
∑a
=
j =1
≤ max
i =1,...,m
k, j
xj
n
∑a
i, j
xj
j =1
= Ax ∞
This hold for a particular ∞ norm unit vector x , so by taking a maximum
n
max ∑ | ai , j |≤ max Ax ∞ .
i =1,...,m
x ∞ =1
j =1
We have now
n
max Ax
x ∞ =1
∞
≤ max ∑ | ai , j |≤ max Ax ∞ ,
i =1,...,m
x ∞ =1
j =1
so
Ax
∞
= max Ax
x ∞ =1
n
∞
= max ∑ | ai , j | . ▪
i =1,...,m
j =1