ME(EE) 550
Foundations of Engineering Systems Analysis
Chapter Three: Normed Vector Spaces
The concepts of convergence and continuity in models of physical processes
have been discussed in Chapter One on metric spaces and topological spaces. The
notion of linearity in algebraic structures of vector spaces has been presented in
Chapter Two. The treatment of analysis in these two chapters was intentionally
made independent of each other. In other words, Chapter Two could have been
covered before Chapter One. The purpose of this Chapter Three is to present
a special class of vector spaces that would combine the two notions: continuity
and linearity. This would ultimately lead to continuous linear transformations
that include linear functionals and linear operators as they are extensively used in
Engineering, Physics, and other disciplines.
The underlying vector space, presented in this chapter, synergistically combines
the topological structure of metric spaces and the algebraic structure of vector
spaces to yield normed vector spaces. The combination is made in such a way that
these two structures, namely, topological and algebraic, are compatible in the sense
that the addition and multiplication operations are continuous and the resulting
metric is readily related to the scalar field of the vector space. This chapter should
be read along with Chapter 5 Part A of Naylor & Sell. Specifically, both solved
examples and exercises in Naylor & Sell are very useful.
1
Basic Concepts
Let (V, ⊕) be a vector space over a (complete) field F, where we choose the field
F = (F, +, ·) to be either R or C. [Note: Both R and C are complete fields. The
field Q is not considered because it is not complete in the usual metric.] We wish
to introduce a distance d : V × V → [0, ∞) between a pair of points in V . This will
allow us to define various mathematical entities such as limits, convergent sequences
and series, and continuous mappings. Since V has the algebraic structure of a
vector space, the distance that we will define must be consistent with this algebraic
structure. To this end, we will require the following properties:
Property 01 Translation invariance: d(x, y) = d(x ⊕ z, y ⊕ z) ∀x, y, z ∈ V .
Property 2 Positive homogeneity: d(λx, λy) = |λ|d(x, y) ∀ x, y ∈ V and ∀λ ∈ F.
Property 3 Convexity: Every open ball B(x0 , r) , {x ∈ V : d(x, x0 ) < r} is a
convex set.
1
The invariance under translation implies that the distance d is entirely determined if we specify the function x 7→k x k= d(x, 0), i.e., the distance of a point x
from the origin. This is what we call the norm of a vector, which is formally defined
below.
Definition 1.1. (Norm) Let (V, ⊕) be a vector space over a (complete) field F.
Then, a function k • k : V → [0, ∞) is called a norm if the following conditions hold
∀x, y ∈ V ∀α ∈ F:
• Strict positivity: kxk = 0 iff x = 0V
• Homogeneity: kαxk =| α | kxk
• Triangular Inequality: kx ⊕ yk ≤ kxk + kyk
A vector space with a norm is called a normed vector space and is denoted as the
pair (V, k • k).
Remark 1.1. If all of the above conditions of a norm except kxk = 0 ⇒ x = 0V
is satisfied, then we introduce the notion of seminorm p(•) by setting d(x, y) =
p(x − y), where it is not guaranteed that we obtain a distance on the vector space.
Indeed, we may have d(x, y) = p(x − y) = 0 even if x 6= y. There are, however,
cases such as the Lp -space where one may convert a seminorm into a norm by
constructing equivalence classes of seminorms. See, for example, p. 66 of Real and
Complex Analysis by Rudin.
Definition 1.2. (Separating seminorm) A sequence {pn } of seminorms is called
separating if, for every x ∈ V \ {0}, there exists at least one index k such that
pk (x) > 0.
Lemma 1.1. (Distance generated by seminorms) Let {pn } be a sequence of separating seminorms on a vector space V . Then, a distance on V can be defined
as:
∞
X
pk (x − y)
d(x, y) ,
2−k
(1)
1 + pk (x − y)
k=1
Proof. The identities d(x, x) = 0 and d(x, y) = d(y, x) are immediate consequences
of strict positivity and homogeneity in Definition 1.1. The assertion that the sequence {pn } is separating guarantees that d(x, y) > 0 if x 6= y.
Now, what remains to complete the proof is triangular inequality. We observe
that if the real numbers a, b, c ≥ 0 and c ≤ (a + b), then it follows that
a
c
a+b
b ≤
≤
+
1+c
1+a+b
1+a 1+b
s
because the function s 7→ 1+s
is increasing and concave down. By using the property of triangular inequality in Definition 1.1 with a ≡ pk (x − y), b ≡ pk (y − z) and
c ≡ pk (z − x), it follows that
d(x, z) =
∞
X
k=1
2
−k
∞
X
pk (x − y)
pk (y − z)
pk (x − z)
−k
≤
2
+
1 + pk (x − z)
1 + pk (x − y) 1 + pk (y − z)
k=1
i.e., d(x, z) ≤ d(x, y) + d(y, z). Hence, d(•, •) is indeed a distance.
2
If the vector space V with the distance in Eq. (1) is a complete metric space,
then V is called a Frechét space.
Example 1.1. Examples of Normed Spaces:
1. Let V = Fn and p ∈ [1, ∞). Then, kxkp ,
T
the vector x ≡ [x1 , x2 , · · · , xn ] .
P
n
k=1
|xk |p
1/p
is a norm, where
2. Let V = Fn . Then, the sup norm k•k∞ is defined as: kxk∞ , maxk∈{1,2,··· ,n} |xk |.
Remark 1.2. It follows from the triangular inequality of a norm in V that
| kxk − kyk |≤ kx − yk ∀x, y ∈ V
Remark 1.3. A norm can be considered to be a metric of the vector space by
setting d(x, y) = kx − yk. However, a metric may not satisfy the homogeneity
condition of a norm because there is no notion of a (scalar) field related to a metric
space. So, every normed vector space is a metric space but the converse is not true,
in general.
Proposition 1.1. The norm is a uniformly continuous function.
Proof. Choose δ(ε, x) = ε ∀x. Then, ∀ε > 0 ∀x, y ∈ V , |kxk − kyk| ≤ kx − yk <
δ = ε ⇒ |kxk − kyk| < ε.
Definition 1.3. (Lipschitz Continuity) Let V and W be two normed vector spaces
over the same field F. A function f : V → W is called Lipschitz continuous if
there exists a constant c ∈ (0, ∞), called the Lipschitz constant, such that kf (x) −
f (y)kW ≤ ckx − ykV ∀x, y ∈ V .
Proposition 1.2. Lipschitz continuity ⇒ uniform continuity.
Proof. Choose δ(ε, x) =
δ ⇒ c|kxk − kyk| < ε.
ε
c
∀x. Then, ∀ε > 0 ∀x, y ∈ V , |kxk − kyk| ≤ kx − yk <
Remark 1.4. Uniform continuity ; Lipschitz continuity, in general.
Definition 1.4. (Contraction Mapping) Let V a normed vector space over the field
F. A mapping f : V → V is called a contraction if there exists a constant ρ ∈ (0, 1),
such that kf (x) − f (y)kV ≤ ρkx − ykV ∀x, y ∈ V .
Definition 1.5. (Norm Equivalence) Let V a normed vector space over the field
F. Then, two norms k ∗ ka and k ∗ kb on V are said to be equivalent if there exists
αm , αM ∈ (0, ∞) such that
0 < αm kxka ≤ kxkb ≤ αM kxka ∀ x ∈ V \ {0}
Alternatively,
0<
1
1
kxkb ≤ kxka ≤
kxkb ∀ x ∈ V \ {0}
αM
αm
Remark 1.5. From a topological perspective, the implication of norm equivalence
in Definition 1.5 is that the two norms, considered as respective metrics in the
underlying vector space, generate the same topology,
3
1.1
Finite-dimensional Normed Spaces
This section deals with finite-dimensional normed spaces. The set of finite-dimensional
normed spaces is a proper subset of the set of all normed spaces that include infinitedimensional normed spaces. Nevertheless finite-dimensional normed spaces are useful for analysis of many engineering systems. To this end we present below several
important theorems and lemmas.
Theorem 1.1. (Norm Equivalence in Finite-dimensional Spaces) On a finitedimensional vector space, all norms are equivalent.
Proof. Let V be an n-dimensional vector space over a field F, where n ∈ N. Since
V is isomorphic to Fn , it suffices to show norm equivalence on Fn .
Let a bounded set in Fn be defined as: San−1 , {x ∈ Fn : kxka = 1}. It follows
by continuity of a norm and San−1 being the inverse image of the closed set {1} (in
the usual topology on R) that San−1 is closed in Fn , k • ka . Notice that 0 ∈
/ San−1 .
Let k • kb be an arbitrary norm in Fn . Thus, k • kb is a continuous function with z 7→ kzkb, which is restricted to the closed and bounded set San−1
(which is compact in the finite-dimensional space Fn ). Therefore, by following
the boundedness theorem on real-valued continuous functions on compact sets (see
Theorem 4.16 (page 89) of Principles of Mathematical Analysis by W. Rudin),
k • kb reaches its minimum and maximum values at zm and zM , respectively, where
0 < kzm kb ≤ kzkb ≤ kzM kb ∀z ∈ San−1 .
x
Let x ∈ Fn \ {0}. Then, z , kxk
∈ San−1 . Therefore, 0 < kzm kb kxka ≤
a
kxkb ≤ kzM kb kxka ∀x ∈ San−1 .
Next we present an alternative proof of Theorem 1.5 for equivalence of all norms
in a finite-dimensional vector space, which requires the following lemma.
Lemma 1.2. (Linear combination in Normed Spaces) Let V be a vector space
over a (complete) field F that is either R or C. Let {v 1 , · · · , v n } be a linearly
independent set of vectors in the normed space (V, k • k). (Note that the space V
could be either finite-dimensional or infinite-dimensional). Then, there exists a real
number c ∈ (0, ∞) such that, for an arbitrary choice of scalars αj ∈ F, j = 1, · · · , n
with at least one αj 6= 0, the following inequality holds:
n
n
X
X
| αj |
αj v j ≥ c
j=1
j=1
Pn
Proof. Let us denote s = j=1 | αj |. If s = 0, then each αj is zero and the proof
follows trivially. Letting s > 0 (i.e., ensuring that not all αj ’s are zero) and setting
βj = αj /s, it suffices to show that
n
n
X
X
|βj | = 1
βj v j ≥ c where
j=1
j=1
We establish the above assertion by contradiction. Suppose it is false. Then,
there exists a sequence {y k } of vectors such that
4
n
n
X
X
yk = βjk v j where
|βjk | = 1
j=1
j=1
Pn
such that ky k → 0 as k → ∞. Having j=1 |βjk | = 1, it follows that |βjk | ≤
1 ⇒ {βjk } is bounded for each j = 1, · · · , n. Consequently, by Bolzano-Weierstrass
theorem, it follows that the bounded sequence of scalars {β1k } has a convergent
subsequence; let this subsequence converge to β1 . Let the corresponding subsequence of the sequence of vectors {y k } be denoted as {y1k }. By the same argument,
the sequence of vectors {y1k } has a subsequence {y2k } for which the corresponding
subsequence of scalars {β2k } converges to β2 . Continuing in this way for n steps,
we obtain a subsequence of vectors {ynk } whose terms are of the form
k
ynk =
n
X
j=1
γjk v j
where
n
X
j=1
|γjk | = 1
and the scalars γjk → βj as k → ∞. Hence, as k → ∞ ynk → y =
Pn
where
j=1 |βj | = 1 so that not all βj ’s are zero.
Pn
j=1
βj v j
Since {v 1 , · · · , v n } is a linearly independent set of vectors, it is ensured that
y 6= 0. On the other hand, by continuity of the norm, ynk → y implies kynk k → kyk.
Since ky k k → 0 by the original assumption and {ynk } is a subsequence of {y k }, it
follows that kynk k → 0, Hence, kyk → 0 ⇒ y = 0. This is a contradiction of the
original assumption that y 6= 0. The proof of the lemma is complete.
By setting αm , kzm kb and αM , kzM kb , equivalence of the norms k • ka and
k • kb is established.
Now, by using Lemma 1.2, we present the alternative proof of Theorem 1.5.
Proof. Let {v 1 , · · · , v n } be a basis for Fn . Then, a vector x ∈ Fn \ {0} is expressed
Pn
as x = j=1 αj v j , where αj ∈ F.
Given a norm k • ka on Fn , there exists a real ca ∈ (0, ∞) such that kxka ≥
Pn
ca j=1 | αj | by Lemma 1.2.
Next we consider the norm k • kb and apply the triangular inequality to obtain
Pn
kxkb ≤ kb j=1 | αj |,
where kb , max{kv 1 kb , · · · , kv n kb }, which implies that kxkb ≤ kcab kxka
By repeating the process with the norms k • ka and k • kb reversed, we obtain
kxka ≤ kcab kxkb ⇒ kxkb ≥ kcab kxka . By setting αm , kcab and αM , kcab , equivalence
of the norms k • ka and k • kb is established.
2
Summable Sequences and Integrable Functions
Definition 2.1. (Space of summable sequences ℓp and ℓ∞ ) Let x , {xk } and
y , {yk } be two sequences in a normed space V, k • k over the field F, where the
vector addition and scalar multiplication are denoted as: z = x ⊕y ⇒ zk = xk +yk
and w = α ⊗ x ⇒ wk = αxk ∀x , y , z ∈ V ∀α ∈ F.
A subspace of the normed space of summable sequences is defined to be ℓp for a
P∞
given p ∈ [1, ∞) if k=1 |xk |p < ∞ for all x ∈ ℓp ; and the resulting norm is denoted
5
P
1
∞
p p
as: kx kℓp ,
.
k=1 |xk |
A subspace of the normed space of summable sequences is defined to be ℓ∞ if
supk∈N |xk | < ∞ for all x ∈ ℓ∞ ; and the resulting norm is denoted as: kx kℓ∞ ,
supk∈N |xk |.
Definition 2.2. (Space Lp of Lebesgue-integrable functions) In a non-measuretheoretic setting, the space Lp of Lebesgue-integrable functions on a set T ⊆ R is
defined as:
R
o
n
1
R
p p
for
and kxkLp ,
Lp (T ) , x : T dt |x(t)|p < ∞
T dt |x(t)|
p ∈ [1, ∞).
In a measure-theoretic setting, the space Lp of Lebesgue-integrable functions
in the measure space R, B, µ is defined as:
R
o
n
p1
R
p
Lp (µ) , x : R dµ(t) |x(t)|p < ∞ and kxkLp ,
dµ(t)
|x(t)|
<
∞
R
for p ∈ [1, ∞).
Remark 2.1. The measure space is not restricted to R, B, µ . For example, the
space R could be replaced by Rn and accordingly the σ-algebra B by B n and the
Lebesgue measure µ by an appropriate product measure π.
Definition 2.3. (Equivalence Class) The equivalence
class generated by a bounded
n
o
function x in R, B, µ is defined as: E(x) , y : y = x µ-almost everywhere (ae) ,
where y = x µ-ae ⇒ y(t) 6= x(t) only on a set of zero µ-measure.
Definition 2.4. (Essential Supremum) Let E(x) be the equivalence class generated
by a bounded function x in R, B, µ . Then, the essential supremum of the function
x is defined as: ess sup x , inf x̃∈E(x) supt∈R |x̃(t)|.
Definition 2.5. (Space L∞ of essentially bounded functions) Let E(•) be the
equivalence class in R,nB, µ . Then, the o
space of essentially bounded functions is
defined as: L∞ (µ) ,
x : ess sup x < ∞
and the associated norm is defined as:
kxkL∞ , ess sup x.
Remark 2.2. To make kxkLp and kxkL∞ valid norms (instead of being semi-norms)
in a measure space R, B, µ , a necessary condition is that kxk = 0 if and only if
x = 0 for both Lp and L∞ . This equality is true in the µ-almost everywhere sense.
In essence, the vectors in both Lp and L∞ are equivalence classes of functions,
not individual functions. In other words, a quotient space is constructed from the
vector space of individual functions under the equivalence relation E(•).
Lemma 2.1. If f ∈ L1 (µ), then
Z
Z
dµ(t)f (t) ≤ dµ(t)f (t)
R
Proof. Let z = dµ(t)f (t) which is, in general, a complex number, i.e., z ∈ C.
There exists α ∈ C with |α| = 1 such that αz = |z|. Let u be the real part of αf .
Then, u ≤ |αf | = |f |. Hence,
Z
Z
Z
Z
Z
dµ(t)f (t) = α dµ(t)f (t) = dµ(t) αf (t) = dµ(t) u(t)) ≤ dµ(t)f (t)
The third of the above equalities holds because the preceding equalities show that
R
dµ(t)αf (t) is real.
6
"# a¢#¢s¢$¢ #s¢ ¢w /¢¢
# ¢ 1i¢ ‰UI¢ ¢
Bpi„ZFI~¢ ‰UI¢ B{’I¢ ¢ p{¢ IxŽZ”9`Ij¢
¡%y 3I‰¢
2.1
Inequalities in ℓp and Lp Spaces for p ∈ [1, ∞]
9iD¢
The goal of this subsection is to introduce Minkowski inequality that is the triangular inequality in ℓp and Lp Spaces for p ∈ [1, ∞]. In this context, we establish
˜L¢ZkŠIr{I‰¢(
( 9Ģas9I9Ģ
ƒUp™i¢ Zi¢
+ZQ‘{I¢'2¢
‰UIi¢Z‹¢Zƒ¢@bI9{¢‰V9Œ¢
9iG¢
another inequality,
known
Hölder9ƒ¢
inequality,
which
is extensively
used in real and
(¢ ¢analysis
6functional
as well as bears significant importance in engineering analysis.
‡UIh¢
*WRJ¢&-¢ of scalar inequality
Figure 1: Geometric interpretation
Lemma 2.2. (Scalar Inequality) Given p ∈ (1, ∞), let q ∈ (1, ∞) be such that
1
1
p + q = 1. Note that p and q are called conjugate exponents.
p
q
Let a ∈ [0, ∞) and a ∈ [0, ∞). Then, ab ≤ ap + bq .
Proof. It follows from the given conditions on p and q that:
1=
p+q
⇒ pq = p + q ⇒ (p − 1)(q − 1) = 1
pq
In the (ξ, η)-plane, consider the curve η = ξ p−1 , or equivalently, ξ = η q−1 . Let
A1 ,
Z
0
a
dξ ξ
p−1
ap
=
and A2 ,
p
Z
b
dη η q−1 =
0
bq
q
By interpreting A1 and A2 as areas, as shown in Figure 1, it follows that
ab ≤ A1 + A2 .
Theorem 2.1. (Hölder Inequality) Given p ∈ (1, ∞), let q ∈ (1, ∞) be such that
1
1
p + q = 1. Then, the following inequalities hold.
1. Finite Sums:
n
n
n
n
X
X
1 X
X
1q
p p
|x
|
|x
y
|
≤
≤
x
y
|yj |q
j
j j
j j
j=1
j=1
j=1
2. Infinite Sums: Given x ∈ ℓp (µ), i.e.,
P∞
q
j=1 |yj | < ∞,
P∞
j=1
j=1
|xj |p < ∞ and y ∈ ℓq (µ), i.e.,
∞
∞
∞
∞
X
X
p1 X
X
1q
|xj |p
|xj yj | ≤
xj yj ≤
|yj |q
j=1
j=1
j=1
7
j=1
R
3. Integrals over Ω ⊆ R: Given x ∈ Lp (µ), i.e., t∈Ω dµ(t)|x(t)|p < ∞ and
R
y ∈ Lq (µ), i.e., t∈Ω dµ(t)|y(t)|q < ∞,
Z
Z
dµ(t)|x(t)y(t)|
dµ(t)x(t)y(t) ≤
t∈Ω
t∈Ω
≤
Z
dµ(t)|x(t)|p
t∈Ω
! p1
Z
dµ(t)|y(t)|q
t∈Ω
! q1
Proof. See Naylor & Sell p. 550, where Hölder inequality is proved by making use
of the scalar inequality in Lemma 2.2
Remark 2.3. Hölder inequality is extendable to p = 1 and p = ∞, which implies
that and q = ∞ and q = 1, respectively, by using the relation p1 + q1 = 1 on the
extended real line R, as explained below.
1. Finite Sums:
n
X
|xj yj | ≤
n
X
|xj |
j=1
j=1
max |yj |
j
P∞
2. Infinite Sums: Given x ∈ ℓ1 (µ), i.e., j=1 |xj | < ∞ and y ∈ ℓ∞ (µ), i.e.,
supj |yj | < ∞,
∞
∞
X
X
|xj |
sup |yj |
|xj yj | ≤
j
j=1
j=1
3. Integrals over Ω ⊆ R: Given x ∈ L1 (µ), i.e.,
y ∈ L∞ (µ), i.e., ess supt∈Ω |y(t)| < ∞,
!
Z
Z
dµ(t)|x(t)y(t)| ≤
R
t∈Ω
dµ(t)|x(t)|
t∈Ω
dµ(t)|x(t)| < ∞ and
!
ess sup |y(t)| < ∞
t∈Ω
t∈Ω
Remark 2.4. In Theorem 2.1, for the special case p = 2 that mandates q = 2 by
the relation p1 + 1q = 1, Hölder inequality is called the Schwarz inequality.
Theorem 2.2. (Minkowski Inequality) Let p ∈ [1, ∞). Then, the following inequalities hold.
1. Finite Sums:
n
n
n
X
p1 X
p1
X
1
xj ± yj p p ≤
|xj |p
|yj |p
+
j=1
j=1
2. Infinite Sums: Given x ∈ ℓp (µ), i.e.,
P∞
p
j=1 |yj | < ∞,
j=1
P∞
j=1
|xj |p < ∞ and y ∈ ℓp (µ), i.e.,
∞
∞
∞
p1 X
p1
X
X
1
xj ± yj p p ≤
|xj |p
|yj |p
+
j=1
j=1
j=1
3. Integrals over Ω ⊆ R: Given x ∈ Lp (µ), i.e.,
R
y ∈ Lp (µ), i.e., t∈Ω dµ(t)|y(t)|p < ∞,
Z
t∈Ω
dµ(t)|x(t)± y(t)|
p
p1
≤
Z
dµ(t)|x(t)|
t∈Ω
8
p
R
t∈Ω
! p1
dµ(t)|x(t)|p < ∞ and
+
Z
t∈Ω
dµ(t)|y(t)|
p
! p1
Proof. See Naylor & Sell pp. 550-551, where Minkowski inequality is proved by
making use of Hölder inequality in Theorem 2.1.
Remark 2.5. Minkowski inequality is extendable to p = ∞ on the extended real
line R, as explained below.
1. Finite Sums:
max |xj ± yj | ≤ max |xj | + max |yj |
j
j
j
2. Infinite Sums: Given x ∈ ℓ∞ (µ), i.e., supj |xj | < ∞ and y ∈ ℓ∞ (µ), i.e.,
supj |yj | < ∞,
sup |xj ± yj | ≤ sup |xj | + sup |yj |
j
j
j
3. Integrals over Ω ⊆ R: Given x ∈ L∞ (µ), i.e., ess supt∈Ω |x(t)| < ∞ and
y ∈ L∞ (µ), i.e., ess supt∈Ω |y(t)| < ∞,
ess sup |x(t) ± y(t)| ≤ ess sup |x(t)| + ess sup |y(t)|
t∈Ω
t∈Ω
t∈Ω
Remark 2.6. Theorem 2.2 establishes the triangular inequality in ℓp and Lp as
well as spaces ℓ∞ and L∞ spaces. Thereby, it is established that these spaces are
normed vector spaces. In the next section, we show that these spaces are complete
under the respective norms (used as metrics0, i.e., they are Banach spaces.
Theorem 2.3. (Inclusion Theorem 1) Given 1 < p < q < ∞, the following condition holds: ℓ1 ℓp ℓq ℓ∞ .
Proof. If the sequence x = {x1 , x2 , · · · } ∈ ℓ1 , then we first show that kxkℓ∞ ≤
kxkℓq k ≤ kxkℓp k ≤ kxkℓ1 k < ∞. For p > 1 and x ∈ ℓ1 , it follows that, for ∀n ∈ N,
n
X
|xk |p ≤
k=1
n
X
|xk |
k=1
p
≤
∞
X
k=1
|xk |
p
= kxkℓ1
p
p
p
As n → ∞, the above equations yield: kxkℓp ≤ kxkℓ1 , which implies kxkℓp ≤
kxkℓ1 .
For q > p > 1 and x ∈ ℓp , it follows that, for ∀n ∈ N,
n
X
k=1
|xk |q ≤
n
X
k=1
|xk |p
pq
≤
∞
X
k=1
|xk |p
pq
= kxkℓp
q
q
q
As n → ∞, the above equations yield: kxkℓq ≤ kxkℓp , which implies kxkℓq ≤
1 kxkℓp . Then, we consider a sequence k(1/p)
that belongs to {ℓq } but not to {ℓp }
for 1 < p < q < ∞. This establishes the strict inclusion as needed in the proof of
the theorem.
P
1
∞
q q
= kxkℓq . Taking
Next, let x ∈ ℓq for q ∈ [1, ∞); then, |xk | ≤
|x
|
k
k=1
supremum over all k ∈ N, it follows that kxkℓ∞ ≤ kxkℓq . We note that the discrete
step function (i.e., the sequence {1, 1, · · · }) belongs to ℓ∞ but not to ℓq for q < ∞.
This establishes the strict inclusion as needed in the proof of the theorem.
Theorem 2.4. (Inclusion Theorem 2) Let (R, B, ν) be a measure space, where the
measure ν is finite. Then, given 1 < p < q < ∞, the following condition holds:
L1 (ν) ) Lp (ν) ) Lq (ν) ) L∞ (ν).
9
Proof. The proof follows by an application of the integral form of Theorem 2.1
(Hölder Inequality).
Theorem 2.5. (Inclusion Theorem 3) Let (R, B, µ) be a measure space and the
measure µ is either finite or σ-finite. Then, given p ∈ [1, ∞], the following condition
T
holds: L1 (µ) L∞ (µ) is a subspace of Lp (µ).
T
Proof. Let x ∈ L1 (µ) L∞ (µ). If p = 1 or if p = ∞, the the proof is obvious that
x ∈ Lp (µ). Therefore, let p ∈ (1, ∞). Let S ⊆ R such that |x(t)| ≥ 1 ∀t ∈ S. Since
R
x ∈ L1 (µ), i.e., kxkL1 , R dµ(t) |x(t)| < ∞, it follows that the measure µ(S) < ∞.
Furthermore, since x ∈ L∞ (µ), i.e., kx(t)kL∞ ≡ ess supt∈R |x(t)| < ∞, it follows
that |x(t)| ≤ kx(t)kL∞ < ∞ µ-a.e. on S. Therefore, for any p ∈ (1, ∞), it follows
that
p
R
R
R
= R dµ(t) |x(t)|p = S dµ(t) |x(t)|p + R\S dµ(t) |x(t)|p
kxkLp
R
R
≤ S dµ(t) |x(t)|p + R\S dµ(t) |x(t)| because |x(t)| < 1 on R \ S
R
R
≤ S dµ(t) kxkpL∞ + R\S dµ(t) |x(t)|
R
≤ µ(S) kxkpL∞ + R dµ(t) |x(t)|
= µ(S) kxkpL∞ + kxkL1 < ∞
T
The above inequality implies that L1 (µ) L∞ (µ) ⊆ Lp (µ). Since both L1 (µ) and
T
L∞ (µ) are vector spaces, their intersection L1 (µ) L∞ (µ) is also a vector space.
T
Hence, L1 (µ) L∞ (µ) is a subspace of the vector space Lp (µ) for all p ∈ [1, ∞].
Figure 2: Comparison of profiles of unit disk under different norms in R2
((Courty: Fig. 16 in Kreyszig p. 65))
Example 2.1. Let us consider the space R2 and the closed & bounded sets Sp1 ,
1
{x ∈ R2 : kxkp = 1} for p ∈ [1, ∞], where kxkp , |x1 |p + |x2 |p p for p ∈ [1, ∞)
and kxk∞ , max{|x1 |, |x2 |}. Given the vector space V = R2 , let us construct the
10
regions kxkℓp for p ∈ [1, ∞]. Referring to Figure 2, we notice that
{x ∈ R2 : kxkℓ1 ≤ 1} a rhombus
{x ∈ R2 : kxkℓ2 ≤ 1} a circle
{x ∈ R2 : kxkℓ4 ≤ 1} a smooth distorted circle
{x ∈ R2 : kxkℓ∞ ≤ 1} a square
3
Completion of Normed Spaces
The notion of Hamel basis in finite-dimensional vector spaces has been introduced
in Chapter 2. It is a purely algebraic concept. The concept of convergence of a
series can be used to define a “basis” in the following definition.
Definition 3.1. (Schauder Basis) Let (V, k • k) be a normed space and let {ek } be
a sequence in V such that, for every x ∈ V , there exists a sequence of scalars {αk }
with the property
n
X
αk ek → 0 as n → ∞.
x−
k=1
k
Then, the sequence {e } is called a Schauder basis for (V, k • k). the series
Definition 3.2. (Banach Space) A complete normed vector space (i.e., where every
Cauchy sequence converges in the space) is called a Banach space.
Lemma 3.1. (Boundedness of a Cauchy Sequence) Every Cauchy sequence is
bounded in a normed space.
Proof. Let (V, k•k) be a normed space. Let {xk } be a Cauchy sequence in (V, k•k).
Then, for any given ε > 0, there exists m(ε) ∈ N such that kxℓ −xn k ≤ ǫ ∀ ℓ, n ≥ m.
Since kxm k < ∞, it follows by the triangular inequality and the fact that
∀ n > m, kxn k = kxm + (xn − xm )k ≤ kxm k + kxn − xm k ≤ kxm k + ε < ∞
Theorem 3.1. (Completion of Finite-dimensional Normed Spaces) Let (V, k • k)
be an n-dimensional normed vector space over a (complete) field F, where n ∈ N.
Then, (V, k • k) is a Banach space.
Proof. We make use of the fact that any n-dimensional space V over the field F is
isomorphic to Fn for any n ∈ N. Then, we use the method of induction to prove
the theorem based on the convergence of an arbitrary Cauchy sequence in Fn to a
point in Fn . It follows that if n = 1, then F is complete in the sense of the usual
metric, because of the hypothesis of completeness of the field F. Next, we show
that, for all n ∈ N \ {1}, if the space Fn−1 is complete in the sense of the usual
metric, then the space Fn is also complete in the sense of the usual metric. Let
1
e , · · · , en be a basis for the vector space Fn and let
X
δk , inf n kek −
αj ej k > 0 ∀k ∈ {1, · · · , n},
αj ∈F
j6=k
11
which is the distance from the vector ek to an (n − 1)-dimensional subspace of Fn
generated by the the remaining basis vectors, e1 , · · · , ek−1 , ek+1 , · · · , en . Since the
basis e1 , · · · , en forms a linearly independent set, δk > 0. Then,
δ , min δk > 0 because δk > 0 ∀k ∈ {1, · · · , n}
k
Let {xk } be a Cauchy sequence in Fn , where each vector can be uniquely
represented in terms of the basis {e1 , · · · , en } as
xk =
k
X
λkj ej where λkj ∈ F
j=1
Then, for any ℓ, m ∈ N, it follows that
n
ℓ
X
j
ℓ
m
x − xm = λℓj − λm
j e ≥ |λk − λk | δ
∀k ∈ {1, · · · , n}
j=1
Given that limℓ,m→∞ xℓ − xm = 0 for the Cauchy sequence {xk } and the
= 0, i.e., the sequence {λℓ }
scalar δ > 0, it follows that limℓ,m→∞ λℓk − λm
k
k
converges to a scalar λk ∀k ∈ {1, · · · , n}.
P
Let us construct a vector x , nj=1 λj ej ∈ Fn and then show that xj → x.
For all ℓ ∈ N, it follows by the triangular inequality and homogeneity properties
of a norm that
n
n
ℓ
X
j
X ℓ
ℓ
x − x = λj − λj kej k
e
≤
λ
−
λ
j
j
j=1
j=1
Since limℓ→∞ |λℓj −λj = 0 ∀j ∈ {1, · · · , n}, it follows that limℓ→∞ xℓ −x = 0.
Hence, the Cauchy sequence {xk } converges to x ∈ Fn .
We present an alternative proof of Theorem 3.1.
Proof. Let {y k } be an arbitrary Cauchy sequence of vectors in V . We will show
that {y k } converges in V , i.e., there exists y ∈ V such that limℓ→∞ xℓ − x = 0;
in other words, the limit of the sequence {y k } is y ∈ V .
Since dim V = n, we select a basis for V as {v 1 , · · · , v n }. Then each y k is
Pn
expressed in the form: y k = j=1 αkj v j , where αkj ∈ F ∀j, k. Since {y k } is a Cauchy
sequence of vectors in V , it follows that ∀ε > 0 ∃ n ∈ N such that kyj − yℓ k < ε
for all j, ℓ > n. Then, using Lemma 1.2, we assert that there exists a real constant
c ∈ (0, ∞) such that
n
n
X
X
| (αkj − αℓj ) |
(αkj − αℓj ) v j ≥ c
ε > ky k − y ℓ k = where k, ℓ > N
j=1
j=1
Division by c ∈ (0, ∞) yields
| (αkm − αℓm ) |≤
n
X
| (αkj − αℓj ) |<
j=1
ε
∀m = 1, · · · , n ∀k, ℓ > N
c
Therefore, each of the n sequences αkm , m = 1, · · · , n is Cauchy in F. Since the
field F is complete, each of these sequences of scalars converges; let us denote
12
the respective limits as αm , m = 1, · · · , n. Using these n limits, we construct
Pn
y = m=1 αm v m . therefore, y ∈ V and
n
n
X
X
| (αkj − αj ) | kv j k
(αkj − αj ) v j ≤
ky k − yk = j=1
j=1
As as k → ∞, on the right side of the above equation, αkj → αj ; hence, ky k −yk → 0,
i.e., y k → y. this shows that {y k } is convergent in V . Since {y k } was originally
assumed to be an arbitrary Cauchy sequence in V . The proof is now complete.
Theorem 3.2. The ℓp -space over a complete field F (i.e., R or C) is complete for
p ∈ [1, ∞].
Proof. First let us consider p ∈ [1, ∞). Given a Cauchy sequence {xk } in ℓp , we
will show that {xk } converges to a sequence x ∈ ℓp . Then, if xk = {ξ1k , ξ2k , · · · },
P
1
∞
k
ℓ p p
and it follows that
then kxk − xℓ kℓp ,
j=1 |ξj − ξj |
k
ℓ
lim |ξm
− ξm
| ≤ lim
k,ℓ→∞
k,ℓ→∞
∞
X
j=1
|ξjk − ξjℓ |p
p1
= 0 ∀m ∈ N
k
Hence, the sequence {ξm
} is Cauchy in F for each m ∈ N and therefore converges to
a unique ξm ∈ F. Now we show that the constructed sequence x , {ξ1 , ξ2 , · · · } ∈ ℓp .
It follows from Lemma 3.1 that ∃ M ∈ (0, ∞) which bounds the sequence {xk },
i.e., kxk kℓp ≤ M ∀ k ∈ N. Hence,
n
X
|ξjk |p ≤ kxk kp ≤ M p ∀n ∈ N
j=1
Letting k → ∞ in the finite sum in the left hand term of the above equation, the
Pn
inequality j=1 |ξj |p ≤ M p holds uniformly for all n ∈ N. Then, letting n → ∞, it
P
p
p
follows that ∞
j=1 |ξj | ≤ M < ∞, i.e., x ∈ ℓp .
k
Next we show that
x → x aspk → ∞. Given ε > 0, ∃ ℓ ∈ N such that
Pk
n
m p
n
kx − xm kℓp
≤ εp ∀ n, m > ℓ and ∀ k ∈ N. Letting
j=1 |ξj − ξj | ≤
m → ∞ in the finite sum in the left hand term of the inequality, it follows that
Pk
n
p
p
j=1 |ξj − ξj | ≤ ε ∀ n > ℓ and ∀ k ∈ N. Now letting k → ∞, it follows that
kxn − xkℓp < ε ∀ n > ℓ, which implies that xn → x ∈ ℓp . This part completes the
proof for p ∈ [1, ∞).
It remains to show completion of ℓp for p = ∞, i.e., if {xk } is a Cauchy sequence in ℓ∞ , then xk → x ∈ ℓ∞ . Let {xk } be a Cauchy sequence in ℓ∞ . Then,
k
ℓ
| ≤ limk,ℓ→∞ kxk − xℓ kℓ∞ = 0. Hence, ∀j ∈ N ∃ ξj ∈ F such that
− ξm
limk,ℓ→∞ |ξm
ξjk → ξj uniformly in j as k → ∞. Now we show that the constructed sequence
x , {ξ1 , ξ2 , · · · } ∈ ℓ∞ .
Since {xk } is a Cauchy sequence, it follows from Lemma 3.1 that ∃ M ∈ (0, ∞)
which bounds the sequence {xk }, i.e., kxk kℓ∞ ≤ M ∀ k ∈ N. Hence, |ξjk | ≤
kxk kℓ∞ ≤ M , from which it follows that kxkℓ∞ ≤ M , implying that x ∈ ℓ∞ . The
convergence of xk to x follows directly from the fact that ξjk → ξj uniformly in j
as k → ∞.
Definition 3.3. (Space of Convergent Sequences) A subspace of ℓ∞ , where all
sequences are convergent, is called the c-space. A subspace of c is called c0 if all
sequences converge to zero.
13
Remark 3.1. By Lemma 3.1, it follows that all sequences in the c-space are
bounded. Therefore, c is a subspace of ℓ∞ .
Theorem 3.3. (Completion of the c Space) The space c of all convergent sequences
over a complete field F is complete.
Proof. Since c is a subspace of ℓ∞ that is a complete space, it suffices to establish
completeness of the space c by showing that c is closed in ℓ∞ . Let c̄ be the closure
of c in ℓ∞ . Then, there exists a Cauchy sequence {xk : k ∈ N} of sequences in c,
where xk = {ξ1k , ξ2k , · · · } and xk → x in the ℓ∞ -induced metric, i.e., x ∈ c̄, where
x = {ξ1 , ξ2 , · · · }. We will show that x ∈ c.
For any ε > 0 ∃ n1 ∈ N such that kxk − xkℓ∞ < 3ε ∀ k ≥ n1 , which implies that
|ξjk − ξj | < 3ε ∀j. Since xk , {ξ1k , ξ2k , · · · } is convergent in the space c, it follows
that, ∀ k ∈ N and any ε > 0 ∃ n2 (ε) ∈ N such that |ξik − ξjk | < 3ε ∀ i, j ≥ n2 . Then,
it follows by triangular inequality that ∀ i, j, k ≥ max(n1 , n2 )
|ξi − ξj | ≤ |ξi − ξik | + |ξik − ξjk | + |ξjk − ξj | <
ε ε ε
+ + =ε
3 3 3
which implies that the sequence x = {ξ1 , ξ2 , · · · } is convergent in c ⇒ x ∈ c. Since
the choice of x is arbitrary, it follows that c = c̄ and hence c is closed in ℓ∞ .
Corollary 3.1. (Completion of the c0 Space) The space c0 of all sequences converging to 0 over a complete field F is complete.
Proof. The proof follows from that of Theorem 3.3.
Definition 3.4. (Space of Continuous Functions) Having the domain [a.b]
R,
the space of all continuous functions that belong to the Lp [a, b] space, p ∈ [1, ∞],
is called the Cp [a, b]-space. Note that Cp [a, b] Lp [a, b].
Theorem 3.4. (Completion of the C∞ [a, b] Space) Let [a, b]
R. The space
C∞ [a, b] of all continuous functions which belong to L∞ [a, b] is complete, where
the metric on C∞ [a, b] is defined as:
d∞ (x, y) = max |x(t) − y(t)|
t∈[a,b]
Proof. Let {xk } be a Cauchy sequence in C∞ [0, 1] ⇒ ∀ε > 0 ∃ n(ǫ) ∈ N such
that d∞ (xk , xℓ ) < ε ∀ k, ℓ ≥ n . Then, ∀ t0 ∈ [0, 1], the sequence xk (t0 ) of scalars
converges to a scalar that we call x(t0 ). In this way, we construct a scalar-valued
function x whose domain is the compact set [0, 1]. Since xk → x uniformly on [0, 1]
and since xk ’s are continuous on [0, 1] and the convergence is uniform, it follows
that the limit function x is continuous on [0, 1]. Therefore, x ∈ C∞ [0, 1].
Example 3.1. The space Cp [0, 1] is not complete for any p ∈ [1, ∞). To see this,
let us consider a sequence of continuous functions {1, t, t2 , · · · } whose domain is
[0, 1], i.e., xk (t) = tk ∀ t ∈ [0, 1]; it is verified from the fact
Z 1
lim
dt | tk − tℓ |p = 0
k,ℓ→∞
0
2
that {1, t, t , · · · } is a Cauchy sequence in the Cp [0, 1] space for any p ∈ [1, ∞). It
is also seen that xk → x pointwise on [0, 1] in the metric induced by the Lp -norm
for p ∈ [1, ∞). The limit function x is obtained as:
14
x(t) =
(
0 if t ∈ [0, 1)
1 if t = 1
Clearly, the limit function x is not continuous on [0, 1] and hence x ∈
/ Cp [0, 1]
but x ∈ Lp [0, 1]. That is, the Cauchy sequence {1, t, t2 , · · · } does not converge in
Cp [0, 1] but it does converge to a discontinuous function in Lp [0, 1].
Remark 3.2. The above example reveals the fact all norms are not equivalent
in an infinite-dimensional normed space. This result is different from the result
derived in Theorem 1.5 for finite-dimensional vector spaces.
4
Density and Separability of Normed Spaces
Definition 4.1. (Density) Let (V, k • k) be a normed vector space. A set S ⊆ V is
called dense in V if any one (or both) of the following two conditions are satisfied
for all x ∈ V : (i) x ∈ S, or (ii) x is a limit point of S. Equivalently, ∀x ∈ V and
∀ε > 0 ∃ y ∈ S such that kx − yk < ε.
Remark 4.1. For a dense set S in V , the closure of S in V , denoted as S, is the
space V itself, i.e., S = V .
Definition 4.2. (Separability) Let (V, k • k) be a normed vector space. Then, V
is called separable if there exists a countable and dense subset S ⊆ V , i.e., S is
countable and S = V .
Example 4.1. The space R1 of the real line is separable because Q
and Q = R.
R is countable
Proposition 4.1. The ℓp space is separable for p ∈ [1, ∞).
Proof. Without loss of generality, we assume that R (which can be naturally extended to the complex field C) is the field of the vector space ℓp . Let S ⊆ ℓp be
the set of all sequences with finitely many non-zero rational elements and the rest
being zeros. Hence S is a countable set because the countable union of countable
sets forms a countable set (see Chapter One).
To show that S is dense in ℓp , let x , {ξ1 , ξ2 , · · · } ∈ ℓp for a given p ∈ [1, ∞),
P∞
i.e., k=1 |ξk |p < ∞. Then, it follows that, for any fixed ε > 0 ∃ n(ε) ∈ N such
P∞
p
that k=n+1 |ξk |p < ε2 . It is also seen that, for ∀ k ∈ {1, 2, · · · , n}, if qk ∈ Q such
εp
and if y , {q1 , q2 , · · · , qn , 0, 0, · · · }, then y ∈ S and it follows
that |ξk − qk |p < 2n
that
kx − ykpℓp =
n
X
j=1
|ξj − qj |p +
∞
X
εp
εp |ξj |p < n
+
= εp ⇒ kx − ykℓp < ε
2n
2
j=n+1
Therefore, the countable S is dense in ℓp and hence ℓp is separable for p ∈ [1, ∞).
Proposition 4.2. The ℓ∞ space is not separable.
Proof. We need the following lemma to prove the proposition.
Lemma 4.1. Let S be the set of all (infinite) sequences whose elements belong to
the binary alphabet {0, 1}. Then, S is an uncountable set.
15
Proof. We prove the lemma by contradiction. Let us assume that S is a countable
set. Then, S must consist of countably many sequences, i.e., S , s1 , s2 , · · · , and
the ith sequence is si = si1 , si2 , · · · , where sij ∈ {0, 1}. Let us construct a sequence
s̃ , {s̃1 , s̃2 , · · · } such that the ith element s̃i of the sequence s̃ is 0 (or 1) if the ith
element sii of the sequence si ∈ S is 1 (or 0), respectively. So, s̃ 6= si ∀ i ∈ N.
Hence, s̃ ∈
/ S. This is a contradiction of the original assumption that S is countable.
Therefore, S is uncountable.
Now we proceed to prove the proposition by making use of Lemma 4.1. Let
x and y be two distinct sequences in S, i.e., x, y ∈ S such that x 6= y. Then,
kx − ykℓ∞ = 1, i.e., the distance between any two distinct points in S is 1 in the
ℓ∞ metric. Let D ℓ∞ be nonempty and countable. Let U be an open ball in ℓ∞
with radius 12 and its center at one of the points in D. Therefore, U ∩ S contains
at most one point of S. Since D is countable and S is uncountable, the closure D
cannot contain all points of S. Since S is a subset of ℓ∞ , D is a strictly proper
subset of ℓ∞ . That is, no countable subset of ℓ∞ is dense in ℓ∞ . So, ℓ∞ is not
separable.
Remark 4.2. It follows from Theorems 4.1 and 4.2 that the ℓp -space and the ℓ∞ space cannot have the same topology because separability is a topological property
(see Proposition 3.2 in Chapter 1).
16