ME(EE) 550
Foundations of Engineering Systems Analysis
Chapter Two: Algebraic Structure of Vector Spaces
This chapter briefly presents the algebraic structure of vector spaces starting
with the rudimentary concepts of groups and fields. This chapter should be read
along with Chapter 4 Part A of Naylor & Sell. Both solved examples and exercises
in Naylor & Sell would be very useful.
1
Monoids and Groups
Let S be a nonempty set. With m ∈ N, an m-ary operation ⊛ is a function
⊛ : S m → S, i,e., mapping S × S × · · · S into S. Most commonly encountered
|
{z
}
m times
operations are binary, i.e., m = 2, where ⊛ : S × S → S.
This section focuses on binary algebras that are algebraic systems with a single
binary operation. We first move from a primitive algebra called semigroup to
monoid, and then to a highly structured algebraic system called group. Concepts
of groups are extensively used in Physics and Engineering.
Definition 1.1. (Binary Algebra) Let S be a nonempty set. The operation ⊛ :
S × S → S is called a binary algebra and is referred to as (S; ⊛).
Definition 1.2. (Semigroup) A binary algebra (S; ⊛) is called a semigroup if it
satisfies the associativity property, i.e., (α ⊛ β) ⊛ γ = α ⊛ (β ⊛ γ) ∀α, β, γ ∈ S. A
semigroup is called commutative if α ⊛ β = β ⊛ α ∀α, β ∈ S.
Definition 1.3. (Identity Element) An element 1ℓ ∈ S is said to be a left identity
of the binary algebra (S; ⊛) if 1ℓ ⊛ α = α ∀α ∈ S. Similarly, an element 1r ∈ S
is said to be a right identity of the binary algebra (S; ⊛) if α ⊛ 1r = α ∀α ∈ S.
Finally, an element 1 ∈ S is said to be an identity of the binary algebra (S; ⊛) if
1 ⊛ α = α ⊛ 1 = α ∀α ∈ S.
Definition 1.4. (Zero Element) An element 0ℓ ∈ S is said to be a left zero of
the binary algebra (S; ⊛) if 0ℓ ⊛ α = 0ℓ ∀α ∈ S. Similarly, an element 0r ∈ S
is said to be a right zero of the binary algebra (S; ⊛) if α ⊛ 0r = 0r ∀α ∈ S.
Finally, an element 0 ∈ S is said to be a zero of the binary algebra (S; ⊛) if
0 ⊛ α = α ⊛ 0 = 0 ∀α ∈ S.
Definition 1.5. (Monoid) A semigroup with an identity element is called a monoid.
That is, a monoid is a binary algebra that satisfies the associativity property and
has an identity element. A monoid is called commutative if α⊛β = β⊛α ∀α, β ∈ S.
1
Definition 1.6. (Invertible Element) Let (S; ⊛) be a monoid with an identity
element 1. Then, an element α ∈ S is called left invertible if there exists α−1
∈S
ℓ
−1
such that αℓ ⊛ α = 1. Similarly, an element α ∈ S is called right invertible if
−1
there exists α−1
r ∈ S such that α ⊛ αr = 1. An element α ∈ S is called invertible
if there exists α−1 ∈ S such that α−1 ⊛ α = α ⊛ α−1 = 1.
Definition 1.7. (Group) A monoid whose every element is invertible is called a
group. That is, a binary algebra (S; ⊛) is called a group if the following properties
are satisfied:
• Closure Property : α ⊛ β ∈ S ∀α, β ∈ S
• Associativity Property : (α ⊛ β) ⊛ γ = α ⊛ (β ⊛ γ) ∀α, β, γ ∈ S.
• Existence of an Identity : There exists 1 ∈ S such that 1 ⊛ α = α ⊛ 1 =
α ∀α ∈ S.
• Existence of an Inverse for each Element: There exists α−1 ∈ S such
that α−1 ⊛ α = α ⊛ α−1 = 1 ∀ α ∈ S.
Definition 1.8. (Abelian Group) A group (S; ⊛) is called commutative (also called
Abelian) if the following additional property: α ⊛ β = β ⊛ α ∀α, β ∈ S is satisfied.
It is customary to denote the binary operator ⊛ as + for commutative groups. That
is, (S, +) indicates an Abelian group. The inverse of an element α is denoted as
−α and the identity as 0.
Example 1.1. Let S = R , (−∞, ∞) and let + be the usual addition operation.
Then, (S, +) is an Abelian group whose identity element is 0.
Example 1.2. Let S = R , (−∞, ∞) \ {0} and let • be the usual multiplication
operation. Then, (S, •) is an Abelian group whose identity element is 1.
Example 1.3. Let S be the set of all n × n invertible matrices with real entries.
Let ⊛ be the operation of matrix multiplication. Then, (S; ⊛) is a group whose
identity element is the n × n identity matrix, but it is NOT Abelian. Note that
if M is the set of all n × n matrices with real entries, then (M ; ⊛) is not a group
because some of the matrices in M are not invertible; however, (M ; ⊛) is a monoid.
Example 1.4. Let S be the set of all m × n matrices (m is not necessarily equal
to n) with real entries. Let ⊛ be the operation of matrix addition. Then, (S; ⊛) is
an Abelian group whose identity element is the (m × n) zero matrix.
Definition 1.9. (Subgroup) Let (S; ⊛) be a group and let Se ⊆ S be closed under
e If (S,
e ⊛) satisfies the properties of a group as delineated
⊛, i.e., ⊛ : Se × Se → S.
e ⊛) is called a subgroup of the group (S; ⊛). In this case,
in Definition 1.7, then (S,
e
e
e ⊛) is called a proper subgroup of
if S ( S, i.e., S is a proper subset of S, then (S,
the group (S; ⊛). If Se = S or if Se = {1}, where 1 is the identity element of S, then
e ⊛) is called a trivial subgroup of (S; ⊛). The largest proper subgroup of (S; ⊛)
(S,
is called the maximal subgroup.
Example 1.5. (R, +) is a proper subgroup of (C, +) and (Q, +) is a proper subgroup of (R, +).
2
Definition 1.10. (Cosets and Normal Subgroup) Let (G; ⊛) be a group and (S; ⊛)
be a subgroup of (G; ⊛). If a ∈ G, then the set S ⊛ a , {x = b ⊛ a : b ∈ S} is
called a right coset of S in (G; ⊛) and the set a ⊛ S , {x = a ⊛ b : b ∈ S} is called
a left coset of S in (G; ⊛). If, for all a ∈ G, we have S ⊛ a = a ⊛ S, then (S; ⊛) is
called a normal subgroup of (G; ⊛), and the right and left cosets are simply called
cosets.
Remark 1.1. Every subgroup of an Abelian group is a normal subgroup.
2
Rings and Fields
This section focuses on rings and fields that deal with two binary operations. Additional algebraic structures are imposed on rings to construct fields.
Definition 2.1. (Ring) Let (S; +, •) be an algebraic system, i.e., + : S × S → S
and • : S × S → S. Then, (S; +, •) is called a ring if (S; +) is an Abelian group
(with 0 ∈ S as the additive identity element) and if, for every α, β, γ ∈ S, the
following conditions are satisfied:
• Associative property : (α • β) • γ = α • (β) • γ).
• Distributive property : α•(β+γ) = α•β+α•γ and (β+γ)•α = β•α+γ•α.
A ring is called commutative if, in addition, it satisfies the commutative property,
i.e., α • β = β • α ∀α, β ∈ S.
Remark 2.1. A ring (S; +, •) may or may not satisfy the following properties:
• Existence of multiplicative Identity : There exists 1 ∈ S such that
1 • α = α • 1 = α ∀α ∈ S
• Cancelation: If α 6= 0, then ∀ β, γ ∈ S
(
(α • β = α • γ) ⇒ (β = γ)
(β • α = γ • α) ⇒ (β = γ)
Definition 2.2. (Ideals) Let (S; +, •) be a ring and let D be a nonempty subset
of S that is closed under the operations of + and • in S. If (D; +, •) is itself a
ring, then it is called a subring of (S; +, •). A subring (D; +, •) of a ring (S; +, •)
is called a left ideal if
s ∈ S and x ∈ D ⇒ s • x ∈ D
and is called a right ideal if
s ∈ S and x ∈ D ⇒ x • s ∈ D
The subring (D; +, •) is called an ideal if it is both a left ideal and a right ideal.
Remark 2.2. Just as normal groups play an important role in the theory of groups
(see Definition 1.10), so do ideals in the theory of rings.
3
Definition 2.3. (Zero-divisor) Let (S; +, •) be a ring. Then, an element α ∈ S\{0}
is called a left [resp. right] zero-divisor if there exists β ∈ S \{0} such that α•β = 0
[resp. β • α = 0]. An element of S, which is both a left and a right zero-divisor, is
called a zero-divisor.
Definition 2.4. (Invertible Element) Let (S; +, •) be a ring with 1 as the multiplicative identity (i.e., the identity with respect to the operation •). Then, an
element α ∈ S is called a left [resp. right] invertible if there exists β ∈ S [resp.
γ ∈ S] such that β • α = 1 [resp. α • γ = 1]. The element β [resp. γ] is called a
left [resp. right] inverse of . If an element α ∈ S is both left and right invertible,
then α is called an invertible element.
Definition 2.5. (Integral Domain) Let (S; +, •) be a commutative ring with the
multiplicative identity 1 6= 0 and no zero divisors. Then, (S; +, •) is called an
integral domain.
Definition 2.6. (Division Ring) Let (S; +, •) be a ring with the multiplicative
identity 1 6= 0 and let (S \ {0}, •) be a group with 1 as the identity. Then, (S; +; •)
is called a division ring.
Definition 2.7. (Field) A commutative division ring is called a field.
Remark 2.3. A field (F ; +, •) has the following properties.
• (F ; +, •) is a commutative ring.
• There exists 1 ∈ F such that 1 • α = α • 1 = α ∀α ∈ F .
• (F \ {0}; •) is an Abelian group with 1 ∈ F \ {0} as the identity element.
• If α ∈ (F \ {0}, then there exists α−1 ∈ (F \ {0} such that
α−1 • α = α • α−1 = 1
Example 2.1. (R : +; •) and (C : +; •) are fields that are very commonly used
in engineering analysis. Note that (Q : +; •) is also a field but it is seldom used
because, as we have seen, Q is not a complete set whereas R and C are. Note that
(Z : +; •) is a commutative ring but it is not a field because no element of Z \ {1}
has a multiplicative inverse.
Example 2.2. Fields can be both infinite and finite. Examples of infinite fields
are: (R : +; •) and (C : +; •). An example of a finite field (also called Galois field)
is GF (2) , {0, 1}; ⊕2; ⊗2 , where ⊕2 is the addition operation under modulo 2
and ⊗2 is the multiplication operation under modulo 2. This is the smallest field.
Example 2.3. A polynomial over a field (F ; +, •) is defined as an expression of
the form: a0 + a1 x + a1 x2 + · · · + an xn where ai ∈ F and n is a non-negative
integer; and x is called the indeterminate. Examples of the indeterminate include
real numbers, complex numbers, and square matrices.
Definition 2.8. (Homomorphism) Let (U ; ⊛, ⊚) and (V ; ⋆, •) be two algebraic
systems and let h : U → V . Then, h is called a homomorphism from U to V
(carrying ⊛ to ⋆ and ⊚ to •) if, for all (u1 , u2 ) ∈ U × U , the following conditions
hold: h(u1 ⊛ u2 ) = h(u1 ) ⋆ h(u2 ) and h(u1 ⊚)u2 ) = h(u1 ) • h(u2 ).
4
If the function h is injective, then h is called a monomorphism; if h is surjective,
then it is called a epimorphism; and if h is bijective, then it is called an isomorphism.
In case of isomorphism, the inverse function h−1 exists and it is legitimate to say
that (U ; ⊛, ⊚) and (V ; ⋆, •) are isomorphic to each other.
3
Modules and Vector Spaces
This section focuses on modules and vector spaces. Modules over a ring are a
generalization of Abelian groups that are necessary for further study of algebra.
Important examples of modules are vector spaces that are of prime focus in this
course. In particular, a vector space is an Abeian group defined over a field whose
elements are called scalars.
Definition 3.1. (Module) Let (S; +, •) be a ring. Then, a (left) S -module is
an additive Abelian (i.e., commutative) group (M, ⊕) together with a function
S × M → M , where the image of (s, m) ∈ S × M is denoted as sm ∈ M such that
∀ r, s ∈ S and ∀ u, v ∈ M the following conditions hold:
• r(u ⊕ v) = ru ⊕ rv
• (r + s)u = ru ⊕ su
• r(su) = (rs)u
If the ring (S; +, •) has a (multiplicative) identity element 1, and if 1u = u ∀u ∈ M ,
then M is called a unitary (left) S-module. If (S; +, •) is a division ring, then a
unitary (left) S-module is called a left vector space. Similar definitions hold for
right S -module, unitary right S -module, and right vector space.
Remark 3.1. Modules over a ring are generalization of Abelian groups that, in
turn, are modules over (Z; +, •) .
Definition 3.2. (Vector Space) Let (V ; ⊕) be an Abelian group and (F ; +, •) be
a field whose multiplicative identity is 1 (i.e., α • 1 = 1 • α = α ∀α ∈ V ) and the
additive identity (i.e., zero-element) is 0 (i.e., α ⊕ 0 = 0 ⊕ α = α ∀α ∈ V ). Then,
(V ; ⊕) is called a vector space over the field (F ; +, •) if the following properties are
satisfied:
• Closure: There exists a function ⊗ : F × V → V , called scalar multiplication,
such that
(
α ⊗ x ∈ V ∀α ∈ F ∀x ∈ V
1 ⊗ x = x ∀x ∈ V
• Associativity: α ⊗ (β ⊗ x) = (α • β) ⊗ x ∀α, β ∈ F ∀x ∈ V
• Distributivity: ∀α, β ∈ F ∀x, y ∈ V
(
α ⊗ (x ⊕ y) = (α ⊗ x) ⊕ (α ⊗ y)
(α + β) ⊗ x = (α ⊗ x) ⊕ (β ⊗ y)
The elements of V are called vectors and the elements of F are called scalars. Often
the multiplicative operators • and ⊗ are simply omitted, i.e., we write α • β as αβ
and α ⊗ x as αx. However, it is important to distinguish between the operators of
5
the scalar addition + and the vector addition ⊕. We denote the zero (i.e., additive
identity) of the field (F ; +, •) as 0 and the zero vector (i.e., the identity of the
Abelian group (V ; ⊕)) as 0.
Remark 3.2. It follows from the definition of a vector space that ∀x ∈ V ∀α ∈ F
• 0x = 0 because 0x = (1 + (−1))x = 1x ⊕ (−1)x = x ⊕ (−x) = 0
• (−1)x = −x because x ⊕ (−1)x = 1x ⊕ (−1)x = (1 + (−1))x = 0x = 0
• α0 = 0 because α0 = α(x⊕(−x)) = αx⊕α(−x) = αx⊕α(−1)x = αx⊕(−α)x
= (α + (−α))x = 0x = 0
Remark 3.3. Every vector space must contain the zero vector 0. That is, the set
V in a vector space (V ; ⊕) can never be empty. A geometric interpretation of this
fact is that every coordinate frame of a vector space must have the origin.
Definition 3.3. (Subspace) Let (V ; ⊕) be a vector space over a field (F ; +, •) .
Let U ⊆ V such that (U ; ⊕) is a vector space over a field (F ; +, •). Then, (U ; ⊕)
is a subspace of (V ; ⊕). In addition, if U is a proper subset of V , then (U ; ⊕) is a
proper subspace of (V ; ⊕).
Remark 3.4. If (U ; ⊕) is a subspace of (V ; ⊕), then it follows that (U ; ⊕) is a
subgroup of the group (V ; ⊕).
Proposition 3.1. (U ; ⊕) is a subspace of a vector space (V ; ⊕) over a field (F ; +, •)
if and only if the following condition holds: (αx ⊕ y) ∈ U ∀x, y ∈ U ∀α ∈ F .
Proof. The proof follows from the definition of a subspace.
Definition 3.4. (Linear Combination) Let (V ; ⊕) be a vector space over a field
(F ; +, •) and let S be a nonempty (finite or countably infinite or uncountable) set
of vectors. Then, x ∈ V is said to be a linear combination of vectors in S if there
exists a finite set of vectors {u1 , · · · , un } and a finite set of scalars {α1 , · · · , αn },
Ln
where n ∈ N, such that x = j=1 αj uj .
Definition 3.5. (Linear Dependence) Let (V ; ⊕) be a vector space over a field
(F ; +, •) and let S be a nonempty (finite or countably infinite or uncountable) set
of vectors. Then, the vectors in S are said to linearly independent if, for each
x ∈ S, x is not a linear combination of the vectors in S \ {x} (i.e., the set S with
x removed). The set S in the vector space (V ; ⊕) is linearly dependent if it is
not linearly independent, i.e., there exists a vector x ∈ S such that x is a linear
combination of vectors in S \ {x}.
Theorem 3.1. Let (V ; ⊕) be a vector space over a field (F ; +, •) and let S be
a nonempty (finite or countably infinite or uncountable) set of vectors. Then, S
is linearly independent if and only if, for each nonempty finite subset of S, say,
Ln
j
{u1 , · · · , un }, the only n-tuple of scalars satisfying the equation:
j=1 αj u = 0,
is the trivial solution: α1 = · · · = αn = 0.
Proof. See Naylor & Sell pp. 177-178.
6
Remark 3.5. A (nonempty) set S in a vector space (V ; ⊕) is linearly dependent if
and only if there exists a nonempty finite subset of S, say, {u1 , · · · , un } and scalars
Ln
α1 , · · · , αn , where not all αi ’s are zero, such that j=1 αj uj = 0.
Definition 3.6. (Spanning) Let (V ; ⊕) be a vector space over a field (F ; +, •) and
let S be a nonempty (finite or countably infinite or uncountable) set of vectors.
Then, the set of all (finite) linear combinations of vectors in S is the space spanned
by S and is denoted as span(S).
Remark 3.6. It follows from Definition 3.6 that span(S) is the smallest subspace
of (V ; ⊕) containing the set S, i.e., span(S) is the intersection of all subspaces of
(V ; ⊕) that contain the set S.
3.1
Hamel Basis and Dimension of Vector Spaces
Hamel basis is a purely algebraic concept and is not the only concept of basis that
arises in the analysis of vector spaces. There are other concepts of basis (e.g.,
Schauder basis and orthonormal basis) that involves both algebraic and topological
concepts. These issues will be dealt with in later chapters.
Definition 3.7. (Hamel Basis) Let (V ; ⊕) be a vector space over a field (F ; +, •)
and let B be a nonempty set of vectors. Then, B is said to be a Hamel basis for
(V ; ⊕) if the following conditions hold: (i) B is a linearly independent set, and (ii)
span(B) = V .
Remark 3.7. Let (V ; ⊕) be a vector space and let S be a linearly independent
set of vectors in V . Then, there exists a Hamel basis B such that S ⊆ B, and it
follows that every vector space has a Hamel basis.
Remark 3.8. Let B1 and B2 be two Hamel bases of a vector space (V ; ⊕). Then,
B1 and B2 have the same cardinal number. (See Naylor & Sell, p. 184).
Definition 3.8. (Dimension) Let (V ; ⊕) be a vector space over a field (F ; +, •).
The dimension of (V ; ⊕) is defined to be the cardinal number of its Hamel basis
and is denoted by dim(V ).
Remark 3.9. It follows from the above definition that dimension is a property of
the vector space and is independent of any particular Hamel basis.
Definition 3.9. (Vector Space Isomorphism) Let (V ; ⊕) and (W ; ⊛) be two vector
space over the same field (F ; +, •). Then, (V ; ⊕) and (W ; ⊛) are said to be isomorphic if there exists a linear bijective mapping h : V → W . That is, h : V → W
satisfies the following conditions.
• Linearity: h(αx ⊕ y) = αh(x) ⊛ h(y) ∀x, y ∈ V ∀α ∈ F
• Injectivity: h(x) = h(y) ⇒ x = y ∀x, y ∈ V
• Surjectivity: {h(x) : x ∈ V } = W
Remark 3.10. The role of the field in Definition 3.9 is very critical in the sense
that a given Abelian group could generate two different vector spaces when defined
over two different fields. For example, the Abelian group of complex numbers C
7
forms a two-dimensional vector space over the real field R, which is isomorphic
to the two-dimensional real vector space R2 . The same Abelian group of complex
numbers C forms a one-dimensional vector space over the complex field C. The
implications of this fact are illustrated below by a physical example.
Let an unforced single-degree-of-freedom underdamped linear time-invariant
mass-spring-damper (or inductance-capacitance-resistance) system, with non-zero
initial conditions, be governed by the following equation.
d2 y(t)
dy(t)
+ 2 ξ ωn
+ ωn2 y(t) = 0
2
dt
dt
where y(t) is the time-dependent displacement (or charge); ωn is the natural frequency; and ξ is the damping coefficient (0 ≤ ξ < 1). A state-space representation
of the above equation in the two-dimensional vector space R2 over the real field R
is given below.
#"
#
p
1 − ξ 2 ωn
x1
=
−ξ ωn
x2
p
2
where a choice for the states
is: x1 = ξωn y + dy
and x#
2 = − 1 − ξ ωn y and the
"
p dt
−ξ ωn
1 − ξ 2 ωn
p
state transition matrix
∈ R2×2 maps R2 into R2 .
− 1 − ξ 2 ωn
−ξ ωn
d
dt
"
x1
x2
#
"
−ξ ωn
p
− 1 − ξ 2 ωn
Let us construct a vector space over the complex field C instead of the real field
R. To this end, let us define a complex-valued state z(t) , x1 (t) + ix2 (t), where
√
i = −1; and x1 and x2 are as defined above. Then it follows that
p
p
dx2
dx1
+i
= − ξωn x1 + 1 − ξ 2 ωn x2 + i − 1 − ξ 2 ωn x1 − ξωn x2 ,
dt
dt
which reduces to
h
i
p
d
[x1 + ix2 ] = − (ξ + i 1 − ξ 2 )ωn [x1 + ix2 ].
dt
Consequently, the state-space representation of the above equation in the onedimensional vector space C1 over the complex field C is as given below.
h
i
p
d
[z] = − (ξ + i 1 − ξ 2 )ωn [z]
dt
i
h
p
where the state transition matrix − (ξ + i 1 − ξ 2 )ωn ∈ C1×1 maps C1 into
C1 . Note that, in the disciplines of Physics and Engineering, such a 2nd order
underdamped system is often referred to as a single degree-of-freedom system. Here,
we have shown that an underdamped system is represented on a vector space of
dimension 2 over the real field R, or on a vector space of dimension 1 over the
complex field C.
Next we consider an overdamped system, i.e., the damping coefficient ξ > 1. A
state-space representation of the overdamped system in the two-dimensional vector
space R2 over the real field R is given below.
d
dt
"
x1
x2
#
=
"
−ξ ωn
p
ξ 2 − 1 ωn
8
#"
#
p
x1
ξ 2 − 1 ωn
−ξ ωn
x2
p
dy
where a choice for the states
is:
x
=
ξω
y
+
ξ 2 − 1 ωn y and the
and
x
=
1
n
2
dt
"
#
p
2
−ξ ωn
ξ − 1 ωn
state transition matrix p 2
∈ R2×2 maps R2 into R2 .
ξ − 1 ωn
−ξ ωn
Similar to what was done for the underdamped system, the vector space is
constructed over the complex field C instead of the real field R. The complex√
valued state is defined as z(t) , x1 (t) + ix2 (t), where i = −1; and x1 and x2 are
as defined above. Then it follows that
p
p
dx1
dx2
ξ 2 − 1 ωn x1 − ξωn x2 ,
+i
= − ξωn x1 + ξ 2 − 1 ωn x2 + i
dt
dt
which reduces to
p
p
d
[x1 + ix2 ] = ωn (−ξx1 + ξ 2 − 1 x2 ) + i( ξ 2 − 1 x1 − ξx2 )
dt
Consequently, the state-space representation of the above equation in the onedimensional vector space C1 over the complex field C as given below.
h
i
hp
ih
i
p
d
ξ 2 − 1 ωn (z − z̄)
[z] = − (ξ + i ξ 2 − 1)ωn [z] −
dt
where z̄ is the complex conjugation of z ∈ C1 . Note that the operation of complex conjugation may not be linear because, given any z, z̃, γ ∈ C, it follows that
¯
z + γ z̃ = z̄ + γ̄ z̃¯ instead of z + γ z̃ = z̄ + γ z̃.
For the critically damped system (i.e., ξ = 1), The system dynamics is given as
d
dt
"
x1
x2
#
=
"
−ωn
0
ωn
−ωn
#"
x1
x2
#
where a choice for" the states is:# x1 = ωn y and x2 = ωn y +
−ωn ωn
transition matrix
∈ R2×2 maps R2 into R2 .
0
−ωn
dy
dt
and the state
Letting z(t) , x1 (t) + ix2 (t), it follows that
i
h ω ih
d
n
(z − z̄)
[z] = [−ωn ][z] − i
dt
2
Theorem 3.2. Let (V ; ⊕) and (W ; ⊗) be two vector spaces over the same field F.
Then, (V ; ⊕) and (W ; ⊗) are isomorphic if and only if dim(V ) = dim(W ).
Proof. Let (V ; ⊕) and (W ; ⊗) be isomorphic and let BV be a basis for V . Let
h : V → W be an isomorphism (i.e., linear bijection) of V onto W . Then, due to
linearity of h, the image h(BV ) of BV under h generates linear combinations that
would span W . Furthermore, since h is surjective, it follows that span(h(BV )) = W .
Therefore, h(BV ) is a Hamel
basis
of W . Finally, since
h is injective, it follows that
card(BV ) = card(h(BV )) ⇒ dim(V ) = dim(W ) .
To prove the other way, let dim(V ) = dim(W ) be given. Therefore, there
exists a one-to-one correspondence h̃ between the spaces V and W . Using this
correspondence h̃, let us define a linear mapping h : V → W .
Let BV and BW be Hamel bases of the spaces V and W , respectively. Let
x ∈ V that is expressed uniquely as a (finite) linear combination of vectors in BV
9
Ln
in the form: x = j=1 αj v j , where v j ∈ BV and the scalar αj ∈ F and at least one
αj 6= 0. Now we construct h such that
(
Nn
h(x) = j=1 αj h̃(v j )
h(0) = 0
Since BW is a basis, h is a bijective mapping from v onto V . Furthermore, h is
linear and hence h is an isomorphism.
Theorem 3.3. (Vector Space Isomorphism) (V ; ⊗) be a finite-dimensional vector
space over the field F, where dim(V ) = n ∈ N. Let (Fn , ⊕) denote the vector
space made up of ordered n-tuples of scalars in F. Then, the vector space (V ; ⊗) is
isomorphic to (Fn , ⊕).
Proof. Given that (V ; ⊗) is an n-dimensional vector space with scalars in F, let
{v 1 , · · · , v n } be a basis of V , i.e., the vectors v j , j = 1, · · · , n are linearly independent and span the space (V ; ⊗).
To establish an isomorphism between the spaces (Fn , ⊕) and (V ; ⊗), it is necessary to find a function h : Fn → V that is linear and bijective. Let h(a) ,
Nn
j
n
′
j=1 aj v , where a ∈ F , i.e., a = [a1 · · · an ] with aj ∈ F.
n
Linearity of h: Let α ∈ F and a ∈ F and b ∈ Fn . Then, αa ⊕ b ∈ Fn and
N
h(αa ⊕ b) = nj=1 (αaj + bj ) v j = (αh(a)) ⊗ h(b) ⇒ linearity is established.
Injectivity of h: Let h(a) = h(b) for a ∈ Fn and b ∈ Fn . By linearity of h and
Nn
using −1 ∈ F, it follows that 0 = h(a ⊕ (−1)b) ⇒ j=1 aj + (−1)bj v j . Because
of linear independence of v j ’s, it follows that aj + (−1)bj = 0 ⇒ aj = bj ∀j.
Therefore, a = b ⇒ injectivity is established.
Surjectivity of h: Since v j ’s form a basis of (V ; ⊗), there exist unique scalars
Nn
cj ’s such that x = j=1 cj v j = h(c) ∀x ∈ V , i.e., there exists c ∈ Fn ∀x ∈ V ⇒
surjectivity is established.
Remark 3.11. Since a finite-dimensional (say n ∈ N) vector space V over a field
F (e.g., the real field R or the complex field C) is isomorphic to Fn , it is natural
to study the properties of Fn in lieu of those of V . The vectors in Fn can be
interpreted as: (i) n-tuples of scalars in F, or (ii) linear mappings from F to Fn .
Let us present a generalization of the second concept. Let N , {1, 2, · · · , n} be
the set of all integers from 1 to n. Then, a function x : N → F implies that a rule
assigns a scalar x(j) to each j ∈ N and let the set of all such functions be denoted
as F(N ) , i.e., specifying a function x ∈ F(N ) amounts to presenting n consecutive
scalars x(1), x(2), · · · , x(n), which could be arranged as a column vector or as a
row vector in Fn . This provides a bijectivity between F(N ) and Fn . For all c ∈ F
and x, y ∈ F(N ) , it is meaningful to define the operations of scalar multiplication
and vector addition in F(N ) respectively by
(cx)(j) = cx(j) and (x ⊕ y)(j) = x(j) + y(j)
These operations make F(N ) a vector space over the field F, which reveals that there
is no essential difference between F(N ) and Fn . Now we generalize this concept to
infinite-dimensional vector spaces.
Let S be an arbitrary (i.e., finite, countable, or uncountable) nonempty set that
serves as an index set and let FS denote the set of all functions f : S → F. If S = N,
10
then FS is the vector space of all possible sequences s : N → F which includes all
(i.e., real or complex) scalar-valued sequences. Similarly, if S = R, then FS is
the vector space of all possible (not necessarily linear) functions f : R → F that
includes all (i.e., real or complex) scalar-valued time-dependent signals; for example,
f (t) could be the output voltage of an operational amplifier. This concept allows
extension of the tools of linear algebra to functions spaces. In general, if S is a
countably infinite set, i.e., S ∼ N, the the vector x ∈ FS is called a sequence; if S
is an uncountably infinite set, i.e., S ∼ R, the vector x ∈ FS is called a function.
3.2
Linear Transformation of Vector Spaces
This subsection introduces the rudimentary concepts of linear transformation of
vector spaces. More advanced materials on transformation of vector spaces will be
presented in later chapters.
Definition 3.10. (Linear Transformation) Let (V ; ⊕) and (W ; ⊗) be two vector
spaces over the same field F. Then, L : V → W is called a linear transformation if
the following condition holds for all xk ∈ V and all αk ∈ F:
L ⊕nk=1 (αk xk ) = ⊗nk=1 αk L(xk ) for any arbitrary n ∈ N
If W = V , then the transformation L : V → V is called an operator. If W = F,
then the transformation L : V → F is called a functional. The space of all linear
bounded functionals of a vector space V is called the dual space V ⋆ .
Definition 3.11. (Null Space) The null space N (L) of a linear transformation
L : V → W is the subspace of V that satisfies the following condition:
N (L) , x ∈ V : Lx = 0W
Definition 3.12. (Range Space) The range space R(L) of a linear transformation
L : V → W is the subspace of W that satisfies the following condition:
R(L) , Y ∈ W : y = Lx and x ∈ V
Definition 3.13. (Sum, Direct Sum, and Algebraic Complement) Let V be a
vector space, where the operation of vector addition is ⊛, and let U and W be
two subspaces of V . Then, the sum of U and W is a subspace of V , denoted as
Y = U + W , which is spanned by all vectors in U and W , i.e.,
U + W , x = u ⊛ w : u ∈ U and w ∈ W
The direct sum of of U and W is a subspace of V , denoted as Y = U
L
W , if
∀y ∈ Y ∃ u ∈ U and w ∈ W such that there is a unique representation y = u⊛w.
In this setting, U is called the algebraic complement of W (alternatively, W is the
algebraic complement of U ) in Y .
Definition 3.14. (Projection) Let V be a vector space. Then, a linear transformation P : V → V is called a projection on V if P 2 = P , i.e., P (P x) = P x ∀x ∈ V .
11
Remark 3.12. Linearity is a necessity for projection on a vector space. A nonlinear
mapping f : V → V , which satisfies the condition f 2 = f is not necessarily a
projection. For example, let f : R → R be defined as:


if x ≥ +1
 +1
f (x) =
0 if −1 < x < +1


−1
if x ≤ −1
In this case, f ◦ f = f but f is not a projection because it is nonlinear.
Lemma 3.1. (Decomposition of projection) Let P be a projection defined on a vector space V . Then, the null space N (P ) and the range space R(P ) of the projection
P are algebraic complements of each other in V , and the following conditions are
satisfied.
T
1. R(P ) N (P ) = {0V }.
L
2. R(P ) N (P ) = V .
T
Proof. First part: Let x ∈ R(P ) N (P ). Since x ∈ R(P ), there exists y ∈ V such
that P y = x ⇒ P x = x. Since x ∈ N (P ), it follows that P x = 0V ⇒ x = 0V .
T
Therefore, R(P ) N (P ) = {0V }.
Second part: Let x ∈ V and define y = P x and y ∈ R(P ). Having z = x − y
which implies x = y + z, it follows that
P z = P (x − y) = P x − P y = P x − P 2 x = P x − P x = 0V .
Therefore, z ∈ N (P ). Since the choice of x ∈ V is arbitrary, it follows that
L
R(P ) N (P ) = V .
Remark 3.13. Let U and W be two subspaces of a vector space V such that
T
L
U W = {0V } and U
W = V . Then, there exists a projection P on V such
that R(P ) = U and N (P ) = W . This issue will be further dealt with in Chapter
Four (see also Naylor & Sell, p. 200).
Remark 3.14. Some of the important observations on sum, direct sum, and algebraic complement of subspaces are listed below: (see Naylor & Sell, pp.198-199.)
L
T
1. The sum U + W is the direct sum U
W if and only if U W = 0V .
2. The natural mapping ϕ, denoted as ϕ[(x, y)] = x ⊛ y, of the direct sum
L
T
U
W is an isomorphism if and only if U W = 0V .
3. If U is a subspace of a vector space V , then U must have an algebraic complement in V .
3.3
Matrix Representation of Finite-dimensional Spaces
Let V and W be two finite-dimensional vector spaces over the same field F, where
dim(V ) = n and dim(W ) = m; let A ∈ L(V, W ) be a linear transformation from
V to W . If B , {b1 , b2 , · · · , bn } is a basis for V , then each vector x ∈ V can be
Pn
uniquely represented as x = k=1 bk βk , where βk ∈ F. Let the vector of coordinates
βk be denoted as [x]B , [β1 β2 · · · βn ]T , which implies that [x]B ∈ Fn . Similarly, if
C , {c1 , c2 , · · · , cm } is a basis for W , then the image of x ∈ V under A ∈ L(V, W ),
12
x V
A
Ax W
B
[x]B
C
Fn
[A]B,C
[Ax]C Fm
Figure 1: Isomorphism between L(V, W ) and Fm×n
Pm
i.e., the vector Ax ∈ W can be uniquely represented as Ax = k=1 ck γk , where
γk ∈ F; thus, the vector of coordinates γk is denoted as [Ax]C , [γ1 γ2 · · · γm ]T ,
which implies that [Ax]C ∈ Fm .
Definition 3.15. (Matrix representation) The linear transformations from the coordinates, [x]B ∈ Fn , of a vector x ∈ V to the coordinates, [Ax]C ∈ Fm , of the
vector Ax ∈ W is an (m × n) matrix. This matrix, denoted by A ∈ Fm×n , serves
as the coordinates of the linear transformation A ∈ L(V, W ) relative to the bases
(B, C).
Theorem 3.4. (Matrix Representation Theorem) Let B , {b1 , b2 , · · · , bn } and
C , {c1 , c2 , · · · , cm } be bases for the vector spaces V and W , respectively. Let the
k th column of a matrix A ∈ Fm×n be defined as ak , [Abk ]C for 1 ≤ k ≤ n and
k ∈ N. Then, L(V, W ) is isomorphic to Fm×n and [Ax]C = A[x]B ∀x ∈ V .
P
Proof. Let x ∈ V and x = nk=1 bk βk . Then, [x]B , [β1 β2 · · · βn ]T . Since A is a
linear transformation, it follows that
C X
X
n
n
n
X
k C
k
=
[Ax] = A
b βk
βk =
ak βk = A[x]B
Ab
C
k=1
k=1
k=1
Next, to show that the spaces L(V, W ) and Fm×n are isomorphic, let us define a
linear transformation T : L(V, W ) → Fm×n such that, for each A ∈ L(V, W ), there
exists a unique matrix A ∈ Fm×n whose k th column is ak = [Abk ]C . What remain
to be shown are that T is linear, injective, surjective. This part of the proof is
similar to that of Theorem 3.3.
In view of Theorem 3.4, every finite-dimensional linear transformation can be
represented by a matrix of appropriate dimension as seen in Figure 1 that pictorially
illustrates the isomorphic relationship between the spaces L(V, W ) and Fm×n . The
matrix A ∈ Fm×n is the matrix representation of the linear transformation A
induced by the bases B and C.
Example 3.1. (Integration of a Polynomial) Let the vector spaces V and W over
the real field R be defined as: V = P n−1 ((0, ∞)) and W = P n ((0, ∞)), where
P n ((0, ∞)) is the space of polynomials of degree n or less and the indeterminate
13
is a real scalar in the range of (0, ∞). Let us consider the integral transform
A ∈ L(V, W ), i.e., for each x ∈ V and each t ∈ (0, ∞),
Z t
(Ax)(t) ,
dτ x(τ )
0
Let B be the basis of elementary polynomials, i.e., B = {1, t, t2 , · · · } for t ∈ (0, ∞).
Then, the matrix representation of A induced by the bases of elementary polynomials is computed as:
#B
"Z
t
h tk iB
1
k
k−1 B
k−1
a = [At
] =
=
dτ τ
= 1k+1
k
k
0
where 1k denotes the k th column of the identity matrix. Thus, for n = 3, it
follows that the matrix representation A of the integral transform A mapping the
coordinates of a polynomial into the coordinates of its integral is given as;



A=


0 0 0
1 0 0 


1
0 2 0 
0 0 13
For example, if x(t) = 4 − 3t + t2 , then [x]B = [4 − 3 1]T and
Ax are obtained as




0
0 0 0 
4
 4
 1 0 0 
 


[Ax]B = A[x]B = 
  −3  =  3
 −2
 0 12 0 
1
1
1
0 0 3
3
Thus, the integral of 4 − 3t + t2 is 4t − 32 t2 + 13 t3 .
the coordinates of





Example 3.2. (Differentiation of a Polynomial) Let the vector spaces V and W
over the real field R be defined as: V = P n ((0, ∞)) and W = P n−1 ((0, ∞)), where
P n ((0, ∞)) is the space of polynomials of degree n or less and the indeterminate
is a real scalar in the range of (0, ∞). Let us consider the derivative transform
A ∈ L(V, W ), i.e., for each x ∈ V and each t ∈ (0, ∞),
(Ax)(t) ,
d
x(t)
dt
Let B be the basis of elementary polynomials, i.e., B = {1, t, t2 , · · · }, t ∈ (0, ∞).
Then, the matrix representation of A induced by the bases of elementary polynomials is computed as:
#B
"
iB
h
d k
k
k B
= ktk−1 = k 1k
a = [At ] =
t
dt
where 1k denotes the k th column of the identity matrix. Thus, for n = 3, it
follows that the matrix representation A of the derivative transform A mapping
the coordinates of the polynomial into the coordinates of its derivative is given as;

0 1 0 0


A= 0 0 2 0 
0 0 0 3

14
[x]B
[Ax]D
[A]B,D
B
D
x
P
Ax
C
Q
E
[A]C,E
[x]C
[Ax]E
Figure 2: Coordinate Transformation in Fm×n
For example, if x(t) = −4 + 5t − 2t2 + t3 , then [x]B = [−4 5 − 2 1]T and the
coordinates of Ax are obtained as




 −4

5
0 1 0 0 


 5  

[Ax]B = A[x]B =  0 0 2 0  
 =  −4 
 −2 
3
0 0 0 3
1
Thus, the derivative of − 4 + 5t − 2t2 + t3 is 5 − 4t + 3t2 .
3.4
Coordinate Transformation in Finite-dimensional Spaces
This subsection introduces the basic concepts of coordinate transformation in finitedimensional vector spaces through the usage of linear transformation. The concept
of matrix representation in Theorem 3.4 leads to the fact that, by choosing bases
for two finite-dimensional vector spaces V ∼ Fn and W ∼ Fm over the same field F,
each vector in L(V, W ) can be represented by its coordinates as a vector in Fm×n .
To develop a strategy for selecting bases of V and W , we encounter the problem of
transforming from one matrix representation to another. That is, given the matrix
representation of A ∈ L(V, W ) relative to a given pair of bases, the problem is
how to find its matrix representation relative to another pair of bases. Here each
matrix represents the same linear transformation in a different coordinate system.
Consequently, moving from one matrix representation to another is a coordinate
transformation problem.
Theorem 3.5. (Coordinate Transformation Theorem) Let V ∼ Fn and W ∼ Fm
be two finite-dimensional vector spaces over the same field F and let A ∈ L(V, W ).
Let B = {b1 , b2 , · · · , bn } and C = {c1 , c2 , · · · , cn } be two bases for V , and let
D = {d1 , d2 , · · · , dm } and E = {e1 , e2 , · · · , en } be two bases for W . Let the matrix
P ∈ Fn×n whose k th column is [ck ]B , and let the matrix Q ∈ Fm×m whose k th
column is [dk ]E . Then, it follows that
[A]C,E = Q [A]B,D P
Proof. First we show that the matrix P maps C-coordinates into B-coordinates.
For each x ∈ V , we have
[x]B =
n
hX
k=1
ck [x]C
k
iB
=
n
X
[ck ]B [x]C
k =
k=1
n
X
k=1
15
C
pk [x]C
k = P [x]
By using a similar argument we show that the matrix Q maps D-coordinates into
E-coordinates. That is, for each y ∈ W , we have [y]E = Q [y]D . Therefore, for
each x ∈ V , it follows that
Q [A]B,D P [x]C = Q [A]B,D [x]B = Q [Ax]D = [Ax]E = [A]C,E [x]C
Since x ∈ V is arbitrarily chosen, the proof follows by application of Theorem 3.4.
Pre-multiplication and post-multiplication by the respective transformation matrices is needed to obtain one matrix representation from another as seen pictorially
in Figure 2 that summarizes the relationship between a linear transformation and
two of its matrix representations. A matrix itself is a linear transformation. As
such, it is possible to find a matrix representation of a matrix.
Example 3.3. (Coordinate Transformation) This example deals with a linear
transformation, namely, differentiation on the space of polynomials, and two of
its matrix representations. Let V = P n ((0, ∞) and let A be the derivative transformation on V . That is, for each x ∈ V and each t ∈ (0, ∞),
(Ax)(t) ,
d
x(t)
dt
Then, A ∈ L(V, W ), where W = P n−1 ((0, ∞)). Let B be the standard basis
of elementary polynomials, i.e., B = {1, t, t2 , · · · }. If A ∈ F n×(n+1) is the matrix
representation of A induced by the bases of elementary polynomials, then it follows
from Example 3.2 for the case n = 3 that


0 1 0 0


A= 0 0 2 0 
0 0 0 3
Next, let us consider the following alternative basis for the space of polynomials:
D = {1, 2t, 3t2, · · · }. If D ∈ F n×(n+1) is the matrix representation of A induced by
the the bases associated with D, then it follows from Theorem 3.5 that D = QAP ,
where the coordinate transformation matrix P is computed as pk = [ktk−1 ]B = k1k
for 1 ≤ k ≤ (n + 1), and the coordinate transformation matrix Q is computed as
q k = [ktk−1 ]D = k1 1k for 1 ≤ k ≤ n). For n = 3, it follows that




1 0 0 0
1 0 0
 0 2 0 0 




P =
 and Q =  0 12 0 
 0 0 3 0 
0 0 13
0 0 0 4
Therefore,

0 2 0 0


D = QAP =  0 0 3 0 
0 0 0 4

As a check, let us use Theorem 3.4 to compute the matrix D directly. For
1 ≤ k ≤ (n + 1), we have
iD
hd
(ktk−1 ) = [k(k − 1)tk−1 ]D = k 1k−1 and 10 = 0
dk = [A(ktk−1 )]D =
dt
16
4
Tensor Product of Vector Spaces
The notion of tensor product is rather a language than a substantial mathematical
theory. Nevertheless this language is very useful in the disciplines of Physics and
Engineering and is often indispensable. In particular tensors provide a uniform
description of different aspects of linear algebra within a single algebraic structure.
Definition 4.1. (Multilinearity) Let V1 , · · · , Vp , where p ∈ N, and U be vector
spaces over a given field F. Then, the map
ϕ : V1 × · · · × Vp → U
is called multilinear (specifically, p-linear) if it is linear in each of the p arguments when the other arguments are held constant. Such a map is denoted as
Hom(V1 , · · · , Vp , U ) and forms a vector space, which is a subspace of the space of
all maps from V1 , · · · , Vp to U .
If the spaces V1 , · · · , Vp and U are finite-dimensional (but not necessarily of
the same dimension), then the space Hom(V1 , · · · , Vp , U ) is also finite-dimensional
and dim(Hom(V1 , · · · , Vp , U )) = dim(V1 ) · · · dim(Vp ) dim(U ). This is so because
a multilinear map ϕ : V1 × · · · × Vp → U is determined by the images of the
basis vectors of the vector spaces V1 , · · · , Vp , which, in turn, are determined by
their coordinates in a basis of U . Furthermore, if U = F, then Hom(V1 , · · · , Vp , F)
is a space of multilinear functions on V1 , · · · , Vp . In particular, if p = 1, then
Hom(V, F) is called the dual space V ⋆ of the vector space V provided that Hom(V, F)
is bounded.
The tensor products of two vector spaces V and W (over the same field) arises
naturally for bilinear maps ϕ : V × W → U , and the corresponding space U is
called the tensor product of V and W .
Theorem 4.1. (Bilinear Properties) Let V and W be two vector spaces (over the
same field) with bases {vi , i ∈ I} and {wj , j ∈ J}. The following properties of a
bilinear map ϕ : V × W → U are equivalent.
(i) The set of vectors {ϕ(vi , wj ), i ∈ I, j ∈ J} forms a basis of U .
P
(ii) Every vector u ∈ U decomposes uniquely as u = i ϕ(vi , yi ), yi ∈ W .
P
(iii) Every vector u ∈ U decomposes uniquely as u = j ϕ(xj , wj ), xj ∈ V .
If the spaces V and W are infinite-dimensional, the respective sums are required to
be finite.
P
P P
Proof. Let u = i j uij ϕ(vi , wj ). Then, by setting yi = j uij wj , it follows
P
that u = i ϕ(vi , yi ), yi ∈ W and vice versa. Therefore, equivalence of the propP
erties (i) and (ii) is established. Similarly, by setting xj = i uij vi , it follows that
P
u = j ϕ(xj , wi ), xj ∈ V . Thus, equivalence of (i) and (iii) is established.
Remark 4.1. If the property (i) of Theorem 4.1 holds for some bases of V and W ,
then it holds for any bases of V and W .
17
Definition 4.2. (Tensor Product) A tensor product of two vector spaces V and
W , over the same field F, is a vector space T over F with a bilinear map
⊗ : V × W → T, (x, y) 7→ (x ⊗ y)
that satisfies the following condition: If {vi : i ∈ I} and {wj : j ∈ J} are bases of
V and W , respectively, then {vi ⊗ wj : i ∈ I, j ∈ J} is a basis of T . This condition
is independent of the choice of the bases in V and W .
Remark 4.2. The tensor product of vector spaces V and W in Definition 4.2 is
denoted as V ⊗ W ; if it is necessary to specify the field F over which the vector
spaces are defined, then the tensor product is denoted as V ⊗F W . Furthermore, if
both V and W are finite-dimensional, then it follows that
dim(V ⊗ W ) = dim V × dim W.
A tensor product exists for any two vector spaces over the same field. For
example, let a vector space T have a basis {tij : i ∈ I, j ∈ J} and let a bilinear map
be defined as: ⊗ : V × W → T so that vi ⊗ wj = tij for the basis vectors vi and wj .
A tensor product is unique in the following sense: If (T1 , ⊗1 ) and (T2 , ⊗2 )
are two tensor products of vector spaces V and W , then there exists a unique
isomorphism ψ : T1 → T2 such that ψ(x⊗1 y) = x⊗2 y ∀x ∈ V and ∀y ∈ W . Indeed,
for basis vectors, the isomorphism can be constructed as: ψ(vi ⊗1 wj ) = vi ⊗2 wj .
Example 4.1. Let a bilinear map ⊗ : F[x] × F[y] → F[x, y] be defined as:
(v ⊗ w)(x, y) = v(x)w(y)
(Note that F[•] denotes a polynomial algebra over the field F.) The products
xi ⊗ y j = xi y j , where i, j = 0, 1, 2, · · · , form a basis for F[x, y]. Hence,
F[x, y] = F[x] ⊗ F[y].
Similarly, a multi-linear tensor will follow the rule:
F[x1 , · · · , xm ; y1 , · · · , yn ] = F[x1 , · · · , xm ] ⊗ F[y1 , · · · , yn ]
Theorem 4.2. Let V and W be vector spaces over the same field F. Then, for any
bilinear map ϕ : V × W → U , there exists a unique linear map ψ : V ⊗ W → U
such that ϕ(x, y) = ψ(x ⊗ y) ∀x ∈ V ∀y ∈ W .
Proof. On basis vectors of the vector space V ⊗ W , the linear map ψ : V ⊗ W → U
is determined as:
ψ(vi ⊗ wj ) = ϕ(vi , wj ).
Every element u ∈ V ⊗ W decomposes uniquely as:
XX
uij vi ⊗ wj where uij ∈ F
u=
i
j
where the scalars uij are called the coordinates of u with respect to the given bases
of V and W . Specifically, in finite-dimensional vector spaces, u is described by a
(m × n) matrix (uij ), where m = dim(V ) and n = dim(W ).
18
Definition 4.3. (Decomposition) A vector u ∈ V ⊗ W is called decomposable if it
decomposes as:
u = x ⊗ y for some x ∈ V and y ∈ W.
P
P
Remark 4.3. Let x = i xi vi and y = j yj wj ; then, for u = x ⊗ y, it follows
that uij = xi yj . This implies that, in the finite-dimensional case, rank(uij ) ≤ 1.
Thus, decomposable elements comprise a small part of the space V ⊗ W unless
either V or W is one-dimensional; nevertheless they span the whole space.
Example 4.2. (Tensor Product) Let V and W be two finite-dimensional vector
spaces over the same field F with dim(V ) = n and dim(W ) = n. Let A = (aij ) be
the matrix of a linear operator A in a basis {v1 , · · · , vn } of V , and let B = (bkℓ ) be
the matrix of a linear operator B in a basis {w1 , · · · , wm } of W . Then, the matrix
of linear operator A ⊗ B in the basis {v1 ⊗ w1 , · · · , v1 ⊗ wm , v2 ⊗ w1 , · · · , v2 ⊗
wm , · · · , vn ⊗ w1 , · · · , v2 ⊗ wm } of the space V ⊗ W is





a11 B
a21 B
···
an1 B
a12 B
a22 B
···
an2 B

· · · a1n B
· · · a1n B 


··· ··· 
· · · ann B
which is called the tensor product of matrices A and B and is denoted as A ⊗ B.
It follows that
• T race(A ⊗ B) = T race(A) × T race(B).
• Det(A ⊗ B) = Det(A)m × Det(B)n .
19