6
Linear Transformations
1
6
Hopfield Network Questions
• The network output is repeatedly multiplied by the weight
matrix W.
• What is the effect of this repeated operation?
• Will the output converge, go to infinity, oscillate?
• In this chapter we want to investigate matrix multiplication,
which represents a general linear transformation.
2
6
Linear Transformations
A transformation consists of three parts:
1. A set of elements X = {xi}, called the domain,
2. A set of elements Y = {yi}, called the range, and
3. A rule relating each x i  X to an element yi  Y.
A transformation is linear if:
1. For all x 1, x 2  X, A(x 1 + x 2 ) = A(x 1) + A(x 2 ),
2. For all x  X, a   , A(a x ) = a A(x) .
3
6
Example - Rotation
Is rotation linear?
1.
2.
4
6
Matrix Representation - (1)
Any linear transformation between two finite-dimensional
vector spaces can be represented by matrix multiplication.
Let {v1, v2, ..., vn} be a basis for X, and let {u1, u2, ..., um} be
a basis for Y.
n
x
=
m
 xi v i
y
i=1
=
 y iu i
i= 1
Let A:XY
A x 
n
=


A   x j v j =
j = 1

y
m
 y iu i
i =1
5
6
Matrix Representation - (2)
Since A is a linear operator,
n
 x j A v j
m
=
j=1
 y iu i
i =1
Since the ui are a basis for Y,
m
A v j 
(The coefficients aij will make
up the matrix representation of
the transformation.)
 aij ui
=
i= 1
n
m
m
j=1
i =1
i= 1
 x j  a i ju i =  y iu i
6
Matrix Representation - (3)
6
m
n
m
i=1
j=1
i= 1
 u i  a ij x j =  y iu i
m
n


 u i   aij x j – yi =
i=1
j= 1
0
Because the ui are independent,
a 11 a 12  a 1n x 1
n
 a ij x j =
j= 1
a 21 a 22  a 2n x 2
yi
This is equivalent to
matrix multiplication.
a m1 a m2  a mn x n
y1
=
y2
ym
7
6
Summary
• A linear transformation can be represented by matrix
multiplication.
• To find the matrix which represents the transformation we
must transform each basis vector for the domain and then
expand the result in terms of the basis vectors of the range.
m
A v j 
=
 aij ui
i= 1
Each of these equations gives us
one column of the matrix.
8
6
Example - (1)
Stand a deck of playing cards on edge so that you are looking
at the deck sideways. Draw a vector x on the edge of the deck.
Now “skew” the deck by an angle q, as shown below, and note
the new vector y = A(x). What is the matrix of this transformation in terms of the standard basis set?
9
6
Example - (2)
To find the matrix we need to transform each of the basis vectors.
m
A v j 
=
 aij ui
i= 1
We will use the standard basis vectors for both
the domain and the range.
2
As j 
=
 a ij si =
a 1 js 1 + a 2 j s 2
i=1
1
Example - (3)
6
We begin with s1:
If we draw a line on the bottom card and then skew the
deck, the line will not change.
2
As 1 
= 1 s1 + 0 s2 =
 ai1 si
= a 11 s1 + a21 s 2
i=1
This gives us the first column of the matrix.
1
Example - (4)
6
Next, we skew s2:
2
A s 2 
= tan  q s1 + 1 s2 =
 ai2si
= a 12 s1 + a 22 s 2
i=1
This gives us the second column of the matrix.
1
6
Example - (5)
The matrix of the transformation is:
A = 1 tan  q
0
1
1
Change of Basis
6
Consider the linear transformation A:XY. Let {v1, v2, ..., vn} be
a basis for X, and let {u1, u2, ..., um} be a basis for Y.
n
=
 xi v i
y
=
i=1
 y iu i
i= 1
A x 
=
y
The matrix representation is:
a 11 a 12  a 1n x 1
a m1 a m2  a mn x n
y1
=
y2




a 21 a 22  a 2n x 2

x
m
ym
Ax = y
1
New Basis Sets
6
Now let’s consider different basis sets. Let {t1, t2, ..., tn} be a
basis for X, and let {w1, w2, ..., wm} be a basis for Y.
n
x
=
m
 x'i t i
y
i= 1
=
 y'iw i
i =1
The new matrix representation is:
a' 11 a' 12  a' 1n x' 1
a' m1 a' m2  a' mn x' n
=
y' 2





a' 21 a' 22  a' 2n x' 2
y' 1
y' m
A'x' = y'
1
How are A and A' related?
Expand ti in terms of the original basis vectors for X.
t 1i
ti
=
 t j iv j
ti =
j=1
t 2i

n
t ni
Expand wi in terms of the original basis vectors for Y.
w 1i
m
wi
=
 w j iu j
j= 1
wi =
w 2i

6
w mi
1
6
How are A and A' related?
Bt = t1 t2  tn
x = x'1 t1 + x'2 t2 +  + x' nt n = Btx'
Bw = w 1 w2  w m
y = Bwy'
Ax = y
ABt x' = Bwy'
–1
Bw ABt x' = y'
A' =  B–w1 ABt 
A'x' = y'
Similarity
Transform
1
Example - (1)
6
Take the skewing problem described previously, and find the
new matrix representation using the basis set {s1, s2}.
t1
= 0.5 s1 + s2
t2
= – s1 +
s2
0.5
t1 =
1
–1
t2 =
1
Bt = t t = 0.5 – 1
1 2
1
1
Bw = Bt = 0.5 –1
1
1
(Same basis for
domain and range.)
1
Example - (2)
6
–1
A' =  Bw AB t =
2  3 2  3 1 tan q 0.5 – 1
–2  3 1  3 0 1
1 1
A' =  2  3  tanq + 1
 2  3  tanq
– 2  3  tan q – 2  3  tan q + 1
For q = 45°:
A' =
53 23
–2  3 1  3
A= 11
01
1
Example - (3)
6
x = 0.5
Try a test vector:
1
y = Ax = 1 1 0.5 = 1.5
01
1
1
y' = A'x' =
x' = 1
0
5 3 2 3 1 = 5 3
–2  3 1  3 0
–2  3
Check using reciprocal basis vectors:
y' = B– 1 y = 0.5 – 1
1 1
–1
1.5 = 2  3 2  3 1.5 = 5  3
1
–2  3 1  3 1
–2  3
2
6
Eigenvalues and Eigenvectors
Let A:XX be a linear transformation. Those vectors
z  X, which are not equal to zero, and those scalars
l which satisfy
A(z) = l z
are called eigenvectors and eigenvalues, respectively.
Can you find an eigenvector
for this transformation?
2
Computing the Eigenvalues
6
Az = l z
A – l I z = 0
 A – l I = 0
Skewing example (45°):
A= 11
01
1 –l 1 z = 0
0 1 –l
0
1 –l 1
0 1 –l
= 0
2
1 – l = 0
0 1 z = 0 1 z 11 = 0
1
00
0 0 z 21
0
l1 = 1
l2 = 1
z 21 = 0
z1 = 1
0
For this transformation there is only one eigenvector.
2
Diagonalization
Perform a change of basis (similarity transformation) using
the eigenvectors as the basis vectors. If the eigenvalues are
distinct, the new matrix will be diagonal.
{z 1  z 2  , z n}
B = z1 z 2  z n
{l1  l 2  , ln}
Eigenvectors
Eigenvalues
l1 0  0
0 l2  0

–1
 B AB  =


6
0 0  ln
2
Example
6
A= 11
11
1 –l 1
1 1 –l
l1 = 0
l2 = 2
= 0
l1 = 0
2
l – 2l =  l  l – 2 = 0
l2 = 2
1 1 z = 1 1 z 11 = 0
1
11
1 1 z 21
0
z 21 = –z 11
– 1 1 z = – 1 1 z 12 = 0
1
1 –1
1 – 1 z 22
0
Diagonal Form:
1 –l 1 z = 0
1 1 –l
0
z 22 = z 12
z1 =
1
–1
z2 = 1
1
–1
A' = B AB = 1  2 – 1  2 1 1 1 1 = 0 0
12 12
1 1 –1 1
02
2