Matrix Multiplication
A2 x 4
 1  2 0 3


  1 2 3 0
B4 x 2 
 1

 2
 0

 2
3

4 
1

3
C2 x 2
?
 (_ _1 ) (_4_ )


 (_ _5 ) (_17_ )
( 1 )( 1 )  (  2 )(  2 )  ( 0 )( 0 )  ( 3 )(  2 )   1
(1) ( 3 )  (  2 ) ( 4 )  ( 0 ) (1)  ( 3 ) ( 3 )  ?
(  1)( 1)  ( 2 )(  2 )  ( 3 )( 0 )  ( 0 )(  2 )  ?
(  1 )( 3 )  ( 2 )( 4 )  ( 3 )( 1 )  ( 3 )( 3 )  ?
Rank and Dependency
Columns of a matrix:
 C1  2 C2  C3
 Matrix A:

1 2 3  1 0123 







C C 
C


1
3
1
0
3
2

?




1
1


A


3
1
2








2


1
0



4


2
1



4

1


1
2
Linear Combinations
( s c a la r s   
1
,

, C
2

, ... , C
, ... ,
C 1
Columns:
2
 Linear Combination:
 1 C 1   2 C 2  ...  k C k

)
k
k
“Nontrivial” if at least one  is not 0
** Columns are dependent if
there is a nontrivial linear combination for which:
 1 C 1   2 C 2  ...  k C k  
Rank
The RANK of a matrix is the maximum
number of linearly independent columns
that can be selected from the columns of the
matrix.
 Rank of X is same as rank of X’X.
 A matrix is invertible only if



It is square and
It is of FULL RANK
Example - Look for Dependency
11 11 99 11 00 
      
   
22  33  88 77 045

(2) (?)
(2)   (?)
(1) ((?)
9)    
(?)
99
11
11
11
00
          
 11
 11
44
33  019
          
DEPENDENT?
MAYBE,
MAYBE NOT!
Same Example (con.t)
11 1
 9
11 00
   
 
   
22  33 
8
77 00
(?)
(2) (?)
(1)   ((?)
0)   ((?)
1)  
11
1
11 00
9
   
 
   
11  11 
 4
33 00
   
 
   
DEPENDENT?
YES !!!!
*
*
*
*
*
* *
Y
S ri2
*
*
* *
* *
*
*
*
* *
*
*
*
* *
*
* *
* *
*
* *
*
*
X
*
*
Class Variables
• Data Set
 Y1 
 14 
 
 
Y
 2
 16 
 Y3 
 20
 
 
 Y4    22
Y 
 25
 
 5
 28
 Y6 
 
 
 31
 Y7 
Y
drug
14
A
16
A
20
A
22
A
25
B
28
B
31
B






















0
 1
 1

 
 
0
1
 
 1
 1
 1
0

 
 
0    1   1 
 1
 0
1

 
 
1
 1
 0

 
 
1
 1
 0
X
MATRIX
0d
1A A
e
0
1A A
p
0e
1A A
0n
1A A
0B B1d
e
B
1
0B
n
0B B1t
1
1
1
1
1
1
1











A
A
A
A
A
B
B
B
B
B
C
C
C
C
C
D
D
D
D
D
X
































1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
A
1
A
1
A
0
A
0
A0
A0
A
0
A
0
A
1
A
1
A
1
B0
A
0
0
A
0
A
B1
B0
B0
B0
B0
B1
B1
B1
B1
A0
A0
A0
B0
B0
A
0
A
0
A
0
B0
B0
C0
C0
C0
C0
C0
C0
C0
C0
C0
C0
B0
B0
B0
C1
B0
B0
B0
C0
D0
D0
D0
D0
D0
D0
D0
D0
D0
D0
C1
C1
C1
C1
D0
C0
C0
C0
C0
D1
D0
D0
D0
D0
D1
D1
D1
D1






























































Y
Y
Y
Y
Y
Y
Y
Y
Y
Y
Y
Y
Y
Y
Y
Y
Y
Y
Y
Y































X 
































1
1
0
0
0
1
1
0
0
0
1
1
0
0
0
1
1
0
0
0
1
1
0
0
0
1
0
1
0
0
1
0
1
0
0
1
0
1
0
0
1
0
1
0
0
1
0
1
0
0
1
0
0
1
0
1
0
0
1
0
1
0
0
1
0
1
0
0
1
0
1
0
0
1
0
1
0
0
0
1
1
0
0
0
1
1
0
0
0
1
1
0
0
0
1
1
0
0
0
1































 0
 

 11 
 22 
  
33
 
 44 
 
Y  X e































Y
Y
Y
Y
Y
Y
Y
Y
Y































X 
Y
Y
Y
Y
Y
Y
Y
Y
Y
Y
Y































1
1
0
0
0
1
1
0
0
0
1
1
0
0
0
1
1
0
0
0
1
1
0
0
0
1
0
1
0
0
1
0
1
0
0
1
0
1
0
0
1
0
1
0
0
1
0
1
0
0
1
0
0
1
0
1
0
0
1
0
1
0
0
1
0
1
0
0
1
0
1
0
0
1
0
1
0
0
0
1
1
0
0
0
1
1
0
0
0
1
1
0
0
0
1
1
0
0
0
1
Y  X e








































  
04
1  4
1
2  4
2
3  4
3








Treatment 1:
Y1 j  01  11 X 1 j  e j
Treatment 2:
Y2 j  02  12 X 2 j  e j
Treatment 3:
Y3 j  03  13 X 3 j  e j
A 2 X20Y
A 5 24
A 8 22
A 11 28
A 14 26
A 17 25
B 2X 15
Y
B 6 19
B 7 20
B 12 26
B 14 30
B 17 28
C 1X Y
2
C 5 8
C 9 12
C 11 10
C 14 18
C 15 20
30
25
20
15
10
5
0
0
5
10
15
20
A 2 20
A 5 24
A 8 22
A 11 28
A 14 26
A 17 25
B 2 15
B 6 19
B 7 20
B 12 26
B 14 30
B 17 28
C 1 2
C 5 8
C 9 12
C 11 10
C 14 18
C 15 20
30
25
20
15
X=
10
5
0
0























1
1A
0 0
1
1A
0 0
1
1A
0 0
1
1A
0 0
1
1A
0 0
1
B0
1 0
1
B0
1 0
1
B0
1 0
1
B0
1 0
1
B0
1 0
1
C
0
0 1
C
0
0101
1
C
0
0 1
1
C
0
0 1
1
C
0
0 1
1
5
2 
5
8
11 

14 
17 
2  <== One column
6
One slope

7
12 
14 
17 
1
5  15
20

9

11 
14 
15 
A 2
A 5
A 8
A 11
A 14
A 17
B 2
B 6
B 7
B 12
B 14
B 17
C 1
C 5
C 9
C 11
C 14
C 15
20
24
22
28
26
25
15
19
20
26
30
28
2
8
12
10
18
20
X =
1 1 0
 30
1 1 0
1 1 0
1251 0
1 1 0
 20
1 1 0
1 0 1
1150 1
1 0 1
 10
1 0 1
1 0 1
1 50 1
1 0 0
 0
1 0 00
1 0 0
1 0 0

1 0 0
1 0 0
0
0
0
0
0
0
0
0
0
0
0
0
1
1
1
1
1
1
2 2 0
5 5 0
8 8 0
11 11 0
14 14 0
17 17 0
2 0 2
6 0 6
7 0 7
12 0 12
14 0 14
17 0 17
1 0 0
5 0 0
5
10
0
0
9
11 0 0
14 0 0
15 0 0
0
0
0
0
0
0
0
0
0
0
0
0
1
5
9
11
14
15

03 



  01  03 

  02  03 


<== Add


13
 TRT*X
Interaction
   

11
13



  1 2  1 3 




 15
20





PROC GLM;
CLASS TRT;
MODEL Y = TRT X TRT*X;
• F test to delete TRT*X
• What is being tested?
• Key =>
• What is REDUCED MODEL?
• Y = TRT X ==> Single slope
• Testing ……
• H0: Arbitrary lines
• H1: Parallel lines
PROC GLM;
CLASS TRT;
MODEL Y = TRT TRT*X;
1

1
1

1
1

1
1

1

1
1

1

1
1 0 0 X
0
1 0 0 X
0
1 0 0 X
0
1 0 0 X
0
0 1 0
0
X
0 1 0
0
X
0 1 0
0
X
0 1 0
0
X
0 0 1
0
0
0 0 1
0
0
0 0 1
0
0
0 0 1
0
0
0

0
0

0
0

0
0

0

X
X

X

X
• One dependency in TRT
• NO dependencies in
TRT*X
• F for TRT*X
• What is it testing
NOW?
• Reduced model = ?
• MODEL Y=TRT
• All slopes 0
•
H0: All slopes 0
• (not just equal)
• H1: Arbitrary slopes
PROC GLM;
CLASS TRT;
MODEL Y = TRT TRT*X;
1

1
1

1
1

1
1

1

1
1

1

1
1 0 0 X
0
1 0 0 X
0
1 0 0 X
0
1 0 0 X
0
0 1 0
0
X
0 1 0
0
X
0 1 0
0
X
0 1 0
0
X
0 0 1
0
0
0 0 1
0
0
0 0 1
0
0
0 0 1
0
0
0

0
0

0
0

0
0

0

X
X

X

X
• One dependency in TRT
03







03 
 01
 02  03 





11


1 2


1 3


?
Covariance adjusted means
30
Parameter
Estimate
Estimate
INTERCEPT 4.0123
TRT A
12.2217
B
10.9158
C
0.0000
X
0.8350
B
B
B
B
25
• <=
• <=
20
15
• <=
10
5
NOTE: The X'X matrix
has been found to be
singular ...
0
0
5
10
15
20
X  9 .44
grade
^
Y  b0  b1 ( IQ )  b2 ( study )
b1
b
2
b0
study
IQ
^
Y  b0  b1 ( IQ )  b2 ( study )  b3 ( IQ * study )
grade
“Interaction”!
IQ
study
^
G  72.21 4.11(study)  0.13( IQ)  0.053( IQ * study)
110
100  0.053(____*
100
110 study)
G  72.21 4.11(study)  0.13(____)
^
-14.300
-13.00
+5.30
+5.83
IQ=100
IQ=110
^
G  59.21 1.19(study)
^
G  57.91 1.72(study)
IQ=100
IQ=110
at IQ  100
at IQ  110