Backgrounds-Convexity
 Def: line segment joining two points x1 , x 2  R n is the collection
of points x  x1  (1   ) x 2 , 0    1
( x  1 x1  2 x 2 , 1  2  1, 1 , 2  0)
(Generally, x  im1 i x i ,
im1 i  1, i  0 i, called convex combination)
x2
x 2   ( x1  x 2 )
x2
x1
x1
( x1  x 2 )
OR-1 2007
1
 Def: C  R n is called convex set iff x1  (1   ) x 2  C
whenever x1  C , x 2  C , and 0    1.
Convex sets
OR-1 2007
Nonconvex set
2
C  R n be a convex set and for k  R , define
kC  { y  R n | y  kx for some x  C}
Then kC is a convex set.
Proposition: Let
Pf) If k = 0, kC is convex. Suppose k  0.
For any x, y kC,  x' , y' C such that x'  (1 / k ) x, y'  (1 / k ) y
Then x  (1   ) y  kx'(1   )ky'  k (x'(1   ) y' )  kC

Hence the property of convexity of a set is preserved under
scalar multiplication
Consider other operations.
OR-1 2007
3
Convex function
 Def: Function f : R n  R is called a convex function iff
f (x1  (1   ) x 2 )  f ( x1 )  (1   ) f ( x 2 ), 0    1,
Also called strictly convex function iff
f (x1  (1   ) x 2 )  f ( x1 )  (1   ) f ( x 2 ), 0    1,
f (x)
( x1, f ( x1))  R n 1
( x 2 , f ( x 2 ))
f ( x1)  (1   ) f ( x 2 )
f (x1  (1   ) x 2 )
1
x
OR-1 2007
x
x1  (1   ) x 2
x2
4
 Equivalent definition: f is called convex if the points above or on
the graph of f(x) (epigraph of f) is a convex set
 Def: f is a concave function if –f is a convex function.
 Def: xC is an extreme point of a convex set C if x cannot be
expressed as y  (1   ) z, 0    1 for distinct y, z C ( x  y, z )
: extreme points
OR-1 2007
5
Review-Linear Algebra
2 x1  x2  5x3  x4  20
x1  5x2  4 x3  5x4  30
3x1  x2  6 x3  2 x4  20
2  1 5 1 
A  1 5
4 5


3 1  6 2

OR-1 2007
 x1 
x 
x   2
 x3 
 
 x4 
20
b  30 
 
20
Ax  b in matrix, vector notation
6
 Submatrices multiplication
 a11 a12
A  a21 a22

 a31 a32
a13
a23
a33
a14 
a
a24    11
  A21
a34 
A12 
A22 
 b1 
b 
2   b1 

B

b3   B2 
 
b4 
 a11b1  A12 B2 
AB  

 A21b1  A22 B2 
OR-1 2007
7
 자주 사용 하게 될 submatrices multiplication
 a11 a12
a
a22
A   21
 

am1 am 2
Ax  b   A1
y' A  y'A1
y ' A   y1
OR-1 2007
a1n 
 a1' 
 a '
 a2n 
   A1 A2  An    2 

 
  

 
 amn 
am '
 x1 
x 
A2  An  2   A1x1  A2 x2    An xn  b
 x3 
 
 x4 

A2  An    y' A1
y' A2  y' An 
 a1' 
 a '
y2  ym  2    y1a1' y2a2 '   ym am '
  
 
am '
8
B 1 Ax  B 1b
B 1 A  B 1  A1
Suppose A  B

A2  An   B 1 A1
B 1 A2  B 1 An
N  where B : m  m, (nonsingul ar) N : m  (n  m)

 
Then B 1 A  B 1 B N   B 1B B 1 N  I
OR-1 2007

B 1 N

9
 Def: {x1 , x 2 ,, x m } is said to be linearly dependent if  c1 , c2 ,, cm
1
2
m
, not all equal to 0, such that c1 x  c2 x    cm x  0
( i.e., there exists a vector in {x1 , x 2 ,, x m } which can be expressed
as a linear combination of the other vectors. )
 Def: {x1 , x 2 ,, x m } linearly independent if not linearly dependent.
In other words, im1 ci x i  0 implies ci  0 for all i
(i.e., none of the vectors in {x1 , x 2 ,, x m } can be expressed as a
linear combination of the remaining vectors.)
 Def: Rank of a set of vectors : maximum number of linearly
independent vectors in the set.
 Def: Basis for a set of vectors : collection of linearly independent
vectors from the set such that every vector in the set can be
expressed as a linear combination of them. (maximal linearly
independent subset, minimal generator of the set)
OR-1 2007
10
 Thm) r linearly independent vectors form a basis iff the set
has rank r.
 Def: row rank of a matrix : rank of its set of row vectors
column rank of a matrix : rank of its set of column vectors
 Thm) row rank = column rank
 Def : nonsingular matrix : rank = number of rows = number of
columns.
Otherwise, called singular
 Thm) If A is nonsingular, then unique inverse exists.
( AA1  I  A1 A)
OR-1 2007
11
Simutaneous Linear Equations
Thm: Ax = b has at least one solution iff rank(A) = rank( [A, b] )
Pf) ) rank( [A, b] )  rank(A). Suppose rank( [A, b] ) > rank(A).
Then b is lin. ind. of the column vectors of A. Hence b
cannot be expressed as a linear combination of the
columns of A, i.e. Ax = b does not have a solution.
 ) There exists a basis in columns of A which generates b.
So Ax = b has a solution.

Suppose rank(A) = rank [A, b] = r.
Then Ax = b has a unique solution if r = n.
has infinitely many solutions if r < n. (m-r equations are
redundant)
OR-1 2007
12
 Operations that do not change the solution set of the linear
equations
(Elementary row operations)
Change the position of the equations
Multiply a nonzero scalar k to both sides of an equation
Multiply a scalar k to an equation and add it to another
equation
X  {x | a1 ' x  b1 , a2 ' x  b2 ,, am ' x  bm }
Y  {x | (a1 'ka2 ' ) x  (b1  kb2 ), a2 ' x  b2 ,, am ' x  bm }
Show that x *  X implies x *  Y ( X  Y )
and x *  Y implies x *  X (Y  X )
Hence X = Y
OR-1 2007
13
Solving systems of linear equations (Gauss-Jordan Elimination, 변
수의 치환) (simplex method 에서 이용)
x1
 x2
 4 x3
 x1  3 x2
2 x2
 10
 10
 5 x3
x2
 22


x1
 6 x3
 20
 2 x3
 10
x3

2

x1  x2
2 x2
 4 x3
 10
 4 x3
 20
2 x2
 5 x3
 22
x1  x2
x2
 4 x3
 10
 2 x3
 10
2 x2
 5 x3
 22
OR-1 2007
 8
x1
 6
x2
x3
 2
14
 무한히 많은 해를 갖는 경우
 x2
x1
 4 x3
 x1  3 x2
2 x2
 5 x3
 x4
 10
 x4
 10
 x4
 22
 x2
 4 x3
 x4
 10
 2 x2
 4 x3
 2 x4
 20
2 x2
 5 x3
 x4
 22
x1  x2
x2
 4 x3
 x4
 10
 2 x3
 x4
 10
2 x2
 5 x3
 x4
 22
x1
x1
x2
 6 x3
 2 x4
 20
 2 x3
 x4
 10
 x3
 x4

x1
x2
 x3
 8 x4
 8
 3 x4
 6
 x4
 2
2
 8  8 x4
x1
 6  3 x4
x2
 x3
 2
 x4
Assign x4  t for arbitrary t and get
x1  8  8t , x2  6  3t , x3  2  t
OR-1 2007
15
 8  8 x4
x1
 6  3 x4
x2
 x3
 2
 x4
x4 is independent variable and x1 , x2 , x3 are dependent variables.
Particularly, the solution obtained by setting the indepent variables to 0
and solving for the dependent variables is called a basic solution.
Here x1  8, x2  6, x3  2, x4  0
(will be used in the simplex method)
OR-1 2007
16
x1
x2
x1
x2
 x3
 8 x4
 8
 3 x4
 6
 x4
 2
 6 x3
 2 x4
 20
 2 x3
 x4
 10
 x3
 x4

x1  8  8x4 , x2  6  3x4 , x3  2  x4
2
x1
x2
 8 x3

24
 3 x3

12
 x3
 x4
 2
x1  24  8x3, x2  12  3x3, x4  2  x3
Both systems have the same set of solutions, but representation is different
e.g.) x1  8, x2  6, x3  2, x4  0
OR-1 2007
17
 Elementary row operations are equivalent to premultiplying a
nonsingular square matrix to both sides of the equations
x1
 x2
 4 x3
 x1  3 x2
2 x2
 10
 10
 5 x3
 22
x1  x2
2 x2
 4 x3
 10
 4 x3
 20
2 x2
 5 x3
 22
 1  1 4  x1  10
 1 3 0  x   10

 2   
2 5  x3  22
 0
1
  1  1 4  x1   1
 10
 1 1   1 3 0  x    1 1  10


 2  
 
1  0
2 5  x3  
1 22

 1  1 4  x1  10
 0
2 4  x2   20

   
2 5  x3  22
0
OR-1 2007
18
x1  x2
2 x2
 4 x3
 10
 4 x3
 20
2 x2
 5 x3
 22
x1  x2
x2
 4 x3
 10
 2 x3
 10
2 x2
 5 x3
 22
1
  1  1 4  x1   1
 10
 1 1   1 3 0  x    1 1  10


 2  
 
1  0
2 5  x3  
1 22

1
 1
  1  1 4  x1   1
 1
 10
 1 / 2   1 1   1 3 0  x    1 / 2   1 1  10



 2  

 
1 
1  0
2 5  x3  
1 
1 22

OR-1 2007
19
1
 1
  1  1 4  x1   1
 1
 10
 1 / 2   1 1   1 3 0  x    1 / 2   1 1  10



 2  

 
1 
1  0
2 5  x3  
1 
1 22

1
  1  1 4  x1   1
 10
  1 / 2  0
2 4  x2    1 / 2  20


  
 
1 0
2 5  x3  
1 22

 1
  1  1 4  x1   1
 10
 1 / 2 1 / 2   1 3 0  x2   1 / 2 1 / 2  10


  
 
1  0
2 5  x3  
1 22

OR-1 2007
20
 So if we multiply all elementary row operation matrices, we
get the matrix having the information about the elementary
row operations we performed
x1
 x2
 4 x3
 x1  3 x2
2 x2
 10
 10
 5 x3
 22

 8
x1
 6
x2
x3
 2
7.5 6.5  6  1  1 4  x1  7.5 6.5  6 10
2.5 2.5  2  1 3 0  x   2.5 2.5  2 10


 2  
 
1  0
2 5  x3    1  1
1 22
  1  1
 1 0 0  x1   8
 0 1 0  x2   6

   
0 0 1  x3  2
OR-1 2007
7.5 6.5  6
A1  2.5 2.5  2


1
  1  1
21
 Finding inverse of a nonsingular matrix A.
Perform elementary row operations (premultiply elementary
row operation matrices) to make [A : I ] to [ I : B ]
Let the product of the elementary row operations matrices be
C.
Then C [ A : I ] = [ CA : C ] = [ I : B]
Hence CA = I  C = A-1 and C = B
OR-1 2007
22