AM 121: Intro to Optimization
Models and Methods
Lecture 2: Intro to LP, Linear algebra review.
Yiling Chen
SEAS
TexPoint fonts used in EMF.
Announcement
•  Andrew will teach a section on AMPL and
CPLEX installation on Friday at 3pm-4pm at
MD 119
2
Lecture 2: Lesson Plan
• 
• 
• 
• 
What is an LP?
LP in matrix form. Matrix review.
Graphical and algebraic correspondence
Problems in canonical form
Jensen & Bard: 2.1-2.3, 2.5, 3.1 (can ignore the two
definitions for now),3.2
Recommended text is available at Gordon McKay
library (3rd floor of Piece Hall).
3
Linear Programming
•  Maximizing (or minimizing) a linear function
subject to a finite number of linear constraints
n
j=1
4
Terminology
•  Decision variables:
•  Parameters:
–  Fixed, known
•  Objective function
•  Constraints
5
A Little History
•  The field of linear programming started in 1947
when George Dantzig (1914-2005) designed the
“simplex method” for solving LP formulations of
U.S. Air Force planning problems
•  Dantzig was deciding how to use the limited
resources of the Air Force
•  planning == programming
–  “program” was a military term that referred to plans or
proposed schedules for training, logistical supply, or
deployment of combat units.
•  Some later came to call this “Dantzig’s great
mistake”
6
LP in Matrix Form
7
Review: Matrices (1/3)
•  Matrix: rectangular array of numbers
–  Dimension:
• 
• 
: column vector
: row vector
, scalar
• 
• 
“inner product”
8
Review: Matrices (2/3)
• 
transpose:
•  Partitions
(“usual rules of matrix algebra”)
9
Review: Matrices (3/3)
•  Square matrix:
•  Identity matrix: square matrix w/ diagonal elements
all 1 and all non-diagonal are 0.
•  For square A, Inverse:
Write
•  Given
Finds unique solution to a square linear system.
10
Terminology for Solutions of LP
•  A feasible solution
–  A solution that satisfies all constraints
•  An infeasible solution
–  A solution that violates at least one constraint
•  Feasible region
–  The region of all feasible solutions
•  An optimal solution
–  A feasible solution that has the most favorable
value of the objective function
11
Example: Marketing Campaign
•  Comedy– 7 million high-income women, 2
million high-income men. Cost $50,000
•  Football – 2 million high-income women and
12 million high-income men. Cost $100,000
•  Goal: reach at least 28 million high-income
women and 24 million high-income men at
MINIMAL cost
12
graphical version of problem
(solution is x1=3.6, x2=1.4, value 320)
How would you establish the
optimality of this algebraically?
13
Example: Alt. Optimal Solutions
Note that there ARE still
extremal optimal solutions
14
Example: Unbounded Objective
How would you show this algebraically?
15
Example: Infeasible Problem
•  ..\..\Desktop\17.bmp
How would you show this algebraically?
16
Thinking Computationally
•  canonical form
•  basic feasible solutions
•  solution improvement
17
Putting in Canonical Form
1.  All decision variables are non-negative
2.  All constraints are equalities
3.  The RHS coefficients are all non-negative
4.  One decision variable is “isolated” in each constraint with a
+1 coefficient. These variables do not appear in any other
constraint and have a zero coefficient in objective function
Why might this be useful??
18
Basic Feasible Solution
Have an associated “basic feasible solution” in which the
isolated decision variables are non-zero and the rest are zero.
Here, set
Clearly optimal in this example as well. (Why?)
Optimality criterion: “if every non-basic variable has a non19
positive coefficient in the objective function”
Solution Improvement
Current basic feasible solution: x1 = 6, x2=4, x3=0, x4=0.
Now, increase x4 and decrease x2 (keep x3=0) until second
constraint binds. Obtain new solution: x1=3, x2=0, x3=0, x4=1.
Can transform into a new canonical form in which the isolated
variables are x1 and x4. (“pivot on x4 in the second constraint”).
! “pick something to come in, something forced to leave”
20
New Canonical Form
After linear transformations:
Basic feasible solution x1=3, x2=0, x3=0, x4=1 is
optimal
21
Geometric Interpretation of Solution
Improvement
x1 = 6, x2=4, x3=0, x4=0
x1=3, x2=0, x3=0, x4=1
22
Can anything be put in canonical
form?
(a) maximization, (b) positive RHS, (c) equality
constraints, (d) non-negative variables, (e) isolated
variables
23
Reduction to canonical form (I)
• 
•  if a RHS is negative then multiply the
constraint by -1
24
Reduction to canonical form (II)
•  Inequality constraints
slack variable
surplus variable
25
Reduction to canonical form (III)
•  Free variables (i.e. without non-negativity
constraints)
•  y = u – v
•  u ! 0
•  v ! 0
•  whenever y ! 0, u=y and v = 0
•  whenever y < 0, u = 0, v=-y
•  Replace y with (u-v) wherever it appears
26
•  Make sure one variable is “isolated” in each
constraint with a +1 coefficient
•  Some will be already OK. E.g.,
•  But, some not OK. E.g.,
isolated and can function as
an initial basic variable
doesn’t work (coeff -1)
•  Solution: introduce a new artificial variable (make
sure x6=0 in final solution)
27
Next Time
•  Applications, Examples, Exercises.
28