Design and Analysis
of Algorithms
Lecture 7
Yoram Moses
April 22, 2010
http://www.ee.technion.ac.il/courses/046002
1
Linear Programming
2
Standard Form



Maximization
All constraints are ≤ inequalities
All variables are non-negative
Objective function
Constraints
Non-negativity constraints
3
Slack Form







Maximization
n+m variables and m constraints
n “regular variables” and m “slack variables”
All variables are non-negative
All constraints, except for non-negativity, are equalities
Slack variables appear only on LHS of equality constraints
Each equality constraint has a single unique slack variable
4
Conversion to Slack Form:
Example
Standard LP
Slack LP
5
Basic vs. Non-basic Variables



Slack variables  basic variables
Regular variables  non-basic variables
B = set of basic variables, N = set of non-basic variables



Ex: B = { 4,5,6 }, N = { 1,2,3 }
|B| = m, |N| = n
For a solution x, xB = basic part of x, xN = non-basic part of x
6
Short Matrix Form



|B| = m, |N| = n
Constraints can be indexed by members of B
For a solution x,
= basic part of x (i.e., xi for all i  B)
 xN = non-basic part of x (i.e., xj for all j  N)
 xB
7
Basic Solutions

Every slack form is associated with a basic
solution:
 All
non-basics are set to 0 (i.e., xN = 0)
 All basics are set to corresponding free coefficients
(i.e., xB = b)
 Ex: x1 = 0, x2 = 0, x3 = 0, x4 = 30, x5 = 24, x6 = 36
8
Tight Constraints & Constraint Violation

Tight constraint: One in which the basic
variable is forced to 0 in the basic solution
 The

corresponding free coefficient is 0.
Violated constraint: One in which the basic
variable is forced to be negative in the
basic solution.

the corresponding free coefficient is negative.
9
Basic Feasible Solutions
Basic Feasible Solution (BFS): A basic
solution, which is feasible
 Easy fact: A basic solution is feasible if
and only if all free coefficients are nonnegative.
 Lemma: Every BFS corresponds to a
vertex of the feasible region polytope.

10
Simplex Algorithm: Overview
Works in iterations
 At each iteration: transform one slack form
P into an equivalent slack form P’
 Objective value of basic solution of P’ is
always at least as good as that of P
 Stop when reaching a local optimum

11
Moving to an Equivalent Slack
Form

How to increase the objective value of the BFS?


Increase from 0 the value of some non-basic variable, whose
coefficient in the objective function is positive.
By how much?

As much as possible without violating any of the constraints.
objective: 0


Can increase x1 by at most 9.
Objective increases to 27.
12
Switching Basic with Non-basic

Suppose that we increase a non-basic variable xi until
some constraint j becomes tight






xj, the basic variable of the constraint j, becomes 0
We can thus switch between xi and xj
xi will become the basic variable of constraint j
xj will become a non-basic
When increasing x1 to 9, x6 becomes 0.
We switch between x1 and x6.
13
Switching Basic with Non-basic

We write x1 as a function of other non-basics and x6:
14
Switching Basic with Non-basic

We rewrite the objective function:
15
Switching Basic with Non-basic

We rewrite all the constraints as well, and obtain the
following equivalent linear program:
16
Switching Basic with Non-basic

The new LP is equivalent to the previous LP



We just rewrote x1 in terms of other variables
Basic solution: x1 = 9, x2 = 0, x3 = 0, x4 = 21, x5 = 6, x6 = 0
New objective value: 27
17
Example continued
objective:
27


Choose x3
First constraint to become tight is constraint 3.
objective:
27.75
18
Example continued
objective:
27.75


Choose x2
First constraint to become tight is constraint 2.
objective:
28
19
Example continued
objective:
28



No more non-basics whose coefficient in the objective
function is positive
We stop and output basic solution as the optimal
solution
Solution: x1 = 8, x2 = 4, x3 = 0, x4 = 18, x5 = 0, x6 = 0.
Value: 28.
20
Back to the Standard Form
objective:
28

Solution: x1 = 8, x2 = 4, x3 = 0. Value: 28.
21
Pivoting

Pivot: a single iteration of the simplex algorithm
1. Choose a non-basic variable xi whose coefficient in
the objective function is > 0
 xi is called the “entering variable”
 If more than one exists, choose one according to some
pivoting rule
2. Find the first constraint j that will be violated when we
increase the value of xi from 0
 If more than one exists, choose one according to some
pivoting rule
3. Make xi the basic variable of constraint j, and make xj
a non-basic variable
 xj is called the “leaving variable”
4. Write xi as a function of xj and the other non-basics
5. Rewrite the objective function and the constraints
22
Pivoting: Geometric Intuition
Lemma: Pivoting corresponds to moving
from one vertex of the feasible region to a
neighbor vertex, whose objective value is
at least as good.
23
Unbounded Programs

Sometimes it is possible to increase the value of the
entering value unboundedly, without violating any
constraint

In this case the optimal solution of the LP is unbounded
Pivot will return “unbounded”

24
The Simplex Algorithm
1.
2.
3.
4.
5.


find an initial BFS
while there is a non-basic variable whose coefficient in
the objective function is > 0
run pivot
if pivot returns “unbounded” return “unbounded”
return BFS of current slack form as the optimal solution
Geometric view: Repeatedly move from a vertex of the
feasible region to a better neighbor vertex, until a local
maximum is reached.
Initial BFS is found by solving an auxiliary linear
program (read section in book)
25
Simplex Analysis: Correctness



If LP is infeasible, S’x will fail to find an initial BFS
If LP is unbounded, Pivot will return “unbounded”
If LP has a bounded optimal solution, it has one at
a vertex
 Simplex
will reach a local maximum vertex
 Local maximum vertex must be a global maximum
 Hence, Simplex will output an optimal solution
26
Simplex Analysis: Running Time

We have not specified the two “pivoting
rules”

For choosing the entering variable
 For choosing the leaving variable
Degeneracy: objective value of BFS does
not improve in an invocation of Pivot
 Unwise pivoting rules may lead to infinite
loops (i.e., everlasting degeneracy)

27
Degeneracy
objective: 0
entering: x1
leaving: x4
objective: 8
entering: x3
leaving: x5
objective: 8
entering: x2
leaving: x1
28
Pivoting Rules





Bland’s rule: choose entering/leaving variable
with smallest index.
Lemma: If Simplex uses Bland’s rule, it never
cycles.
Conclusion: Simplex has at most
iterations.
Theorem [Kalai]: There is a randomized pivoting
rule, with which Simplex runs for a subexponential number of iterations in expectation.
Open problem: Is there a pivoting rule with which
Simplex runs in polynomial time?
29
End of Lecture 8
30