Chapter 5. The Duality Theorem
 Given a LP, we define another LP derived from the same data, but
with different structure. This LP is called the dual problem (쌍대문제).
 The main purpose to consider dual is to obtain an upper bound
(estimate) on the optimal objective value of the given LP without
solving it to optimality. Also dual problem provides optimality
conditions of a solution x* for a LP and help to understand the
behavior of the simplex method.
Very important concept to understand the properties of the LP and the
simplex method.
OR-1 2009
1
 Approach to find optimal value of an opt. problem (max form):
Find lower bound z and upper bound z so that for optimal
value z *, we have z  z *  z
Lower bound : usually obtained from a feasible solution.
If x* is a feasible solution to LP, c’x* provides a lower bound.
Upper bound : usually obtained by solving a relaxation of the problem or
by finding a feasible solution to a dual problem.
ex) linear programming relaxation of an integer program:
Given IP, max c’x, Ax  b, x  0 and integer, consider the linear
programming relaxation, max c’x, Ax  b, x  0.
Let z* be the optimal value of IP and z’ be the optimal value of LP relaxation.
Then z*  z’. ( Let x* be the optimal solution of IP. Then x* is a feasible
solution of LP relaxation. So there may exist a feasible solution to LP which
provides better objective value)
OR-1 2009
2
(continued)
Dual of the LP problem : Let y be a feasible solution to the dual problem.
Then the dual objective value provided by y gives an upper bound on the
optimal LP objective value. (study in Chapter 5.)
 If the lower and upper bound are the same, we know that z* is optimal
value. We may need to find the optimal solution additionally, but the
optimal value is found.
 Although the lower and upper bound may not be the same, from the gap
(= upper bound – lower bound), we can estimate the quality of the
solution we have.
OR-1 2009
3
 Taking nonnegative linear combination of inequality constraints:
Consider two constraints x1  2 x2  3 and 2 x1  x2  4 …. (1’)
In vector notation :
 1
 2
a1 ' x  3, a 2 ' x  4, where a1   , a 2   
 2
 1
If we multiply scalars y1  0 to the 1st constraint and y2  0 to the 2nd
constraint and add the l.h.s. and r.h.s. respectively,
we get ( y1  2 y 2 ) x1  (2 y1  y 2 ) x2  (3 y1  4 y 2 ) ….. (2’)
In vector notation, ( y1a1 ' y2 a 2 ' ) x  (3 y1  4 y2 )
 Any vector that satisfies (1’) also satisfies (2’), but converse is not true.
Moreover, the coefficient vector in l.h.s. of (2’) is obtained by taking the
nonnegative linear combination of the coefficient vectors in (1’)
OR-1 2009
4

x2
2x1+x24
y1(1, 2)+y2(2, 1), y1, y2  0
x1+2x23
(1, 2)
(2, 1)
(5/3, 2/3)
(0, 0)
OR-1 2009
(y1a1’+y2a2’)x  (3y1+4y2)
x1
5
 ex)
max
4 x1
 x2
 5 x3
 3 x4
s.t.
x1
 x2
 x3
 3 x4

5 x1
 x2
 3 x3
 8 x4
 55
 x1  2 x2  3 x3
x1 , x2 , x3 , x4  0
 5 x4

1
3
Lower bound : consider feasible solution (0, 0, 1, 0)  z*  5
(3, 0, 2, 0)  z*  22
Upper bound : consider inequality obtained by multiplying 0 to the
1st , 1 to the 2nd, and 1 to the 3rd constraints and add the l.h.s. and
r.h.s. respectively
…… (1)
4 x  3x  6 x  3x  58
1
OR-1 2009
2
3
4
6
4 x1  3x2  6 x3  3x4  58 ....... (1)
Since we multiplied nonnegative numbers, any vector that satisfies the
three constraints (including feasible solutions to the LP) also satisfies
(1). Hence any feasible solution to LP, which satisfies the three
constraints and nonnegativity, also satisfies (1).
 Note that all the points satisfying 4x1 + 3x2 + 6x3 + 3x4  58 do not
necessarily satisfy the three constraints in LP.
 Further, any feasible solution to the LP must satisfy
4 x1  x2  5x3  3x4  4 x1  3x2  6 x3  3x4 (  58)
since any feasible solution must satisfy nonnegativity constraints on the
variables and the coefficients in the second expression is greater than
or equal to the corresponding coefficients in the first expression.
So 58 is an upper bound on the optimal value of the LP.
OR-1 2009
7
max
4 x1
 x2
 5 x3
 3 x4
s.t.
x1
 x2
 x3
 3 x4

5 x1
 x2
 3 x3
 8 x4
 55
 x1  2 x2  3 x3
x1 , x2 , x3 , x4  0
 5 x4

1
3
 Now, we may use nonnegative weights yi for each constraint.
( y1  5 y2  y3 ) x1  ( y1  y2  2 y3 ) x2  ( y1  3 y2  3 y3 ) x3  (3 y1  8 y2  5 y3 ) x4
 y1  55 y2  3 y3
In vector notation,
{ y1 (1,  1,  1, 3)  y2 (5, 1, 3, 8)  y3 (1, 2, 3,  5)}' x  ( y1  55 y2  3 y3 )
Objective function of the LP is 4 x1  x2  5x3  3x4
OR-1 2009
8
 Hence as long as the nonnegative weights yi satisfy
y1  5 y2  y3  4,  y1  y2  2 y3  1
 y1  3 y2  3 y3  5, 3 y1  8 y2  5 y3  3
we can use y1  55 y2  3 y3 as an upper bound on optimal value.
 To find more accurate upper bound (smallest upper bound), we want
to solve
min
y1
 55 y 2
 3 y3
s.t.
y1
 5 y2
 y3
 4
 y1
 y2
 2 y3
 1
 y1
 3 y2
 3 y3
 5
 8 y 2  5 y3
y1 , y2 , y3  0
 3
3 y1
max
4 x1
 x2
 5 x3
 4 x4
s.t.
x1
 x2
 x3
 3 x4

5 x1
 x2
 3 x3
 8 x4
 55
 x1
 2 x2
 3 x3
 5 x4

1
3
x1 , x2 , x3 , x4  0
 Dual problem obtained. Note that the objective value of any feasible
solution to the dual problem provides an upper bound on the optimal
value of the given LP.
OR-1 2009
9
 General form
n
maximize
(P)
subject to
cjxj
maximize c' x
j 1
subject to
n
 aij x j  bi ,
j 1
x j  0,
(D)
j  1,2,..., n
subject to
 bi yi
minimize b' y
i 1
m
 aij yi  c j ,
i 1
x0
i  1,..., m
m
minimize
Ax  b
subject to
j  1,..., n
y' A  c
y0
yi  0, i  1,2,..., m
OR-1 2009
10
 Thm: (Weak duality relation)
Suppose (x1, …, xn) is a feasible solution to the primal problem (P) and
(y1, …, yn) is a feasible solution to the dual problem,
then  nj 1 c j x j  im1 bi yi
pf)
nj 1 c j x j  nj 1 (im1 aij yi ) x j im1 (nj 1 aij x j ) yi im1 bi yi

 Cor: If we can find a feasible x* to (P) and a feasible y* to (D) such that
*
*
nj1 c j x j  im1 bi yi , then x* is an optimal solution to (P) and y* is
an optimal solution to (D).
pf) For all feasible solution x to (P), we have
*
*
nj1 c j x j  im1 bi yi  nj1 c j x j
Similarly, for all feasible y to (D), we have
*
*
im1 bi yi  nj1 c j x j  im1 bi yi
OR-1 2009

11
 [Strong Duality Theorem] If (P) has an optimal solution (x1*, …, xn* ),
then (D) also has an optimal solution, say (y1*, …, ym* ), and
*
*
nj1 c j x j  im1 bi yi
(i.e., no duality gap, dual optimal value – primal optimal value = 0)
Note that strong duality theorem says that if (P) has an optimal
solution, the dual (D) is neither unbounded nor infeasible, but always
has an optimal solution.
OR-1 2009
12
Idea of proof: Read the optimal solution of the dual problem from the
coefficients of the slack variables in the z-row from the optimal
dictionary (tableau).
ex)
 z  4x
x
 5 x  3x
 0
1
2
3
4
x1
 x2
 x3
 3 x4
5 x1
 x2
 3 x3
 8 x4
 x1
 2 x2
 3 x3
 5 x4
 x5

 x6
1
 55
 x7

3
Note that the dual variables y1, y2, y3 matches naturally with slack
variables x5, x6, x7. For example x5 is slack variable for the first
constraint and y1 is dual variable for the first constraint, and so on.
OR-1 2009
13
At optimality, the tableau looks
z
 x1
2 x1
 x2
 2 x3
 11x5
 6 x7
  29
 4 x3
 5 x5
 3 x7

14
 2 x5
 x7

5
 11x7

1
x1
 x3
 5 x1
 9 x3
x4
 21x5
 x6
In the z-row of the dictionary, the coefficients of the slack variables are
–11 for x5, 0 for x6, -6 for x7.
Assigning these values with reversed signs to the corresponding dual
variables, we obtain desired optimal solution of the dual:
y1 = 11, y2 = 0, y3 = 6.
OR-1 2009
14
 Idea of the proof:
Note that the coefficients of x5, x6, and x7 in the z-row show what
numbers we multiplied to the corresponding equation and add them
to the z-row in the elementary row operations (net effect of many row
operations)
ex)
 z  4 x1  x2  5 x3  3 x4
 0
x1
 x2
 x3
 3 x4
5 x1
 x2
 3 x3
 8 x4
 x1
 2 x2
 3 x3
 5 x4
 x5

 x6
1
 55
 x7

3
Suppose we performed row operations
(row2)  2*(row 1) +(row 2), then ( z-row)  3*(row 2) + (z-row).
The net effect in z-row is adding 6*(row 1) + 3*(row 2) to the z-row
and the scalar we multiplied can be read from the coefficients of x5
and x6 in the z-row.
OR-1 2009
15
(ex-continued)
 z  4 x1
 x2
 5 x3
 3 x4
x1
 x2
 x3
 3 x4
5 x1
 x2
 3 x3
 8 x4
 x1
 2 x2
 3 x3
 5 x4
 z  4 x1
(row 1)2 +
(row 2)
OR-1 2009
 x5
 x6
 5 x3
 3 x4
x1
 x2
 x3
 3 x4
 x5
7 x1
 1x2
 x3
 14 x4
 2 x5
 x1
 2 x2
 3 x3
 5 x4
0

1
 55
 x7
 x2

 x6

3

0

1
 57
 x7

3
16
 (ex-continued)
 z  4 x1
 x2
 5 x3
 3 x4
x1
 x2
 x3
 3 x4
 x5
7 x1
 1x2
 x3
 14 x4
 2 x5
 x1
 2 x2
 3 x3
 5 x4
 z  25x1
(row 2)  3 +
z-row
OR-1 2009
 x6
 8 x3
 45x4
 6 x5
x1
 x2
 x3
 3 x4
 x5
7 x1
 1x2
 x3
 14 x4
 2 x5
 x1
 2 x2
 3 x3
 5 x4
0

1
 57
 x7
 2 x2

 3 x6

3
 171
 x6
 x7

1

57

3
17
 Example – Initial tableau
 z  4 x1
 x2
 5 x3
 3 x4
x1
 x2
 x3
 3 x4
5 x1
 x2
 3 x3
 8 x4
 x1
 2 x2
 3 x3
 5 x4
 x5
 x6

0

1
 55
 x7

3
Optimal tableau
z
 x1
2 x1
OR-1 2009
 x2
 2 x3
 11x5
 6 x7
  29
 4 x3
 5 x5
 3 x7

14
 2 x5
 x7

5
 11x7

1
x1
 x3
 5 x1
 9 x3
x4
 21x5
 x6
18
 Let yi be the scalar we multiplied to the i-th row and add to the z-row in
the net effect.
Then the coefficient of slack variables in the z-row represent the yi
values we multiplied to the i-th row for i = 1, … ,m.
Also the coefficients of structural variables in the z-row are given as
c j  im1 yi aij ,
j  1, ... , n
Now in the optimal tableau, all the coefficients in the z-row are  0, which
implies
yi  0, i  1, ... , m
c j  im1 yi aij  0,
(for slack variables)
j  1, ... , n
(for structural variables)
If we take – yi as a dual solution, they are dual feasible.
( yi )  0, i  1, ... , m
im1 ( yi )aij  c j ,
OR-1 2009
j  1, ... , n
19
 Also the constant term in the z-row gives the value im1 yi bi
So it is the negative of the dual objective value of the dual solution (yi ), i = 1, ,,, , m
Note that the constant term in the z-row also gives the negative of the
objective value of the current primal feasible solution.
So we have found a feasible dual solution which gives the same dual
objective value as the current primal feasible solution.
From previous Corollary, the dual solution and the primal solution are
optimal to the dual and the primal problem, respectively.
It is the idea of the proof.
OR-1 2009
20
 pf of strong duality theorem)
Suppose we introduce slack variables
xn i  bi   nj 1 aij x j (i  1, ..., m)
and solve the LP by simplex and obtain optimal dictionary with
z  z *  nk1m ck xk , ck  0 k , z *  nj1 c j x*j
*
Let yi  cn i , i  1,..., m
We claim that yi*, i = 1, … , m is an optimal dual solution, i.e. it
satisfies dual constraints and im1 bi yi*  z *
Equate z  nj1 c j x j and z  z *  nj1 c j x j  nj nm1 c j x j
 nj1 c j x j  z *  nj1 c j x j  nj nm1 c j x j
This equation must be satisfied by any x that satisfies the dictionary
(excluding the nonnegativity constraints) since the set of feasible
solutions does not change for any dictionary.
OR-1 2009
21
Have nj1 c j x j  z *  nj1 c j x j  nj nm1 c j x j
Substitute xn i  bi   nj 1 aij x j , i  1,..., m into above
since any feasible solution to dictionary should satisfy this.
 nj1 c j x j  z *  nj1 c j x j  im1 yi* (bi  nj1 aij x j )
 nj1 c j x j  z *  im1 bi yi*  nj1 (c j  im1 aij yi* ) x j
Now this equation should hold for all feasible solutions to the
dictionary. From the initial dictionary, we know that any feasible
solution to the dictionary can be obtained by assigning arbitrary
values to x1, … , xn and setting
xn+i = bi - j=1n aijxj , ( i = 1, … , m).
Use these solutions. Note that, in the above equation, the variables
xn+i do not appear. So it must hold for any choices of xj , j = 1, … , n.
OR-1 2009
22
*
*
*
nj1 c j x j  z  im1 bi yi  nj1 (c j  im1 aij yi ) x j
Equality must hold for all choices of x1, …, xn.
Hence z *  im1 bi yi* , c j  c j  im1 aij yi* , j  1,..., n
ck  0  k 
c j  im1 aij yi*  0

 yi*  0

*
im1 aij yi  c j , j  1,..., n
y*i  0, i  1,..., m
Hence y* is dual feasible.
*
*
*
Also we have that im1 bi yi  z  nj1 c j x j
Since
*
*
im1 bi yi  nj1 c j x j  im1bi yi for all dual feasible y (weak duality)
yi*, i = 1, … , m is an optimal dual solution and im1 bi yi*  nj1 c j x*j

OR-1 2009
23