4. Duality and Sensitivity
For every instance of an LP, there is an associated LP known as the dual problem. The
original problem is known as the primal problem.
There are two de…nitions of the dual pair of problems, which are equivalent. Duality can
also be shown to be a `symmetric’ relationship (property of involution) in that “the dual of
the dual is the primal problem”.
4.1 Primal Dual Formulations
CANONICAL FORM
P1: Minimize cT x
subject to Ax
b
x
0
D1: Maximize
or bT y
yT b
cT
subject to y T A
y
T
or A y
c
0
STANDARD FORM
P2 Minimize cT x
subject to Ax = b
x
0
D2 Maximize y T b
or bT y
T
subject to y T A
cT
(or A y
y
free variables (u.r.s.)
c)
Note that if the number of variables in the primal problem is n and the number of
constraints is m then the the number of variables in the dual problem is m and the number
of constraints is n: In fact variables in the dual problem correspond to constraints in the
primal problem and vice-versa.
In other words, if x2Rn and A is (m
n) then y 2 Rm and
T
A y
1
c
represents a system of n inequalities (the dual constraints) in m new unknown dual variables
(y1 ; :::; ym ) :
Example 1
Primal:
Minimize
10x1 + 5x2 + 4x3
subject to
3x1 + 2x2 3x3
4x1
+ 2x3
3
10
x 1 ; x2 ; x 3
0
Dual:
Maximize
3y1 + 10y2
subject to
3y1 + 4y2
2y1
3y1 + 2y2
10
5
4
y 1 ; y2
0
Example 2
Primal:
Minimize
6x1 + 8x2
subject to 3x1 + x2
s1
5x1 + 2x2
=4
s2
=7
x1 ; x2 ; s1 ; s2
0
Dual:
Maximize
4w1 + 7w2
subject to 3w1 + 5w2
6
w1 + 2w2
8
w1 ; w2
0
2
The primal in this example is stated in standard form P2. However the variables s1 ; s2
play the part of non-negative surplus variables since they do not appear in the objective
function and it is easy to write down an equivalent primal problem in the canonical form
P1. The dual problem is the same, whether written in the form of D1 or as D2.
Economic interpretation of the dual
The dual of the diet problem introduced earlier
cT x
min
x
s.t.
Ax
x
r
0
is
yT r
max
y
s.t.
yT A
y
cT
0
We can provide an interesting economic interpretation of the dual in this case:
Recall that aij represents the amount of nutrient i contained in unit amount of the j th
food.
Consider a pill maker (an alternative supplier of nutrients) who wants to set a price for
each nutrient. Let i be the price per unit of the ith nutrient (i = 1; :::; m) forming a price
vector . The pill maker’s price to supply the required amount of each nutrient in the diet
is
m
X
i ri :
i=1
The price of all nutrients needed to make up a unit amount of the j th food should be
competitive, therefore not be more than the price of unit amount of the j th food. Hence
m
X
i aij
cj
j = 1; :::; n
i=1
Identifying y with we see that the dual problem maximizes the income of the pill maker
subject to a competitiveness constraint.
3
4.2 Duality - basic properties
4.2.1 Equivalence of dual forms
The duality relationships P1-D1, P2-D2 are equivalent.
Proof
We …rst take the dual pair in standard form P2-D2 as the de…nition and …nd the dual of
a problem in the canonical form P1:
Minimize cT x
subject to Ax
x
Introduce a vector of m surplus variables s
0 so the constraints can be written
Ax
or
A
b
0
s=b
!
x
Im
s
=b
which is of the form A0 x0 = b where A0 and x0 are partitioned matrices. Applying the
duality result for the primal in standard form P2, the dual problem is
Maximize
subject to
bT y
AT
Im
!
y
c
0
!
In this dual problem, although the D2 dual variables are not explicitly unrestricted in
sign (free variables), the constraints imply that AT y c and y
0, which provides the
dual problem D1.
We can also take the dual pair in canonical form P1-D1 as the de…nition and …nd the
dual of a problem in the canonical form P2. (Exercise)
4.2.2 Involution property
The dual of the dual is the primal.
Proof
4
Consider the dual in canonical form D1 (y2Rm ; A(m
n))
Maximize
bT y
T
subject to A y
c
y
0
We may write this in the form of a canonical P1 primal problem
Minimize
bT y
subject to
A
T
y
c
y
0
The corresponding canonical dual problem in variables w2Rn is
Maximize
subject to
cT w
T
AT w
w
0
b
which is equivalent to the primal problem P1
Minimize cT w
subject to Aw
b
w
0
4.2.3. Mixed Constraints
The dual of LP problems that are neither in standard form nor in canonical form can be
written down using rules summarized in the following table.
Min Problem
Max problem
Type of Constraint
aTi x bi
aTi x bi
aTi x = bi
Type of Variable
yi 0
yi 0
yi u.r.s.
Type of Variable
xj 0
xj 0
xj u.r.s.
(P1, D1)
(P2, D2)
Type of Constraint
yT Aj cj
(either case)
T
y Aj cj
yT Aj = cj
5
Notes:
1. u.r.s. denotes “unrestricted in sign”or free variable
2. aTi denotes ith primal constraint coe¢ cients
i.e. the ith row of A-matrix
3. Aj denotes j th dual constraint coe¢ cients
i.e. the jth column of A-matrix
Example 3 (mixed constraints)
Primal:
Maximize z = 5x1 + 6x2
subject to x1 + 2x2 = 5
x1 + 5x2 3
4x1 + 7x2 8
x1 u.r.s., x2
0
Dual:
Minimize w = 5y1 + 3y2 + 8y3
subject to
y1 y2 + 4y3 = 5
2y1 + 5y2 + 7y3 6
y1 u.r.s., y2
0; y3
0
Use of the Table:
m = 3; n = 2 so dual has 3 variables y1 ; y2 ; y3 and 2 constraints
Since primal is a maximization, refer to Table 2nd column for primal properties read o¤ dual properties from Table 1st column.
Variable x1 is u.r.s. )1st dual constraint is ‘=’
Variable x2 0 ) 2nd dual constraint is ‘ ’(for minimization)
1st primal constraint is ‘=’, ) y1 is u.r.s.
2nd primal constraint is ‘ ’) y2 0
3rd primal constraint is ‘ ’) y3 0
6
4.3 Duality Theorems
The primal and dual LP problems are so intimately related that solving either problem
provides the solution to the other. For the following results we assume the primal is a
minimization in standard form P2 with dual D2.
4.3.1 Weak duality lemma
If x, y are feasible for the primal and dual problems respectively, then)
yT b
cT x
(4.1)
Proof
Let x, y be feasible for problems P2, D2 respectively. Then
yT A
cT
(4.2a)
cT
0
(4.2b)
cT x
0
(4.2c)
yT A
)
yT A
(4.2c) follows from (4.2b) because x
0 (feasible for P2) so
Now Ax = b; again because x is feasible for P2, so
y T Ax
cT x
0
yT b
cT x
0
yT b
Pn
i=1
yT A
cT
i
xi
0.
cT x
as required.
Corollary 1
If x, y are feasible for P2, D2 respectively and cT x = y T b then x and y are optimal for
their respective problems.
i.e. If a pair of feasible solutions for the primal and dual problems have the same objective
function value, then they are optimal for their respective problems.
Corollary 2
7
If either problem is unbounded (optimal solution is at in…nity) the other problem in
infeasible.
[ NB The converse is not true as both problems can be infeasible! ]
Example
In Example 2 of Section 4.1 the primal and dual objective functions are
z = 6x1 + 8x2
and
w = 4y1 + 7y2 :
It can easily be veri…ed (e.g. graphically) that zmin = wmax =
achieved at x =
T
7
;0
5
and y = 0; 56
T
42
and these optima are
5
.
Let us solve the dual problem through simplex iterations. In standard form constraints
are
3y1 + 5y2 + v1 = 6
y1 + 2y2 + v2 = 8
A basis of dual slacks is available:
Max y1
y2
v1
v2
3
1
5
2
6
8
z
4
7 0
After one pivot, the optimal tableau is
y1
y2
v2
z
v1
3
5
1
5
1
5
1
5
2
5
7
5
6
5
28
5
42
5
We may con…rm the elements of the tableau as follows:
!
!
5 0
3 1
B=
;
N =
2 1
1 0
8
B
B
1
1
3
1
1
N =
5
b=B
(zj
1 0
2 5
1
=
5
1
b=
1
5
cj ) = cTB y j
z0 = cTB b =
!
1
2
!
cj =
6
5
28
5
7; 0
=
!
1 0
2 5
1
5
2
5
3
5
1
5
6
8
1
;
5
!
=
!
=
!
= (yij )
!
6
5
28
5
y2
v2
7
5
42
5
The matrix form of the reduced tableau is
Non-basic variables
Basic vars Y = B
z
where zj
1
N = (yij ) b
fzj
cj g
(4.3)
z0
cTN and z0 = cTB b.
cj are the elements of cTB B 1 N
Simplex multipliers (dual variables)
The components of the m-vector
y T0 = cTB B
1
(4.4)
known as simplex multipliers are the dual variables associated with a particular basis B: The
dual vector is generally infeasible for the dual problem during intermediate tableau iterations.
Only when the bottom row satis…es primal optimality does y T0 become dual feasible.
We will show below that
1. the dual variables y 0 corresponding to an optimal basis are feasible
2. the dual objective w0 = y T0 b equals z0 , the minimum value of the primal.
9
Therefore by Corollary 1 to the weak duality lemma we have constructed an optimal
dual vector y 0 .
4.3.2 Strong duality theorem
If either the primal or the dual has a …nite optimal solution then so has the other and the
corresponding objective function values are equal. If either problem is unbounded the other
problem is infeasible.
Proof
Let y T0 = cTB B
1
where B is an optimal basis.
h
i
y T0 A = y T0 B N = cTB B
h
i
1
T
T
= cB cB B N
i
h
cTB cTN = cT
1
h
B
N
i
(4.5a)
(4.5b)
so that y T0 is dual feasible, with objective value
w0 = y T0 b
= cTB B 1 b
= cTB xB
= z0
Notice that in going from (4.5a) to (4.5b) we have assumed that cTB B 1 N cTN .
This follows from the optimality conditions zj cj 0 8j, which are simply the components of cTB B 1 N cTN .
Example (continued)
For the above example
y T0 = cTB B
1
=
5
1
7;
=
7
;
5
=
x1 ;
1 0
2 5
0
0
x2
10
:
!
Hence the primal optimal solution is
7
x1 = ; x2 = 0:
5
4.3.3 Complementary slackness
The complementary slackness (C-S) conditions, which are a consequence of duality, relate
the values of the primal and dual vectors at a common optimum. They can be stated for
either the asymmetric (P2,D2) or the symmetric (P1 D1) primal dual pair.
Theorem (Complementary slackness - asymmetric case)
Let x; y be feasible for P2, D2 respectively. A necessary and su¢ cient condition (NSC)
that they both be optimal is that for each j (j = 1; :::; n)
i) If xj > 0 then y T Aj = cj
ii) If y T Aj < cj then xj = 0
Proof.
Since x; y are feasible for their respective problems, we have
Ax = b ,
yT A
Thus y T A + v T = cT with v T
x
0
cT
0. Postmultiply by x and apply Ax = b
y T Ax + v T x = cT x
y T b + v T x = cT x
cT x
yT b = vT x
(4.6)
By corollary 2 to the Weak Duality theorem, x; y are optimal if and only if r.h.s.= 0.
P
Since v T x = nj=1 vj xj with vj ; xj 0; r.h.s.= 0 if and only if vj xj = 0 for each j.
So i) xj > 0 ) vj = 0 and ii) vj > 0 ) xj = 0 as required:
[In fact ii) follows from i) and feasibility of y ]
Theorem (Complementary slackness - symmetric case)
11
Let x; y be feasible for P1, D1 respectively. A necessary and su¢ cient condition (NSC)
that they both be optimal is that
i) xj vj = 0 8j
ii) si yi = 0 8i
where vj is the j th dual slack (corresponding to xj ) and si is the ith primal slack
Comment: By analogy with the asymmetric case, we can write each equation as a pair
of implications, so
8j; xj > 0 ) vj = 0; vj > 0 ) xj = 0
(4.7a)
8i; si > 0 ) yi = 0; yi > 0 ) si = 0
(4.7b)
Proof. Since x; y are feasible for their respective problems, we have
Ax
b,
x
0
yT A
cT ,
y
0
0 , vT
or, introducing slack variables s
Ax
0
s=b,
y T A + v T = cT ,
x; s
0
y; v
0
Then
y T Ax
yT s = yT b
y T Ax + v T x = cT x
so
cT x
yT b = vT x + yT s
The result follows by analogy to the previous (asymmetric) case.
Example 2 (continued)
Recall the primal was
Minimize
6x1 + 8x2
subject to
3x1 + x2
5x1 + 2x2
4
7
x 1 ; x2
0
12
(4.8)
and the dual
Maximize
4w1 + 7w2
subject to 3w1 + 5w2
w1 + 2w2
6
8
w1 ; w2
0
The optimal solution to the primal is
(x1 ; x2 ; s1 ; s2 ) =
7
1
; 0; ; 0
5
5
From the C-S conditions, the optimal solution to the dual takes the form
(v1 ; v2 ; w1 ; w2 ) = (0; ; 0; )
i.e. the 1st dual variable is zero at the optimum (w1 = 0). Hence 5 = 6 from the 1st dual
constraint which is active (satis…ed with equality) since v1 = 0: Hence
(w1 ; w2 ) =
0;
6
5
4.4 The Dual Simplex Algorithm
Suppose we have a tableau that satis…es the bottom row optimality conditions but which
corresponds to an infeasible solution.
i.e. we have
non-basic variables
with zj
cj
xB
Y =(yij )
b
z-row
zj
cj ’s
z0
0; each j (for minimization) and bi < 0 for some i.
Such a tableau is said to be dual feasible because the associated dual vector y T0 =
cTB B 1 satis…es
y T0 A=y T0 [BjN ] cT
The dual simplex algorithm is applicable. The rules for selecting a pivot element are:
0; stop; the current basis is optimal.
1. If b
mini bi = bp
13
Otherwise choose pivot row p by
(bp < 0)
2. If ypj
0 8j stop; the problem is infeasible (because dual is unbounded).
Otherwise, choose pivot column q by the minimum ratio rule
min
zj
cj
ypj
j
: ypj < 0
=
zq
cq
ypq
(4.9)
3. Pivot on the element ypq (< 0) according to usual Simplex rules and return to Step 1.
Notes
i) The dual simplex algorthm is in fact the primal simplex algorithm applied to the dual
problem.
ii) The dual simplex algorithm can be useful
(a) in avoiding a Phase I calculation when an infeasible but “optimal” tableau is
available “by inspection”,
(b) in sensitivity analysis when the current optimal solution becomes infeasible through
the addition of an extra constraint or a small change in the right hand side vector
b:
in integer programming by Gomory’s cutting planes .
Example (Dual Simplex)
Minimize
2x1
+3x2
+4x3
subject to
x1
2x1
+2x2
x2
+ x3
+3x3
3
4
x1 ;
x2 ;
x3
0
An initial (infeasible) tableau using s1 ; s2 as basic variables is
s1
s2
x1
1
2
2
"
x2
2
1
3
x3
1
3
4
14
3
4
0
After 1st dual simplex pivot
s2
x2
s1
x1
x3
5
2
1
2
1
2
1
2
3
2
1
2
1
4
"
1
2
4
1
After 2nd dual simplex pivot
x2
x1
s2
s1
x3
1
5
2
5
1
5
2
5
1
5
8
5
1
5
7
5
9
5
Optimal solution is (x1 ; x2 ; x3 ; s1 ; s2 ) =
Optimal solution to dual (w1 ; w2 ) =
2
5
11
5
28
5
11 2
; ; 0; 0; 0
5 5
8 1
;
5 5
can be con…rmed using C-S conditions (Ex.)
N.B.
1. The OF value z is monotonic increasing in successive tableaux (min problem).
2. Since
I m appears in the the initial A matrix and y T0 = cTB B
w1 ; w2 also appear negatively as the zj
1
=
cTB B
1
cj values for surplus variables s1 ; s2 .
4.5 Sensitivity Analysis
,How is an optimal solution (of a LP in standard form) in‡uenced by
1. changes to the objective function coe¢ cients c
2. changes to the rhs. of the constraints b
3. changes to the constraint matrix A; in particular
(a) adding a new constraint aTm+1 x = bm+1
(b) adding a new variable (or activity) xm+1 ?
15
( I m)
From a knowledge of the simplex tableau at optimality
non-basic variables
Y =B 1 N
xB
z-row
cTN
1
cTB B N
b = B 1b
(4.10)
1
z0 = cTB B b
we see that
1. small changes to c a¤ect only the bottom row
2. small changes to b a¤ect only the rhs.
3. changes to A can potentially in‡uence all sections of the tableau
(we will only consider in detail adding a new constraint)
Example (Furniture manufacturer)
A manufacturer of furniture makes three products: desks, tables and chairs. Each item of
furniture made consumes amounts of three resources: timber, …nishing time and carpentry
time.
To manufacture one desk requires 8 ft. of timber, 4 hours …nishing time and 2 hours
carpentry time A table requires 6 ft. of timber, 2 hours …nishing time and 1.5 hours carpentry
time A chair requires 1 ft. of timber, 1.5 hours …nishing time and 0.5 hours carpentry time
The total weekly availability of timber is 48 ft., of …nishing time is 20 hrs and of carpentry
time is 8 hrs. The unit pro…t from selling a desk is $60, from a table is $30, from a chair is
$20.
The LP formulation to maximize the manufacturers pro…t subject to the given resource
constraints is as follows:
Let x1 ; x2 ; x3 = no. of units of each product (desks, tables and chairs respectively) to
make per week.
Maximize 60x1
subject to
+30x2
+20x3
8x1
4x1
+6x2
+2x2
+ x3
+1:5x3
48
20
2x1
x1 ;
+1:5x2
x2 ;
0:5x3
x3
8
0
16
resulting in the following tableaux
Initial Tableau
x1
s1
s2
s3
x2
Final Tableau
x3
8
4
2
6
2
60
30
s3
1 48
3
20
2
1
8
2
3
2
20
s1
x3
x1
0
x2
s2
8
4
2
2
2
2
3
2
5
4
1
2
10
5
24
8
2
10 280
The optimal solution is to manufacture 2 desks and 8 chairs resulting in a pro…t of $280.
There is a surplus of 24 ft. of timber, but all available hours of …nishing time, carpentry
time are used up.
Often the data assumed in the calculation are imprecise or subject to change. We now
ask
Q1. What range of variation in unit pro…t leaves the current solution optimal?
Q2. What is the optimal product mix if the unit pro…t for a table increases to $40?
Q3. What if the total available …nishing time increases to 30 hours?
These questions can be answered through a sensitivity analysis.
1. Changes to c
From (4.10) we see these a¤ect the optimality conditions zj cj 0 (for max) each j,
and the optimal value z0 = cTB b (though not of course the polytope representing the
feasible region).
Consider increasing a single component of cB ; say c1 by an amount
The modi…ed …nal tableau is
s3
x2
s1
x3
x1
8
4
2
2
2
2
3
2
5
4
1
2
10
+ 32
s2
5
+ 54
17
10
1
2
24
8
2
280
+2
, i.e. c1
c1 + :
The condition for the current solution to remain optimal is
3
2
5
5+
4
1
10
2
10 +
0
0
0
which need to be satis…ed simultaneously. Hence
4
20
or equivalently
56
c1
80
For example, if c1 = 56 ( = 4) we …nd an alternative optimal manufacturing plan
is to stop manufacturing desks and instead to produce 1.6 tables and 11.2 desks per
week
s3 x 2 s2
s3 x 1 s2
s1
x3
8
4
x1
3
2
4
2
2
5
4
0
2
2
24
8
1
2
2
136
5
56
5
8
5
s1
x3
x2
4
12 272
0
12
(27:2)
(11:2)
(1:6)
272
NB Non-integer production values can still have meaning in a continuous production
setting, although integer programming (IP) techniques (see Section 5) will provide ways
of …nding optimal integer solutions. Similarly we can establish ranges for c2 and c3
(leaving the other components unchanged) that ensure the current solution remains
optimal.
2. Suppose c2 = 40 ( = 10 in c2 ). As x2 is nonbasic in the optimal tableau, only
that column’s zj cj value is changed. It becomes negative ( 5) and after one pivot
optimality is restored
s3
s1
x3
8
4
x1
3
2
10
x2
2
2
5
4
5
s2
s3
2
2
24
8
1
2
2
x1
136
5
56
5
8
5
s1
x3
x2
16
10 280
18
s2
4
8 288
(27:2)
(11:2)
(1:6)
This is the same BFS as before.
3. Changes to b
These a¤ect the rhs. of an optimal tableau (4.10). Consider an increase of
second resource limitation b2 , b2
b2 + :
to the
We consider the e¤ect on b of the rhs. becoming b0 = b + b with b = (0; ; 0)T =
e2 where e2 = (0; 1; 0), the 2nd column of the identity matrix I 3 :
b = B 1b
0
b =B
1
(b + b)
= b +B
1
=b+
B 1 e2
b
Notice that the product B 1 e2 picks out the second column of B 1 which appears in
the …nal tableau under s2 . In the original A matrix the column Aj corresponding
to s2 is e2 . Hence B 1 Aj = B 1 e2 appears in the body of the …nal tableau under s2 :
Hence
2
3
2
6
7
b=4 2 5
1
2
The rhs. remains feasible while
24 + 2
0; 8 + 2
1
2
0; 2
0
i.e. while
4 () 16
4
b2
24
e.g. if b2 = 30 ( = 10) the …nal tableau becomes infeasible. In this example feasibility
(hence the optimal solution) is obtained after one iteration of the dual simplex algorithm
s3
x2
s1
x3
8
4
2
2
x1
3
2
5
4
10
5
s2
s3
2
2
1
2
44
28
s1
x3
s2
3
x2
x1
32
16
6
40 30 20 320
10 380
19