Compulsory Project 2 – with answers
MAT-INF3100 Linear optimization, Spring 2015
• NOTE. Hand in your assignment using Devilry (devilry.ifi.uio.no) in a
single PDF file that you name “username.pdf” (your username!). You
must also hand in the source code you wrote for Problem 3 (You are expected to discuss it as well as hand it in). Moreover, you should read the
general information about compulsory projects at the course web page.
Your submission should be written by yourself and reflect your own understanding.
Problem 1
1.1
Write down the dual problems for the following LP problems:
min
subject to
3x1 + 5x2 − x3 ,
x1 − x2 + x3 ≤ 3,
2x1 − 3x2 ≤ 4,
x1 , x2 ≥ 0.
(1)
and
max
subject to
3x1 + 2x2 ,
4x1 + 2x2 ≤ 16,
x1 + 2x2 ≤ 8,
x1 + x2 ≤ 5,
x1 , x2 ≥ 0.
(2)
Answer: For (1), the dual problem reads
max
−3y1 − 4y2 ,
subject to
−y1 − 2y2 ≤ 3,
y1 + 3y2 ≤ 5,
−y1 ≤ −1,
y1 , y2 ≥ 0.
1
(3)
Answer: For (2), the dual problem reads
min
16y1 + 8y2 + 5y3 ,
subject to
4y1 + y2 + y3 ≥ 3,
2y1 + 2y2 + y3 ≥ 2,
y1 , y2 ≥ 0.
(4)
1.2
Write (1) and its dual problem in matrix form. Do the same for (2).
Answer: The primal problem (1) can be written as
min c> x,
>
Ax ≥ b, x ≥ 0,
>
where x = (x1 , x2 , x3 ) , c = (3, 5, −1) , b = (3, 4)> , and
1 −1 1
A=
.
2 −3 0
The dual problem can be written as
max b> y,
A> y ≤ c, y ≤ 0,
where y = (y1 , y2 )> .
Similarly, the primal problem (2) can be written as
max c> x,
>
Ax ≥ b, x ≥ 0,
>
where x = (x1 , x2 ) , c = (3, 2) , b = (16, 8, 5)> , and


4 2
A = 1 2 .
1 1
The dual problem can be written as
max b> y,
A> y ≤ c, y ≤ 0,
where y = (y1 , y2 , y3 )> .
1.3
Show that x? = (3, 2) is feasible for the primal problem (2) and y ? = (1/2, 0, 1)
is feasible for the corresponding dual problem. Moreover, show that x? is in fact
the optimal solution of (2).
Answer:: Straightforward computations reveal that x? = (3, 2) and y ? = (1/2, 0, 1)
satisfy their respective constraints. Moreover, the objective function values of
the primal and dual problems coincide:
η ? := 3 ∗ 3 + 2 ∗ 2 = 9 + 4 = 13,
ξ ? := 16 ∗ 1/2 + 8 ∗ 0 + 5 ∗ 1 = 13.
We then conclude by the weak duality theorem.
2
Problem 2
Consider the function
(x, y) ∈ R2 ,
S(x, y) = f (x) − xy + g(y),
R = (−∞, ∞),
where f is a strongly concave function and g is a strongly convex function.
A function h is called strongly convex if h00 (y) > 0 for all y ∈ R, h0 (y) → −∞
as y → −∞, and h0 (y) → ∞ as y → ∞. A function is strongly concave if −h is
strongly convex.
2.1
Show that the gradient ∇S = (∂S/∂x, ∂S/∂y) vanishes at a unique point
(x? , y ? ) ∈ R2 .
Answer: Computing partial derivatives,
∂S/∂x = f 0 (x) − y = 0,
∂S/∂y = g 0 (y) − x = 0,
from which it follows that
φ(y) = y,
ψ(x) = x,
where
φ(y) = f 0 (g 0 (y)),
ψ(x) = g 0 (f 0 (x)).
These functions are decreasing and maps R onto R:
φ0 (y) = f 00 (g 0 (y))g 00 (y) < 0,
ψ 0 (x) = g 00 (f 0 (x))f 00 (x) < 0,
as f is concave and g is convex. Hence, the graph of φ(y) must intersect the
straight line y at a unique point y ∗ . Similarly, ψ(x) must intersect the straight
line x at a unique point x∗ .
2.2
With the stationary point (x? , y ? ) ∈ R2 given above, show that
max min S(x, y) = S(x? , y ? ),
min max S(x, y) = S(x? , y ? ).
x∈R y∈R
y∈R x∈R
Hint: Verify second order derivative tests for both inner and outer optimisations.
Answer: Let us first consider maxx∈R miny∈R S(x, y). We compute
∂ 2 S/∂y 2 = g 00 (y) > 0
(as g is convex).
Hence, miny∈R S(x, y) = S(x, y ? ), for each fixed x. Next, we compute
∂ 2 S/∂x2 = f 00 (x) < 0
(as f is concave),
and so maxx∈R S(x, y ? ) = S(x? , y ? ). Hence, maxx∈R miny∈R S(x, y) = S(x? , y ? ).
A similar argument gives that miny∈R maxx∈R S(x, y) = S(x? , y ? ).
3
Problem 3
Assume we have an LP problem on standard form:
max {cT x : Ax ≤ b, x ≥ O}
For simplicity we only work with matrices A ≥ 0 and assume that b ≥ 0, so
x = 0 is a feasible solution (A is an m times n matrix, and b, c are appropriately
sized). Given this, implement the following two algorithms:
• Algorithm S: the primal simplex algorithm (Section 6.2 in the book)
• Algorithm I: the path-following interior point algorithm (Figure 18.1 in
the book, see chapters 17, 18 and 19 for various details)
The choice of language is up to you, but we recommend MATLAB (or perhaps Python). Hand in your code (in separate files) and a test run for both
algorithms, using the data file on the course web page as a test. Print a line or
two of useful information every iteration. Discuss your results.
Some further points:
• The optimal value in the example problem is 212.6539.
• When solving linear systems of equations V x = d do not use the inv(V)
command – rather use the backslash operator, i. e. x = V \ d.
• The major part of the work for algorithm I is solving the KKT-system
(Karush-Kuhn-Tucker system, the optimality conditions for the barrier
problem) to find the steps ∆x, ∆y, ∆z and ∆w. This can be done in
different ways, but we suggest using the normal equations in primal form,
see start of section 19.2, eq. (19.9) for finding ∆y, then find ∆x etc.
• If you are using matlab and have downloaded the .mat file from the webpage, you can use the command load(’lptestproblemoblig.mat’) to
have the correct A, b, c appear in your MATLAB workspace.
• The test problem is quite large, so it might be a good idea to develop
against a simpler problem at first – see for example pages 96-101 in Vanderbei (2nd ed.).
Answer: Note: We do not include the code in this file – in a proper solution it
would be attached as two separate files, as described in the assignment.
The following is an example of output when running the two algorithms.
Running s i m p l e x method . . .
I t e r a t i o n 1 complete , c u r r e n t
I t e r a t i o n 2 complete , c u r r e n t
I t e r a t i o n 3 complete , c u r r e n t
I t e r a t i o n 4 complete , c u r r e n t
I t e r a t i o n 5 complete , c u r r e n t
I t e r a t i o n 6 complete , c u r r e n t
4
value
value
value
value
value
value
108.0549
139.9851
160.3693
163.0008
164.7146
172.6429
Iteration
Iteration
Iteration
Iteration
Iteration
Iteration
Iteration
Iteration
Iteration
Iteration
Iteration
Iteration
Iteration
Iteration
Iteration
Iteration
Iteration
Iteration
Iteration
7 complete ,
8 complete ,
9 complete ,
10 complete
11 complete
12 complete
13 complete
14 complete
15 complete
16 complete
17 complete
18 complete
19 complete
20 complete
21 complete
22 complete
23 complete
24 complete
25 complete
current value 186.0797
current value 192.3676
current value 196.2962
, current value 201.5087
, current value 202.8033
, current value 203.2862
, current value 203.6421
, current value 203.7756
, current value 208.0495
, current value 209.4698
, current value 209.4733
, current value 209.6871
, current value 211.3956
, current value 212.0439
, current value 212.0981
, current value 212.1842
, current value 212.5634
, current value 212.5669
, current value 212.6539
Running i n t e r i o r p o i n t method . . .
I t e r a t i o n 1 complete , c u r r e n t c o m p l e m e n t a r i t y 1 1 4 . 2 9 4 4
I t e r a t i o n 2 complete , c u r r e n t c o m p l e m e n t a r i t y 8 5 . 5 5 4 1
I t e r a t i o n 3 complete , c u r r e n t c o m p l e m e n t a r i t y 6 3 . 6 6 8 8
I t e r a t i o n 4 complete , c u r r e n t c o m p l e m e n t a r i t y 4 4 . 0 3 3 2
I t e r a t i o n 5 complete , c u r r e n t c o m p l e m e n t a r i t y 3 4 . 9 9 2 2
I t e r a t i o n 6 complete , c u r r e n t c o m p l e m e n t a r i t y 2 9 . 1 3 1 9
I t e r a t i o n 7 complete , c u r r e n t c o m p l e m e n t a r i t y 2 5 . 7 2 2 4
I t e r a t i o n 8 complete , c u r r e n t c o m p l e m e n t a r i t y 2 2 . 6 4 7 3
I t e r a t i o n 9 complete , c u r r e n t c o m p l e m e n t a r i t y 1 7 . 3 6
I t e r a t i o n 10 complete , c u r r e n t c o m p l e m e n t a r i t y 1 4 . 0 1 9 8
I t e r a t i o n 11 complete , c u r r e n t c o m p l e m e n t a r i t y 1 1 . 1 1 3 1
I t e r a t i o n 12 complete , c u r r e n t c o m p l e m e n t a r i t y 1 0 . 3 2 3 8
I t e r a t i o n 13 complete , c u r r e n t c o m p l e m e n t a r i t y 8 . 2 4 2 1
I t e r a t i o n 14 complete , c u r r e n t c o m p l e m e n t a r i t y 5 . 8 7 8 2
I t e r a t i o n 15 complete , c u r r e n t c o m p l e m e n t a r i t y 4 . 0 6 3 4
I t e r a t i o n 16 complete , c u r r e n t c o m p l e m e n t a r i t y 2 . 8 1 3 3
I t e r a t i o n 17 complete , c u r r e n t c o m p l e m e n t a r i t y 1 . 9 1 4 7
I t e r a t i o n 18 complete , c u r r e n t c o m p l e m e n t a r i t y 1 . 1 8 5 4
I t e r a t i o n 19 complete , c u r r e n t c o m p l e m e n t a r i t y 0 . 7 7 2 9 4
I t e r a t i o n 20 complete , c u r r e n t c o m p l e m e n t a r i t y 0 . 3 9 5 2 6
I t e r a t i o n 21 complete , c u r r e n t c o m p l e m e n t a r i t y 0 . 1 8 3 2 2
I t e r a t i o n 22 complete , c u r r e n t c o m p l e m e n t a r i t y 0 . 0 8 5 9 4 5
I t e r a t i o n 23 complete , c u r r e n t c o m p l e m e n t a r i t y 0 . 0 3 5 1 5 3
I t e r a t i o n 24 complete , c u r r e n t c o m p l e m e n t a r i t y 0 . 0 0 9 7 9 8 5
I t e r a t i o n 25 complete , c u r r e n t c o m p l e m e n t a r i t y 0 . 0 0 1 3 5 2 8
I t e r a t i o n 26 complete , c u r r e n t c o m p l e m e n t a r i t y 0 . 0 0 0 1 7 3 2
I t e r a t i o n 27 complete , c u r r e n t c o m p l e m e n t a r i t y 1 . 9 8 9 6 e −05
I t e r a t i o n 28 complete , c u r r e n t c o m p l e m e n t a r i t y 2 . 0 6 0 6 e −06
5
Runs c o m p l e t e . Values :
SIMPLEX : 2 1 2 . 6 5 3 9
INTERIOR : 2 1 2 . 6 5 3 9
Max . d e v i a t i o n : 9 . 4 8 4 4 e −07
In terms of iteration count, the two methods were comparable here, but
we know from the theory that for larger problems this would not be the case.
We also see that the two found solutions are more or less exactly the same, as
the maximal deviation between the two is of order 10−6 – this could indicate
that the interior point algorithm converges to the same solution as the simplex
method in this case.
6