8
CHAPTER 1. STATIC OPTIMISATION AND EXTENSIONS
A theorem can be proved that
min cT x = max bT y
•
x
y
,
and
• the dual variables corresponding to the constraints which are active at the minimum point
are positive and
• the dual variables, which correspond to non-active constraints are zero.
An economic interpretation of this fact will be given at the end of this section.
Moreover, there is a series of “mirror” properties which link the primal and the dual
problems. Below, we list a few of them, which are important from the point of view of
economic applications. In our case [...]
• the dual problem is a maximisation problem whereas the primal is a minimisation one;
• the dual restrictions are “from above” whereas the primal are “from below”;
• the number of primal variables equals the number of dual restrictions and the number
of primal restrictions equals the number of dual variables;
• the dual problem may be interpreted as production maximisation subject to restricted
resources where AT is the technological coefficients matrix;
• the primal problem will consist of minimisation of the cost of resource utilisation subject
to fulfillment of some production quota;
• in that situation the primal variables will be interpreted as shadow prices5 for resources:
those which correspond to “abundant” resources (restrictions inactive) will be zero while
those which refer to “scarce” resources (restrictions active) will usually be positive.
The third bullet point provides us with an obvious motivation for examining a dual solution.
If a solution procedure is sensitive to the dimension of the problem to be solved (as it certainly
is in the case of the graphical method) changing a problem from the primal to the dual may
be beneficial. Indeed, suppose that we have a minimisation problem of six variables and two
constraints; the graphical solution to this problem is impossible. However, the dual problem
can be graphically analysed and solved. (Obviously, in the dual variables; passing from the
dual to the primal solution may be difficult.)
1.1.4
Concavity criterion
Establishing concavity of a function using the Definition 1.1.5 might be difficult for complicated
functions. If a function is twice continuously differentiable the following Lemma 1.1.1 can be
useful. The lemma is formulated as a concavity criterion. A similar criterion for a convex
function could be easily formulated (see Remark 1.1.1,’s first bullet point).
5
The multipliers interpretation will be discussed in Section 1.2.4.
1.1.
9
STATIC OPTIMISATION
Before we state the concavity lemma a few definitions have to be formulated.
Definition 1.1.7 Suppose f : X → R1 is twice continuously differentiable and X ⊂ Rn . The
n × n matrix of the second order partial derivatives
2
∂ f (x)
H(x) ≡
(1.13)
∂xi ∂xj i=1,...n;j=1,...n
is the Hessian of function f .
p
x21 + x2 , [x1 , x2 ]T ∈ R2 . The Hessian of J is


x2
− 2 x1 3
3
2
(x1 +x2 ) 2
2(x1 +x2 ) 2 

.
H(x) = 


x1
1
− 2
− 2
3
3
Example 1.1.2 Consider J(x) =
2(x1 +x2 ) 2
(1.14)
4(x1 +x2 ) 2
Definition 1.1.8 The expression
xT Ax
where x ∈ Rn and A ∈ Rn
×n
is called the quadratic form of matrix A.
Definition 1.1.9 Symmetric matrix A quadratic form is called
• negative definite iff xT Ax < 0, ∀x ∈ Rn \ {0}
• negative semi-definite iff xT Ax ≤ 0, ∀x ∈ Rn , x 6= 0 (but for some x 6= 0, xT Ax < 0)
• positive definite iff xT Ax > 0, ∀x ∈ Rn \ {0}
• positive semi-definite iff xT Ax ≥ 0, ∀x ∈ Rn , x 6= 0 (but for some x 6= 0, xT Ax > 0).
If the form was to be definite in a particular set X rather than in Rn , ∀x ∈ Rn would have to
be replaced by ∀x ∈ X ⊂ Rn .
For simplicity, one can call the symmetric matrix A negative definite (semi-definite etc. )
if the associated quadratic form is negative definite (semi-negative, etc. ).
It might be difficult to compute and assess whether a quadratic form of a composite
function is negative definite (semi-negative, etc. .) The following criterion that uses Silvester’s
conditions might be practical.
Criterion 1.1.1
• Quadratic form xT H(x)x is positive definite iff all principal minors6 of H(x) are positive.
• Quadratic form xT H(x)x is negative definite iff quadratic form xT [−H(x)]x is positive definite.
6
An ij-th minor of matrix A is the determinant of a (square) submatrix of A where the i-th row and j-the
column were removed. Principal minors are such that contain elements of the main diagonal. Principal leading
minors are principal minors that contain element {1, 1}.
10
CHAPTER 1. STATIC OPTIMISATION AND EXTENSIONS
The conditions for semi-definiteness are a bit more involved.
Criterion 1.1.2
• Quadratic form xT H(x)x is positive semi-definite iff all principal minors of H(x) and of all
n × n matrices obtained by permutating both the rows and columns of H(x) are non negative.
• Quadratic form xT H(x)x is negative semi-definite iff quadratic form xT [−H(x)]x is positive
semi-definite.
Finally, we can formulate the concavity criterion as the following lemma.
Lemma 1.1.1 Suppose f : Rn → R1 is twice continuously differentiable. Function f is
concave iff if Hessian A(x) of the function f is negative semi-definite ∀x ∈ Rn . Function f is
strictly concave iff if Hessian A(x) of the function f is negative definite ∀x ∈ Rn .
If concavity of f is to be checked on a particular convex set X rather than in Rn , ∀x ∈ Rn
would have to be replaced by ∀x ∈ X ⊂ Rn .
Convexity can always be proved as concavity of negative function (i.e., , if f is convex, −f
is concave). However, a companion lemma for convex functions can also be formulated.
Lemma 1.1.2 Suppose f : Rn → R1 is twice continuously differentiable. Function f is
convex iff if Hessian A(x) of the function f is positive semi-definite ∀x ∈ Rn . Function f is
strictly convex iff if Hessian A(x) of the function f is positive definite ∀x ∈ Rn .
If convexity of f is to be checked on a particular convex set X rather than in Rn , ∀x ∈ Rn
would have to be replaced by ∀x ∈ X ⊂ Rn .
p
Example 1.1.3 Consider J(x) = x21 + x2 , [x1 , x2 ]T ∈ R2 from Example 1.1.2 defined on
X = {x : x1 + 2x2 ≤ 8, xT ∈ R2+ }. Check if J is concave, convex or else in X.
The Hessian of J is computed as (1.14). According to the above criterion, to decide about
J’s concavity we need to compute the signs of the minors of −H. They all should be positive,
or non negative, ∀x ∈ X (see the first bullet points in Criteria 1.1.1 and 1.1.2).
!
x2
= −1.
(1.15)
sign −
3
(x21 + x2 ) 2
This is sufficient for J to be non concave. For convexity, all principal minors of H should
be non negative. The first minor’s sign is the reverse of (1.15). However, the second minor’s
sign is
x21
x2
sign −
−
= −1.
(1.16)
4(x21 + x2 )3 4(x21 + x2 )3
So, J is neither convex nor concave in X.
1.1.
11
STATIC OPTIMISATION
However, observe that, along the x1 axis i.e., , for x2 = 0, J is negative semi-definite and can
have a local maximum there.
1.1.5
The Lagrange multiplier method
In this section, the only section in this chapter, we waive the assumption about non-negativity
of x. Consequently, the set Ω here (i.e., , (1.18)) is defined more generally than (1.2).
The Lagrange multiplier method is a well known method for optimisation (see, for example,
the QUAN 101 handbook) of a function subject to equality-type constraints. We will revise
the method briefly.
The crucial notion of this method is the Lagrange function (Lagrangean). Notice that
the Lagrangean of an optimisation problem with inequality constraints looks identical to the
Lagrangean formulated for the problem with equality constraints.
Definition 1.1.10 The Lagrange function of the optimisation problem
x̂ ≡ arg max J(x)
(1.17)
Ω ≡ {x : Gj (x) ≥ 0, j = 1, 2...m}
(1.18)
Ω ≡ {x : Gj (x) = 0, j = 1, 2...m})
(1.19)
x∈Ω
(or
where J : Rn → R1 , Gj : Rn → Rm , is defined as the following function of x ∈ Rn and
µ ∈ Rm
m
X
L(x, µ) ≡ J(x) +
µj Gj (x) = J(x) + µT G(x)
(1.20)
j=1
]T
Rm
where µ = [µ1 , ...µm ∈
are called the Lagrange multipliers and G(x) is the
vector function of constraints defined in (1.19), (or (1.19)); x ∈ Rn stands, obviously, for
[x1 , .. xn ]T .
As noted above, the Lagrange method of solution of constrained optimisation problems applies
to equality-type constraints; it is summarised in the following theorem.
Theorem 1.1.1 If x̂ is a solution7 of the problem (1.17), (1.19)8 , J(x) and Gj (x) are
continuously differentiable9 and such that the Jacobian matrix of the constraints G(x) i.e.,
the matrix defined as follows
 ∂G1

1
... ∂G
∂x1
∂xn

.. ...
..  ,
∂Gm
∂Gm
... ∂xn
∂x1
7
Watch notation: x may denote a scalar (x ∈ R1 ) or a vector (x ∈ Rn ) depending on in which context it
has been defined.
8
Equality constraints.
9
I.e., differentiable with continuous derivatives.