Convex sets and convex functions
Convexity
REM: A set D ⊂ Rn is convex if
∀x, y ∈ D ⇒ λx + (1 − λ)y ∈ D
∀λ ∈ [0, 1].
Closure properties:
• If {Di } is a collection of convex sets then D = ∩i Di is convex.
• If D is convex in Rn and f : Rn → Rm is a linear function then f (D) is convex in
Rm .
• if D is convex then int(D) and D̄ are convex.
• C = {x : x ∈ C ⇒ ax ∈ C, ∀a ≥ 0} (cone with vertex in the origin) is convex iff
when x, y ∈ C then x + y ∈ C.
• If C1 and C2 are convex cones then C1 ∩ C2 and C1 + C2 are too.
Subgraph and epigraph
Let f : D ⊂ Rn → R where D is a convex set.
{(x, z) ∈ D × R : f (x) ≥ z}
is called subgraph and denoted by sub(f );
{(x, z) ∈ D × R : f (x) ≤ z}
is called epigraph and denoted by epi(f ).
Concave/convex functions
DEFINITION 1. f is a concave function on D if the sub(f ) set is convex;
f is a convex function on D if the epi(f ) set is convex.
THEOREM 2. f is a concave function on D iff ∀x, y, ∈ D, ∀λ ∈ [0, 1]:
f (λx + (1 − λ)y) ≥ λf (x) + (1 − λ)f (y);
f is a convex function on D iff ∀x, y, ∈ D, ∀λ ∈ [0, 1]:
f (λx + (1 − λ)y) ≤ λf (x) + (1 − λ)f (y).
Strict convexity/concavity
We speak of strictly concave (convex) function
when the involved inequalities are strict.
1
THEOREM 3. f is (strictly) concave iff −f is (strictly) convex;
f is (strictly) convex iff −f is (strictly) concave.
Closure Properties
Pn
• If
Pnfi (i = 1, . . . , n) are concave, then 1 λi fi (x) is concave for all λi ≥ 0;
1 λi fi (x) is strictly concave if at least one fi is strictly concave and the corresponding λi > 0.
• fPis concave on D iff ∀x1 , . . . , xp ∈ D, ∀α1 , . . . , αp where αi ≥ 0 (i = 1, . . . , p)∧
p
1 αi = 1
f (α1 x1 + · · · + αp xp ) ≥ α1 f (x1 ) + · · · + αp f (xp ).
• If f is concave on D and φ : R → R is increasing and concave, then φ ◦ f is
concave.
• f (x) = ax + b where a ∈ Rn , b ∈ R is both concave and convex.
Exercise
Give the following definitions:
• A function f : Rn → R is affine if f (x) = a · x + b, where a ∈ Rn and b ∈ R.
• X is called linear space (on R) if on X it is defined a sum operation such that
– f, g ∈ X then f + g = g + f ∈ X;
– α ∈ R, f ∈ X then αf ∈ X.
Prove that:
• a function is affine iff it is simultaneously concave and convex;
("⇒" easy; "⇐" tricky: try with f : R → R)
• the set of concave functions on a set D is a cone in the set of all the functions on D.
(Is it a linear space?)
Regularity conditions: Continuity
THEOREM 4. Let f : D → R be a concave function where D is an open convex set.
Then f is continuous on D.
2
REM: A concave function is necessarily continuous everywhere on its domain, except
perhaps at the boundary points. So it is possible to state that
THEOREM 5. Let f : D → R be a concave function on D.
Then f is continuous on int(D).
Examples
Let f (x) =
−ex , 0 < x < 1;
.
−3, x = 0, 1.
f is concave but non continuous on [0, 1].
REM: Economic life often takes places at the boundaries of convex sets: the possibility
of discontinuities must be taken into account.
Regularity conditions: Differentiability
LEMMA 6. Let f : D ⊂ R → R be concave on the open convex set D. Then for all
x1 < x2 < x3 we have
f (x3 ) − f (x1 )
f (x3 ) − f (x2 )
f (x2 ) − f (x1 )
≥
≥
.
x2 − x1
x3 − x1
x3 − x2
If f is concave then
f (x+h)−f (x)
h
is decreasing in h for each fixed x.
THEOREM 7. Let f : D ⊂ R → R be concave on the open convex set D.
0
0
Then there exist f+
(x) and f−
(x) for all x ∈ D.
More generally in Rn
THEOREM 8. Let f : D ⊂ Rn → R be concave on the open convex set D. Then
• Df (x, h) exists for all x ∈ D and for all h;
• f is differentiable almost everywhere on D;
• the differential df is continuous (where it exists).
REM: A property for a function f : D ⊂ Rn → R holds almost everywhere on D iff
it holds on D \ X, where X has Lebesgue measure zero.
Examples: A singleton has measure zero in R. A curve has measure zero in R2 . A
surface has measure zero in R3 .
3
Convexity and differentiability
Framework 1 : The case of differentiable functions.
THEOREM 9. Let f : D ⊂ Rn → R be differentiable on the open convex set D.
Then f is concave on D iff ∀x, y ∈ D,
df (x)(y − x) ≥ f (y) − f (x);
f is convex on D iff ∀x, y ∈ D,
df (x)(y − x) ≤ f (y) − f (x).
Case n = 1: A differentiable function f on an open interval D ⊂ R is concave iff
f (y) ≤ f (x) + f 0 (x)(y − x).
Convexity and differentiability, cont’d
THEOREM 10. Let f : D ⊂ Rn → R be differentiable on the open convex set D.
Then f is concave on D iff ∀x, y ∈ D,
(df (y) − df (x))(y − x) ≤ 0;
f is convex on D iff ∀x, y ∈ D,
(df (y) − df (x))(y − x) ≥ 0.
Case n = 1: A differentiable function f on an open interval D ⊂ R is concave iff f 0
is decreasing.
Convexity and definiteness
Framework 2 : The case of C 2 functions.
THEOREM 11. Let f : D ⊂ Rn → R be of class C 2 on the open convex set D. Then
1. f is concave on D iff
d2 f (x) is a negative semidefinite matrix for all x ∈ D.
2. f is convex on D iff
d2 f (x) is a positive semidefinite matrix for all x ∈ D.
3. f is strictly concave on D if
d2 f (x) is a negative definite matrix for all x ∈ D.
4. f is strictly convex on D if
d2 f (x) is a positive definite matrix for all x ∈ D.
4
Examples
• Q(x) = xAx where A is a symmetric matrix.
D2 Q(x) = 2A independently from x.
Concavity (convexity) of Q depends on positive (negative) semidefiniteness of A.
• h(x) = min{f (x), g(x)} is concave if f and g are too.
• f (x1 , . . . , xn ) = xa1 1 xa2 2 . . . xann , (ai > 0), is the Cobb-Douglas function and it is
P
– strictly concave if i ai < 1;
P
– concave if i ai ≤ 1.
Exercises
• Suppose g(x) is a concave (convex) function of one variable in the interval I and let
us consider f (x, y) = g(x) for all x ∈ I and all y in some interval J. Prove that
f (x, y) is concave (convex) for x ∈ I and y ∈ J. Suppose now that g(x) is strictly
concave (convex): will f (x, y) also be strictly concave (convex)?
• Given the function f (x, y) = (ln(x))a (ln(y))b defined for x > 1 and y > 1, where
a > 0, b > 0 and a + b < 1, prove that f (x, y) is strictly concave.
P
• The profit function Π(p) = supy∈Y i pi yi gives the measure of the maximum
profit which the firm can earn given prices p and technology Y . Prove that Π is a
convex function of p.
• Let f : Rn+ → R be defined by f (x1 , . . . , xn ) = log(xa1 . . . xan ) where a > 0. Study
the concavity of f .
5
Convexity and optimization
Local/global Theorem
THEOREM 12. Any local maximum (minimum) of a concave (convex) function is
a global maximum (minimum) of the function.
THEOREM 13. If f is concave on D then the set arg max{f (x) : x ∈ D} is
either empty or convex;
If f is convex on D then the set arg min{f (x) : x ∈ D} is
either empty or convex.
Strict concavity and uniqueness
THEOREM 14. If f is strictly concave on D then the set arg max{f (x) : x ∈ D} is
either empty or it contains a single point;
If f is strictly convex on D then the set arg min{f (x) : x ∈ D} is
either empty or it contains a single point.
Necessary and sufficient conditions (1)
C ASE 1: Unconstrained Optimization
THEOREM 15. x∗ is an interior maximum of a function f which is concave and differentiable in D, iff df (x∗ ) = 0;
x∗ is an interior minimum of a function f which is convex and differentiable in D, iff
df (x∗ ) = 0;
The concavity/convexity assumption
results in a
necessary and sufficient condition!
6
Necessary and sufficient conditions (2)
C ASE 2: Equality constraints
The second order conditions are always satisfied when the Lagrangian is concave, in
which case the stationarity is also sufficient for a global maximum.
COROLLARY 16. If (x∗ , λ) is a stationary point of the Lagrangian L(x, λ) and L(x, λ)
is concave in x, then x∗ is a global solution of the problem
max f (x) subject to g(x) = 0.
x∈D
In the particular case of concave objective function and affine constraints, the Lagrangian is always concave.
Necessary and sufficient conditions (3)
C ASE 3: Inequality constraints
THEOREM 17. Let f : D → R be a concave C 1 function, D open and convex. Let
gi , i = 1, . . . , m be concave and C 1 on D. Suppose there is some x̄ ∈ D such that
g(x̄) < b. Then x∗ is solution of
max f (x) subject to g(x) ≤ b
x∈D
iff there is λ∗ ∈ Rm such that
Pm
KT1. df (x∗ ) − j=1 λ∗j dgj (x∗ ) = 0;
KT2. λ∗j ≥ 0 and λ∗j (bj − gj (x∗ )) = 0 for all j.
Remark
The Theorem that gives second order sufficient conditions for GLOBAL optimality
under inequality constraints, follows as a corollary of the previous theorem.
7
Final remarks, examples
and exercises
1. Saddle points of the Lagrangian
Rem: In constrained optimization problems with equality constraints there is a link
between solution points and stationary points of the Lagrangian.
Is it possible to characterize (x∗ , λ∗ ) in relation with the solutions of optimization
problems with inequality constraints?
DEFINITION 18. Assume X ⊂ Rn and Y ≤ Rm . Let h : X × Y → R. The point
(x∗ , y ∗ ) is a saddle point of h if
h(x, y ∗ ) ≤ h(x∗ , y ∗ ) ≤ h(x∗ , y)
for all (x, y) ∈ X × Y .
THEOREM 19. Let (x∗ , λ∗ ) be a saddle point of L(x, λ) defined on D × Rm
+.
Then x∗ is a solution of
max f (x) subject to g(x) ≤ b.
x∈D
Moreover
λ∗j (bj − gj (x∗ )) = 0 ,
f or all j.
Rem: The theorem gives sufficient conditions for optimality without any regularity
condition on the functions involved.
Sketch of the proof
• L(x∗ , λ∗ ) ≤ L(x∗ , λ) for all λ ≥ 0 ⇒
λ∗ · (b − g(x∗ )) ≤ λ · (b − g(x∗ )).
(∗)
• (∗) implies λ · (b − g(x∗ )) ≥ 0.
• Put now λ = 0 in (∗) to obtain λ∗ · (b − g(vx∗ )) = 0.
• L(x, λ∗ ) ≤ L(x∗P
, λ∗ ) ⇒
m
∗
f (x ) ≥ f (x) + j=1 λ∗j (bj − gj (x∗ )) ⇒
f (x∗ ) ≥ f (x), ∀x ∈ D.
THEOREM 20. (x∗ , λ∗ ) is a saddle point of L(x, λ) iff
• gj (x∗ ) ≤ bj ;
λ∗j (bj − gj (x∗ )) = 0 ∀j;
Pm
• f (x∗ ) ≥ f (x) + j=1 λ∗j (bj − gj (x∗ )),
∀x ∈ D.
Rem: The hypothesis D ⊂ Rn is not necessary in the proof. The statement is also true
in case of not necessarily finite-dimensional linear space.
8
2. A generalization of concavity
Rem: concave and convex functions relax the additivity requirement of linearity and
dispense with homogeneity. Nevertheless this is too restrictive for many economic models,
so a further generalization is commonly accepted.
E XAMPLE: The utility function
αn
1
u(x1 , . . . , xn ) = xα
1 · · · xn ,
is not concave unless
P
i
αi ≤ 1.
Quasi convexity
DEFINITION 21. A function f : D ⊂ Rn → R where D is a convex set, is
• quasiconcave on D if every upper contour set Uf (a) = {x ∈ D : f (x) ≥ a} is a
convex set;
• quasiconvex on D if every lower contour set Lf (a) = {x ∈ D : f (x) ≤ a} is a
convex set.
Analogous definitions for strict quasiconcavity(convexity).
αn
1
E XAMPLE: u(x1 , . . . , xn ) = xα
1 · · · xn is quasiconcave.
Generalizations: Quasi convexity, cont’t
Rem: If f is quasiconcave then the function along the line joining any two points in
the domain lies above at least one of the endpoints.
E XAMPLES:
• The surface of a bell is quasiconcave.
• Any concave function is quasiconcave.
• A quasiconcave function is not necessarily concave.
• Any monotone function on D ⊂ R is both quasiconcave and quasiconvex.
• If f is quasiconcave and g is increasing then g ◦ f is quasiconcave.
9
Characterization of quasi convexity
THEOREM 22. f is a quasiconcave function on D iff ∀x, y, ∈ D, ∀λ ∈ [0, 1]:
f (λx + (1 − λ)y) ≥ min{f (x), f (y)}.
f is a quasiconvex function on D iff ∀x, y, ∈ D, ∀λ ∈ [0, 1]:
f (λx + (1 − λ)y) ≤ max{f (x), f (y)}.
Rem: quasiconcavity and quasiconvexity do not exhibit many of the regularity properties of concave and convex functions.
Quasi convexity and optimization
THEOREM 23. Let f and gj (j = 1, . . . , m) be C 1 quasiconcave functions mapping
U ⊂ Rn into R where U is open and convex. Define
D = U ∩ {x ∈ Rn | g(x) ≤ b}.
Suppose there exists x∗ ∈ D and λ∗ ∈ Rm such that
Pm
KT1. df (x∗ ) − j=1 λ∗j dgj (x∗ ) = 0;
KT2. λ∗j ≥ 0 and λ∗j (bj − gj (x∗ )) = 0 for all j.
Then x∗ maximizes f over D provided at least one of the following conditions holds:
QC1. df (x∗ ) 6= 0;
QC2. f is concave.
For more details see Sundaram.
10