APPENDIX
B
CONVEX AND
CONCAVE FUNCTIONS
Convex Function
A function of n variables ƒ(x) defined on a convex set D is said to be a convex
function if and only if for any two points x (1) and x (2) 僆 D and 0 ⭐ ␭ ⭐ 1,
ƒ[␭ x (1) ⫹ (1 ⫺ ␭)x (2)] ⭐ ␭ƒ(x (1)) ⫹ (1 ⫺ ␭)ƒ(x (2))
Figure B.1 illustrates the definition of a convex function of a single variable.
Properties of Convex Functions
1. The chord joining any two points on the curve always falls entirely on
or above the curve between those two points.
2. The slope or first derivative of ƒ(x) is increasing or at least nondecreasing as x increases.
3. The second derivative of ƒ(x) is always nonnegative for all x in the
interval.
4. The linear approximation of ƒ(x) at any point in the interval always
underestimates the true function value.
5. For a convex function, a local minimum is always a global minimum.
Figure B.2 illustrates property 4. The linear approximation of ƒ at the point
x 0, denoted by ƒ̃(x; x 0), is obtained by ignoring the second and other higher
order terms in the Taylor series expansion
ƒ̃(x; x 0) ⫽ ƒ(x 0) ⫹ ⵜƒ(x 0)(x ⫺ x 0)
For a convex function, property 4 implies that
648
Engineering Optimization: Methods and Applications, Second Edition. A. Ravindran, K. M. Ragsdell and
G. V. Reklaitis © 2006 John Wiley & Sons, Inc. ISBN: 978-0-471-55814-9
CONVEX AND CONCAVE FUNCTIONS
649
Figure B.1. Convex function.
ƒ(x) ⭓ ƒ(x 0) ⫹ ⵜƒ(x 0)(x ⫺ x 0)
for all x
The gradient of a function ƒ(x1, . . . , xn) is given by
ⵜƒ(x1, . . . , xn) ⫽
冋
␦ƒ ␦ƒ
␦ƒ
,
,...,
␦x1 ␦x2
␦xn
册
The Hessian matrix of a function ƒ(x1, . . . , xn) is an n ⫻ n symmetric
matrix given by
Hƒ(x1, . . . , xn) ⫽
冋 册
␦2ƒ
⫽ ⵜ2ƒ
␦xi ␦xj
Figure B.2. Linear approximation of a convex function.
650
CONVEX AND CONCAVE FUNCTIONS
Test for Convexity of a Function. A function ƒ is convex function if the
Hessian matrix of ƒ is positive definite or positive semidefinite for all values
of x1, . . . , xn.
Concave Function
A function ƒ(x) is a concave function in D if and only if ⫺ƒ(x) is a convex
function in D.
Test for Concavity of a Function. A function ƒ is concave if the Hessian
matrix of ƒ is negative definite or negative semidefinite for all values of x1,
. . . , xn.
Example B.1
ƒ(x1, x2, x3) ⫽ 3x 12 ⫹ 2x 22 ⫹ x 32 ⫺ 2x1x2 ⫺ 2x1x3
⫹ 2x2 x3 ⫺ 6x1 ⫺ 4x2 ⫺ 2x3
ⵜf(x1, x2, x3) ⫽
冢
冤
6x1 ⫺ 2x2 ⫺ 2x3 ⫺ 6
4x2 ⫺ 2x1 ⫹ 2x3 ⫺ 4
2x3 ⫺ 2x1 ⫹ 2x2 ⫺ 2
6 ⫺2
4
Hƒ(x1, x2, x3) ⫽ ⫺2
⫺2
2
冣
冥
⫺2
2
2
To show that ƒ is a convex function, we test for whether H is positive definite
or positive semidefinite. Note that:
1. H is symmetric.
2. All diagonal elements are positive.
3. The leading principal determinants are
兩6兩 ⬎ 0
冏
冏
6 ⫺2
⫽ 20 ⬎ 0
4
⫺2
兩Hƒ兩 ⫽ 16 ⬎ 0
Hence H is a positive-definite matrix, which implies ƒ is a convex function.
(As a matter of fact, when Hƒ is positive definite, ƒ is said to be strictly
convex with a unique minimum point.)