Optimization
Daniel A. Menasce
Department of Computer Science
The Volgenau School of IT&E
1
© 2007 D.A. Menasce. All Rights Reserved.
Basic Problem Statement
Minimize f (x1,..., x n )
Subject to
g1 (x1,..., x n ) " 0
...
gm (x1,..., x n ) = 0
Objective or cost function
constraints
!
Alternatively, one may want to maximize a function subject to
constraints.
2
© 2007 D.A. Menasce. All Rights Reserved.
1
Types of Optimization Problems
Minimize f (x1,..., x n )
Subject to
g1(x1,..., x n ) op K1
...
gm (x1,..., x n ) op K m
where
!
constraints
If the objective function and all
constraint functions are linear,
we have a Linear Programming
(LP) problem.
An LP in which the variables
can only take integer values is
an Integer Programming
Problem.
op " {<,#,>,$ }
!
A LP in which some of the variables are restricted to be
integers is called a mixed Integer Programming Problem.
If the objective function and/or constrains are non linear we
have a non-linear programming problem (NLP).
3
© 2007 D.A. Menasce. All Rights Reserved.
Example of a NLP
What is the number n in a cluster that maximizes the throughput
X(n) = n*X subject to a) cost C(n)=n*Co+C1 ≤Cmax and b) the
probability Pall-up that all servers are up must be at least 95%.
Maximize X(n) = n * X
Subject to
n*Co + C1 - $6,000 ≤ 0
Pall-up (n) = pn ≥ 0.75
4
© 2007 D.A. Menasce. All Rights Reserved.
2
Excel Solver
Uses the Generalized Reduced Gradient (GRG2) algorithm for
optimizing nonlinear problems.
objective
function
variables
constraints
5
© 2007 D.A. Menasce. All Rights Reserved.
Adding a Constraint
6
© 2007 D.A. Menasce. All Rights Reserved.
3
Solution
Throughput of an individual server
Co
$
500.00
C1
$ 1,000.00
Cmax
$ 6,000.00
Pfailure
0.05
Min-Prob-Up
0.75
Objective function
n
Cost (n)
Prob-all-up
10 tps
50 tps
5
$ 3,500.00
0.77
7
© 2007 D.A. Menasce. All Rights Reserved.
Example of an Optimization
Problem
f(x,y)
y
Maximize f (x,y)
0<x ≤ 4
x+2*y > 3
x
Problem of finding local optimum and not global optimum 8
© 2007 D.A. Menasce. All Rights Reserved.
4
Grid Resource Allocation and
SLAs
• A computation requires NC million CPU
cycles and has to be completed in at most
Tmax time units and its computational cost
cannot exceed Cmax dollars.
• There are three available computing
resources with speed si in 106 cycles/sec and
cost ci in dollars/sec.
• Possible co-allocations: (1), (2), (3), (1,2),
(1,3), (1,3), and (1,2,3)
9
© 2007 D.A. Menasce. All Rights Reserved.
Grid Resource Allocation and
SLAs
• N: number of computing
resources
• NCi: number of cycles
allocated to computing
resource i.
• T: execution time of a
given allocation
• C: cost of an allocation.
Constraints:
N
) NC
i
= NC
i =1
N ( NC %
i
T = max '
$ ! Tmax
i =1
s
& i #
N
NCi
C=)
" ci ! Cmax
s
i =1
i
10
© 2007 D.A. Menasce. All Rights Reserved.
5
Grid Resource Allocation and
SLAs
• If there are no cost
constraints, the
solution that
minimizes the total
execution time is the
one that allocates
more cycles to the
faster resources in
proportion to its
speed:
si
NCi = NC "
N
!s
j
j =1
11
© 2007 D.A. Menasce. All Rights Reserved.
Grid Resource Allocation
Optimization Problems
• Problem 1 (execution time minimization): “Find
the feasible solution that satisfies the cost
constraint at minimum execution time.”
• Problem 2 (cost minimization): “Find the feasible
solution that minimizes the cost C and that
satisfies the execution time constraint.”
12
© 2007 D.A. Menasce. All Rights Reserved.
6
Execution Time Minimization
Inputs:
Node
1
2
3
speed (si)
Mcycles/sec
1,000
2,000
3,000
cost/sec
(ci) $/sec
0.1
0.25
0.6
NC
Tmax
Cmax
$
$
10,000,000
4,800
1,500
Mcycles
sec
Solution: Obtained using MS Excel’s Solver Tool.
Allocation
1
2
3
1,2
1,3
2,3
1,2,3
Cycle Allocation (NCi)
10,000,000
10,000,000
10,000,000
3,333,333
6,666,667
0
4,800,000
0
5,200,000
0
6,666,667
3,333,333
2,592,592.41 5,185,185.39 2,222,222.20
T
10,000
5,000
3,333
3,333
4,800
3,333
2,593
C
1,000
1,250
2,000
1,167
1,520
1,500
1,352
13
© 2007 D.A. Menasce. All Rights Reserved.
14
© 2007 D.A. Menasce. All Rights Reserved.
7
Cost Minimization
Inputs:
Node
1
2
3
speed (si)
Mcycles/sec
1,000
2,000
3,000
cost/sec
(ci) $/sec
0.1
0.25
0.6
NC
Tmax
Cmax
$
$
10,000,000
4,800
1,500
Mcycles
sec
Solution: Obtained using MS Excel’s Solver Tool.
1
2
3
1,2
1,3
2,3
1,2,3
Cycle Allocation (NCi)
10,000,000
10,000,000
10,000,000
4,800,000
5,200,000
0
5,000,000
0
5,000,000
0
9,600,000
400,000
4,800,000
5,200,000
-
T
10,000
5,000
3,333
4,800
5,000
4,800
4,800
C
1,000
1,250
2,000
1,130
1,500
1,280
1,130
15
© 2007 D.A. Menasce. All Rights Reserved.
Optimization Example in Class
• A computer system has ND disks on which
NF files need to be stored. The capacity of
disk i is Ci bytes, the size of file j is Sj, the
arrival rate of requests to file j is λj, and the
average service time of a request to file j
when it is stored on disk i is si,j.
• Assume ND = 2. Find the optimal allocation of
files to disks that minimizes the difference in
utilization between the two disks.
16
© 2007 D.A. Menasce. All Rights Reserved.
8
Optimization Example in Class
minimize U1 " U 2
subject to
NF
U i = % a j,i # $ j # si, j utilization of disk i
j=1
NF
%a
j,i
# S j & Ci disk capacity constraint
j=1
0 & Ui < 1
utilization of disk i constraint
a j,i ' {0,1}
0 indicates that file j is not stored on disk i.
17
!
© 2007 D.A. Menasce. All Rights Reserved.
Optimization Example in Class
File number
Lamba
(req/sec)
Size (GB)
1
2
3
4
5
Disk capacity (in GB)
1
2
5
20
12
14
7
20
30
15
9
10
Service time Service time
on disk 1
on disk 2
(sec)
(sec)
0.010
0.014
0.008
0.010
0.015
0.017
0.020
0.022
0.025
0.030
30
35
18
© 2007 D.A. Menasce. All Rights Reserved.
9
Combinatorial Search
• Space of solutions: vectors of the type
c = (c1, …, cn)
• A cost function
is associated with each
r
vector: C(c )
• Number of possible valuesn of ci: |ci|
• Size of the search space: " c i
r
i=1
C(
c
)
•
Search:
find
point
c*
that
optimizes
!
• If the size of the search space is too large, an
exhaustive search !is not feasible: need to use
a heuristic approach.
!
19
© 2007 D.A. Menasce. All Rights Reserved.
Heuristic Combinatorial
Search Techniques
• Purpose:
– Reduce significantly the number of points
visited in the solution space
– Find a point whose cost is very close to the
optimal solution.
• Two examples of heuristic search
techniques:
– Hill climbing
– Beam search
20
© 2007 D.A. Menasce. All Rights Reserved.
10
Hill Climbing Search
Set initial point: c ← co;
Initialize iteration counter: i ← 0;
ThereIsLargestNeighbor ← True
While (i < MaxIterations &
ThereIsLargestNeighbor)
If there is a neighbor c’ of c with higher cost than c
then c ← c’ else ThereIsLargestNeighbor ← False;
i←i+1
EndWhile
Solution is point c
21
© 2007 D.A. Menasce. All Rights Reserved.
Hill-Climbing Search
C
8
B
11
J
12
A
10
D
E
F
15
G
17
13
I
25
30
H
27
22
© 2007 D.A. Menasce. All Rights Reserved.
11
Hill-Climbing Search
C
8
B
11
J
A
10
D
E
F
13
15
12
G
I
30
25
H
17
27
23
© 2007 D.A. Menasce. All Rights Reserved.
Hill-Climbing Search
C
8
B
11
J
12
A
10
D
E
F
15
G
17
13
I
25
30
H
27
24
© 2007 D.A. Menasce. All Rights Reserved.
12
(k,d) Beam Search
25
© 2007 D.A. Menasce. All Rights Reserved.
(k,d) Beam Search
26
© 2007 D.A. Menasce. All Rights Reserved.
13
(2,4) Beam Search
level
1
10
2
15
11
12
18
3
16
22
17
18
40
28
31
27
25
21
19
20
4
29
26
39
32
27
©©
2007
D.A.
All Rights
Reserved.
2003
D. Menasce.
A. Menascé.
All Rights
Reserved.
Applications of Combinatorial
Search Techniques
• Autonomic Computing (self-configuring, selforganizing)
– Search space: space of all possible configurations
of software and hardware parameters of a
computer system.
– Associated with each point, a QoS function (using
a performance model) computes the QoS for that
configuration.
– Goal: find the configuration that optimizes the QoS
of the system.
28
© 2007 D.A. Menasce. All Rights Reserved.
14
Example: optimal selection of CPU
share in VMs
state
(30,45,25)
(25,50,25)
(25,60,15)
(25,45,30)
(35,30,35)
(15,60,25)
(15,50,35)
(35,15,50)
(35,25,40)
(15,70,15)
(15,45,40)
(30,25,45)
29
© 2007 D.A. Menasce. All Rights Reserved.
Example: optimal selection of CPU
share in VMs
state
(30,45,25)
0.2
0.5
(35,30,35)
utility
0.3
0.8
(35,25,40)
0.6
(25,50,25)
(15,60,25)
(15,70,15)
0.9
0.6
0.1
0.95
(25,60,15)
(15,50,35)
(15,45,40)
-0.2
-0.1
0.5
(25,45,30)
(35,15,50)
(30,25,45)
30
© 2007 D.A. Menasce. All Rights Reserved.
15
Example: optimal selection of CPU
share in VMs
state
(30,45,25)
0.2
0.5
(35,30,35)
utility
0.3
0.8
(35,25,40)
0.6
(25,50,25)
(15,60,25)
(15,70,15)
0.9
0.6
0.1
0.95
(25,60,15)
(15,50,35)
(15,45,40)
-0.2
-0.1
0.5
(25,45,30)
(35,15,50)
(30,25,45)
31
© 2007 D.A. Menasce. All Rights Reserved.
Example: optimal selection of CPU
share in VMs
state
(30,45,25)
0.2
0.5
(35,30,35)
utility
0.3
0.8
(35,25,40)
0.6
(25,50,25)
(15,60,25)
(15,70,15)
0.9
0.6
0.1
0.95
(25,60,15)
(15,50,35)
(15,45,40)
-0.2
-0.1
0.5
(25,45,30)
(35,15,50)
(30,25,45)
32
© 2007 D.A. Menasce. All Rights Reserved.
16
Example: optimal selection of CPU
share in VMs
state
(30,45,25)
0.2
0.5
(35,30,35)
utility
0.3
0.8
(35,25,40)
0.6
(25,50,25)
(15,60,25)
(15,70,15)
0.9
0.6
0.1
0.95
(25,60,15)
(15,50,35)
(15,45,40)
-0.2
-0.1
0.5
(25,45,30)
(35,15,50)
(30,25,45)
33
© 2007 D.A. Menasce. All Rights Reserved.
Example: optimal selection of CPU
share in VMs
state
(30,45,25)
0.2
0.5
(35,30,35)
utility
0.3
0.8
(35,25,40)
0.6
(25,50,25)
(15,60,25)
(15,70,15)
0.9
0.6
0.1
0.95
(25,60,15)
(15,50,35)
(15,45,40)
-0.2
-0.1
0.5
(25,45,30)
(35,15,50)
(30,25,45)
34
© 2007 D.A. Menasce. All Rights Reserved.
17
Example: optimal selection of CPU
share in VMs
state
(30,45,25)
0.2
utility
0.3
(25,50,25)
0.6
(25,60,15)
-0.2
(25,45,30)
State: CPU share allocated to each VM:
(35,30,35)
(15,60,25(a1,
) …, aM)
(15,50,35)
(35,15,50)
0.8 at each state
0.1is computed-0.1
0.5 utility function
The
with the help of
a predictive queuing network model. The best state to
move to is determined using combinatorial search
(35,25,40)
(15,70,15
)
(15,45,40)
(30,25,45)
techniques.
0.95
0.5
0.6
0.9
35
© 2007 D.A. Menasce. All Rights Reserved.
18