4. Design Space Exploration of
Embedded Systems
© Lothar Thiele
ETH Zurich, Switzerland
Swiss Federal
Institute of Technology
4-1
Computer Engineering
and Networks Laboratory
Contents of Lectures (Lothar Thiele)
1. Introduction to Embedded System Design
2. Software for Embedded Systems
3. Real-Time Scheduling
4. Design Space Exploration
5. Performance Analysis
Swiss Federal
Institute of Technology
4-2
Computer Engineering
and Networks Laboratory
Topics
Multi-Objective Optimization
! Introduction
! Multi-Objective Optimization
! Algorithm Design
•
•
•
•
Design Choices
Representation
Fitness and Selection
Variation Operators
! Implementation
Design Space Exploration
! System Design
! Problem Specification
! Example
Swiss Federal
Institute of Technology
4-3
Computer Engineering
and Networks Laboratory
Evolutionary Multiobjective Optimization Algorithms
What are Evolutionary Algorithms?
randomized, problem-independent search heuristics
→ applicable to black-box optimization problems
How do they work?
by iteratively improving a population of solutions by
variation and selection
→ can find many different optimal solution in a single run
Swiss Federal
Institute of Technology
4-4
Computer Engineering
and Networks Laboratory
Black-Box Optimization
objective function
Decision-Module
Stretch-Module
Handling-Module
DX
100
ψ
lane
DV
lane
v, a
a require
α Br
α Dr
v
v=f(t)
decision
vector x
t
lane
x, v, a
point of
gear
change
n
gear
2
4
R
?
α Dr
gear
1
3
5
Vehicle-Module
gear
s clutch
objective
vector f(x)
lane, x, v
a,gear, n
(e.g. simulation model)
Optimization Algorithm:
only allowed to evaluate f
(direct search)
Swiss Federal
Institute of Technology
4-5
Computer Engineering
and Networks Laboratory
The Knapsack Problem
weight = 750g
profit = 5
weight =
1500g
profit = 8
weight = 300g
profit = 7
weight =
1000g
profit = 3
?
Goal: choose subset that
•
maximizes overall profit
•
minimizes total weight
Swiss Federal
Institute of Technology
4-6
Computer Engineering
and Networks Laboratory
The Solution Space
profit
20
15
10
5
weight
500g
Swiss Federal
Institute of Technology
1000g
1500g
2000g
4-7
2500g
3000g
3500g
Computer Engineering
and Networks Laboratory
The Trade-off Front
Observations: " there is no single optimal solution, but
# some solutions ( ) are better than others ( )
profit
selecting a
solution
20
15
finding the good
solutions
10
5
weight
500g
Swiss Federal
Institute of Technology
1000g
1500g
2000g
4-8
2500g
3000g
3500g
Computer Engineering
and Networks Laboratory
Decision Making: Selecting a Solution
Approaches:
profit more important than cost (ranking)
weight must not exceed 2400g (constraint)
profit
20
15
too heavy
10
5
weight
500g
Swiss Federal
Institute of Technology
1000g
1500g
2000g
4-9
2500g
3000g
3500g
Computer Engineering
and Networks Laboratory
Optimization Alternatives
Use of classical single objective optimization methods
simulated annealing, tabu search
integer linear program
other constructive or iterative heuristic methods
Decision making (weighting the different objectives) is
done before the optimization.
Population based optimization methods
evolutionary algorithms
genetic algorithms
Decision making is done after the optimization.
Swiss Federal
Institute of Technology
4 - 10
Computer Engineering
and Networks Laboratory
Optimization Alternatives
scalarization
weighted sum
population-based
SPEA2
y2
y2
y1
y1
parameter-oriented
scaling-dependent
Swiss Federal
Institute of Technology
set-oriented
scaling-independent
4 - 11
Computer Engineering
and Networks Laboratory
Scalarization Approach
multiple
objectives
(y1, y2, …, yk)
parameters
transformation
example: weighting approach
y2
single
objective
y
maximization problem
(w1, w2, …, wk)
y = w1y1 + … + wkyk
y1
Swiss Federal
Institute of Technology
4 - 12
Computer Engineering
and Networks Laboratory
A Generic Multiobjective EA
population
archive
evaluate
sample
vary
update
truncate
new population
new archive
Swiss Federal
Institute of Technology
4 - 13
Computer Engineering
and Networks Laboratory
An Evolutionary Algorithm in Action
max. y2
hypothetical trade-off front
min. y1
Swiss Federal
Institute of Technology
4 - 14
Computer Engineering
and Networks Laboratory
Design Space Exploration
Specification
Specification
Optimization
Optimization
Evaluation
Evaluation
power
consumption
Implementation
Implementation
latency
cost
Swiss Federal
Institute of Technology
4 - 15
Computer Engineering
and Networks Laboratory
Packet Processing in Networks
©UCB Rabaey
Embedded
Embedded
Internet
Internet
Devices
Devices
method (a)(fsd)
for I=1 to n
do nothing
call comm(a,dsf,*e);
end for
Wearable
WearableComputing
Computing
Core
Access
Mobile
MobileInternet
Internet
Swiss Federal
Institute of Technology
4 - 16
Computer Engineering
and Networks Laboratory
Network Processors
Network processor = high-performance, programmable device
designed to efficiently execute communication
workloads [Crowley et al.: 2003]
incoming flows
(packet streams)
routing / forwarding
transcoding
encryption / decryption
outgoing flows
(processed packets)
real-time flows
network
processor
(NP)
e.g., voice
e.g., sftp
non-real-time flows
Swiss Federal
Institute of Technology
?
4 - 17
Computer Engineering
and Networks Laboratory
Optimization Scenario: Overview
Given:
Sought:
" specification of the task structure (task model) =
for each flow the corresponding tasks to be executed
# different usage scenarios (flow model) =
sets of flows with different characteristics
network processor implementation (resource model)
= architecture + task mapping + scheduling
Objectives: " maximize performance
# minimize cost
Subject to: " memory constraint
(performance model)
# delay constraints
Swiss Federal
Institute of Technology
4 - 18
Computer Engineering
and Networks Laboratory
Topics
Multi-Objective Optimization
Introduction
Multi-Objective Optimization
Algorithm Design
Design Choices
Representation
Fitness and Selection
Variation Operators
Implementation
Design Space Exploration
System Design
Problem Specification
Example
Swiss Federal
Institute of Technology
4 - 19
Computer Engineering
and Networks Laboratory
Dominance, Pareto Points
A (design) point Jk is dominated by Ji, if Ji is
better or equal than Jk in all criteria and
better in at least one criterion.
A point is Pareto-optimal or a Pareto-point, if it is not
dominated.
The domination relation imposes a partial order on all
design points
We are faced with a set of optimal solutions.
Divergence of solutions vs. convergence.
Swiss Federal
Institute of Technology
4 - 20
Computer Engineering
and Networks Laboratory
Multi-objective Optimization
Swiss Federal
Institute of Technology
4 - 21
Computer Engineering
and Networks Laboratory
Multi-objective Optimization
Maximize (y1, y2, …, yk) = ƒ(x1, x2, …, xn)
y2
y2
Pareto optimal = not dominated
better
incomparable
worse
dominated
incomparable
y1
y1
Pareto set = set of all Pareto-optimal solutions
Swiss Federal
Institute of Technology
4 - 22
Computer Engineering
and Networks Laboratory
Randomized (Black Box) Search Algorithms
Idea:
find good solutions without investigating all solutions
Assumptions:better solutions can be found in the neighborhood
of good solutions
information available only by function evaluations
Randomized
search algorithm
t = 1:
(randomly) choose a
solution x1 to start with
Swiss Federal
Institute of Technology
ƒ
4 - 23
t ≥ t+1:
(randomly) choose a
solution xt+1 using solutions
x1, …, xt
Computer Engineering
and Networks Laboratory
Types of Randomized Search Algorithms
memory
selection
variation
mating
selection
environmental
selection
≥1
EA
both
evolutionary algorithm
TS
1
no mating selection
1
no mating selection
tabu search
SA
simulated annealing
Swiss Federal
Institute of Technology
4 - 24
Computer Engineering
and Networks Laboratory
Limitations of Randomized Search Algorithms
The No-Free-Lunch Theorem
All search algorithms provide in average the same
performance on a all possible functions
with finite search and objective spaces.
[Wolpert, McReady: 1997]
Remarks:
Not all functions equally likely and realistic
We cannot expect to design the algorithm beating all others
Ongoing research: which algorithm suited for which class of
problem?
Swiss Federal
Institute of Technology
4 - 25
Computer Engineering
and Networks Laboratory
Topics
Multi-Objective Optimization
Introduction
Multi-Objective Optimization
Algorithm Design
Design Choices
Representation
Fitness and Selection
Variation Operators
Implementation
Design Space Exploration
System Design
Problem Specification
Example
Swiss Federal
Institute of Technology
4 - 26
Computer Engineering
and Networks Laboratory
Design Choices
representation
fitness assignment
0011
mating selection
0111
parameters
0011
0100
0000
1011
environmental selection
Swiss Federal
Institute of Technology
0011
variation operators
4 - 27
Computer Engineering
and Networks Laboratory
Issues in Multi-Objective Optimization
y2
How to maintain a diverse
Pareto set approximation?
Diversity
# density estimation
How to prevent nondominated
solutions from being lost?
$ environmental selection
y1
How to guide the population
towards the Pareto set?
Convergence
" fitness assignment
Swiss Federal
Institute of Technology
4 - 28
Computer Engineering
and Networks Laboratory
Comparison of Three Implementations
2-objective knapsack problem
SPEA2
VEGA
extended
VEGA
Trade-off between
distance and diversity?
Swiss Federal
Institute of Technology
4 - 29
Computer Engineering
and Networks Laboratory
Topics
Multi-Objective Optimization
Introduction
Multi-Objective Optimization
Algorithm Design
Design Choices
Representation
Fitness
Variation Operators
Implementation
Design Space Exploration
System Design
Problem Specification
Example
Swiss Federal
Institute of Technology
4 - 30
Computer Engineering
and Networks Laboratory
Representation
search space
decoder
solution space
1 0 1 1 1 0
objective space
ƒ
m
1 0 1 1 1 0
objectives
1 0 1 1 1 0
solutions encoded by vectors, matrices, trees, lists, ...
fixed length
variable length
Issues:
completeness (each solution has an encoding)
uniformity (all solutions are represented equally)
redundancy (cardinality of search space vs. solution space)
feasibility (each encoding maps to a feasible solution)
Swiss Federal
Institute of Technology
4 - 31
Computer Engineering
and Networks Laboratory
Example: Binary Vector Encoding
Given: graph
Goal: find minimum subset of nodes such that each edge
is connected to at least one node of this subset
(minimum vertex cover)
A B C D E
nodes A B C D E F …
selected? 1 0 1 1 1 0 …
Swiss Federal
Institute of Technology
4 - 32
Computer Engineering
and Networks Laboratory
Example: Integer Vector Encoding
Given: graph, k colors
Goal: assign each node one of the k colors such that the
number of connected nodes with the same color is
minimized (graph coloring problem)
A B C D E
nodes A B C D E F …
colors 1 2 1 3 1 2 …
Swiss Federal
Institute of Technology
4 - 33
Computer Engineering
and Networks Laboratory
Example: Real Vector Encoding
parameters
values
x1
x2
x3
x4
…
0.33 0.53 1.03 3.25 …
xn
9.83
[Michalewicz, Fogel: How to Solve it. Springer 2000]
Swiss Federal
Institute of Technology
4 - 34
Computer Engineering
and Networks Laboratory
Tree Example: Parking a Truck
steering
angle
u
cab
d
dock
trailer
t
position (x,y)
constant speed
Swiss Federal
Institute of Technology
4 - 35
Goal:
find function c with
u = c(x, y, d, t)
Computer Engineering
and Networks Laboratory
Search Space for the Truck Problem
Operators:
Arguments:
X
position x
Y
position y
DIFF
cab angle d
TANG
trailer angle t
Search space : set of symbolic expression using the above
operators and arguments
Swiss Federal
Institute of Technology
4 - 36
Computer Engineering
and Networks Laboratory
Example Solution: Tree Representation
MULT
MINUS
PLUS
X
Y
DIFF
TANG
encodes the function (symbolic expression): u = (x – d) * (y + t)
Swiss Federal
Institute of Technology
4 - 37
Computer Engineering
and Networks Laboratory
A Solution Found by an EA
truck simulation
Swiss Federal
Institute of Technology
encoded tree
4 - 38
Computer Engineering
and Networks Laboratory
Topics
Multi-Objective Optimization
Introduction
Multi-Objective Optimization
Algorithm Design
Design Choices
Representation
Fitness and Selection
Variation Operators
Implementation
Design Space Exploration
System Design
Problem Specification
Example
Swiss Federal
Institute of Technology
4 - 39
Computer Engineering
and Networks Laboratory
Fitness Assignment
Fitness = scalar value representing quality of an individual (usually).
Used for mating and environmental selection
Fi = f ( m (i) )
Difficulties:
multiple objectives have to be considered (Pareto set is sought)
multiple optima need to be approximated (how to consider
diversity?)
constraints are involved which have to be met
Swiss Federal
Institute of Technology
4 - 40
Computer Engineering
and Networks Laboratory
Example Pareto Ranking
Fitness function:
execution time
F (1) = 3
2
F ( 2) = 1
6
1
F ( 3) = 1
3
F ( 4) = 2
F (5) = 1
4
F (6) = 0
5
cost
Swiss Federal
Institute of Technology
4 - 41
Computer Engineering
and Networks Laboratory
Constraint Handling
feasible
Constraint = g(x1, x2, …, xn) ≥ 0
g≥0
infeasible
g<0
Approaches:
construct initialization and variation such that infeasible
solutions are not generated (resp. not inserted)
representation is such that decoding always yields a feasible
solution
calculate constraint violation (- g(x1, x2, …, xn) ) and incorporate it
into fitness, e.g., Fi = f ( m (i) ) - g(x1, x2, …, xn) (fitness to be
maximized)
code constraint as a new objective
Swiss Federal
Institute of Technology
4 - 42
Computer Engineering
and Networks Laboratory
Selection
Two conflicting goals:
trade-off
exploitation
(converge fast)
exploration
(avoid getting stuck)
Two types of selection:
mating selection = select for variation
environmental selection = select for survival
Swiss Federal
Institute of Technology
4 - 43
Computer Engineering
and Networks Laboratory
Example: Tournament Selection
= integrated sampling rate assignment and sampling
population
mating pool
"
uniformly choose
T individuals at
random independently
of fitness
#
compare fitness
and copy best
individual
in mating pool
T = tournament size (binary tournament selection means T=2)
Swiss Federal
Institute of Technology
4 - 44
Computer Engineering
and Networks Laboratory
Topics
Multi-Objective Optimization
Introduction
Multi-Objective Optimization
Algorithm Design
Design Choices
Representation
Fitness and Selection
Variation Operators
Implementation
Design Space Exploration
System Design
Problem Specification
Example
Swiss Federal
Institute of Technology
4 - 45
Computer Engineering
and Networks Laboratory
Example: Vector Mutation
Bit vectors:
1 0 1 1 1 0
each bit is flipped with probability 1/6
1 0 0 1 1 0
Permutations:
1 2 3 4 5 6
1 4 3 2 5 6
swap
1 2 3 4 5 6
1 3 4 2 5 6
rearrange
Swiss Federal
Institute of Technology
4 - 46
Computer Engineering
and Networks Laboratory
Example: Vector Recombination
Bit vectors:
1 1 0 0 1 0
1 1 0 0 0 1
1 0 1 0 0 1
Permutations:
child
1 2 3 4 5 6
1 2 3 6 4 5
parents
6 2 3 4 1 5
Swiss Federal
Institute of Technology
4 - 47
Computer Engineering
and Networks Laboratory
Example: Recombination of Trees
PLUS
MULT
MINUS
MINUS
X
PLUS
DIFF
Y
TANG
MINUS
TANG
PLUS
DIFF
Y
MULT
exchange
Swiss Federal
Institute of Technology
4 - 48
Y
DIFF
TANG
Computer Engineering
and Networks Laboratory
Example: SPEA2 Algorithm
Step 1:
Step 2:
Step 3:
Step 4:
Step 5:
Step 6:
Generate initial population P0 and empty archive
(external set) A0. Set t = 0.
Calculate fitness values of individuals in Pt and At.
At+1 = nondominated individuals in Pt and At.
If size of At+1 > N then reduce At+1, else if
size of At+1 < N then fill At+1 with dominated
individuals in Pt and At.
If t > T then output the nondominated set of At+1.
Stop.
Fill mating pool by binary tournament selection.
Apply recombination and mutation operators to
the mating pool and set Pt+1 to the resulting
population. Set t = t + 1 and go to Step 2.
Swiss Federal
Institute of Technology
4 - 49
Computer Engineering
and Networks Laboratory
SPEA2 Fitness Assignment
Idea (Step 2): calculate dominance rank weighted by
dominance count
y2
non-dominated solutions:
F = #dominated solutions
2
4+2
dominated solutions
F = # of non-Pareto solutions +
∑ strengths of dominators
2
3
4+2+3
4+3
4+2+2+3
y1
Note: lower raw fitness = better quality
Swiss Federal
Institute of Technology
4 - 50
Computer Engineering
and Networks Laboratory
Topics
Multi-Objective Optimization
Introduction
Multi-Objective Optimization
Algorithm Design
Design Choices
Representation
Fitness and Selection
Variation Operators
Implementation
Design Space Exploration
System Design
Problem Specification
Example
Swiss Federal
Institute of Technology
4 - 51
Computer Engineering
and Networks Laboratory
What Is Needed…
A framework that
Provides ready-to-use modules (algorithms / applications)
Is simple to use
Is independent of programming language and OS
Comes with minimum overhead
Idea: separate problem-dependent from problemindependent part
cut
Selection
Fitness assignment
Representation
Recombination
Archiving
Swiss Federal
Institute of Technology
Objective functions
Mutation
4 - 52
Computer Engineering
and Networks Laboratory
PISA: Implementation
shared
shared
file
file
system
system
selector
process
application independent:
mating / environmental
selection
individuals are
described
by IDs and objective
vectors
Swiss Federal
Institute of Technology
text
files
variator
process
handshake protocol: application dependent:
state / action
variation operators
individual IDs
stores and manages
individuals
objective vectors
parameters
4 - 53
Computer Engineering
and Networks Laboratory
PISA Installation
Swiss Federal
Institute of Technology
4 - 54
Computer Engineering
and Networks Laboratory
Topics
Multi-Objective Optimization
Introduction
Multi-Objective Optimization
Algorithm Design
Design Choices
Representation
Fitness and Selection
Variation Operators
Implementation
Design Space Exploration
System Design
Problem Specification
Example
Swiss Federal
Institute of Technology
4 - 55
Computer Engineering
and Networks Laboratory
Design Space Exploration
Application
Architecture
Mapping
Estimation
Swiss Federal
Institute of Technology
4 - 56
Computer Engineering
and Networks Laboratory
Embedded System Design
architecture
template
task
graph
binding
restrictions
evaluation
allocation
bindings
performance /
cost vector
construct
architecture
map
application
estimate
performance
multiobjective
optimization
architecture
Swiss Federal
Institute of Technology
4 - 57
performance
Computer Engineering
and Networks Laboratory
Example: System Synthesis
Given:
algorithm
mappings
architectures
f 1, f 2, K
Goal:
schedule
1
mapping
architecture
3
2
4
5
Objectives: cost, latency, power consumption
Swiss Federal
Institute of Technology
4 - 58
Computer Engineering
and Networks Laboratory
Evolutionary Algorithms for DSE
individual
EA
allocation
" selection
# recombination
$ mutation
binding
decode allocation
decode binding
scheduling
“chromosome” = encoded
allocation + binding
design point
(implementation)
fitness evaluation
fitness
user constraints
Swiss Federal
Institute of Technology
4 - 59
Computer Engineering
and Networks Laboratory
Topics
Multi-Objective Optimization
Introduction
Multi-Objective Optimization
Algorithm Design
Design Choices
Representation
Fitness and Selection
Variation Operators
Implementation
Design Space Exploration
System Design
Problem Specification
Example
Swiss Federal
Institute of Technology
4 - 60
Computer Engineering
and Networks Laboratory
Basic Model – Problem Graph
Problem graph GP(VP,EP):
2
1
Interpretation:
• VP consists of functional
nodes VPf (task, procedure) and communication
nodes VPc .
6
5
3
• EP represent data dependencies
7
4
Swiss Federal
Institute of Technology
4 - 61
Computer Engineering
and Networks Laboratory
Basic model – architecture graph
Architecture graph GA(VA,EA):
HWM1
RISC
RISC
shared
bus
HWM1
PTP bus
shared bus
HWM2
PTP bus
HWM2
Architecture graph
Architecture
• VA consists of functional resources VAf (RISC, ASIC) and
bus resources VAc. These components are potentially allocatable.
• EA model directed communication.
Swiss Federal
Institute of Technology
4 - 62
Computer Engineering
and Networks Laboratory
Basic model – specification graph
Definition: A specification graph is a graph
GS=(VS,ES) consisting
of a problem graph GP,
an architecture graph
GA, and edges EM. In
particular, VS=VP∪VA,
ES=EP∪EA∪EM
1
5
RISC
3
SB
7
HWM1
2
PTP
6
HWM2
4
GP
Swiss Federal
Institute of Technology
4 - 63
EM
GA
Computer Engineering
and Networks Laboratory
Basic model - synthesis
Three main tasks of synthesis:
• Allocation α is a subset of VA.
• Binding β is a subset of EM, i.e., a mapping of functional
nodes of VP onto resource nodes of VA.
• Schedule τ is a function that assigns a number (start time) to
each functional node.
Swiss Federal
Institute of Technology
4 - 64
Computer Engineering
and Networks Laboratory
Basic model - implementation
0
Definition: Given a
specification graph GS
an implementation is a
triple (α,β,τ), where α
is a feasible allocation,
β is a feasible binding,
and τ is a schedule.
1
1
0
1
21
29
RISC
5
3
7
8
SB
1
HWM1
20
1
α
2
1
21
β
6
RISC
HWM1
2
30
shared
4
bus
τ
Swiss Federal
Institute of Technology
4 - 65
PTP bus
HWM2
Computer Engineering
and Networks Laboratory
Example
P
Swiss Federal
Institute of Technology
4 - 66
Computer Engineering
and Networks Laboratory
Challenges
Encoding of (allocation+binding)
simple encoding
eg. one bit per resource, one variable per binding
easy to implement
many infeasible partitionings
encoding + repair
eg. simple encoding and modify such that for each vp ∈ VP there
exists at least one va ∈ VA with a β(vp) = va
reduces number of infeasible partitionings
Generation of the initial population, mutation
Recombination
Swiss Federal
Institute of Technology
4 - 67
Computer Engineering
and Networks Laboratory
Topics
Multi-Objective Optimization
Introduction
Multi-Objective Optimization
Algorithm Design
Design Choices
Representation
Fitness and Selection
Variation Operators
Implementation
Design Space Exploration
System Design
Problem Specification
Example
Swiss Federal
Institute of Technology
4 - 68
Computer Engineering
and Networks Laboratory
Exploration - Case Study (1)
Swiss Federal
Institute of Technology
4 - 69
Computer Engineering
and Networks Laboratory
Exploration - Case Study (2)
Swiss Federal
Institute of Technology
4 - 70
Computer Engineering
and Networks Laboratory
Exploration - Case Study (3)
frame memory
dual ported frame memory
block matching module
output module
subtract/add module
DCT/IDCT module
Swiss Federal
Institute of Technology
input module
4 - 71
Huffman encoder
Computer Engineering
and Networks Laboratory
EA Case Study - Design Space
350
300
250
power consumption 200
150
1000
200
50
300
100
latency
400
cost
150
500
200
Swiss Federal
Institute of Technology
4 - 72
Computer Engineering
and Networks Laboratory
Exploration Case Study - Solution 1
INM
OUTM
FM
RISC2
SBS
Swiss Federal
Institute of Technology
4 - 73
Computer Engineering
and Networks Laboratory
Exploration Case Study - Solution 2
INM
OUTM
DPFM
HC
DCTM
BMM
SAM
SBF
Swiss Federal
Institute of Technology
4 - 74
Computer Engineering
and Networks Laboratory