Set-Based Design:
A Decision-Theoretic Perspective
Chris Paredis, Jason Aughenbaugh,
Rich Malak, Steve Rekuc
Systems Realization Laboratory
Systems Realization Laboratory
Product and Systems Lifecycle Management Center
G.W. Woodruff School of Mechanical Engineering
Georgia Institute of Technology
www.srl.gatech.edu
www.pslm.gatech.edu
Objectives
ƒ Make you familiar with the concepts of
Set-Based Design
ƒ Help you think about the characteristics of
set-based design in terms of Decision Theory
Systems Realization Laboratory
What is Set-Based Design?
Set-based Design
“Point-based” Design
x2
x2
x1
Marketing
Propose
Design
Analyze
& Critique
Engineering
x1
Eliminate Dominated Alternatives
Styling
Marketing
Styling
Modify
Manufacturing
Manufacturing
Engineering
Systems Realization Laboratory
Foundations by Ward, Sobek & Liker
ƒ Ward (1989):
• Mechanical Design Compiler
• Compile high-level description
into set of possible solutions
• Eliminate through labeled
interval propagation
• Eliminate only alternatives that
can be proven not to work
ƒ Sobek, Ward & Liker (1999):
• Case study: Toyota Production
system
• Engineers communicate in terms
of sets
• Multiple design alternatives are
developed in parallel
• Paradox: value despite apparent
"inefficiencies"
Systems Realization Laboratory
Simple Example: Design of Pneumatic System
(adapted from Ward, 1989)
ƒ Catalog of Components:
• 50 motors
• 30 compressors, …
ƒ Interval-Based Characterization of Components
• Motor: RPM (nom load) = [1740,1800]
• Cylinder: Force = [0,100] N
ƒ Design Requirements
• Load: Velocity = every [0,2] m/s
• Power-supply: 110V AC
ƒ Propagation of set-based requirements
• Yields relatively small set of feasible solutions
Motor
Compressor
Valve
Cylinder
Load
Systems Realization Laboratory
Some Short-Comings in Current State of the Art
ƒ Only algebraic equations
• No differential or partial differential equations
• No black boxes – equations need to be expressed symbolically
ƒ Only for catalog design – configuration of discrete options
• Significant extensions are needed to support continuous variables
ƒ Only pure intervals – no probabilities
• Ignoring probability information often leads to overly conservative designs
ƒ Weak on optimization beyond constraint satisfaction
• What if satisfying all the constraints still leaves many alternatives?
Systems Realization Laboratory
Need for a Strong Foundation
ƒ Our approach: Build on the foundation of Decision Theory
O11
O12
O1k
U (O11 )
U (O12 )
U (O1k )
A2
O21
O22
O2k
U (O21 )
U (O22 )
U (O2 k )
An
On1
On 2
Onk
U (On1 )
U (On 2 )
U (Onk )
A1
Decision
Select
Ai
where
E [U ]
is maximum
+
Normative Decision Theory
Reality of Design Context
• Bounded rationality
• Limited resources
• Incomplete knowledge
• Diverse information and
knowledge needs
• Collaboration among
geographically distributed
stakeholders
• …
Formal, Systematic but Practical Methods for Engineering Design
Formalisms
Representations
Methods
Tools
Systems Realization Laboratory
Trade-off Between Information Cost and Value
Information Economics
Benefit
Cost
Information is only valuable
to the extent that it leads to better decisions
No change in the decision Æ benefit of information is zero
Systems Realization Laboratory
Overview of Presentation
ƒ What is Set-Based Design?
ƒ Current Limitations of Set-Based Design
ƒ Set-Based Design from a Decision-Theoretic Perspective
•
•
•
•
Set-based design and sequential decision making
Expressing the utility of decision alternatives as intervals
Decision policy: eliminate non-dominated design alternatives
Searching through a set of non-dominated alternatives: Branch & Bound
ƒ Implication for Modeling and Simulation in Design
ƒ Implications for PLM
ƒ Conclusion
Systems Realization Laboratory
Decision 1
Sequential
decisions:
Vehicle type
decision alternatives
Decision 2
Set-Based Design and Sequential Decisions
Engine/motor type
decision alternatives
bike
car
gas
electric
diesel
Vehicle type decision alternatives
car
In the first decisions,
the designer chooses
from a set
Gas car
bike
Gas bike
Electric car
Diesel car
Electric bike
Diesel bike
Each decision alternative is a set of design alternatives
Æ Decision alternatives are imprecisely defined
Systems Realization Laboratory
Set-Based Decision Alternatives
Cost = ?
$ [400,1000]
Efficiency = ?
[25,30] %
SET OF
150 hp Gas Engines
Mass = ?
[100,300] kg
Reliability = ?
[95,99] %
Imprecise Alternative Æ Imprecise Performance
Systems Realization Laboratory
Imprecision and Variability Æ P-Box
Deterministic
0
Imprecise
cdf
1
1
1
cdf
1
0.5
X
0
0.4
0.5
0.5
X
0
Precise Distribution
-
0
-3
0.6
Precise Scalar
1
2
0
1
2
3
-
1
1
1
cdf
Precise
cdf
1
Probabilistic
0.5
0.5
-
0
0
-1
0
1
P-box of interval
2
X
0
0
3
-2
-1
0
1
2
3
4
X
Probability-box
Systems Realization Laboratory
Other Sources of Imprecision in Design
ƒ Some other sources of uncertainty best represented by intervals
•
•
•
•
•
•
Simulations and analysis models – abstractions of reality
Statistical data – finite samples of environmental factors
Bounded rationality – imprecise subjective probabilities
Expert opinion – lack or conflict of information
Preferences – incomplete or non-stationary
Numerical implementation – limited machine precision
ƒ Consequence:
The performance (expected utility) of a decision alternative
is best expressed in terms of intervals
e.g. mass = [100,300] kg, cost = $ [400,1000] Æ utility = […,…]
Systems Realization Laboratory
Decision Making Under Interval Uncertainty
ƒ In normative decision theory:
Decision Policy = Maximize Expected Utility
ƒ In set-based design:
Uncertainty expressed as intervals or probability boxes
Expected Utility Æ Interval of Expected Utility
How to make a decision when expected utilities are intervals?
Systems Realization Laboratory
Decision Making for Intervals of Expected Utility
How can a decision be reached?
Expected utility
Consider 3 design alternatives {A, B, C}
with expected utility intervals as shown:
Which alternative is
the most preferred?
Possible policies include:
A
B
C
• Interval Dominance
• Maximality
• Γ-maximin
• Γ-maximax
• Hurwicz criterion η
• E-admissibility
Systems Realization Laboratory
Interval Dominance Decision Policy
Eliminate only alternatives that are provably dominated
Expected utility
Consider 3 design alternatives {A, B, C}
with expected utility intervals as shown:
C dominates A
Æ eliminate A
X
A
upper bound of A
<
lower bound of C
B
C
B and C continue to be
considered (set-based design)
Systems Realization Laboratory
The Myth of "Optimal Design"
ƒ Due to uncertainty,
"optimal design"
cannot be determined
ƒ When uncertainty is large,
selecting only the
"optimal design" often
leads to back-tracking
1500
1000
500
ELIMINATE
ELIMINATE
Expected Utility ($)
ƒ Set of non-dominated
solutions
Interval Dominance: Expected Utility vs. Gear Ratio
2000
0
non-dominated set
-500
-1000
0.5
1
1.5
2
2.5
3
3.5
4
Gear Ratio, Ng
Systems Realization Laboratory
4.5
5
What If Non-Dominated Set is Too Large?
cost
[
bounds
on cost
]
Set of All
Gas Engines
[
bounds on mass
]
mass
Search non-dominated set using
Branch and Bound approach
Systems Realization Laboratory
What If Non-Dominated Set is Too Large?
cost
]
Large
Gas Engines
[
[
]
Small
Gas Engines
[
small
]
[
mass
large
]
Refine design alternatives
Æ Reduces imprecision in performance
Æ Allows for additional elimination
Systems Realization Laboratory
Γ-maximin Decision Policy
Avoid a very bad outcome for sure.
Expected utility
Consider 3 design alternatives {A, B, C}
with expected utility intervals as shown:
*
*A
B
*
C
Chose the alternative with the
highest lower-bound
Æ Robust Solution
Should only be used
as a tie-breaker
Systems Realization Laboratory
Consequences for Set-Based Design
Decision Alternatives and their Expected Utilities are Sets
ƒ Unlikely that a single "point solution" will dominate
• "Point solutions" are often greedy Æ result in expensive back-tracking
• "Point solutions" force us to make assumptions that are not supported by
current information
ƒ Constraint propagation versus non-domination
• Intersection of feasible sets for individual disciplines or sub-systems
= elimination of dominated solutions
• Infeasible = overall utility is unacceptably low = dominated
• But: set of feasible solution is likely to be large Æ need for efficient search
ƒ Uncertain information should be represented accurately
• Without overstating what is known
• But also without omitting much information
Æ need for probabilistic or even hybrid (p-box) representations
Systems Realization Laboratory
Challenges for Modeling and Simulation
ƒ Uncertainty quantification
• Every model is an abstraction of reality and thus wrong
• Model accuracy (systematic error) must be stated in terms of intervals
• Uncertainty quantification of model parameters / inputs / outputs
ƒ Need for abstract models
• Allow designers to quickly eliminate large portions of the design space
• Currently not addressed Æ opportunity: abstraction through data mining
ƒ How to capture abstract models without losing much information?
• Capturing interval dependence is critical
Systems Realization Laboratory
Challenges for PLM
ƒ Representations of design alternatives in terms of sets
• Most important at systems engineering level
• Set-based geometric representations – leverage GD&T support
ƒ Representations for communicating preferences
• Requirements are too limiting
• Better communication mechanism than requirements flow-down
ƒ Methods for efficiently propagating constraints
• Interval arithmetic may yield hyper-conservative results
ƒ Methods for efficiently searching set-based design spaces
• Branch and bound: How to branch efficiently?
Systems Realization Laboratory
Conclusions
ƒ Set-Based Design
• Foundation developed by Ward et al. starting in late 80's
• Empirical evidence of superior results: Toyota
• Many remaining limitations and research issues
ƒ Need for a strong foundation: Decision Theory
•
•
•
•
All sequential design methods are set-based
Expected utility of decision alternatives should be expressed as intervals
Decision policy: eliminate non-dominated design alternatives
Searching through a set of non-dominated alternatives: Branch & Bound
Systems Realization Laboratory
References
ƒ
ƒ
ƒ
ƒ
ƒ
ƒ
Ward, "A Theory of Quantitative Inference Applied to a Mechanical Design
Compiler", Ph.D. Thesis, MIT AI Lab, 1989.
Sobek, Ward, and Liker, "Toyota’s Principles of Set-based Concurrent
Engineering" Sloan Management Review, 1999
Malak and Paredis, "An Investigation of Set-Based Design from a Decision
Analysis Perspective," ASME Design Automation Conference, submitted.
Rekuc, Aughenbaugh, Bruns, and Paredis, "Eliminating Design Alternatives
Based on Imprecise Information," SAE World Congress, 06M-269, 2006.
Panchal, Fernández, Allen, Paredis, and Mistree, "An Interval-Based
Focalization Method for Decision-Making in Decentralized, Multi-Functional
Design," Design Automation Conference, DETC2005-85322, Long Beach, CA,
September 24–28, 2005.
Aughenbaugh and Paredis, "The Value of Using Imprecise Probabilities in
Engineering Design," ASME Journal of Mechanical Design, to appear in July
2006.
http://www.srl.gatech.edu/Members/cparedis
Systems Realization Laboratory