Physical Experiments &
Computer Experiments,
Preliminaries
Chapters 1&2
“Design and Analysis of Experiments“
by Thomas J. Santner, Brian J. Williams and William I. Notz
EARG presentation Oct 3, 2005
by Frank Hutter
1
Preface
What they mean by Computer Experiments


Code that serves as a proxy for physical
process
Can modify inputs and observe how process
output is affedcted
Math

Should be understandable with a Masters level
training of Statistics
2
Overview of the book
Chapter 1: intro, application domains
Chapter 2:


Research goals for various types of inputs (random, controlled,
model parameters)
Lots of definitions, Gaussian processes
Chapters 3-4:
Predicting Output from Computer Experiments
Chapter 5:
Basic Experimental Design (similar to last term)
Chapter 6:
Active Learning (sequential experimental design)
Chapter 7: Sensitivity analysis
Appendix C: code
3
Physical experiments: a few
concepts we saw last term (2f)
Randomization

In order to prevent unrecognized nuisance variables
from systematically affecting response
Blocking


Deals with recognized nuisance variables
Group experimental units into homogeneous groups
Replication

Reduce unavoidable measurement variation
Computer experiments


Deterministic outputs
“None of the traditional principles [...] are of use“
4
Types of input variables (15f)
Control variables xc


Can be set by experimenter / engineer
Engineering variables, manufactoring variables
Environmental variables Xe




Depend on the environment/user
Random variables with known or unknown distribution
When known for a particular problem: xe
Noise variables
Model variables xm


Parameters of the computer model that need to be set
to get the best approximation of the physical process
Model parameters, tuning parameters
5
Examples of Computer Models (6ff)
ASET (Available Safe Egress Time)

5 inputs, 2 outputs
Design of Prosthesis Devices


3 environment variables, 2 control variables
2 competing outputs
Formation of Pockets in Sheet Metal

6 control variables, 1 output
Other examples


Optimally shaping helicopter blade – 31 control
variables
Public policy making: greenhouse gases – 30 input
variables, some of them modifiable (control variables)
6
ASET (Available Safe Egress Time) (4f)
Evolution of fires in enclosed areas
Inputs





Room ceiling height and room area
Height of burning object
Heat loss fraction for the room (depends on
insulation)
Material-specific heat release rate
Maximum time for simulation (!)
Outputs


Temperature of the hot smoke layer
Distance of hot smoke layer from fire source
7
Design of Prosthesis Devices (6f)
2 control variables


b, the length of the bullet tip
d, the midstem parameter
3 environment variables



, the joint angle
E, the elastic modulus of the
surrounding cancellous bone
Implant-bone interface friction
2 conflicting outputs



Femoral stress shielding
Implant toggling
(flexible prostheses minimize stress,
but toggle more  loosen)
8
Formation of Pockets in Steel (8ff)
6 control variables






Length l
Width w
Fillet radius f
Clearance c
Punch plan view
radius p
Lock bead distance d
Output

Failure depth (depth at
which the metal tears)
9
Research goals for
homogeneous-input codes (17f)
Homogeneous-input: only one of the three
possible variable types present
All control variables: x=xc


Predict y(x) well for all x in some domain X
Global perspective
Integrated squared error sX [y‘(x) - y(x)]2 w(x) dx
Can‘t be computed since y(x) unknown, but in Chapter 6 we‘ll
replace [y‘(x)-y(x)]2 by a computable posterior mean squared
value

Local perspective
Level set: Find x such that y(x) = t0
t0 = maximum value
10
All environmental variables: x=Xe (18)
How does the variability in Xe transmit
through the computer code ?
Find the distribution of y(Xe)
When the problem is to find the mean:

Latin hypercube designs for choosing the
training sites
11
All model variables x=xm (18f)
Mathematical modelling contains unknown
parameters (unknown rates or physical
constants)
Calibration (parameter fitting)

Choose the model variables xm so that the
computer output best matches the output from
the physical experiment
12
Research goals for mixed inputs
(19ff)
Focus on case with control and
environmental variables x=(xc,Xe) where
Xe has a known distribution
Example: hip prosthesis
y(xc,Xe) is a random variable whose
distribution is induced by Xe


Mean (xc) = E{y(xc,Xe)}
Upper alpha quantile  = (xc):
P{y(xc,Xe) >= } = 
13
Research goals for mixed inputs:
simple adaption of previous goals (20ff)
Predict y well over its domain:


Minimize sX [‘(x) - (x)]2 w(x) dx
(again, there is a Bayesian analog with
computable mean)
Maximize the mean output: maxxc (xc)
14
When the distribution of Xe is
unknown
Various flavours of robustness
G-robust: minimax


Want to minimize your maximal loss
Pessimistic
(.)-robust

Minimize weighted loss (weighted by prior density on
distribution over Xe)
M-robust



Suppose for a given xc, y(xc,xe) is fairly flat
Then value of Xe doesn‘t matter so much for that xc
Maximize (xc) subject to constraints on variance
w.r.t. Xe
15