Two stage group screening with
noise factors, unequal group
sizes and differing probabilities
of active effects
Anna Vine
University of Southampton, UK
Susan Lewis, Angela Dean
Funded by EPSRC
Summary
• Two-stage group screening
– for control and noise factors
– Interaction group screening
• Criteria for comparing strategies
• Example
– Software to guide experimenters
Screening in industrial
experiments
• Response can depend on a large number of
different factors of two types:
– control: can be set by engineers during
manufacturing
– noise: can’t be controlled in
manufacturing/use but can be controlled in
experiments
• Aim of screening experiments is to identify
important factors
– judged by engineers to give a substantial
improvement, Δ > 0, in product performance
Two-stage group screening
• Individual two-level factors divided into
groups:
– F groups of control factors with sizes
(c)
g1 ,..., g F
(c)
– N groups of noise factors with sizes
( n)
1
g
,..., g N
( n)
• Define a grouped factor for each group by
setting all factors within a group to their high
(low) level
• Two stages of experimentation
– Stage I on grouped factors
– Stage II on factors in groups found to be
important at stage I
Active factors
• In interaction group screening main effects and
interactions of the grouped factors are estimated
at stage I
• A grouped control factor is declared active
– if it has a detected main effect
– if it is in a detected control x control
interaction
– if it is in a detected control x noise
interaction
• A grouped noise factor is only declared active if
in a detected interaction with a control factor
• Detection of active grouped factors is not
certain
Probabilities of effects being
active
• The total number of effects S that require
estimation over the two-stage experiment is a
random variable
• Based on experts’ opinions assign prior
probabilities
– to each individual main effect being active
– to each individual interaction being active
– or use heredity principle (Chipman 1996)
• Calculate probabilities of each grouped effect
being active from individual probabilities
Criteria for choice of groupings
Aim to
1.
minimise expected total number of effects
that require estimation, E(S)
2.
minimise the probability of exceeding a target
number u of factorial effects requiring
estimation, P(S > u)
3.
minimise the risk of failing to detect
important main effects and interactions
•
Conflict between aims
•
Lewis and Dean (2001) and Dean and Lewis
(2002)
Formulating the criteria
• Expected total number of individual effects
requiring estimation
E(S) = 2N +
F
F +1
+ FN + ∑∑g i (c)g j(n )p i,j(cn )
2
N
+ [∑g j ][1
(n )
i =1 j=1
+ 1) / 2][1
N
F
F
)] + ∑∑g i ( c )g k (c )p i,k (cc )
i =1 k = i +1
∏ (1
F
p i, j
( cn )
)
∏ (1
p i,k ( cc ) )(1 p i ( c ) )]
k =1,k ≠i
j=1
∏∏ ∏
i =1
p i, j
N
(c)
i =1
F
F 1
( cn )
i =1
F
+ ∑[g i (g i
F
∏ (1
j=1
(c )
N
F
[(1 p i,j(cn ) )(1 p i,k (cc ) )(1 p i (c ) )] + ∏
j=1 k =1,k ≠i
i =1
N
∏ (i
p i,j(cn ) )
j=1
• Probability distribution of the total number of
individual effects requiring estimation
P(S = s) =
∑P(I
R
(c)
t1
, I t 2 (cc ) , I t 3 (cn ) )
Practical situation
• Partition individual control factors into two
types: very likely and less likely
• Control factors believed very likely to be
active are assigned the same high
probability of a main effect being active
• Assign very likely and less likely control
factors to separate groups
Example
To illustrate use of criteria consider:
• 7 individual control factors
- main effects probabilities 1
• 8 individual control factors
- main effects probabilities 0.2
• 4 individual noise factors
- main effects probabilities 0.3
• individual control x noise interaction
probabilities 0.07
• individual control x control interaction
probabilities 0.05
Distribution of S
Risk of exceeding a target
Software
• Interactive web based software
– allows elicitation of information from
experts
– enables comparison of different
groupings under criteria 1 and 2
– allows simulations of group screening
experiments to be run for criterion 3
Use of methods/software
• Planned a group screening experiment on
engine cold start optimisation at Jaguar Cars
Investigation of groupings
Very likely
control
Less likely
control
Noise
E(S)
sd(S)
P(S>120)
P(S>150)
P(S>180)
Grouping
8
2,6
3,5
4,4
2,2,4
2,3,3
2,2,2,2
2,2
2,2
2,2
2,2
2,2
2,2
2,2
174.5
158.41
155.32
154.43
143.16
141.14
134.04
20.31
21.97
23.13
23.56
21.46
21.61
19.51
0.98
0.94
0.92
0.91
0.85
0.83
0.76
0.89
0.70
0.61
0.60
0.38
0.35
0.20
0.55
0.16
0.13
0.12
0.03
0.02
0.01
2,5
2,5
2,5
2,5
2,5
2,5
2,5
8
2,6
3,5
4,4
2,2,4
2,3,3
2,2,2,2
2,2
2,2
2,2
2,2
2,2
2,2
2,2
159.22
142.42
138.49
137.28
127.99
125.79
120.85
21.71
21.19
21.45
21.57
18.97
18.76
16.42
0.94
0.84
0.79
0.78
0.67
0.62
0.52
0.68
0.37
0.29
0.27
0.12
0.09
0.04
0.13
0.02
0.01
0.01
0.00
0.00
0.00
3,4
3,4
3,4
3,4
3,4
3,4
3,4
8
2,6
3,5
4,4
2,2,4
2,3,3
2,2,2,2
2,2
2,2
2,2
2,2
2,2
2,2
2,2
156.64
139.17
134.94
133.63
124.45
122.18
117.41
22.60
21.34
21.27
21.27
18.56
18.23
15.84
0.93
0.80
0.75
0.73
0.60
0.54
0.43
0.63
0.30
0.24
0.21
0.08
0.06
0.02
0.15
0.02
0.01
0.01
0.00
0.00
0.00
2,2,3
2,2,3
2,2,3
2,2,3
2,2,3
2,2,3
2,2,3
8
2,6
3,5
4,4
2,2,4
2,3,3
2,2,2,2
2,2
2,2
2,2
2,2
2,2
2,2
2,2
146.15
130.39
126.19
124.89
117.85
115.69
112.97
20.86
18.86
18.55
18.44
15.72
15.34
13.00
0.88
0.71
0.63
0.61
0.44
0.38
0.27
0.43
0.15
0.09
0.08
0.02
0.01
0.00
0.04
0.00
0.01
0.00
0.00
0.00
0.00
7
7
7
7
7
7
7
Investigation of groupings CGS
Very likely
control
7
7
7
Less likely
control
Grouping
8
2,6
3,5
Noise
E(S)
sd(S)
P(S>80)
P(S>110)
P(S>140)
2,2
2,2
2,2
141.75
118.89
114.16
46.16
42.89
43.21
0.83
0.76
0.79
0.83
0.62
0.59
0.63
0.32
0.25
7
7
7
*7
2,2,3
2,2,3
2,2,3
4,4
2,2,4
2,3,3
2,2,2,2
8
2,6
3,5
2,2
2,2
2,2
2,2
2,2
2,2
2,2
112.68
99.95
97.77
88.07
143.75
120.89
116.16
43.46
36.45
35.80
29.63
46.16
42.89
43.21
0.72
0.62
0.69
0.48
0.83
0.76
0.79
0.47
0.36
0.34
0.19
0383
0.62
0.63
0.26
0.13
0.15
0.04
0.63
0.32
0.34
2,2,3
2,2,3
2,2,3
2,2,3
4,4
2,2,4
2,3,3
2,2,2,2
2,2
2,2
2,2
2,2
114.68
101.95
99.77
90.07
43.46
36.45
35.80
29.63
0.72
0.62
0.69
0.48
0.47
0.36
0.42
0.19
0.26
0.13
0.15
0.04
Classical group screening
Distribution of S
Risk of exceeding a target
Choosing a strategy
• Is CGS a better option?
• IGS is more likely to exceed a target
Choosing a strategy –
Simulation software
Assesses risk of missing active effects
• Provides values of active and non-active
effects sampled from specified
distributions
• Forms the groups of factors at random
• For each selection of effect values,
software simulates an experiment
• Calculates proportions of active individual
main effects and interactions MISSED
• Process repeated 500 times
Choosing a strategy Simulation results
Percentage of active individual effects missed
Strategy
CGS
IGS
control main
effects
6 – 17
1 – 11
control x control
interactions
47 – 73
27 – 63
control x noise
interactions
71 – 96
26 - 78
Conclude: CGS misses more active individual
effects than IGS
Software Results - CGS
Very likely
control
Less likely
control
Noise
E(S)
sd(S)
P(S>80)
P(S>110)
P(S>140)
Grouping
8
2,6
3,5
4,4
2,2,4
2,3,3
2,2,2,2
2,2
2,2
2,2
2,2
2,2
2,2
2,2
141.75
118.89
114.16
112.68
99.95
97.77
88.07
46.16
42.89
43.21
43.46
36.45
35.80
29.63
0.83
0.76
0.79
0.72
0.62
0.69
0.48
0.83
0.62
0.59
0.47
0.36
0.34
0.19
0.63
0.32
0.25
0.26
0.13
0.15
0.04
2,5
2,5
2,5
2,5
2,5
2,5
2,5
8
2,6
3,5
4,4
2,2,4
2,3,3
2,2,2,2
2,2
2,2
2,2
2,2
2,2
2,2
2,2
142.75
119.89
115.16
113.68
100.95
98.77
89.07
46.16
42.89
43.21
43.46
36.45
35.80
29.63
0.83
0.76
0.79
0.72
0.62
0.69
0.48
0.83
0.62
0.59
0.47
0.36
0.42
0.19
0.63
0.32
0.34
0.26
0.13
0.15
0.04
3,4
3,4
3,4
3,4
3,4
3,4
3,4
8
2,6
3,5
4,4
2,2,4
2,3,3
2,2,2,2
2,2
2,2
2,2
2,2
2,2
2,2
2,2
142.75
119.89
105.16
113.68
100.95
98.77
89.07
46.16
42.89
43.21
43.46
36.45
35.80
29.63
0.83
0.76
0.79
0.72
0.62
0.69
0.48
0.83
0.62
0.59
0.47
0.36
0.42
0.19
0.63
0.32
0.39
0.26
0.13
0.15
0.04
2,2,3
2,2,3
2,2,3
8
2,6
3,5
2,2
2,2
2,2
143.75
120.89
116.16
46.16
42.89
43.21
0.83
0.76
0.79
0383
0.62
0.63
0.63
0.32
0.34
2,2,3
2,2,3
2,2,3
4,4
2,2,4
2,3,3
2,2
2,2
2,2
114.68
101.95
99.77
43.46
36.45
35.80
0.72
0.62
0.69
0.47
0.36
0.42
0.26
0.13
0.15
2,2,3
2,2,2,2
2,2
90.07
29.63
0.48
0.19
0.04
7
7
7
7
7
7
7
Investigation of groupings
• 4 noise factors grouped in pairs
Very likely
control
Less likely
control
E(S)
sd(S)
P(S>120)
P(S>150)
P(S>180)
146.15
130.39
126.19
124.89
117.85
115.69
112.97
20.86
18.86
18.55
18.44
15.72
15.34
13.00
0.88
0.71
0.63
0.61
0.44
0.38
0.27
0.43
0.15
0.09
0.08
0.02
0.01
0.00
0.04
0.00
0.01
0.00
0.00
0.00
0.00
Grouping
2,2,3
2,2,3
2,2,3
2,2,3
2,2,3
2,2,3
* 2,2,3
8
2,6
3,5
4,4
2,2,4
2,3,3
2,2,2,2
• Noise factors in two groups of size 2 was
consistently more economical
Conclusions
• Good grouping strategies for minimising
E(S):
– Use groups as small and as equal in size
as possible
– Group together factors with higher
probabilities of their main effects being
active