ST 516
Experimental Statistics for Engineers II
Response Surface Methods
Response surface methodology (RSM) is a combination of statistics
(regression modeling) and mathematics (optimization) used to find
factor settings that optimize the average value of a response, or some
function of that average.
E.g. the yield of a chemical process (y ) is influenced by temperature
(x1 ) and pressure (x2 ):
y = f (x1 , x2 ) + ,
where has mean zero.
1 / 16
Response Surface Methods
Introduction
ST 516
Experimental Statistics for Engineers II
Typically the form of f (x1 , x2 , . . . , xk ) is unknown, so we approximate
it by either
a first-order model
y = β0 + β1 x1 + β2 x2 + · · · + βk xk + or a second-order model
y = β0 +
k
X
i=1
βi x i +
k
X
βi,i xi2 +
i=1
XX
βi,j xi xj + .
i<j
We expect these approximations to work well only for a small region
in (x1 , x2 , . . . , xk ) space.
2 / 16
Response Surface Methods
Introduction
ST 516
Experimental Statistics for Engineers II
Steepest Ascent
Typical approach
Begin with a 2k factorial design augmented with center points,
centered around some initial settings;
Fit the first-order model, then check for interaction and pure
quadratic terms;
If neither interaction nor quadratic effects are significant, use steepest
ascent to improve the mean response.
3 / 16
Response Surface Methods
Steepest Ascent
ST 516
Experimental Statistics for Engineers II
Example
Improving yield as a function of reaction time and temperature
Initial settings: ξ1 = 35 minutes, ξ2 = 155◦ F.
Experimental levels: ξ1 = 30, 40; ξ2 = 150, 160; coded as x1 = −1, 1
and x2 = −1, 1.
Design: unreplicated 2 × 2 with 5 center points.
4 / 16
Response Surface Methods
Steepest Ascent
ST 516
Experimental Statistics for Engineers II
Data:
xi1 xi2 x1 x2 y
30 150 -1 -1 39.3
30 160 -1 1 40.0
40 150 1 -1 40.9
40 160 1 1 41.5
35 155 0 0 40.3
35 155 0 0 40.5
35 155 0 0 40.7
35 155 0 0 40.2
35 155 0 0 40.6
5 / 16
Response Surface Methods
Steepest Ascent
ST 516
Experimental Statistics for Engineers II
First-order model
summary(lm(y ~ x1 + x2, t11d1))
Output
Call: lm(formula = y ~ x1 + x2, data = t11d1)
Residuals:
Min
1Q
-0.244444 -0.044444
Median
0.005556
3Q
0.055556
Max
0.255556
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 40.44444
0.05729 705.987 5.45e-16 ***
x1
0.77500
0.08593
9.019 0.000104 ***
x2
0.32500
0.08593
3.782 0.009158 **
--Signif. codes: 0 *** 0.001 ** 0.01 * 0.05 . 0.1
1
Residual standard error: 0.1719 on 6 degrees of freedom
Multiple R-Squared: 0.941,
Adjusted R-squared: 0.9213
F-statistic: 47.82 on 2 and 6 DF, p-value: 0.0002057
6 / 16
Response Surface Methods
Steepest Ascent
ST 516
Experimental Statistics for Engineers II
Check interaction and pure quadratic effects
summary(aov(y ~ x1 + x2 + I(x1 * x2) + I(x1^2) + I(x2^2), t11d1))
Output
Df
x1
1
x2
1
I(x1 * x2)
1
I(x1^2)
1
Residuals
4
--Signif. codes:
Sum Sq
2.4025
0.4225
0.0025
0.0027
0.1720
Mean Sq F value
Pr(>F)
2.4025 55.872 0.00171 **
0.4225
9.826 0.03503 *
0.0025
0.058 0.82132
0.0027
0.063 0.81374
0.0430
0 *** 0.001 ** 0.01 * 0.05 . 0.1
1
Both are small ⇒ first-order model is adequate.
7 / 16
Response Surface Methods
Steepest Ascent
ST 516
Experimental Statistics for Engineers II
How to improve yield?
Mean yield increases as either x1 or x2 is increased.
If we fix ∆x12 + ∆x22 = ∆2 , the maximum improvement is at
∆
,
+ β22
∆
∆x2 = β2 × 2
.
β1 + β22
∆x1 = β1 ×
β12
This is the direction of steepest ascent.
The engineer chose ∆x1 = 1, ∆x2 = β̂2 /β̂1 = 0.42.
8 / 16
Response Surface Methods
Steepest Ascent
ST 516
Experimental Statistics for Engineers II
But note that this direction depends on the choice of experimental
levels for ξ1 and ξ2 .
Also it’s not clear that the constraint ∆x12 + ∆x22 = ∆2 is relevant.
So other similar directions could also be searched.
In this case, yield increased for 10 steps, then declined.
New initial values are 85 minutes and 175 degrees F.
9 / 16
Response Surface Methods
Steepest Ascent
ST 516
Experimental Statistics for Engineers II
Similar augmented 2 × 2 experiment at the new initial conditions has
significant pure quadratic effect, so first-order model is not adequate.
Engineer added 4 axial points to estimate second-order model.
10 / 16
Response Surface Methods
Steepest Ascent
ST 516
Experimental Statistics for Engineers II
Data:
xi1
80
80
90
90
85
85
85
85
85
92.07
77.93
85
85
xi2
x1
x2
170
-1
-1
180
-1
1
170
1
-1
180
1
1
175
0
0
175
0
0
175
0
0
175
0
0
175
0
0
175
1.414 0
175
-1.414 0
182.07 0
1.414
167.93 0
-1.414
11 / 16
y
76.5
77.0
78.0
79.5
79.9
80.3
80.0
79.7
79.8
78.4
75.6
78.5
77.0
Response Surface Methods
Steepest Ascent
ST 516
Experimental Statistics for Engineers II
Regression model
t11d6Lm <- lm(y ~ x1 + x2 + x1:x2 + I(x1^2) + I(x2^2), t11d6)
summary(t11d6Lm)
Output
Call:
lm(formula = y ~ x1 + x2 + x1:x2 + I(x1^2) + I(x2^2), data = t11d6)
Residuals:
Min
1Q
Median
-0.23995 -0.18089 -0.03995
3Q
0.17758
Max
0.36005
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 79.93995
0.11909 671.264 < 2e-16
x1
0.99505
0.09415 10.568 1.48e-05
x2
0.51520
0.09415
5.472 0.000934
I(x1^2)
-1.37645
0.10098 -13.630 2.69e-06
I(x2^2)
-1.00134
0.10098 -9.916 2.26e-05
x1:x2
0.25000
0.13315
1.878 0.102519
--Signif. codes: 0 *** 0.001 ** 0.01 * 0.05 . 0.1
12 / 16
Response Surface Methods
***
***
***
***
***
1
Steepest Ascent
ST 516
Experimental Statistics for Engineers II
Output, continued
Residual standard error: 0.2663 on 7 degrees of freedom
Multiple R-squared: 0.9827, Adjusted R-squared: 0.9704
F-statistic: 79.67 on 5 and 7 DF, p-value: 5.147e-06
13 / 16
Response Surface Methods
Steepest Ascent
ST 516
Experimental Statistics for Engineers II
ANOVA for second-order model
summary(aov(y ~ x1 + x2 + x1:x2 + I(x1^2) + I(x2^2), t11d6))
Output
Df Sum Sq Mean Sq F value
Pr(>F)
x1
1 7.9198 7.9198 111.6873 1.484e-05 ***
x2
1 2.1232 2.1232 29.9413 0.000934 ***
I(x1^2)
1 10.9816 10.9816 154.8663 4.979e-06 ***
I(x2^2)
1 6.9721 6.9721 98.3225 2.262e-05 ***
x1:x2
1 0.2500 0.2500
3.5256 0.102519
Residuals
7 0.4964 0.0709
--Signif. codes: 0 *** 0.001 ** 0.01 * 0.05 . 0.1
1
14 / 16
Response Surface Methods
Steepest Ascent
ST 516
Experimental Statistics for Engineers II
To test for lack of fit in this quadratic model, we must separate the
sum of squares for “Residuals” into the sum of squares for lack of fit,
and the sum of squares for pure error.
Pure error comes from replicated values (center points).
We can find it by treating x1 and x2 as categorical variables, because
then each unique combination of factor levels has its own mean.
15 / 16
Response Surface Methods
Steepest Ascent
ST 516
Experimental Statistics for Engineers II
Add factor(x1):factor(x2) to the quadratic model:
summary(aov(y ~ x1 * x2 + I(x1^2) + I(x2^2) + factor(x1) : factor(x2), t11d6))
Output
Df Sum Sq Mean Sq F value
x1
1 7.9198 7.9198 149.4303
x2
1 2.1232 2.1232 40.0594
I(x1^2)
1 10.9816 10.9816 207.2009
I(x2^2)
1 6.9721 6.9721 131.5491
x1:x2
1 0.2500 0.2500
4.7170
factor(x1):factor(x2) 3 0.2844 0.0948
1.7885
Residuals
4 0.2120 0.0530
--Signif. codes: 0 *** 0.001 ** 0.01 * 0.05 . 0.1
Pr(>F)
0.0002571
0.0031894
0.0001354
0.0003298
0.0956108
0.2885640
***
**
***
***
.
1
The sum of squares for lack of fit (factor(x1):factor(x2)) is not
significant, so the quadratic model seems to be adequate.
16 / 16
Response Surface Methods
Steepest Ascent