Multivariate Statistical Process
Control and Optimization
Alexey Pomerantsev & Oxana Rodionova
Semenov Institute of Chemical Physics
Russian Chemometrics Society
© Chris Marks
10.02.04
1
Agenda
1. Introduction
2. SPC
3. MSPC
4. Passive optimization (E-MSPC)
5. Active optimization (MSPO)
6. Conclusions
10.02.04
2
Statistical Process Control (SPC)
SPC Objective
To monitor the performance of the process
SPC Concept
To study historical data representing good past process behaviour
SPC Method
Conventional statistical methods
SPC Approach
To plot univariate chart in order to monitor key process variables
10.02.04
3
Historical Process Data (Chemical Reactor)
Production cycles s1, s2, ... ,s54
Key process variables (sensors) X1, X2, ... , X17
s1
s2
s3
s4
s5
s6
s7
s8
s9
s10
s11
s12
s13
s14
s15
s16
s17
s54
10.02.04
X1
-1.19E-01
-1.37E-01
2.51E-02
-1.14E-01
-7.93E-02
1.51E-02
7.44E-02
3.65E-02
1.36E-01
-2.74E-02
7.47E-02
-1.17E-01
1.06E-01
7.39E-02
-9.87E-03
-1.06E-01
-4.76E-02
X2
7.28E-01
7.28E-01
-9.15E-02
6.70E-01
4.14E-01
-6.38E-02
-5.24E-01
-2.66E-01
-7.06E-01
3.60E-01
-3.31E-01
7.02E-01
-2.82E-01
-5.28E-01
1.02E-01
7.68E-01
2.66E-01
X3
-2.15E-02
-2.89E-02
6.73E-03
-2.18E-02
-1.69E-02
3.74E-03
1.11E-02
5.12E-03
2.89E-02
1.82E-03
1.80E-02
-2.16E-02
3.23E-02
1.07E-02
-3.21E-04
-1.52E-02
-9.52E-03
X4
5.22E-01
6.08E-01
-1.13E-01
5.04E-01
3.51E-01
-6.75E-02
-3.24E-01
-1.59E-01
-6.01E-01
1.12E-01
-3.34E-01
5.13E-01
-4.82E-01
-3.21E-01
4.17E-02
4.62E-01
2.10E-01
X5
7.06E-04
7.09E-04
-9.07E-05
6.50E-04
4.04E-04
-6.28E-05
-5.06E-04
-2.56E-04
-6.88E-04
3.42E-04
-3.25E-04
6.81E-04
-2.85E-04
-5.09E-04
9.75E-05
7.41E-04
2.59E-04
X6
7.32E-01
7.02E-01
-7.58E-02
6.65E-01
3.98E-01
-5.67E-02
-5.45E-01
-2.78E-01
-6.77E-01
4.12E-01
-2.99E-01
7.03E-01
-1.87E-01
-5.50E-01
1.13E-01
8.03E-01
2.61E-01
…
X7
3.10E-04
6.58E-04
-2.29E-04
3.83E-04
3.96E-04
-1.15E-04
-1.73E-05
1.43E-05
-6.83E-04
-4.31E-04
-5.30E-04
3.40E-04
-1.25E-03
2.49E-06
-8.29E-05
-2.54E-05
1.92E-04
X8
-6.13E-04
-1.22E-03
4.10E-04
-7.34E-04
-7.35E-04
2.07E-04
7.92E-05
-3.95E-07
1.26E-03
7.24E-04
9.62E-04
-6.63E-04
2.21E-03
4.48E-05
1.36E-04
-2.68E-05
-3.61E-04
X9
-5.92E-05
-1.49E-04
5.65E-05
-7.96E-05
-9.05E-05
2.78E-05
-1.07E-05
-1.14E-05
1.56E-04
1.22E-04
1.28E-04
-6.76E-05
3.14E-04
-1.59E-05
2.44E-05
2.88E-05
-4.19E-05
6.61E-02 -5.40E-01 7.19E-03 -2.85E-01 -5.19E-04 -5.78E-01 1.81E-04 -2.67E-04 -6.23E-05
…
X17
9.74E-03
1.01E-02
-1.43E-03
9.07E-03
5.78E-03
-9.49E-04
-6.79E-03
-3.42E-03
-9.86E-03
4.18E-03
-4.84E-03
9.44E-03
-4.99E-03
-6.81E-03
1.23E-03
9.90E-03
3.65E-03
-6.78E-03
4
Shewart Charts (1931)
1.2
1.5
0.5
X2
Control
Control
0.4
0.8
1.0
0.3
X2 Normal
X1 Control
Sensors
X1X2
X2
Sensor
X1 All
&
Sensors
Normal
0.2
0.4
0.5
X1 Normal
X1 Control
0.1
X1 Normal
0.0
X2 Normal
-0.1
X1 Control
-0.5
-0.4
-0.2
X2 Normal
Normal
-0.3
-1.0
-0.8
-0.4
X1 Control
X2 Normal
Control
X2 Control
s56
s54
s51
s49
s46
s43
s41
s39
s37
s35
s33
s31
s29
s27
s25
s23
s21
s19
s17
s15
s13
s11
s9
s7
s5
s3
s1
-1.5
-0.5
-1.2
Cycles (time)
10.02.04
5
Panel Process Control (just a game)
1
2
3
4
5
6
7
8
9 10 11 12 13 14 15 16 17
0.31 -0.4 0.26 -0.3 -0.3 -0.4 -0.1 0.13 0.08 -0.3 -0.3 #### -0.3 -0.3 #### -0.1 -0.4
3
2
0.31
0.26
0.01
-0.06
-0.09
-0.36
-0.31 -0.27
-0.37
1
-0.01
PC2
0.13 0.08
-0.35 -0.35
-0.28
0
-4
-0.35
-0.38
-3
-2
-1
0
-1
-2
-3
Off
Time till the end of shift:
10.02.04
Exit
On
7:59:10
6
Multivariate Statistical Process Control
(MSPC)
MSPC Objective
To monitor the performance of the process
MSPC Concept
To study historical data representing good past process behavior
MSPC Method
Projection methods of Multivariate Data Analysis (PCA, PCR, PLS)
MSPC Approach
To plot multivariate score plots to monitor the process behavior
10.02.04
7
Projection Methods
Initial Data
Data Plane
Data Center
PCs
Data Projections
10.02.04
8
Low Dimensional Presentation
10.02.04
9
MSPC Charts (Chemical Reactor)
Samples
Key Variables
3
Scores
0.6
PC2
Loadings
PC2
X13
2
s23
s13
s51
-4
-3
-2
1
s10
X16
X15
X12
X9
X8
s16
s1 s50
s27
s26 s12
s4
s40 s47
s21
s15s43
s2
s11
s3
s17
s41 s6
s20
s48 s5
s25
s39
0
s37
s38 s45
s46 s33
s42
s9
s34
s22
s35s44
-1 s52 s8
0 s36
s28 1 s32s312
s24
s29
s19
s7
s14
s18
-1 s53
s49
s30
PC1
3
0.3
X5
X6
X2
X17
X14
X10
X11
X4
4
PC1
X3
-0.4
0
-0.2
0
0.2
0.4
X1
-2
-3
10.02.04
X7
-0.3
10
Panel Process Control (not just a game)
1
2
3
4
5
6
7
8
9 10 11 12 13 14 15 16 17
0.31 -0.4 0.26 -0.3 -0.3 -0.4 -0.1 0.13 0.08 -0.3 -0.3 #### -0.3 -0.3 #### -0.1 -0.4
3
2
0.31
0.26
0.01
-0.06
-0.09
-0.36
-0.31 -0.27
-0.37
1
-0.01
PC2
0.13 0.08
-0.35 -0.35
-0.28
0
-4
-0.35
-0.38
-3
-2
-1
0
-1
-2
-3
Off
Time till the end of shift:
10.02.04
Exit
On
7:59:10
11
Cruise Ship Control (by Kim Esbensen)
10.02.04
12
Key Process Variables
10.02.04
13
PLS1 Prediction of Fuel Consumption
Predicted vs. Measured
3
Scores
PC2
Slope:
Offset:
Correl:
RMSEP:
SEP:
Bias:
2
1
Predicted Fuel
Samples
3
0.95
0.02
0.98
0.23
0.24
-0.005
2
1
PC1
Measured Fuel
0
-4
-3
-2
-1
0
0
1
3
4
-3
-2
-1
0
-1
-1
-2
-2
-3
-3
Weather conditions
X1, X2, X3, X4
10.02.04
2
Cap’s setup
X5, X6, X7
PLS1
1
2
3
Fuel Consumption
Y
14
Passive Optimization
Weather
conditions
Prediction ?
Prediction !
Order!!!
Fuck
censored
X1, X2, X3, X4
X5,X5,
X6,X6,
X7 X7
42
Captain
10.02.04
Computer
Student
15
Active Optimization
Weather
conditions
X5 X6, X7
Advice!!!
X1, X2, X3, X4
Censored
Order?
Optimal
X5, X6, X7
42
Student
10.02.04
Computer
Captain
16
In Hard Thinking about PC and PCs
Forty
two
censored
10.02.04
17
Multivariate Statistical Process
Optimization (MSPO)
MSPO Objective
To optimize the performance of the process (product quality)
MSPO Concept
To study historical data representing good past process behavior
MSPO Methods
Projection methods and Simple Interval Calculation (SIC) method
MSPO Approach
To plot predicted quality at each process stage
10.02.04
18
Technological Scheme. Multistage Process
S
S1
S2
S3
M
M1
W
W1
W2
10.02.04
MR1
MR2
WR1
WR2
M2
A1 A2
A3 A4
A5 A6
CM
M3
P
CM1 CM2 CM3
CW
W3
CW1 CW2 CW3
I
II
III
IV
V
VI
VII
6
8
11
14
16
19
25
19
Y
A6
A5
A4
A3
A2
A1
CM3
CM2
CM1
MR2
MR1
M3
M2
M1
CW3
CW2
CW1
WR2
WR1
W3
W2
W1
S3
S2
Training
Set (102)
XI
XII
XIII
XIV XV XVI
XVII
Y
Test
Set (52)
S1
Historical Process Data
XI
XII
XIII
XIV XV XVI
XVII
Y
X preprocessing
10.02.04
Y preprocessing
20
Quality Data (Standardized Y Set)
1.2
Y
Training Set Samples
Highest Quality Y=+1
0.8
0.4
0.0
-0.4
-0.8
Lowest Quality Y=-1
-1.2
1
1.2
21
Y
41
61
81
101
Test Set Samples
Highest Quality Y=+1
0.8
0.4
0.0
-0.4
-0.8
Lowest Quality Y=-1
-1.2
1
10.02.04
11
21
31
41
51
21
XTRAINING
PLS
6 PCs
XTEST
Y
A6
A5
A4
A3
A2
A1
CM3
CM2
CM1
MR2
MR1
M3
M2
M1
CW3
CW2
CW1
WR2
WR1
W3
W2
W1
S3
S2
S1
General PLS Model
^
Y
^
Y
0.3
Calbration
RMSE
Validation
0.2
0.1
PCs
0
PC_0
10.02.04
PC_2
PC_4
PC_6
PC_8
22
SIC Prediction. All Test Samples
1.0
Y
SIC
PLS1
SIC Prediction
Prediction
SIC
Test
16
0.5
22
3
23
30
27
19
6
31
1
0.0
10
5
2
4
-0.5
18
14
7
37
34
46
40
38
28
25
21
15
12
8
35
24
17
13
11
39
32
44
41
43
49
47
45
33
20
26
9
48
29
36
Test Samples
-1.0
52
50
51
42
1.0
Status plot
plot
Status
26
36
SIC-Residual
0.5
41
52
31
13
13
0.0
43
25
38
50
22
18
51
2
35
1927 4911
28
34 10
37 45
39
14
8
1
6
75
15
21
-0.5
33
16
16
29
-1.0
10.02.04
4 23
9
312
24
30
17
4744
46 40
48 32
20
42
SIC-Leverage
23
SIC Prediction. Selected Test Samples
SIC Prediction
Y
Object Status plot
1.0
Outsiders
0.5
3
0.0
1
5
SIC-Residual
2
0.5
0.0
4
1.0
2
Outsiders
4
Selected Test Samples
Sample No
10.02.04
1
-0.5
-0.5
-1.0
3
Insiders
5
Quality status
-1.0
Abs. Outsiders
1.0
SIC-Leverage
SIC Status
1
Normal
Insider
2
High
Outsider
3
Normal
Absolute outsider
4
Low
Outsider
5
Normal
Insider
24
Passive Optimization in Practice
Objective
To predict future process output being in the middle of the process
Concept
To study historical data representing good past process behaviour
Method
Simple Interval Prediction
Approach
Expanding Multivariate Statistical Process Control (E-MSPC)
10.02.04
25
Y
A6
A5
A4
A3
A2
x
ððxðððð
y
XII
XIII
XIV XV XVI
1
xI
xII
xIII
xIV
Sample 1, Normal Quality Insider
A1
ððð
ðð
ð
Y
XI
1.0
CM3
CM2
CM1
MR2
M3
M2
M1
CW3
CW2
CW1
WR2
WR1
W3
W2
W1
S3
S2
MR1
XVII
Training
Training
Set (102)
S1
Expanding MSPC, Sample 1
xV
VI
VII
SIC
PLS1
Y
x1
0.5
0.0
-0.5
-1.0
10.02.04
26
Expanding MSPC , Samples 2 & 3
Sample 2, High Quality, Outsider
1.0
x2
SIC
PLS1
Y
0.5
0.0
-0.5
-1.0
1.0
Sample 3, Normal Quality, Absolute Outsider
0.5
0.0
-0.5
-1.0
10.02.04
x3
SIC
PLS1
Y
27
Expanding MSPC , Samples 4 & 5
1.0
Sample 4, Low Quality, Outsider
x4
SIC
PLS1
Y
x5
SIC
PLS1
Y
0.5
0.0
-0.5
-1.0
1.0
Sample 5, Normal Quality, Insider
0.5
0.0
-0.5
-1.0
10.02.04
28
Active Optimization in Practice
Objective
To find corrections for each process stage that improve the future
process output (product quality)
Concept
Corrections are admissible if they are similar to ones that
sometimes happened in the historical data in the similar situation
Method
Simple Interval Prediction and Status Classification
Approach
Multivariate Statistical Process Optimization (MSPO)
10.02.04
29
Linear Optimization
Linear function always reaches extremum at the border.
So, the main problem of linear optimization is not to find a
solution, but to restrict the area, where this solution should
be found.
y=a*x
x
10.02.04
30
Optimization Problem
Weather
Fixed variables
conditions
X1, X2, XfixX3, X4
Cap’s
Optimized
setup
X5, X6,
Xopt X7
PLS1
Fuel
Quality
Consumption
measure
YY
Model
Y = X*a = Y0 + Xopt*a2, where Y0 = Xfix*a1 = Const
Task
For given Xfix and a1 to find Xopt that maxi(mini)mizes Y
Solution
max (Y) = Y0 + max (Xopt)*a2, as all a > 0 (by g factor)
10.02.04
31
Interval Prediction of Xopt
XI
Xopt
1.2
0.4
XII
XIII
M1
M3
M2
M1
CW3
CW2
CW1
WR2
X opt
WR1
W3
W2
W1
S3
S2
S1
X fix
XIV
PLS2
PLS±2*RMSEP
Borders
PLS
SIC Prediction
0.3
0.8
0.2
0.4
0.1
0.0
1
1
4
2
2
3
3
44
5
55
-0.1
-0.4
-0.2
-0.8
-0.3
-1.2
-0.4
10.02.04
Selected
Selected Test
Test Samples
Samples
32
Dubious Result of Optimization
1.5
Optimized
1.0
0.5
0.0
-0.5
SIC
Y
Y
x4 Test
Test
x4
-1.0
-1.5
I
II
III
IV
V
PLS
x4 Opt
x4 Opt
Limits
VI
VII
Predicted Xopt variables are out of model!
10.02.04
33
Adjustment with SIC Object Status
Concept
Corrections are admissible if they are similar to ones that sometimes
happened in the historical data in the similar situation.
Optimal variables Xopt should be within the model !
M1
SIC Prediction
0.2
0.0
1
2
3
4
5
0.5
Insiders
4
0.0
-0.5
-0.2
Object Status plot
1.0
SIC-Residual
0.4
51
3
3
3
3
2 2 2
-0.4
10.02.04
-1.0
Selected Test Samples
SIC-Leverage
34
Sample 1 Normal Quality Insider
Optimized
1.0
SIC
PLS
Y
x1
0.5
0.0
-0.5
Object Status plot
-1.0
1.0
SIC-Residual
I
0.5
II
III
IV
V
VI
VII
4
3
0.0
5
1.0
1
1.0
SIC
PLS
Y
x1
2
-0.5
0.5
-1.0
SIC-Leverage
0.0
-0.5
-1.0
Test
10.02.04
I
II
III
IV
V
VI
VII
35
Sample 2 High Quality Outsider
1.0
Optimized
0.5
0.0
-0.5
Object Status plot
1.0
SIC
PLS
Y
x2
SIC-Residual
-1.0
0.5
4
3
0.0
5
1
I
II
III
IV
V
VI
VII
1.0
2
1.0
-0.5
-1.0
SIC-Leverage
0.5
0.0
-0.5
SIC
PLS
Y
x2
-1.0
Test
10.02.04
I
II
III
IV
V
VI
VII
36
Sample 3 Normal Quality Abs. Outsider
Optimized
1.0
0.5
0.0
-0.5
Object Status plot
1.0
SIC
PLS
Y
x3
SIC-Residual
-1.0
0.5
4
3
0.0
5
1
I
II
III
IV
V
VI
VII
1.0
1.0
2
-0.5
-1.0
SIC-Leverage
0.5
0.0
-0.5
SIC
PLS
Y
x3
-1.0
Test
10.02.04
I
II
III
IV
V
VI
VII
37
Sample 4 Low Quality Outsider
1.0
Optimized
SIC
PLS
Y
x4
0.5
0.0
-0.5
Object Status plot
1.0
SIC-Residual
-1.0
0.5
I
4
II
III
IV
V
VI
VII
3
0.0
5
1
1.0
1.0
2
-0.5
-1.0
0.5
SIC
PLS
Y
x4
SIC-Leverage
0.0
-0.5
-1.0
Test
10.02.04
I
II
III
IV
V
VI
VII
38
Sample 5 Normal Quality Insider
1.0
Optimized
SIC
PLS
Y
x5
0.5
0.0
-0.5
Object Status plot
-1.0
1.0
SIC-Residual
I
0.5
II
III
IV
V
VI
VII
4
3
1.0
0.0
5
1
1.0
2
-0.5
SIC
PLS
Y
x5
0.5
-1.0
SIC-Leverage
0.0
-0.5
-1.0
Test
10.02.04
I
II
III
IV
V
VI
VII
39
Food Quality
Philosophy of MSPO. Food Industry
Home-made quality
Intuitive (expert)
control
Home-made quality
MSPO
Restaurant quality
Standard (descriptive)
control
Fast Food quality
ISO-9000
Production Effectiveness
10.02.04
40
Conclusions
Thanks and ...
Bon Appetite!
10.02.04
41