VERIFY Intershot Data Validation

A maximum likelihood analysis of
the L-H transition DB
Darren McDonald
9th ITPA Confinement Database and Modelling Topical Physics Group
meeting in St. Petersburg
1/13
Introduction
•
•
Is L-H scaling sensitive to error models +
if so, is the appropriate one used?
OLS fits are appropriate when
1. Errors in P >> than in other parameters
2. Relative errors same for all experiments
3. Logs of variables ≈ Normally distributed
•
•
All are violated to some extent
Use Maximum-Likelihood to test impact
9th ITPA Confinement Database and Modelling Topical Physics Group
meeting in St. Petersburg
2/13
Maximum-Likelihood method
• Soln is one which makes data most likely
c
c
c


P

f
x
;
c

c

S

B

n
• For
Likelihood is
1
3
2
 I
1 
pP | x, c      eff ,i   exp  12  2
 i 1


I
Pi  f (x i ; c)
i 1
 eff ,i 2
2  
2
4

 eff ,i 2   p ,i 2
2
 f (x i ; c) 
  i, j 2
 
 x

j 1 
i, j

J
• Problem is now Non-Linear, but has been solved
by MINUIT package. Take IAE04R dataset.
9th ITPA Confinement Database and Modelling Topical Physics Group
meeting in St. Petersburg
3/13
OLS model - assumptions
1. Errors in P >> than in other parameter
2. Relative errors same for all experiments
3. Logs of variables ≈ Normally distributed
9th ITPA Confinement Database and Modelling Topical Physics Group
meeting in St. Petersburg
4/13
OLS model - fits
• M-L model + i), ii) and
iii) agrees with OLS
• Now relax
assumptions in turn
Statistical model
c1.102
cS
cB
cn
1. OLS
7.7
0.80
0.65
0.44
2. M-L with i), ii) and iii)
7.7
0.80
0.65
0.44
3. EVOR
7.5
0.85
0.58
0.56
mean
7.5
0.85
0.58
0.56
5. M-L with ii) and iii)
only
7.5
0.85
0.58
0.56
6. M-L with iii) only
7.7
0.97
0.32
0.88
7. OLS adjusted for log
bias
7.6
0.80
0.65
0.44
4.
EVOR with
errors
9th ITPA Confinement Database and Modelling Topical Physics Group
meeting in St. Petersburg
5/13
EVOR model - assumptions
1. Errors in P >> than in other parameter
•
Relax to include all errors
2. Relative errors same for all experiments
3. Logs of variables ≈ Normally distributed
9th ITPA Confinement Database and Modelling Topical Physics Group
meeting in St. Petersburg
6/13
EVOR model - fits
• M-L model + ii) and iii)
agrees with EVOR
• Two methods for
averaging errors ≈
same answer
• Differ from OLS 
OLS biases result
Statistical model
c1.102
cS
cB
cn
1. OLS
7.7
0.80
0.65
0.44
2. M-L with i), ii) and iii)
7.7
0.80
0.65
0.44
3. EVOR
7.5
0.85
0.58
0.56
mean
7.5
0.85
0.58
0.56
5. M-L with ii) and iii)
only
7.5
0.85
0.58
0.56
6. M-L with iii) only
7.7
0.97
0.32
0.88
7. OLS adjusted for log
bias
7.6
0.80
0.65
0.44
4. EVOR
errors
with
9th ITPA Confinement Database and Modelling Topical Physics Group
meeting in St. Petersburg
7/13
Log M-L model - assumptions
1. Errors in P >> than in other parameter
•
Relax to include all errors
2. Relative errors same for all experiments
•
Relax to allow machine-machine variation
3. Logs of variables ≈ Normally distributed
9th ITPA Confinement Database and Modelling Topical Physics Group
meeting in St. Petersburg
8/13
Log M-L model - fits
• M-L model iii) only
differs from OLS and
EVOR  assumption
ii) biases results
• Are we sure about
tokamak error
estimates?
• Easy to extend to
point-point variation
Statistical model
c1.102
cS
cB
cn
1. OLS
7.7
0.80
0.65
0.44
2. M-L with i), ii) and iii)
7.7
0.80
0.65
0.44
3. EVOR
7.5
0.85
0.58
0.56
mean
7.5
0.85
0.58
0.56
5. M-L with ii) and iii)
only
7.5
0.85
0.58
0.56
6. M-L with iii) only
7.7
0.97
0.32
0.88
7. OLS adjusted for log
bias
7.6
0.80
0.65
0.44
4. EVOR
errors
with
9th ITPA Confinement Database and Modelling Topical Physics Group
meeting in St. Petersburg
9/13
M-L model - assumptions
1. Errors in P >> than in other parameter
•
Relax to include all errors
2. Relative errors same for all experiments
•
Relax to allow machine-machine variation
3. Logs of variables ≈ Normally distributed
•
Relax by using real variables
9th ITPA Confinement Database and Modelling Topical Physics Group
meeting in St. Petersburg
10/13
M-L model - fits
• M-L model differs
again  skewing of
logs influences results
• Attempt to correct this
in OLS method (7)
failed
• Are we sure real
errors are Normally
distributed?
Statistical model
c1.102
cS
cB
cn
1. OLS
7.7
0.80
0.65
0.44
2. M-L with i), ii) and iii)
7.7
0.80
0.65
0.44
3. EVOR
7.5
0.85
0.58
0.56
mean
7.5
0.85
0.58
0.56
5. M-L with ii) and iii)
only
7.5
0.85
0.58
0.56
6. M-L with iii) only
7.7
0.97
0.32
0.88
7. OLS adjusted for log
bias
7.6
0.80
0.65
0.44
8. M-L,Errors on P only
5.6
0.86
0.72
0.58
9. M-L
6.0
0.96
0.45
0.80
4.
EVOR with
errors
9th ITPA Confinement Database and Modelling Topical Physics Group
meeting in St. Petersburg
11/13
Consistency, errors and ITER
Statistical
model
c1.102
cS
cB
cn
χ2N
PITER
OLS
7.7 ± 0.3
0.80 ± 0.01
0.65 ± 0.03
0.44 ± 0.03
7.43
35.6
EVOR
7.5 ± 0.3
0.85 ± 0.02
0.58 ± 0.03
0.56 ± 0.03
7.09
43.4
M-L
6.0 ± 0.3
0.96 ± 0.02
0.45 ± 0.04
0.80 ± 0.05
6.26
59.0
• All models differ by more than their errors
• M-L gives lowest χ2N for model, but still >>1 
model still has missing features  must improve
before confidence can be placed in this method
• ITER prediction highest for M-L
9th ITPA Confinement Database and Modelling Topical Physics Group
meeting in St. Petersburg
12/13
Conclusion
• M-L method shown consistent with OLS and
EVOR where assumptions are the same
• All three assumptions looked at biased scaling
• χ2N >> 1  model has missing features  must
have refine error model to use method
• ITER prediction higher for M-L
• Prudent estimates may come from average of a
set of error models
9th ITPA Confinement Database and Modelling Topical Physics Group
meeting in St. Petersburg
13/13