1
THEORY OF SAMPLING
MMEA Certainty Seminar 6.11.2013
Markku Ohenoja
Markku Ohenoja / Control Engineering Laboratory
6.11.2013
2
BACKGROUND
• Literature review on measurement uncertainty (MU)
• Including sampling process, composite samples, digital signal
processing
• Importance of MU estimation well recognized
• ISO standards/guides present the well-established methods
defined in metrology
• The methods basically assume a correct sampling process and
unbiased measurements
 Estimation of measurement uncertainty in the presence of
sampling errors and systematic errors
Markku Ohenoja / Control Engineering Laboratory
6.11.2013
3
OUTLINE
• General approach to MU estimation
• Modeling approach
• Empirical approach
• Theory of sampling
•
•
•
•
Motivation
Sampling strategies
Sampling errors
Variographic analysis
• Other methods for estimating sampling uncertainty
• Uncertainty arising from systematic errors
Markku Ohenoja / Control Engineering Laboratory
6.11.2013
4
GENERAL APPROACH TO THE ESTIMATION OF
MEASUREMENT UNCERTAINTY
Markku Ohenoja / Control Engineering Laboratory
6.11.2013
5
MODELING APPROACH
MATHEMATICAL/THEORETICAL/PREDICTIVE/BOTTOM-UP APPROACH
• Modeling the measurement
•
•
•
•
Defining the measurement model (inputs of the model)
Defining the ”spread” of input values
Correlations and sensitivity
Calculating combined (and expanded) uncertainty
• Well established for analytical measurements
• ISO guide/JCGM100/GUM
Markku Ohenoja / Control Engineering Laboratory
6.11.2013
6
MODELING APPROACH
MATHEMATICAL/THEORETICAL/PREDICTIVE/BOTTOM-UP APPROACH
• Modeling the measurement
•
•
•
•
Defining the measurement model (inputs of the model)
Defining the ”spread” of input values
Correlations and sensitivity
Calculating combined (and expanded) uncertainty
• Well established for analytical measurements
• ISO guide/JCGM100/GUM, EURACHEM guide…
• Some drawbacks in the original guidance found and corrected
• Interpretation with complete Bayesian theory
• Non-Gaussian probability distributions: JCGM101
• Other drawbacks found
• Cannot treat systematic effects
• Usually do not recognize sampling as a source of uncertainty
Markku Ohenoja / Control Engineering Laboratory
6.11.2013
7
EMPIRICAL APPROACH
EXPERIMENTAL/RETROSPECTIVE/TOP-DOWN APPROACH
• Inter-organizational trials, internal methods validations, quality
control
• Duplicate method + statistical analysis
•
•
•
•
•
•
Well established for analytical measurements
EURACHEM guide
Cannot identify the sources of error components
Sampling+analytical precision and analytical bias can be estimated
Applicable for heterogeneous targets, field sampling…
Estimating sampling bias is more challenging
• Other methods
•
•
•
•
Sampling protocols
Collaborative trial in sampling
Sampling proficiency test
Variographic analysis
Markku Ohenoja / Control Engineering Laboratory
6.11.2013
8
GY’S THEORY OF SAMPLING (TOS)
Markku Ohenoja / Control Engineering Laboratory
6.11.2013
9
MOTIVATION OF TOS
• An analyst usually just analyses an object delivered to him
Defining the measurement target (that object or the lot from
where it was taken)
• Heterogeneity of the sampling target ignored
 Grab sampling  non-representative sample
• How to remove a sample from the lot?
• How much material should be selected?
• How often samples should be taken?
• Also applies for secondary sampling etc.
Markku Ohenoja / Control Engineering Laboratory
6.11.2013
Petersen, L., Minkkinen, P. & Esbensen, K.H. 2005, "Representative sampling for reliable data analysis: Theory
of Sampling", Chemometrics and Intelligent Laboratory Systems, vol. 77, no. 1–2, pp. 261-277.
10
EXAMPLES OF POOR SAMPLING
Grab
sample
Composite
sample
Markku Ohenoja / Control Engineering Laboratory
6.11.2013
Paakkunainen, M., Reinikainen, S.-. & Minkkinen, P. 2007, "Estimation of the variance of sampling of process
analytical and environmental emissions measurements", Chemometrics and Intelligent Laboratory Systems, vol.
88, no. 1, pp. 26-34.
11
SAMPLING ERRORS ACCORDING TO TOS
Sample weighting error
Incorrect sampling errors:
• Incremental delimitation error
• Increment extraction error
• Increment preparation error
Point selection errors:
• Long-term PSE / Time fluctuation error
• Periodic PSE / Cyclic fluctuation error
Fundamental sampling error
Markku Ohenoja / Control Engineering Laboratory
Grouping and segregation error
6.11.2013
12
ESTIMATION OF SAMPLING ERRORS
• Modeling approach
•
•
•
•
FSE estimation
Mainly for particulate systems
Expectance value for critical particles in the sample
Gy’s formula
• Empirical approach
 Variographical analysis / Variography
• Central 1-D TOS tool for practical sampling purposes
• FSE+GSE+TAE+IPE = minimum possible error
• Identifying PSEs
• Sampling intervals and sampling strategies
• Compositing of samples in spreadsheets
• Process Analysis Tool
Markku Ohenoja / Control Engineering Laboratory
6.11.2013
Ramsey, M.H. & Ellison, S.L.R. (eds) 2007, Eurachem/EUROLAB/CITAC/Nordtest/AMC Guide: Measurement
uncertainty arising from sampling: a guide to methods and approaches, Eurachem.
SAMPLING STRATEGIES
Markku Ohenoja / Control Engineering Laboratory
6.11.2013
13
14
VARIOGRAPHY - MATHEMATICS
• Collection of the data
• At least 30 samples with systematic sampling
• 1/5 smaller sampling interval than routine samples
• Calculation of “the heterogeneity” of the data
• Calculation of the experimental variogram v(j)
ℎ𝑛 =
𝑎𝑛 − 𝑎𝐿 𝑀𝑛
∙
𝑎𝐿
𝑀𝑛
• Relationship between the samples and the lag distance j
1
𝑣 𝑗 =
2(𝑁 − 𝑗)
• Estimation of the intercept v(0) (nugget)
• Graphically, separate experiment…
• Auxiliary functions for comparing sampling strategies
• Point-to-point calculation, algebraic modeling…
Markku Ohenoja / Control Engineering Laboratory
6.11.2013
𝑁/2
ℎ𝑛+𝑗 − ℎ𝑗
𝑛=1
2
≈
2
Esbensen, K.H., Paoletti, C. & Minkkinen, P. 2012, "Representative sampling of large kernel lots I. Theory of
Sampling and variographic analysis", TrAC Trends in Analytical Chemistry, vol. 32, no. 0, pp. 154-164.
VARIOGRAPHY - VARIOGRAM
Range
• Lag distance where variogram levels off
• Lags below this are more and more
autocorrelated and TSE is smaller
Sill
• Maximum variance of the data series
investigated
• Time series is no longer correlated
Nugget, v(0)
• Variance in 0-D situation
• Minimum possible error
Markku Ohenoja / Control Engineering Laboratory
6.11.2013
15
Petersen, L. & Esbensen, K.H. 2005, "Representative process sampling for reliable data analysis - a tutorial",
Journal of Chemometrics, vol. 19, no. 11-12, pp. 625-647.
VARIOGRAPHY - EXAMPLES
Simulated cross-stream sampling example
Analytical results
Sample mass
Variogram
Markku Ohenoja / Control Engineering Laboratory
6.11.2013
16
Petersen, L. & Esbensen, K.H. 2005, "Representative process sampling for reliable data analysis - a tutorial",
Journal of Chemometrics, vol. 19, no. 11-12, pp. 625-647.
VARIOGRAPHY - EXAMPLES
Simulated cross-stream sampling example
Random
Stratified
Systematic
Sampling variance
3 * Standard deviation (%)
Markku Ohenoja / Control Engineering Laboratory
6.11.2013
17
Paakkunainen, M., Reinikainen, S.-. & Minkkinen, P. 2007, "Estimation of the variance of sampling of process
analytical and environmental emissions measurements", Chemometrics and Intelligent Laboratory Systems, vol.
88, no. 1, pp. 26-34.
VARIOGRAPHY - EXAMPLES
Sulfur discharge in wastewater in one month period
Markku Ohenoja / Control Engineering Laboratory
Confidence interval for average sulfur
measurement over 1-year period:
• 1 sample/week  52 samples 
U=±1.1% (Systematic sampling)
• 2 samples/week  104 samples 
U=±0.57% (Systematic sampling)
• Assuming normally distributed
measurements and U=±1.1%
acceptable  657 samples needed
(Random
sampling)
6.11.2013
18
Esbensen, K.H., Friis-Petersen, H.H., Petersen, L., Holm-Nielsen, J.B. & Mortensen, P.P. 2007, "Representative
process sampling — in practice: Variographic analysis and estimation of total sampling errors (TSE)",
Chemometrics and Intelligent Laboratory Systems, vol. 88, no. 1, pp. 41-59.
VARIOGRAPHY - EXAMPLES
Monitoring of the daily biogas yield (CH4) in a three month period
Markku Ohenoja / Control Engineering Laboratory
6.11.2013
19
20
OTHER APPROACHES FOR ESTIMATING THE
UNCERTAINTY ARISING FROM SAMPLING
Markku Ohenoja / Control Engineering Laboratory
6.11.2013
21
MODELING APPROACH
• Gy’s formula
Heterogeneity invariant
1
1
𝜎 𝐹𝑆𝐸 =
−
𝐻𝐼𝐿
𝑀𝑆 𝑀𝐿
2
• Fish-bone diagrams
Constitutional
parameter
Particle shape
parameter
𝐻𝐼𝐿 = 𝑐𝛽𝑓𝑔𝑑3
Liberation
parameter
Top particle
size
Size range
parameter
Markku D.,
Ohenoja
/ Control
Engineering
Laboratory
6.11.2013
Lambkin,
Nortcliff,
S. & White,
T. 2004,
"The importance of precision in sampling sludges,
biowastes and
treated soils in a regulatory framework", TrAC Trends in Analytical Chemistry, vol. 23, no. 10–11, pp. 704-715.
22
EMPIRICAL APPROACH
• Duplicate method
• Sampling and analytical precision
𝑥 = 𝑋𝑡𝑟𝑢𝑒 + 𝜀𝑡𝑎𝑟𝑔𝑒𝑡 + 𝜀𝑠𝑎𝑚𝑝𝑙𝑖𝑛𝑔 + 𝜀𝑎𝑛𝑎𝑙𝑦𝑠𝑖𝑠
• Analytical bias from certified reference materials
• Sampling bias if reference target available, or inter-organizational
sampling trials
Markku Ohenoja / Control Engineering Laboratory
6.11.2013
23
UNCERTAINTY ARISING FROM SYSTEMATIC ERRORS
Markku Ohenoja / Control Engineering Laboratory
6.11.2013
24
SOURCES OF SYSTEMATIC ERRORS
• Calibration
• Sampling
• Digital signal processing
• Bias from processing algorithms
• Rounding phenomena (finite word length)
• A/D conversions
• Gain and its temperature effect, long-term stability and
temperature drift of onboard calibration reference, integral
nonlinearity, cross-talk, time jitter
• Usually systematic errors are (assumed to be) corrected
• Some errors cannot be identified or quantified
Markku Ohenoja / Control Engineering Laboratory
6.11.2013
25
ESTIMATION OF SYSTEMATIC ERRORS
• Empirical approach of MU estimation
• Reference targets, inter-organizational sampling trials
• Increment of expanded uncertainty to include bias
• Possibility theory / Theory of evidence
• Random-fuzzy variables
Systematic
Random
Ferrero, A. & Salicone, S. 2004, "The random-fuzzy variables: a new approach to the expression of uncertainty
Markku Ohenoja / Control Engineering Laboratory
6.11.2013
in measurement", Instrumentation and Measurement, IEEE Transactions on, vol. 53, no. 5, pp. 1370-1377.
26
SUMMARY
Markku Ohenoja / Control Engineering Laboratory
6.11.2013
27
SUMMARY
• General approach to MU estimation
• Theory of sampling
• Other methods for estimating sampling uncertainty
• Uncertainty arising from systematic errors
Markku Ohenoja / Control Engineering Laboratory
6.11.2013
28
SUMMARY
• General approach to MU estimation
 Sampling is seldom accounted in MU estimation
• Theory of sampling
 TOS provides tools for estimating sampling errors
 Variography for 1-D objects (process streams)
• Estimation of minimum possible error
• Comparison of sampling intervals and strategies
• Process analytical tool
• Other methods for estimating sampling uncertainty
• Uncertainty arising from systematic errors
 Random-fuzzy approach for systematic errors
Markku Ohenoja / Control Engineering Laboratory
6.11.2013