Introduction to measurement and statistical analysis
1
V. Rouillard 2003
ASSESSING EXPERIMENTAL DATA : ERRORS
•
•
Remember: no measurement is perfect – errors always exist.
•
We can only estimate the size of the error or its likelihood that it exceeds a certain
value.
•
•
Errors can be estimated statistically when large number of measurements are taken.
Measurement error is defined as the difference between the true value and the
measured value.
However, must ensure that measurement systems are calibrated.
Introduction to measurement and statistical analysis
2
V. Rouillard 2003
TYPES OF ERROR
•
•
Most errors can be put into two classes: Bias errors and Precision errors.
•
Precision errors are also called random errors and are different for each measurement
made. However, the average value of the random error is zero.
•
If enough measurements are repeated, the distribution of precision errors will be
revealed and the likely size of the error can be estimated statistically.
•
Because bias errors are fixed and do not produce a statistical distribution, they cannot
be estimated using statistical techniques.
•
They can only be estimated by comparison with a standard or another instrument or
even by experience and common sense.
Bias errors are also referred to as systematic errors and remain the same for every
measurement made.
3
Introduction to measurement and statistical analysis
V. Rouillard 2003
TYPES OF ERROR
Frequency of occurrence
Frequency of occurrence
Large bias error &
small random error
Small bias error &
large random error
4
Introduction to measurement and statistical analysis
V. Rouillard 2003
COMMON SOURCES OF ERROR
Bias errors:
•
•
Calibration (eg: zero-offset and scale adjustments)
•
Certain errors caused by defective equipment (eg: poor design, fabrication
and maintenance)
•
•
Loading errors (eg: microphone, vehicle speed gun)
Certain consistently recurring human errors (eg: parallax, poor
synchronisation)
Resolution limitations (eg: lack of significant figures in digital displays)
Random errors:
•
•
•
Certain human errors (eg: lack of concentration)
•
System sensitivity imitations (eg: use bathroom scale to measure mass of
small animal)
Disturbances to equipment (eg: ground vibrations, atmospheric conditions)
Fluctuating experimental conditions (eg: poor experimental design, must
account for inherent oscillations/variations of the measurand)
5
Introduction to measurement and statistical analysis
V. Rouillard 2003
COMMON SOURCES OF ERROR
Combined errors:
•
Backlash, friction and hysteresis (eg: in mechanical indicators such as
pressure gauges)
•
•
Calibration drift or reaction to changing environmental conditions.
Variations in procedure (eg: when short cuts are taken or personnel changes)
Illegitimate errors (mistakes):
•
Blunders and mistakes (eg: forgot to switch on amplifier, write phone number
instead of reading)
•
Computational errors (eg: use wrong calibration constant)
Introduction to measurement and statistical analysis
6
V. Rouillard 2003
INSTRUMENT PERFORMANCE : TERMINOLOGY
•
Accuracy: (expected) closeness with which a measurement approaches the true
value.
•
Precision: indication of the reproducibility of measurements. If a variable is fixed,
precision is the measure of the degree to which successive measurements differ
from one another.
•
•
Resolution: The smallest change in the measurand that the instrument will detect.
•
Error: Difference between the true value and the measured value.
Sensitivity: The ratio of the instrument response to an change in the measured
quantity. Eg: and accelerometer with a sensitivity of 100 mV/g is more sensitive
than one with a sensitivity of 10 mV/g.
Introduction to measurement and statistical analysis
7
V. Rouillard 2003
UNCERTAINTY : ESTIMATING THE LEVEL OF MEASUREMENT ERROR.
•
Total uncertainty, U, combines the bias and random uncertainties as follows:
U x  ( Bx2  Rx2 )
•
This method is based on the assumption that the sources of bias and random
errors are independent and they are therefore unlikely to coincide.
•
Remember: The bias uncertainty is estimated from calibration checks while the
random uncertainty is estimated by statistical analysis of repeat measurements.
Introduction to measurement and statistical analysis
8
V. Rouillard 2003
ESTIMATING THE RANDOM (PRECISION) UNCERTAINTY
Statistical Analysis
•
A measurement sample is drawn from the population to make an estimate of the
measurand.
•
•
In may be that no two samples (4 blades) will have precisely the same value.
But each sample (and specimen) should approximate the average value for the
population
Population
Introduction to measurement and statistical analysis
9
V. Rouillard 2003
ESTIMATING THE RANDOM (PRECISION) UNCERTAINTY
Statistical Analysis
•
A measurement sample is drawn from the population to make an estimate of the
measurand.
•
•
In may be that no two samples (4 blades) will have precisely the same value.
But each sample (and specimen) should approximate the average value for the
population
Population
Sample
(random selection
from population)
Introduction to measurement and statistical analysis
10
V. Rouillard 2003
ESTIMATING THE RANDOM (PRECISION) UNCERTAINTY
Statistical Analysis
•
Manufacturing (production) uncertainty: Analyse repeat measurements from the sample
(each specimen is measured only once).
•
Experimental uncertainty: Analyse repeat measurements from one individual specimen
only.
Introduction to measurement and statistical analysis
11
V. Rouillard 2003
ESTIMATING THE RANDOM (PRECISION) UNCERTAINTY
Statistical Analysis
•
Statistical analysis and interpretation meaningful only of a (relatively) large number of
measurements are made.
•
Systematic errors should be kept small. Statistical treatment cannot remove systematic
(bias) errors.
•
Arithmetic mean:
x
•
x
n
Deviation from the mean: Difference between an individual reading and the mean of
the group of readings: (note: the algebraic sum of all deviations = zero)
d1  x1  x
d2  x2  x .......
d n  xn  x
Introduction to measurement and statistical analysis
12
V. Rouillard 2003
ESTIMATING THE RANDOM (PRECISION) UNCERTAINTY
Statistical Analysis
•
Average deviation: an indication of the precision of the measurements:
D
•
d1  d2  d3  d4 ....... d n
n

d
n
Standard deviation: the root-mean-square (RMS) deviation of the measurements.
For a finite number of readings:
d12  d22  d32  d42  d42 ....... d n2


n 1
•
Variance: mean-square deviation = 2
2
 dt
n 1
Introduction to measurement and statistical analysis
13
V. Rouillard 2003
ESTIMATING THE RANDOM (PRECISION) UNCERTAINTY
Statistical Analysis
•
Case Study: Measurement of the mass of turbine blades for use in jet propulsion
systems. Blades are supplied by different manufacturers. Mass must be established
based on random sample.
Introduction to measurement and statistical analysis
14
V. Rouillard 2003
ESTIMATING THE RANDOM (PRECISION) UNCERTAINTY
Statistical Analysis
Probability Distribution of errors: The frequency distribution of observations can
be calculated and displayed graphically using a histogram or frequency distribution
plot:
Mass [g]
Number of
observations
318.0
318.5
319.0
319.5
320.0
320.5
321.0
321.5
322.0
1
2
12
24
34
27
16
3
1
34
Number of observations
•
27
24
16
12
3
1 2
1 1
Mass [g]
Introduction to measurement and statistical analysis
15
V. Rouillard 2003
ESTIMATING THE RANDOM (PRECISION) UNCERTAINTY
Statistical Analysis
If more observations were made it is expected that the frequency distribution of the
observations will become more defined:
Number of observations
•
Mass [g]
Introduction to measurement and statistical analysis
16
V. Rouillard 2003
ESTIMATING THE RANDOM (PRECISION) UNCERTAINTY
Statistical Analysis
This bell-shaped curve has been shown to approach the distribution function called
the Normal or Gaussian distribution.
2

f ( x)  n exp   x    


Number of observations
•
Mass [g]
Introduction to measurement and statistical analysis
17
V. Rouillard 2003
ESTIMATING THE RANDOM (PRECISION) UNCERTAINTY
Statistical Analysis
This bell-shaped curve has been shown to approach the distribution function called
the Normal or Gaussian distribution.
 1  x   2 
1
p( x ) 
exp   
 
 2
 2    
0.5
0.4
0.3
p(x)
•
0.2
0.1
0
-4
-3
-2
-1
0
(x-)/
1
2
3
4
Introduction to measurement and statistical analysis
18
V. Rouillard 2003
ESTIMATING THE RANDOM (PRECISION) UNCERTAINTY
Statistical Analysis
The normal distribution function characteristically has few observations at the high
and low ends and many in the middle. It has been shown to be very useful in for
evaluating random errors.
Mass [g]
318.0
318.5
319.0
319.5
320.0
320.5
321.0
321.5
322.0
34
Number of
observations
1
2
12
24
34
27
16
3
1
Number of observations
•
27
24
16
12
3
1 2
1 1
Mass [g]
Introduction to measurement and statistical analysis
19
V. Rouillard 2003
ESTIMATING THE RANDOM (PRECISION) UNCERTAINTY
Statistical Analysis
Comments on the normal distribution of random errors:
•
•
•
•
•
All observations include small, disturbing effects called random errors.
Random errors can be positive or negative with equal probability.
Small errors are more likely to occur that large errors.
Very large errors (> 3) are very improbable
The probability of a given error will be symmetrical about zero.
Introduction to measurement and statistical analysis
20
V. Rouillard 2003
ESTIMATING THE RANDOM (PRECISION) UNCERTAINTY
 1  x   2 
Statistical Analysis
1
p( x ) 
exp   
 
2

Interpretation of the normal distribution of random errors:
 2

 

Error Level
Error Level Probability
Terminology
Probable error
Std deviation
90% error
2-Sigma error
3-Sigma error
4-Sigma error







[%]
0.6754
50.0
1
68.3
1.645
90.0
1.96
95.0
3
99.7
4
99.994
Introduction to measurement and statistical analysis
21
V. Rouillard 2003
ESTIMATING THE RANDOM (PRECISION) UNCERTAINTY
Statistical Analysis
Turbine blade mass case study:
Mean:
319.75 g
Number of observations
•
Mass [g]
Standard
deviation:
0.75 g
Introduction to measurement and statistical analysis
22
V. Rouillard 2003
ESTIMATING THE STATISTICAL DISTRIBUTION OF A RANDOM PROCESS
•
Case Study: Analysis of maximum daily wave height for the design of an offshore
structure. Measurements made continuously by a wave rider buoy which stores the
daily maximum wave height and transmits the data to a base station.
Introduction to measurement and statistical analysis
23
V. Rouillard 2003
ESTIMATING THE STATISTICAL DISTRIBUTION OF A RANDOM PROCESS
Daily max. wave height [m]
Sample: record for one year (random?)
Day of year
Introduction to measurement and statistical analysis
24
V. Rouillard 2003
ESTIMATING THE STATISTICAL DISTRIBUTION OF A RANDOM PROCESS
The frequency distribution of observations can be calculated and displayed
graphically using a histogram or frequency distribution plot:
Maximum Daily
Wave Height
[m]
Number of
observations
72
67
62
2
3
4
5
6
7
8
9
10
11
12
13
14
15
4
12
14
46
67
72
62
38
24
13
3
0
0
1
Number of observations
•
46
38
24
12
14
13
4
3
Wave height [m]
1
Introduction to measurement and statistical analysis
25
V. Rouillard 2003
ESTIMATING THE STATISTICAL DISTRIBUTION OF A RANDOM PROCESS
With more observations, it is expected that the frequency distribution will approach
the Normal or Gaussian distribution
Number of observations
•
Wave height [m]
Introduction to measurement and statistical analysis
26
V. Rouillard 2003
ESTIMATING THE STATISTICAL DISTRIBUTION OF A RANDOM PROCESS
The normal distribution has been shown to be very useful in for describing many
random variables such as test scores, people height, weight etc.,
Maximum Daily
Wave Height
[m]
Number of
observations
2
3
4
5
6
7
8
9
10
11
12
13
14
15
4
12
14
46
67
72
62
38
24
13
3
0
0
1
Number of observations
•
Wave height [m]
Introduction to measurement and statistical analysis
27
V. Rouillard 2003
ESTIMATING THE RANDOM (PRECISION) UNCERTAINTY
Statistical Analysis
The normal (Gaussian) distribution is a function of the mean and standard deviation
of the sample:
f(x) 
•
7.5 m
L
O
F
I
M
P
H
K
M
P
N
Q
1
1 x
exp 
2 
 2
2
2.0
Where  is the mean
And  the standard deviation.
In this example:
The mean daily max. height = 7.5 m
The standard deviation is = 2.0 m
3 (99.7%  332 days per 333 days)
the expected ann. max. wave height is:
7.7 + 3(2.0) = 13.7 m
Number of observations
•
4 (99.994%  all but 1 day per 45 yrs)
the expected max. wave height over
45 yrs is: 7.7 + 4(2.0) = 15.7 m
Wave height [m]