Unit 3: Analysis of scientific data and information
.
34
Limitations in
concluding results
Earlier in this unit (Topic 3.2, Section 3) we looked at accounting for how
compound errors affect calculated values. Such errors are derived from
experiments and can thus be identified and controlled. However, errors
caused by mistakes are equally common and need to be carefully managed.
On successful completion of this topic you will:
•• understand limitations in concluding results (LO4).
To achieve a Pass in this unit you need to show that you can:
•• evaluate the total error in a sample of continuous scientific data (4.1)
•• assess the accuracy of a model using the outcomes of processing carried
out on experimental data (4.2)
•• justify the validity of conclusion(s) from the information on a problem
studied (4.3).
1
Unit 3: Analysis of scientific data and information
1 Controlling errors in results
Errors in results can easily arise due to carelessness with regards to how values
are written and used. For example, the volume of a sphere is given by the formula
v = 43 pr3.
The figure 43 is a rational number but its decimal expansion is infinite; therefore
anything other than 43 being used in a calculation will result in a compound error
in the final answer. Values such as π cannot be represented by a fraction of any
kind and so using any approximated value will result in some level of compound
error; in such cases, we simply use the value produced by the technology used in
the calculation.
Similar problems can occur when using a value from one calculation, recording it
truncated but then going on to use it in the reduced form; excessive rounding up
or down of a value also induces compound errors. The solution to these issues is to
never use a truncated or rounded value in calculation (unless the figure has to be
an integer). Although it is perfectly acceptable to display numbers in such a way,
any mathematical procedures should be done to the maximum possible number
of digits. Modern technology, via the use of memory functions, permits this to be
done with ease.
Example
Calculate the distance between planes of atoms using Bragg’s equation if a first
order scattering angle of 22.7° is observed for a wavelength of 524 nm.
2d sin θ = λ
λ
524 3 10–9
d=
=
= 6.7892173551357105396116463829427 x 10–7 m
2 sin θ 2 3 sin(22.7)
Observe the difference when an angle of 23° (i.e. rounded) and a wavelength of
520 nm (i.e. truncated) are used:
d=
λ
520 3 10–9
=
= 6.6541921296433757123768648695213 x 10–7 m
2 sin θ 2 3 sin (23)
There is nearly a 2% difference between the two answers, solely due to numerical
compound errors. However, it should be noted that truncating and rounding are
not necessarily bad things to do per se; you would not normally display answers
to such a large number of decimal places! As a general rule, rounding must never
occur during a calculation (no matter how many stages are involved) but the final
value should be rounded; the guide as to the amount of rounding is given by the
value in the calculation that has the least accuracy.
In the above example, both values (λ and θ) are given to three significant figures
and thus the final answer should be written to the same number of figures, that is,
d = 6.79 3 10–7 m.
By managing errors correctly, through any mathematical analysis of data, the level
of confidence in your results will be naturally greater than if you do not make an
effort to avoid inducing mistakes or unaccounted errors during calculations.
3.4: Limitations in concluding results
2
Unit 3: Analysis of scientific data and information
Checklist
At the end of this section, you should be able to:
 evaluate the total error in a sample of continuous scientific data by understanding when to
use rounding and truncating correctly.
2 Conclusions from work
At school-level science, experiments are typically performed to verify a known
model for explaining a physical process. For levels higher than this, the physical
procedures are much the same but now the models being examined may be
newly formed and need verifying, or are poorly understood and need exploring
– there is no longer an obviously ‘correct’ answer. This is a rather significant
difference! So how do you go about using information gained through the analysis
of experimental data to evaluate hypotheses and models?
Scientific models should allow predictions to be made about the behaviour of the
natural process being examined; such hypotheses can be directly tested to see if
there is any correlation between the variables in question. There is a commonly
used phrase, ’correlation does not equal causation’, when first learning about
correlation and it also applies here.
Imagine an experiment where a student discovers near-perfect positive correlation
between two variables, tested to a 99% confidence level. We might accept the
student’s evaluation that the data matches the prediction from the model, thus
verifying it, because it would be statistically very unlikely for the correlation to
not be genuinely present. But there are some important questions that should be
asked of the work before doing so:
•• Were all variables correctly identified?
•• Were all of the control variables correctly managed and monitored?
•• Were all errors correctly managed in the analysis?
•• Were the correct mathematical procedures used in the numerical analysis?
•• Were the sample sizes large enough to justify the use of the statistical analysis?
•• Was the correct distribution used in the statistical analysis? For example,
was a normal distribution used when a t-distribution would have been more
appropriate in the hypothesis test?
These are just a few of the questions that need to be considered as part of any
evaluation. It could be argued that it is always better to be highly unconvinced
of any correlation observed; attempting to ‘disprove’ it provides the material with
which the evaluation of the experiment and model can be constructed.
But this also needs to be tempered with an element of restraint. For example,
when considering control variables do you need to account for changes in air
pressure during a working day affecting the evaporation rate of a liquid? Certainly
if it is a critical aspect of the investigation but, if not, then this is a variable that,
although identified, does not need to be controlled and monitored.
Perhaps the greatest aid to evaluating results and assessing the validity of a model
is collaboration. By sharing experimental methods, models can be repeatedly
3.4: Limitations in concluding results
3
Unit 3: Analysis of scientific data and information
tested by other students and researchers; by sharing results, separate sources
can be statistically assessed for independence. In the scientific community, the
validity of models is tested and accepted through sheer weight of evidence,
until developments in the theoretical background force the previous data to be
reanalysed or new data, generated with different techniques or equipment, comes
to light.
Take it further
Ruggedness testing is a process whereby a series of experiments are conducted to determine
how significant each variable is, in terms of its impact on the method to be employed in an
investigation. There are several texts on the subject, which explore the topic in detail, for example:
Ellison et al. (2009) Practical Statistics for the Analytical Scientist, RSC and Swartz, M. and Krull, I.
(2012) Handbook of Analytical Validation, CRC Press.
The National Measurement System website also provides an overview:
http://www.nmschembio.org.uk/.
Checklist
At the end of this topic guide you should be able to
 assess the accuracy of a model using the outcomes of processing carried out on
experimental data
 justify the validity of conclusion(s) from the information on a problem studied.
Further reading
Currell, G. and Dowman, A. (2009) Essential Mathematics and Statistics for Science, Wiley, NY.
Acknowledgements
The publisher would like to thank the following for their kind permission to reproduce their
photographs:
Corbis: Radius Images
Every effort has been made to trace the copyright holders and we apologise in advance for any
unintentional omissions. We would be pleased to insert the appropriate acknowledgement in any
subsequent edition of this publication.
About the author
Nick Evanson is a mathematics teacher at Windermere School, UK. He has worked in education for
14 years, at secondary, tertiary and undergraduate levels. He served on the editorial board of the
Physics Education journal, and authored scientific and mathematical modules in Chemistry and
Nuclear Decommissioning courses for Edexcel, University of Cumbria and University of Central
Lancashire. Nick is married and enjoys the outdoor life in Cumbria, regardless of the weather.
3.4: Limitations in concluding results
4