Critique of experiment on baking duration
and moisture content of bread
1. An important aim of most experiments is to remove all other
effects/biases/confounding factors, so that any observed differences are only the result
of the experimental manipulation of interest. In the above example, there are a
number of possible sources of variation, other than duration of baking:
- To reduce error due to differences (e.g. moisture content, yeast levels, flour
quality) among different types of dough, the four pieces of dough should be from
same batch of dough.
- The experimental method involved placing all pieces of dough “in a single oven
at the same time and at a constant temperature of 150° C”. Another possible
source of error is the baking process; the oven door must be opened on three
different occasions before the last sample is removed. Thus, there will be
variations in temperature (due to opening the oven door) that will be experienced
during baking on none, one, two and three occasions by the 12-, 14-, 16- and 18minute samples respectively.
This is a good example of the subtle biases that can creep in to experiments; they can
be extremely difficult to spot! This is another reason why it is such a good idea to
discuss your experimental design with others, as it increases the chance of someone
noticing the introduction of such a bias.
Assuming that the above bias (variation in temperature due to opening the oven
door) is unacceptable, can you think of a methodology that overcomes it (without
introducing a new bias!)?
Using an example from your own research experience, describe an experimental
design that deliberately contains an experimental bias.
2. Four pieces of dough are baked, and only one piece of dough is sampled at each of
the four periods. Although the single piece of dough is sampled five times, this only
improves the accuracy of the measurement for one piece of dough. Essentially, there
is only one replicate per treatment level, which is a major deficiency in the
experimental design when it occurs.
Why is replication necessary? It may help you to consider the repercussions of
relying on one replicate.
What would be an appropriate solution to the lack of replication in the above
experiment?
3. Using these data in an ANOVA is classic pseudoreplication and, generally
speaking, is a very significant weakness in experimental design. Again, unfortunately,
it is a very common occurrence. Parametric analyses such as ANOVA assume that
data points are independent: the five data points in the above experiment are five
subsamples that are dependent on the single piece of dough that was sampled.
For a thorough discussion of pseudoreplication and several examples, read the classic
paper by Hurlbert, S. H. (1984) Pseudoreplication and the design of ecological
experiments. Ecological Monographs 54: 187-211.
4. The following table contains data from two experiments that investigated the effect
of baking duration on the moisture content of bread. The column titled ‘Subsamples’
indicates the data from an experimental design as described above (i.e. five
subsamples from one replicate at each treatment level). The column titled ‘Replicates’
indicates the data from an experimental design that sampled moisture content from
five replicates at each treatment level.
Graph the data for comparison of the results from the two experiments.
Interpret the data, keeping in mind the differences in the design of replication
between the two experiments.
Time
(min)
12
12
12
12
12
14
14
14
14
14
16
16
16
16
16
18
18
18
18
18
Subsampl
es
32
30
30.5
31
29
29
32
30
31
30
26.5
27
25
26
27
26
25
25
25
27
Replica
tes
30
35
33
32
28
27
26
29
30
27
29
25
27
25
28
26
24
21
23
24