©2014 Poultry Science Association, Inc.
The use of a pen-size optimization workbook
for experiment research design using the Visual
Basic for Applications in Excel for poultry1
M. Y. Shim and G. M. Pesti2
Department of Poultry Science, University of Georgia, Athens 30602-2772
Primary Audience: Researchers
SUMMARY
Researchers often seem to choose the number of birds per pen and replications per treatment
somewhat arbitrarily on the basis of cost, availability of animals, housing considerations, convenience, tradition, and so on. Statistical simulation, performed while designing an experiment,
will provide a researcher with an objective estimate of the number of birds per pen and replicates needed for an experiment of known Type I and II errors. In most cases, the time for making power calculations should be before, not after, an experiment is conducted. Here we present
a Microsoft Excel workbook to explore the statistical and economic ramifications of different
combinations of birds per replicate and replicates per treatment for research with poultry.
Key words: replication, statistical power, poultry, sample size
2014 J. Appl. Poult. Res. 23:315–322
http://dx.doi.org/10.3382/japr.2013-00911
DESCRIPTION OF PROBLEM
A fundamental question for poultry nutrition
researchers is how many replicates to use in
their experiments. The question is complicated
by the choice of how many birds to place in each
replicate. Each replicate, the experimental unit,
may be individual birds, but more often pens of
5 to more than 1,000 birds are used.
Grouping birds into pens generally saves on
labor. For instance, having 2 pens of 50 birds
each may require less labor and record keeping
than 10 pens of 10 birds each. Likewise, the variation in the averages of pens should increase as
the number of birds in the pens decreases. That
is, a pen of 50 birds should provide a more reliable sample of the genetic diversity in a pen than
1
say, 10 birds. For a constant number of birds in
an experiment (and total feed costs), having
more birds per pen decreases the total number
of pens. Fewer pens means fewer error degrees
of freedom and reduced statistical power just because of the nature of F- and t- distributions used
to evaluate the results.
When evaluating experiments, most researchers are primarily concerned with Type I
error (α): the probability that they will declare
a difference significant when none really exists.
Most researchers are satisfied if they only declare differences to be real when they are not, 1
chance in 20, or P < 0.05. Poultry nutritionists
are more often concerned with another type of
error, type ii error (β). this error occurs when
we say something is not different when it re-
Presented as a part of the informal nutrition Symposium “From Research measurements to Application: Bridging the Gap”
at the Poultry Science Association’s annual meeting in San diego, California, July 22–25, 2013.
2
Corresponding author: [email protected]
JAPR: Symposium
316
Figure 1. Tool overview.
ally is. Poultry nutritionists want to know how
much of a nutrient they can add before any significant increase in response ceases to exist, or
how much of an alternative ingredient they can
add before no significant decrease in response
can be detected. Poultry nutritionists rarely determine or consider Type II error. The normal
standard is to be correct only 4 times out of
5, or P > 0.80. Paradoxically, if the chance of
committing a Type I error, declaring something
different when it is not, is decreased by increasing the critical probability value, the chance of
committing a Type II error, not declaring a real
difference, is increased [1].
The workbook (Figure 1) using the Visual
Basic for Applications in Excel [2] was developed to calculate and simulate statistical and
economical results by choosing different numbers of birds per pen and different numbers of
replicate pens per treatment on the Type I and II
errors of a simple experiment comparing 2 treatments.
MATERIALS AND METHODS
Pen-size optimization workbook for experiment research design (POWER) for Poultry
Experiments is posted on the Poultry Nutrition
web page [3] (Poultry Nutritionist’s Tool Kit
under Poultry Science: Extension and Outreach
in the College of Agricultural and Environmental Sciences at the University of Georgia). The
Dashboard spreadsheet displays a summary
of inputs and outputs. The Simulation (Single
run) spreadsheet reports results of a simulated
experiment based on the specified inputs. The
Simulation (Multiple runs) spreadsheet presents
the output summaries from many simulated experiments with a normal distribution assumption
of the population mean. The Cost_Estimation
spreadsheet calculates expected costs (Experiment Cost.xls) from experiments of with different numbers of birds per pen and different
numbers of pens per treatment (replicates per
experiment). The TC_vs_DD spreadsheet has
tables and charts about relationships between
(1) total cost and detectable difference and (2)
between birds per pen and detectable difference.
The Chart 1 and Chart 2 spreadsheets display
the same tables as in the TC_vs_DD spreadsheet, and these were included for printing.
The Dashboard spreadsheet contains input,
run, and output sections. All inputs should be
entered on the Dashboard spreadsheet (Figure
Shim and Pesti: INFORMAL NUTRITION SYMPOSIUM
317
Figure 2. Setup inputs.
2). The population mean, SD, Type I error, and
Type II errors are input there. The example values of population mean and population SD are
for a flock of female broilers when they were
48-d-old [4]. The number of birds per pen and
replications per treatment entered are those that
define the experiment that will be simulated. The
current costs to conduct experiments also have
to be entered. Appropriate historical or expected
data should be entered under Feed Consumption
Data. A graph is automatically adjusted and the
coefficients are used to generate costs for different weights of birds. Lastly, a feed-planning
table must be modified. This table is used to allocate feed charges for different times during the
birds’ life. Cost calculations will automatically
be adjusted without any manual corrections to
the other spreadsheets. For example, if a researcher plans to use starter, grower, finisher,
and withdrawal feeds, he or she should fill in the
cells with the days that each feed will be fed and
its cost. If the researcher does not want to use
grower feed, he or she should empty the Grower
column and only enter values into the Starter,
Finisher, and Withdrawal feed columns. The
number of simulations to be run and the mean
for a treated group should be entered just above
the Run button (Figure 3). Each time you want
to simulate a new experiment, just click on the
Run button. The results of one simulated experiment are displayed on the Dashboard under Outputs (Figure 4).
The Simulation (Single run) spreadsheet
shows a simulation to compare 2 treatments
with each replication (Figure 5). In the example
in Figure 5, each replicate contains 15 birds and
each treatment contains 5 replicates. For example, cell B3 contains a randomly generated individual bird weight from a normal distribution
with the mean and SD input.
The Simulation (Multiple runs) spreadsheet
shows multiple simulations (Figure 6). Each
row shows one experiment. The cell B9 presents the average weight of one pen of randomly
generated birds. The G9 and O9 cells present
average weights of 5 pens each for the control
and treated groups, respectively. The H9 and P9
cells present the corresponding SD. These cells
will change based on the number of replications.
This spreadsheet also contains the average mean
difference, t-statistic, and P-values based on the
number of replications. The last column is used
to tabulate the number of individual simulations
meeting the rejection rate probability, which is
summed at the bottom of the page; it was an assumption for the simulation to see how accurate
the result will be based on the input settings.
If the mean values for the control and treated
318
JAPR: Symposium
(Figure 7). Every input value was from the
Dashboard; values should not be overwritten
on this sheet. The TC_vs_DD spreadsheet displays total costs ($366.4048~$3,505.333) and
detectable differences against various numbers of birds per pen and replications, which
range from 120 to 1,920 total birds (Figure 8).
This spreadsheet contains the same graph as
in the Dashboard, and a graph of birds per pen
(5~80) and detectable differences. The Chart
1 and Chart 2 spreadsheets display the detectable difference versus total costs (Figure 9)
and the detectable difference versus birds per
pen (Figure 10) that are also displayed on the
TC_vs_DD spreadsheet, which were included
for printing.
Figure 3. Run button.
groups are identical, then the hypothesis rejection rate should be 5.0%. In the example in Figure 4, it is 5.0%. By clicking on the simulation, it
is clear the P-values vary based on the assumption of a normal distribution for the population
mean, such as in a Monte-Carlo simulation.
The Cost_Estimation spreadsheet contains
the calculation details for experimental costs
Figure 4. Reading outputs.
RESULTS AND DISCUSSION
An easy test exists to demonstrate that the
simulation model is working properly: When the
control and treatments are input with identical
means and standard errors, the null hypothesis
that no differences in means exist should have
a 5% rejection rate (P < 0.05) [5]. The example
Simulation (Multiple runs) spreadsheet shows
5% rejection (Figure 6). Similar tests can be
demonstrated for Type II error by running the
Shim and Pesti: INFORMAL NUTRITION SYMPOSIUM
319
Figure 5. Simulation (single run) spreadsheet.
problem with real differences between the control and treated groups and different levels of
Type II error.
The POWER analysis can be used to calculate the minimum sample size required so that
an effect of a given size is reasonably likely to
be detected. In general, the larger the sample
size, the smaller the sampling error tends to be.
If sample size is too small, gathering the data is
not worthwhile because the results tend to be too
imprecise to be of much use. However, a point
of diminishing returns exists beyond which increasing the sample size provides little benefit.
Once the sample size is large enough to produce
a reasonable level of accuracy, making it larger
Figure 6. Simulation (multiple runs) spreadsheet.
simply wastes time and money. Of course each
researcher has to decide what they think reasonable is and then accept that a certain percentage
of the time their experiment will not yield the
correct conclusions.
The POWER calculations can be made
whenever one performs a statistical test of a hypothesis and one obtains a statistically nonsignificant result. Because of the very large amount
of money at risk from declaring no significant
differences in an experiment, more information
should be presented than the chance of declaring
a significant difference when none exists. The
detectable difference displayed on the Dashboard sheet is from Zar [6].
320
JAPR: Symposium
Figure 7. Cost estimation spreadsheet.
The number of replications required depends
on the experimental design employed, the desired difference from the mean that one would
care to be able to detect, and a specific probability level and type II error. The number of
Figure 8. TC_vs_DD spreadsheet.
replications may be estimated using procedures
described by Zar [6], Cochran and Cox [7], or
Berndtson [8].
When it is assumed that the same numbers of
birds are used in an experiment, for example, 600
Shim and Pesti: INFORMAL NUTRITION SYMPOSIUM
321
Figure 9. Chart 1 spreadsheet.
total birds, the detectable differences decreased
from 84.79 to 28.52 g by increasing from 2 (300
birds/pen) to 12 (50 birds/pen) replications. This
means that more replication numbers generally
give better results than more birds per pen. To
estimate about 50 g of detectable differences in
12 replications, 15 birds are needed. However,
when replications were decreased from 6 to 2,
numbers of birds were increased from 40 to 860
birds per pen to estimate about 50 g of detectable difference.
With 2 replications per treatment, increasing
the number of birds decreased the detectable differences greatly; but with large numbers of birds
per pen, the minimum detectable differences
were quite large (Figure 8). With 12 replications, the increase in the number of birds did not
affect detectable differences as much as with 2
reps, but caused a steep increase in total costs.
To have a relatively small detectable difference with reasonable cost, a proper trade-off
study is necessary. This program compares only
2 treatments. When treatments differences are
added to the program, it will be more powerful.
The POWER for Poultry design should be helpful for making the appropriate calculations and
decisions.
CONCLUSIONS AND APPLICATIONS
1. An Excel (Microsoft) spreadsheet was
effectively developed to calculate and
simulate statistical and economical results by choosing different numbers of
birds per pen and different numbers of
replicate pens per treatment on the Type
I and Type II errors of a simple experiment comparing 2 treatments.
2. The developed program allows the researcher to determine the minimum
sample size required so that one can be
JAPR: Symposium
322
Figure 10. Chart 2 spreadsheet.
reasonably detect an effect of a given
size.
3. More replication numbers generally give
better results than more birds per pen.
4. With 12 replications, the increase in the
number of birds did not affect detectable
differences as much as with 2 reps, but
caused a steep increase in total costs.
REFERENCES AND NOTES
1. Aaron, D. K., and V. W. Hays. 2004. How many pigs?
Statistical power considerations in swine nutrition experiments. J. Anim. Sci. 82:E245–E254.
2. Microsoft Corp., Redmond, WA.
3. Shim, M. Y., and G. M. Pesti. 2012. Pen-size optimization workbook of experimental research design (POWER
for Poultry). University of Georgia College of Agricultural
and Environmental Sciences Extension Bulletin 1417.
Accessed September 2013. http://www.caes.uga.edu/
Publications/pubDetail.cfm?pk_id=8048.
4. Shim, M. Y., G. M. Pesti, L. Jackson, M. Tahir, and
T. D. Pringle. 2012. How many chickens? Statistical power
considerations in experiments with poultry. The University
of Georgia, Athens, GA. Unpublished.
5. Pesti, G. M., and M. Y. Shim. 2012. A spreadsheet to
construct power curves and clarify the meaning of the word
equivalent in evaluating experiments with poultry. Poult.
Sci. 91:2398–2404.
6. Zar, J. H. 1981. Power of statistical testing: Hypothesis about means. Am. Lab. 13:102–107.
7. Cochran, W. G., and G. M. Cox. 1957. Experimental
Designs. 2nd ed. John Wiley & Sons, New York, NY.
8. Berndtson, W. E. 1991. A simple and reliable method
for selecting or assessing the number of replicates for animal
experiments. J. Anim. Sci. 69:67–76.