Power analysis
or
What does it mean
P<0.05
And mainly what does it mean
P>0.05
(According to Scheiner & Gurevitch 2001: Desing and analysis of ecological
experiments. 2nd ed. Oxford Univ. Press)
Decision table
Power = 1-β
Effect size
• Absolute effect size – after fertilization, the
biomass will increase by 100 g.m-2
• Relative effect size – after fertilization, the
biomass will increase by 5%
• Standardized effect size – Absolute effect
size/s.d.
Power of the test depends on the effect size
At small number of
replications, we
have problem to
demonstrate rather
large effect, at large
number of
replications, we are
able to demonstrate
effect that has
nearly no biologicl
meaning.
Power analysis provides
• Power of the test as a function of effect
size, variability of data (these to provide
standardized effect size) and sample(s)
size – i.e. number of replications
• Usefulness of pilot experiment to get
variability and expected effect size
• Useful for experiment/sampling design
planning; useful also, when the test is not
significant, to see whether we even had a
chance to demonstrate the effect
Example 1 – correlation
coefficient
• Relationship between no of species and
biomass (and I expect linear relationship)
• How many quadrat I need to get significant
result?
• Factors which I need to know
– Expected value of correlation coefficient in the
(statistical) „population“ (i.e. The effect size,
the size of deviation from the H0)
– Required power of the test
Example 2 – t-test
• Difference in no of species between mown
and unmown plots (independent samples)
• What I need to know: Expected difference,
„population“ sigma – homoscedascity
expected
• S.E.S.= Difference/sigma
• Required power of the test
• How many quadrats I need
Es = S.E.S. = Difference/Sigma
Possible questions
• Biologically significant is increase of seed production
after competitor removal by 10% (either a number
which is estimated by a rule of thumb, or based on
some evolutionar model); the increase of productivity
by fertilization is economical only, when it is more than
1000kg/ha.
• We assume to know variability in the data (best
professional guess, or from pilot experiment)
• Sample(s) size required to demonstrate the effect (i.e.
To reject H0 at 5% significance level) with probability
at least 90% (power of the test, i.e. 1-β)
• Alternatively: What is/was the chance to demonstrate
the effect with no. of replications available
Missleading word „significant“
• Statistical significance does not imply biological
significance
• (Nearly) each null hypothesis is not correct – its
rejection then depends on the number of
replications we are able to get (we are often
limited in this respect).
• Dangers of careless use of „computerized
sampling“)
• Power analysis
I like the approach (Scheiner & Gurewith):
Vertical lines are confidence intervals (CI)
This approach can be used for estimate of sample size needed (CI size
decreases with N)
Presenting CI in paper provides good indication of biíological significance.