Experimental design
Experiments vs. observational
studies
Manipulative experiments: The only way to proof the
causal relationships
BUT
Spatial and temporal limitation of manipulations
Side effects of manipulations
Laboratory, field, natural trajectory (NTE), and natural
snapshot experiments (Diamond 1986)
Regulation of
indep. variables
Site matching
Ability to follow
trajectory
Maximum
temporal scale
Maximum
spatial scale
Scope (range of
manipulations)
Realism
Generality
Lab
Highest
Field
NTE
Medium/low None
Highest
Yes
Medium
Yes
Medium/low Lowest
Yes
No
Lowest
Lowest
Highest
Highest
Lowest
Low
Highest
Highest
Lowest
NSE
None
Medium/low Medium/hig Highest
h
None/low High
Highest
Highest
None
Low
High
High
NTE/NSE - Natural Trajectory/Snapshot Experiment
Observational studies
(e.g. for correlation between
environment and species, or
estimates of plot characteristics)
Random vs. regular sampling plan
Regular design - biased results, when there is some regular
structure in the plot (e.g. regular furrows), with the same
period as is the distance in the grid - otherwise, better design
providing better coverage of the area, and also enables use
of special permutation tests.
Manipulative experiments
frequent trade-off between feasibility and
requirements of correct statistical design and power of
the tests
To maximize power of the
test, you need to maximize
number of independent
experimental units
For the feasibility and
realism, you need plots of
some size, to avoid the
edge effect
Completely randomized design
Typical analysis: One way ANOVA
Important treatments randomly
assigned to plots
often, chessboard
arrangement or similar
regular pattern
Randomized complete blocks
ENVIRONMENTAL GRADIENT
Block 1
Block 2
Block 3
Block 4
ANOVA, TREAT x BLOCK interaction is error term
TREAT
BLOCK
1
2
3
1
2
3
1
2
3
1
2
3
1
2
3
1
1
1
2
2
2
3
3
3
4
4
4
5
5
5
RESPO1 RESPO2
5
5
6
6
4
4
7
5
9
5
8
4
3
5
5
7
2
4
6
4
7
6
5
5
8
4
11
5
9
6
1
2
If the block has
strong
explanatory
power, the RCB
design is
stronger
1
0
RESPO1
8
6
4
TR
E
A
T:
G
_
1
:1
TR
E
A
T:
G
_
2
:2
TR
E
A
T:
G
_
3
:3
2
0
G
_
1
:1
G
_
2
:2
G
_
3
:3
G
_
4
:4
G
_
5
:5
B
LO
C
K
df
Effect
TREAT
BLOCK
MS
df
Effect
Error
2 6.066667
4
17
TREAT
2 6.066667
MS
Error
8
8
F
p-level
0.4 15.16667 0.001897
0.4
42.5 1.97E-05
12 5.933333 1.022472 0.389016
7
.5
If the block has
no explanatory
power, the RCB
design is weak
7
.0
6
.5
RESPO2
6
.0
5
.5
5
.0
4
.5
4
.0
3
.5
G
_
1
:1
G
_
2
:2
G
_
3
:3
G
_
4
:4
G
_
5
:5
TR
E
A
T:
G
_
1
:1
TR
E
A
T:
G
_
2
:2
TR
E
A
T:
G
_
3
:3
B
LO
C
K
df
Effect
MS
Effect
df
Error
TREAT
BLOCK
2
2.4
4 0.166667
TREAT
2
2.4
MS
Error
F
p-level
8 0.816667 2.938776 0.110435
8 0.816667 0.204082 0.929067
12
0.6
4 0.046656
Latin square design
Most frequent errors - pseudoreplications
1.1. Most frequent errors – Pseudoreplications
Factorial designs
Completely randomised
F for testing effects in various
combination of fixed and random
factors in two-way ANOVA
Tested Both fixed
effect
A
MSA/MSerror
A-fixed,
B-random
MSA/MSAxB
Both random
B
MSB/MSerror
MSB/MSerror
MSB/MSAxB
AxB
MSAxB/MSerror MSAxB/MSerror MSAxB/MSerror
MSA/MSAxB
Fertilization experiment in three countries
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
COUNTRY FERTIL
CZ
0.000
CZ
0.000
CZ
0.000
CZ
1.000
CZ
1.000
CZ
1.000
UK
0.000
UK
0.000
UK
0.000
UK
1.000
UK
1.000
UK
1.000
NL
0.000
NL
0.000
NL
0.000
NL
1.000
NL
1.000
NL
1.000
NOSPEC
9.000
8.000
6.000
4.000
5.000
4.000
11.000
12.000
10.000
3.000
4.000
3.000
5.000
6.000
7.000
6.000
6.000
8.000
Country is a fixed factor (i.e., we are
interested in the three plots only)
Summary of all Effects; design: (new.sta)
1-COUNTRY, 2-FERTIL
df
MS
df
MS
Effect
Effect
Error
Error
F
p-level
1
2
2.16667 12
1.055556 2.05263 .171112
2
1
53.38889 12
1.055556 50.57895 .000012
12
2
26.05556 12
1.055556 24.68421 .000056
Country is a random factor (i.e., the three plots
arew considered as random selection of all plots of
this type in Europe - [to make Brussels happy])
Summary of all Effects; design: (new.sta)
1-COUNTRY, 2-FERTIL
df
MS
df
MS
Effect
Effect
Error
Error
F
p-level
1
2
2.16667 12
1.05556 2.05263 .171112
2
1
53.38889 2
26.05556 2.04904 .288624
12
2
26.05556 12
1.05556 24.68421 .000056
Nested designs („split-plot“)
Two explanatory variables, Treatment and Plot,
Plot is random factor nested in Treatment.
Accordingly, there are two error terms, effect of
Treatment is tested against Plot, effect of Plot
against residual variability:
F(Treat)=MS(Treat)/MS(Plot)
F(Plot)=MS(Plot)/MS(Resid) [often not of interest]
Split plot (main plots and split plots - two error levels)
Plot 1
Plot 2
Plot 3
N
C
N
N
C
P
C
P
Plot 4
Plot 5
N
C
P
C
Plot 6
P
N
P
N
C
P
ROCK
PLOT
1
1
1
1
1
1
1
1
1
2
2
2
2
2
2
2
2
2
TREA
1
1
1
2
2
2
3
3
3
1
1
1
2
2
2
3
3
3
df
Effect
ROCK
PLOT
TREA
ROCK*PLOT
ROCK*TREA
PLOT*TREA
3way
RESP
1
2
3
1
2
3
1
2
3
1
2
3
1
2
3
1
2
3
5
8
6
6
8
6
2
3
3
5
6
5
5
4
3
5
7
6
ROCK is the MAIN PLOT factor,
PLOT is random factor nested in
ROCK, TREATMENT is the within
plot (split-plot) factor.
Two error levels:
F(ROCK)=MS(ROCK)/MS(PLOT)
F(TREA)=MS(TREA)/MS(PLOT*TREA)
MS
df
Effect
Error
1 0.055556
4 8.944445
2 3.166667
MS
Error
F
p-level
4 8.944445 0.006211 0.940968
0
0
8 0.611111 5.181818 0.036018
2 0.722222
8 0.611111
8 0.611111 1.181818 0.355068
0
0
Following changes in time
Non-replicated BACI (Before-after-controlimpact)
Analysed by two-way ANOVA
factors: Time (before/after) and Location (control/impact)
Of the main interest: Time*Location interaction (i.e., the
temporal change is different in control and impact
locations)
13
13
12
11
11
10
10
PB
CD
12
9
8
8
7
6
9
7
C
O
N
T
R
LO
C
A
T
IO
N
IM
P
A
C
T
TIM
E
:
B
E
FO
R
E
T
IM
E
:
6
A
FTE
R
C
O
N
T
R
LO
C
A
T
IO
N
IM
P
A
C
T
TIM
E
:
B
E
FO
R
E
TIM
E
:
A
FTE
R
In fact, in non-replicated BACI, the test is based on
pseudoreplications.
Should NOT be used in experimental setups
In impact assessments, often the best possibility
(The best need not be always good enough.)
Replicated BACI - repeated measurements
T0
Treatment
Control
Impact
Control
Impact
T1
T2
Usually analysed by
“univariate repeated
measures ANOVA”.
This is in fact split-plot,
where TREATment is
the main-plot effect,
time is the within-plot
effect, individuals (or
experimental units) and
nested within a
treatment.
Impact
Control
Of the main interest is
interaction
TIME*TREAT
TRE
T1
5
6
5
4
6
5
13
12
11
10
Height
1
1
1
2
2
2
T214
T3
6
5
7
7
8
9
7
8
7
11
12
15
9
8
7
6
5
4
T
1
T
2
TR
E
:
G
_
1
:1
TR
E
:
G
_
2
:2
T
3
TIM
E
df
Effect
1
2
12
MS
df
Effect
Error
1
24.5
2 35.72222
2 12.16667
MS
Error
4 2.111111
8 0.944444
8 0.944444
F
11.60526
37.82353
12.88235
p-level
0.027111
8.37E-05
0.003151