Measuring Allocation Errors
in Land Change Models in Amazonia
Luiz Diniz, Merret Buurman, Pedro
Andrade, Gilberto Câmara, Edzer Pebesma
Merret Buurman
GeoInfo, Campos do Jordão, 25 November 2013
Measuring Allocation Errors
in Land Change Models in Amazonia
Luiz Diniz
Merret Buurman
Pedro Andrade
+
Gilberto Câmara
Edzer Pebesma
3
&
„Why?“
Merret Buurman, 26.11.2013
4
&
Land change modelling
Simulation
2001
2002
Observed reality
Merret Buurman, 26.11.2013
2003
2004
5
&
Land change modelling
Big responsability
Need to evaluate results
This can only be done afterwards!
2004
Merret Buurman, 26.11.2013
6
&
(1) Goodness of fit metric
(2) Evaluation of models
Merret Buurman, 26.11.2013
7
&
(1) Goodness of fit metric
Merret Buurman, 26.11.2013
8
&
Two complementary views…
Costanza:
Multiple
resolutions
Costanza, R., Model Goodness of Fit - a
Multiple Resolution Procedure.
Ecological Modelling, 1989. 47(3-4): p.
199-215.
Merret Buurman, 26.11.2013
Pontius et al.:
Need to
consider
persistence
Pontius Jr, R.G., E. Shusas, and M.
McEachern, Detecting important categorical
land changes while accounting for
persistence. Agriculture, Ecosystems &
Environment, 2004. 101(2): p. 251-268.
9
&
Two complementary views…
Costanza:
Multiple
resolutions
Costanza, R., Model Goodness of Fit - a
Multiple Resolution Procedure.
Ecological Modelling, 1989. 47(3-4): p.
199-215.
Merret Buurman, 26.11.2013
Pontius et al.:
Need to
consider
persistence
Pontius Jr, R.G., E. Shusas, and M.
McEachern, Detecting important categorical
land changes while accounting for
persistence. Agriculture, Ecosystems &
Environment, 2004. 101(2): p. 251-268.
10
&
Multiple resolutions
Merret Buurman, 26.11.2013
11
&
Multiple resolutions
Merret Buurman, 26.11.2013
12
&
Multiple resolutions
Merret Buurman, 26.11.2013
13
&
Multiple resolutions
Merret Buurman, 26.11.2013
14
&
Multiple resolutions
Merret Buurman, 26.11.2013
15
&
Multiple resolutions
Merret Buurman, 26.11.2013
16
&
Multiple resolutions
Merret Buurman, 26.11.2013
17
&
Multiple resolutions
Merret Buurman, 26.11.2013
18
&
Two complementary views…
Costanza:
Multiple
resolutions
Merret Buurman, 26.11.2013
Pontius et al.:
Need to
consider
persistence
19
&
Two complementary views…
Costanza:
Multiple
resolutions
Merret Buurman, 26.11.2013
Pontius et al.:
Need to
consider
persistence
20
&
Need to consider persistence
• Many cases: Most of the area does not change
• Focus: Predicting the changed area
Example:
• 99% of the area unchanged
• All the change predicted at wrong locations
•  98 % of the area is „correct“!
Merret Buurman, 26.11.2013
21
&
… Combined into one
Change-focusing
multipleresolution
goodness of fit
Merret Buurman, 26.11.2013
22
&
What do we evaluate?
Merret Buurman, 26.11.2013
23
&
What do we evaluate?
Merret Buurman, 26.11.2013
24
&
What do we evaluate?
Equal total
amount!
Merret Buurman, 26.11.2013
25
&
Goodness of fit metric
• (1) Inside sampling window: Compute the
difference in amount of change between both
grids
Merret Buurman, 26.11.2013
26
&
Goodness of fit metric
• (2) Sum this up for all sampling windows
Merret Buurman, 26.11.2013
27
&
Goodness of fit metric
• (3) Divide by twice the total amount of change
– Why twice? In the previous steps, every „wrong“
allocation was counted twice, because too much
change in one cell automatically means too little
change in another, due to the equality of demand
in both grids.
Merret Buurman, 26.11.2013
28
&
Goodness of fit metric
• (4) Subtract from one to get goodness
• … and repeat for all other resolutions
Merret Buurman, 26.11.2013
29
&
Goodness of fit metric
•
•
•
•
•
•
•
•
•
Fw
tw
w
arefi
amodj
i, j
u
s
num
= Goodness of fit at resolution w.
= Number of sampling windows at resolution w.
= Resolution (a sampling window has w2 cells).
= Percent of change in land cover in cell i in the reference cell space.
= Change in land use/land cover in cell j in the model cell space.
= Cells inside a sampling window.
= Cells inside the cell space.
= A sampling window.
= Number of cells in the cell space (tw * w2)
Merret Buurman, 26.11.2013
30
&
(2) Evaluation of models
Merret Buurman, 26.11.2013
31
&
Models
SimAmazonia
2001  2050
BAU and GOV
Soares-Filho, B., et al., Modelling
conservation in the Amazon basin.
Nature, 2006. 440(7083): p. 520-523.
Merret Buurman, 26.11.2013
32
&
Models
SimAmazonia
2001  2050
BAU and GOV
Soares-Filho, B., et al., Modelling
conservation in the Amazon basin.
Nature, 2006. 440(7083): p. 520-523.
Laurance
2000  2020
Optimistic
Non-Opt.
Laurance, W., et al.,
The future of the Brazilian Amazon.
Science, 2001. 291: p.
438-439.
 Compare with PRODES 2011 (25x25km)
Merret Buurman, 26.11.2013
&
Merret Buurman, 26.11.2013
33
&
Merret Buurman, 26.11.2013
34
35
&
Why so weak?
• Neighborhood model: captures only existing
regions (not new frontiers)
• Similarity Neighborhood model & SimAmazonia:
Same reason?  Compare maps!
Merret Buurman, 26.11.2013
&
Merret Buurman, 26.11.2013
36
37
&
Why so weak?
• Neighborhood model: captures only existing
regions (not new frontiers)
• Similarity Neighborhood model & SimAmazonia:
Same reason?  Compare maps!
• Yes! Location of new frontiers difficult to predict
Merret Buurman, 26.11.2013
38
&
Why so weak?
• Laurance
• Overestimates roads
• Assumes same impact of roads everywhere
• Underestimates protected areas
Merret Buurman, 26.11.2013
39
&
Parque
do Xingu
Indigenous areas (FUNAI)
Merret Buurman, 26.11.2013
40
&
Conclusion
• Predicting the locations of future deforestation:
More difficult than expected!
• Problem: Policy recommendation based on
those predictions
• Our hope: Next generation of deforestation
models will capture better the complex human
decision-making
Merret Buurman, 26.11.2013
41
&
Conclusion
• Predicting the locations of future deforestation:
More difficult than expected!
• Problem: Policy recommendation based on
those predictions
• Our hope: Next generation of deforestation
models will capture better the complex human
decision-making
Obrigada!
Merret Buurman, 26.11.2013