Natural Resources Forum 35 (2011) 89–99
Are soil conservation technologies “win-win?” A case study of
Anjeni in the north-western Ethiopian highlands
Menale Kassie, Gunnar Köhlin, Randy Bluffstone and Stein Holden
Abstract
This study measures the impact of fanya juu terraces on the net value of crop income in a high-rainfall area in the Ethiopian
highlands using cross-sectional multiple plot observations. Using propensity score matching methods we find that the net
value of crop income for plots with fanya juu terraces is lower than for plots without fanya juu terraces. This finding makes
it difficult to avoid concluding that while the technologies might reduce soil erosion and associated off-site effects, they do
so at the expense of poor farmers in the Ethiopian highlands. Therefore, fanya juu terraces cannot be characterized as a
“win-win” measure to reduce soil erosion. New agricultural technologies need to be profitable to the farmer if they are to be
adopted and sustained.
narf_1379 89..99
Keywords: Ethiopia; value of crop income; soil conservation; propensity score matching; agro-ecology.
1. Introduction
Land degradation, soil erosion, and nutrient depletion
contribute significantly to low agricultural productivity and
thus food insecurity and poverty in many hilly areas of the
developing world (Pagiola, 1999; Shiferaw et al., 2009).
In response, a considerable amount of public resources
have been mobilized to develop soil and water conservation
(SWC) technologies and promote them to farmers.
The major underlying reason for using SWC
technologies in mountainous regions is to reduce movement
of soils, water flow velocity, and the broader effects of
erosion, such as siltation of rivers, lakes and dams. They
also reduce soil loss from farmers’ plots, preserving critical
nutrients and increasing on-farm yields, and this is the
chief selling point to farmers. Since SWC technologies not
only serve the social good but are also claimed to increase
on-farm yields, they are considered “win-win”.
Whether SWC technologies offer private benefits, social
benefits, private and social benefits, or no benefits at all
Menale Kassie (corresponding author) is at the International Maize and
Wheat Improvement Center, Kenya. E-mail: [email protected]
Gunnar Köhlin is at the Environmental Economic Unit, University of
Gothenburg, Sweden.
Randy Bluffstone is at the Department of Economics, Portland State
University, Oregon, United States.
Stein Holden is at the Department of Economics and Resource
Management, Norwegian University of Life Sciences, Norway.
© 2011 The Authors. Natural Resources Forum © 2011 United Nations
is important for a number of reasons. First, there are
legitimate concerns about the off-site effects of soil erosion,
particularly siltation, which can disrupt a variety of aquatic
ecosystems and cause damage to reservoirs and waterways
(Pagiola, 1999; Scherr and Yadav, 1997). In public and
policy venues, catastrophic floods have also been linked to
soil erosion in Ethiopia, which is the focus of the present
study. For example, flooding in August, October and
December 2006 damaged buildings, killed hundreds of
people, and displaced thousands in the eastern part of
Ethiopia (Mail and Guardian Online, 10 August 2006). The
conventional policy wisdom, in fact, is that if SWC
technologies can reduce these effects, they should be
promoted (Shiferaw et al., 2009; World Food Programme,
2005).
Regarding private benefits, there are real concerns about
the incomes of the farmers to whom SWC technologies
are promoted. Farmers in mountainous areas of developing
countries typically rely almost entirely on agriculture for
their incomes and have some of the lowest incomes and
highest rates of poverty in the world (Jackson and Scherr,
1995). This is also true in Ethiopia. As found by Bluffstone
et al. (2007) and the Ministry of Finance and Economic
Development (MOFED) of Ethiopia (2002; 2006), some
65-85% of incomes in rural Ethiopia, and particularly in the
highlands (home to over 85% of the 75 million Ethiopians),
come from crop agriculture. Furthermore, the incomes and
consumption levels of these, primarily subsistence, farmers
are extremely low. For example, MOFED (2002) found that
90
Menale Kassie, Stein Holden, Gunnar Köhlin and Randy Bluffstone / Natural Resources Forum 35 (2011) 89–99
in 1999-2000, the average rural adult income and
consumption were only about USD 95 and USD 136 per
year, respectively, and about 42% of adults were unable
to obtain 2,200 calories per day on average. A key reason
for these minimal income and consumption levels is that
agricultural productivity is very low by international
standards (World Bank, 2007), with an average yield of
1,000 kg per hectare (Central Statistical Authority of
Ethiopia, 1995). Indeed, low agricultural productivity is a
critical problem throughout Africa (Lufumpa, 2005; Food
and Agricultural Organization, 2002). If SWC technologies
do increase agricultural productivity, they could make a
major contribution to reducing the astounding levels of
poverty observed in rural Ethiopia and other hilly areas of
Africa and, therefore, offer a powerful rationale for their
promotion.
Indeed, international and national initiatives have
promoted SWC technologies in the name of both poverty
alleviation and environmental conservation (Shiferaw
et al., 2009). The problem, however, is that these outreach
programmes often do not allow for the possibility that SWC
technologies may at best only provide social benefits and
could even reduce, rather than increase, farmers’ incomes.
This issue deserves attention — not only so that SWC
technologies can be promoted accurately, but also because
farmers in hilly areas of developing countries cannot afford
to make investments that reduce their incomes.
Another issue is the cost of construction and maintenance
of these technologies, which can be very high. Stocking
and Abel (1989) and Shiferaw and Holden (1998) note
that construction of terraces is arduous and labour
intensive; constructing a bund on a small quarter-hectare
plot may require as many as 100 person days. Furthermore,
opportunity costs can be very high, with terraces taking
up 10 to 20% of the cultivable area (Krüger, 1994), and
even more on sloped plots. Terraces, therefore, actually
significantly reduce the cultivation area. If farmers are to
benefit from installing terraces, productivity must not only
increase, but must increase by more than the production lost
due to reduced cultivation area.
This paper sheds light on farmer incentives to adopt the
fanya juu bund1 by estimating the change in value of crop
production per hectare (ha) in the relatively high-rainfall
(1,690 mm) areas of the northwestern Ethiopian highlands.2
Although there are rigorous studies (Kassie et al., 2008:
2010) on the impact of SWC measures such as stone
terraces and reduced tillage in Ethiopia, to our knowledge
there is no rigorous quantitative evidence at the household
1
Literally, fanya juu means “throw soil uphill” in Swahili. In a fanya juu
bund, a ditch is dug along a contour around a plot, and the soil is thrown
uphill to form a ridge to block soil movements. A natural terrace forms and
increases in size over time, reducing erosion.
2
In Ethiopia, annual mean rainfall ranges from about 100 mm to about
2,000 mm (World Bank, 2009).
level in Ethiopia on the relationship between fanya juu
terraces and agricultural productivity.
2. Literature review
There is no question that soil conservation measures
reduce erosion. For instance, soil loss estimates from
Soil Conservation Research Project experiments in the
northwestern and northeastern highlands of Ethiopia
indicate that fanya juu terraces, on average, could reduce
soil loss by 65%, or by 25-72 metric tons per hectare per
year (Grunder and Herweg, 1991a; 1991b). In spite of what
may be important ecological benefits and substantial efforts
to promote terraces, the reality is that SWC technologies
have not been widely adopted by smallholders in Ethiopia
or many other countries (Okoba et al., 2007; Barrett et al.,
2002; Pender and Kerr, 1998; Herweg, 1993). In Ethiopia,
it has been noted that pilot demonstration projects often
cannot be replicated on smallholder farms (Amede et al.,
2001; Shiferaw and Holden, 1998), and there is even
evidence that conservation structures are often partially or
fully removed after some time (Shiferaw and Holden, 1998;
Tadesse and Belay, 2004). These findings raise questions
about the appropriateness of the technologies and, indeed,
why they were adopted in the first place. The policy
literature is starting to take note of such events.3
Although the empirical literature on the impact of fanya
juu is very thin, there have been some studies that have
estimated the impacts of other SWC measures on mean
yield in developing countries. Byiringiro and Reardon
(1996), using farm-level data in Rwanda, found that farms
with greater investments in soil conservation have much
greater land productivity than other farms. However, they
did not specify the type of conservation. In the Philippines,
Shively (1998) found that conservation via contour
hedgerows has a positive and statistically significant impact
on yield, as assessed using farm-level data. Using stochastic
dominance analysis (SDA) and non-experimental farmlevel data collected in the Philippines, Shively (1999)
compared observed yields obtained from farmers’ fields
with and without contour hedgerows and found that the
use of hedgerow technology did not constitute an
unambiguously dominant production strategy. Yet, Bekele
(2005), using SDA and results from experimental trials of
the Soil Conservation Research Project in a low-rainfall
3
For example, the World Food Programme (2005) recently noted that:
“There is a growing agreement in the area of land rehabilitation and
soil conservation that profitability and cost effectiveness has in the past
been largely neglected. . . . For many years technical soundness and
environmental factors have provided the only guiding principles for
government and donors. . . . The limited success of soil conservation
programmes in Ethiopia in the past was largely a result of the ‘top down’
approach to design and implementation. Many farmers were compelled to
participate in the food-for-work conservation programmes implemented in
the 1980s and consequently failed to maintain the physical structures
adequately.”
© 2011 The Authors. Natural Resources Forum © 2011 United Nations
Menale Kassie, Stein Holden, Gunnar Köhlin and Randy Bluffstone / Natural Resources Forum 35 (2011) 89–99
area of East Ethiopia, found that physical conservation
(level soil terraces) has an unambiguous dominance over
no-conservation condition. Kassie et al. (2008) compared
the performance of stone terraces in high- and low-rainfall
areas of the Ethiopian highlands. Their empirical results
reveal that stone terraces have a significant positive
productivity impact in low-rainfall areas but a negative, yet
not statistically significant, impact in high-rainfall areas.
Similarly, Tiffen et al. (1994) in Kenya (Machakos district)
reported large and statistically significant maize yield
differences between farms with (1854 kg/ha) and without
(1047 kg/ha) fanya juu terraces in the drier agro-ecological
zone of the district, whereas the yield difference was small
and statistically insignificant in water plentiful agroecological zones (p. 199). In drier areas, soil and water
conservation technologies such as fanya juu terraces serve
as a moisture-conserving technology which probably
explains their positive impact on yield. Nyangena and
Köhlin (2008) also evaluate various SWC measures in three
areas (Machakos, Meru and Kiambu) in Kenya. Fanya juu
was combined with other labour-intensive methods into a
category called bench terraces. They found that such
bench terraces had a negative effect on the value of farm
production (significant at the 10% level). Unfortunately,
rainfall was not controlled for in this study. Benin (2006),
based on a survey of 434 households representing the
highlands of the Amhara region of Ethiopia, found that
stone terraces have a significantly positive impact (a 42%
increase in the studied period) on average crop yield in
low-rainfall parts of the Amhara region, but an insignificant
impact in the high-rainfall region. Finally, Pender and
Gebremedhin (2007) conducted a survey of 500 households
representing the semi-arid highlands of Tigray. They found
higher crop yields from plots with stone terraces (by 23%
on average) and estimated the average rate of return of
stone terrace investment at 46%. These results suggest
that the economic returns to soil and water conservation
investments are greater in lower-rainfall than in higherrainfall areas. These studies, however, suffered from
methodological problems that may have led to under- or
over-estimation of the productivity impacts of the analyzed
technologies. First, some of the comparisons (except the
Kassie et al., 2008 study) were not based on comparable
samples, which can yield biased estimates (Heckman et al.,
1998). Second, none of the above studies checked the
sensitivity of estimated adoption effects to hidden bias from
unobserved variables. The current study estimates average
adoption effect controlling for the above econometric
problems.
3. Methodology: Estimation challenges, techniques,
and procedures
conservation. Ignoring these issues may lead to biased
estimates of SWC effects. The first important issue is
that it is difficult to assess productivity gains from soil
conservation based on non-experimental observations,
since the counterfactual outcome, i.e., what the production
would have been without conservation on conserved plots,
is not observed. In experimental studies, this problem is
addressed by randomly assigning plots to treatment and
non-treatment status, which assures that the outcomes
observed on the non-treated plots without conservation are
statistically representative of what would have occurred
without conservation on the treatment plots.4 However, in
real farming situations, farmers and plots are not randomly
assigned to the two groups (treated and non-treated plots);
rather, farmers make their own adoption choices, or are
systematically selected by development agencies and/or
by project administrators based on their propensity to
participate in technology adoption. Additionally, farmers
(or development agencies) are likely to select plots nonrandomly based on their quality attributes, which are often
unobservable by the researcher. Therefore, adopters and
non-adopters may be systematically different and treated
and non-treated plots may also be systematically different,
and these differences may manifest themselves in
differences in farm performance that could be mistakenly
attributed to differences in adoption behaviour. Thus,
possible self-selection due to observed and unobserved plot
and household characteristics makes it difficult to perform
ex post assessment of gains from conservation using
observational data. Failure to account for this potential
selection bias could lead to inconsistent estimates of the
impact of technology adoption.
The standard approaches for dealing with the problem
of self-selection are the two-step Heckman and the
instrumental variable (IV) methods. However, both
approaches address a selection of unobservables by
imposing distributional and functional form assumptions
such as linearity on the outcome equation and extrapolating
over regions of no common support, where no similar
adopter and non-adopter observations exist. The evidence
from Dehejia and Wahba (2002) and Smith and Todd (2005)
suggests that avoiding functional form assumptions and
imposing a common support condition can be important
for reducing selection bias. Moreover, the IV approach
crucially depends on the availability of valid instruments,
which is a challenge in many empirical analyses (Angrist
and Krueger, 2001).
We propose using propensity score matching (PSM),
which does not require linearity, or parametric or
distributional assumptions, which also does not require
exogeneity of covariates to identify the causal effect of
interest. They can be all endogenous (Heckman and
4
There are a number of econometric issues to address
when trying to assess the productivity gains from soil
© 2011 The Authors. Natural Resources Forum © 2011 United Nations
91
We took adoption of fanya juu as the treatment variable, while net value
of crop income per hectare (net of the cost of fertilizer and seed) was the
outcome of interest.
92
Menale Kassie, Stein Holden, Gunnar Köhlin and Randy Bluffstone / Natural Resources Forum 35 (2011) 89–99
Vytlacil, 2007). A limitation of PSM is that unobservable
variables that may affect both the outcome variables
and choice of technology are not accounted for directly;
it assumes selection is based on observable variables.
However, in cross-sectional data, the presence of
unobserved characteristics in the propensity score
estimation can create mismatching and biased estimators.
As noted by Jalan and Ravallion (2003), however, the
assumption of selection of observables is no more
restrictive than assuming away problems of weak
instruments when the Heckman two-step or the IV approach
is employed in cross-sectional data analysis.
Detailed discussion on the propensity score matching
method and related methodology issues are available in the
appendix.
4. Data source and type
The dataset used in this study comes from a farm survey
conducted in 2001 in the northwestern Ethiopian highlands
village of Anjeni. The area is characterized by relatively
high rainfall (1,690 mm or 66 inches per year) and altitudes
of 2,100 to 2,500 metres. The village was selected by the
Soil Conservation Research Project (Anjeni station) to
represent an important agro-ecological zone for agricultural
production in the highlands. The agro-ecological conditions
are representative for a wider area in Ethiopian highlands
(see Figure 1). Although limited in scope, we would like to
argue that this case study is of relevance for an important
agricultural area in Ethiopia.
The dataset includes 148 farm households and about
1,290 plots, after removal of missing observations for
some variables. Enumerators collected a wide range of
information on the households’ production activities, and
on plot-specific characteristics, including SWC status. For
each plot, the respondent recounted the crop or crops
grown during the sample year. In addition, the enumerators
collected a number of other plot attributes, including soil
fertility (the farmer ranked his plot as “poor”, “medium” or
“good”, and a dummy variable was set equal to 1 for the
selected rank and zero for the others ); soil depth (the
farmer ranked his plot as “deep”, “medium deep” or
“shallow”, and a dummy variable was set equal to 1 for
the selected rank and zero for the others); topography
(a dummy variable was set equal to one if the plot was on a
plain and zero if it was on a hill); plot size (measured in
hectares); measured plot slope (in degree), and distance of
the plot from the household (in minutes walking).
Table 1 provides the descriptive statistics of the variables
used in the analysis by adoption status. At the time of our
survey in 2001, about 32.7% of the sampled plots had fanya
juu terraces, 61% of which were over 15 years old. This
technology was introduced to the study area by the Soil
Conservation Research Project (SCRP) established in 1984
in the study area. Because of this research station, fanya juu
terraces are the conservation measure that is mainly used in
cultivated fields, apart from a few instances of traditional
ditch (furrow), an alternative indigenous conservation
measures also practiced in the area. During our field work,
we observed that some farmers were dismantling and/or
reducing the terrace size, even though village officials do
Anjeni station
representative areas
Figure 1. Cost benefit framework for pro-SLM decision-making process: Ethiopian case study, frameworks
for quantifying the biophysical processes of land degradation.
Source: Zeleke (2006).
© 2011 The Authors. Natural Resources Forum © 2011 United Nations
Menale Kassie, Stein Holden, Gunnar Köhlin and Randy Bluffstone / Natural Resources Forum 35 (2011) 89–99
93
Table 1. Descriptive statistics of variables (standard deviation in parentheses)
Variables
Adopters
Non-adopters
Gross crop revenue, in ETB/hectare**
739.068
(617.564)
478.870
(554.912)
888.468
(742.202)
639.175
(662.855)
39.964
(11.494)
1.921
(0.721)
0.533
20.268
(9.587)
1.801
(0.573)
39.878
(11.281)
2.018
(0.885)
0.545
24.770
(10.725)
1.684
(0.655)
0.327
0.274
(0.146)
0.393
N/A
0.242
(0.138)
0.444
0.315
0.291
0.171
0.479
0.351
13.400
(16.199)
17.06
0.064
0.263
0.220
0.329
0.050
0.073
422
0.326
0.230
0.203
0.516
0.281
18.276
(31.746)
17.53
0.154
0.230
0.188
0.303
0.070
0.054
868
Net crop revenue***, in ETB/hectare
Household level variables
Age of household head, in years
Household labour, in man equivalent per ha
Education, [1 = if the household head can read and write; 0 = otherwise]
Distance to extension: Household residence distance to extension office in minutes
Total farm size, in hectares
Plot level variables
Fanya juu (1 = if plot received fanya juu bund, 0 = otherwise)
Plot size, in ha
Deep soil plots [1 = plots with deep soil;
0 = otherwise]
Moderately deep plots [1 = plots with medium soil depth; 0 = otherwise]
Shallow plots, [1 = plots with shallow soil depth; 0 = otherwise] (cf.)*
High-fertility plots, [1 = plots with very fertile soil; 0 = otherwise] (cf.)*
Moderately-fertility plots [1 = plots with moderately fertile soil; 0 = otherwise]
Poor-fertility plots [1 = plots with poor fertility; 0 = otherwise]
Distance from residence to plot, in minutes walking
Plot slope (degree)
Crop1 [1 = if maize crop; 0= otherwise] (cf.)*
Crop2 [1= if pulses and oil crops; 0 = otherwise]
Crop3 [1 = if teff crop; 0 = otherwise]
Crop4 [1 = if barley crop; 0 = otherwise]
Crop5 [1 = if potato crop; 0 = otherwise]
Crop6 [1 = if wheat crop; 0 = otherwise]
Number of observations
* The “cf ” indicates that the variable is used as comparison (reference) group where the other categories are compared; ** ETB, Ethiopian birr;
*** Costs for fertilizer and seed deducted from value of crop production.
Source: Authors’ calculation.
not allow them to do so. Farmers have voiced serious
complaints about terraces. For example, they have been
concerned about water logging and they have reported
difficulties in turning ox-drawn plows due to narrow terrace
spacing. Water-logging might have an effect on soil biota,
eliminating most of the aerobic soil organisms because
of the hypoxic soil conditions. Soil hypoxia reduces the
services of aerobic bacteria, fungi and other organisms, so
soil fertility might be compromised, having repercussions
on productivity.5
We found that the mean net value of crop income
per hectare was USD 80 (ETB 639) on non-treated plots,
compared with USD 60 (ETB 479) on treated plots.6 The
5
We thank an anonymous reviewer for this important point.
Although we could have estimated a separate regression model for each
crop produced, this would have resulted in much smaller sample sizes for
each crop and hence reduced statistical power.
6
© 2011 The Authors. Natural Resources Forum © 2011 United Nations
unconditional mean net value of crop income is higher
on non-treated plots. It is important to emphasize that this
difference may not be a result of fanya juu terraces, but
instead may be due to other factors, e.g., land quality, crop
choice, household characteristics and input use. Therefore,
we needed to conduct careful multivariate analysis to test
the impact of fanya juu terraces adoption on net value of
crop income.
5. Results and discussion
5.1. Estimation of propensity scores
Table 2 reports the results from the logit analysis of
conservation investments and the variables used in the
matching procedures. The adoption regression suggests the
importance of plot size, topography, distance of plot from
94
Menale Kassie, Stein Holden, Gunnar Köhlin and Randy Bluffstone / Natural Resources Forum 35 (2011) 89–99
Table 2. Logit estimates on propensity to use fanya juu bund
Without crop choice variables
With crop choice variables
Variables
Coefficients
Std. error
Coefficients
Std. error
High-fertility plots
Moderate-fertility plots
Deep plots
Moderately deep plots
Ln(plot size)
Plot distance from residence
Plot location
Ln(age)
Ln(distance to extension office)
Family labour supply
Education
Total farm holding
Ln(plot slope)
Joint significance of crop choice variables
Constant
Summary statistics
Model test (F statistics)
R-Squared
Log likelihood
Number of observations
-0.287
-0.250
-0.233
-0.209
0.431***
-0.004**
-3.755***
-0.002
-0.753***
0.008
0.063
0.207*
0.175
0.211
0.165
0.187
0.187
0.105
0.002
0.718
0.006
0.132
0.084
0.139
0.118
0.142
0.215
0.168
0.191
0.184
0.454
0.002
0.713
0.007
0.135
0.211
0.142
0.277
0.008
1.917***
0.714
-0.116
-0.184
-0.167
-0.160
1.455***
-0.006**
-3.3747***
-0.004
-0.806***
0.238
0.045
0.559***
-0.004
27.45***
0.633
96. 539***
0.111
-724.570
1,290
0.556
120.644***
0.129
-710.004
1,290
* Significant at 10%; ** Significant at 5%; *** Significant at 1%; and robust standard errors.
Source: Authors’ calculation.
household residence, distance of household residence from
extension office and total farm size in influencing fanya juu
terraces adoption.
Before discussing the average adoption effect, it is worth
mentioning the quality of the matching process. A visual
inspection of the density distributions of the propensity
scores (Figure 2)7 indicates that the region of common
support is satisfied since there is substantial overlap in
the distribution of the propensity scores of the treated and the
non-treated groups. The bottom half of the graph shows the
propensity score distribution for the non-treated, while
the top half refers to the treated plots. The y-axis indicates the
density of the propensity score distribution.
As shown in Table 3, the unmatched sample fails to satisfy
the balancing properties in that some of covariates have a
standardized difference (SD) greater than 20% and there are
significant differences in the means of some covariates.8 The
fifth column of this table lists the percentage bias between the
groups. As can be seen, all variables have less than 20% SD
after matching. The sixth column lists the results of a t-test of
the equality of means between the groups where there is no
statistically significant mean difference between groups after
matching.
The low pseudo-R2 (0.111 and 0.006 before and after
matching, respectively) and the insignificant likelihood
7
The common support density distribution figures and covariate
balancing test results and the average adoption effect estimates are
obtained using the Stata pstest and psmatch2commands, respectively
(Leuven and Sianesi, 2003).
8
This result is based on NNM, although we reach the same conclusion
using the KM method.
ratio tests of the joint significance of all covariates (LR
chi2 = 180.9 [P = 0.000]*** and 6.91 [P = 0.938] before
and after matching, respectively) also support the
hypothesis that both groups have the same covariate
distribution after matching. These results imply that
there is no systematic difference in the distribution of
covariates between the groups after matching. In the next
subsection we evaluate the fanya juu terraces adoption
effect between groups of plots with similar observed
characteristics.
5.2. Estimation of average adoption effect (ATT):
Matching algorithms
Table 4 reports the estimates of the average adoption effects
estimated by the NNM and KBM methods. As a sensitivity
analysis, the table reports estimates based on the single
and five nearest neighbours, and the Epanechnikov kernel
estimator with two different bandwidths. All analyses were
based on the implementation of common support and
caliper, hence the distributions of treated and non-treated
plots were located in the same domain. As suggested by
Rosenbaum and Rubin (1985), we used a caliper size of
one-quarter of the standard deviation of the propensity
scores. Bootstrap standard errors based on 200 replications
are reported.
The outcome variable is net value of crop income per ha
(hereafter crop income). The matching estimates show that
crop income of non-treated plots is significantly higher than
that of treated plots. The reduction in crop income ranges
© 2011 The Authors. Natural Resources Forum © 2011 United Nations
Menale Kassie, Stein Holden, Gunnar Köhlin and Randy Bluffstone / Natural Resources Forum 35 (2011) 89–99
0
.2
.4
Propensity Score
Untreated
Treated: Off support
.6
95
.8
Treated: On support
Figure 2. The distribution of propensity score and common support region.
Note: “Treated: on support” indicates the observations in the adoption group that have a suitable comparison. “Treated: off support” indicates the
observations in the adoption group that do not have a suitable comparison.
Source: Authors’ calculation.
Table 3. Matching quality indicators
t-test before and
after matching
Mean
Variable
Sample
Propensity score
unmatched
matched
unmatched
matched
unmatched
matched
unmatched
matched
unmatched
matched
unmatched
matched
unmatched
matched
unmatched
matched
unmatched
matched
unmatched
matched
unmatched
matched
unmatched
matched
unmatched
matched
unmatched
matched
Age
Education
Household labour
Ln(Distance to extension office)
Total farm holding
Plot distance to residence
Ln(plot size)
Plot location
Deep soil plots
Moderately deep soil plots
High-fertility plots
Moderate-fertility plots
Plot slope
Treated
Control
% bias (SD)
t
p > |t|
0.40678
0.40505
39.964
40.0000
0.53318
0.53095
1.9210
1.9249
2.8871
2.8932
1.8007
1.7967
13.400
13.452
-1.4475
-1.4523
0.00474
0.00476
0.39336
0.39286
0.31517
0.31429
0.17062
0.17143
0.47867
0.47619
2.7713
2.7692
0.28841
0.40503
39.878
30.733
0.54493
0.47619
2.0183
1.9182
3.1035
2.8939
1.684
1.798
18.276
13.283
-1.5953
-1.4634
0.16014
0.00476
0.44355
0.39048
0.32604
0.29048
0.20276
0.14286
0.51613
0.48095
2.7423
2.7485
80.8
0.0
0.8
2.3
-2.4
11.0
-10.2
5.4
-42.6
-0.1
19.2
-0.2
19.3
0.7
24.0
1.8
-58.9
0.00
-10.2
0.5
-2.3
5.1
-8.3
7.3
-10.2
0.5
6.6
4.7
13.12
0.00
0.07
0.34
-0.40
-1.59
-1.70
0.12
-7.27
-0.02
3.12
0.03
-2.97
0.12
4.00
0.28
-8.62
0.00
-1.71
1.14
-0.39
0.75
-1.37
1.14
-1.26
-0.14
1.05
0.72
0.000
0.998
0.941
0.737
0.691
0.111
0.088
0.902
0.000
0.985
0.002
0.978
0.003
0.902
0.000
0.783
0.000
1.000
0.087
0.256
0.695
0.453
0.169
0.256
0.207
0.890
0.294
0.474
Source: Authors’ calculation.
© 2011 The Authors. Natural Resources Forum © 2011 United Nations
96
Menale Kassie, Stein Holden, Gunnar Köhlin and Randy Bluffstone / Natural Resources Forum 35 (2011) 89–99
Table 4. Impact of adoption on net crop income per hectare with
and without crop choice variables
Matching
algorithm
NNMa
NNMb
KBMc
KBMd
Without crop choice
variables
With crop choice
variables
ATT(in ETB)
ATT (In ETB)
-128.66
-109.81
-100.03
-96.79
(62.67)***
(44,23)***
(33.80)***
(34.80)***
-125.03
-74.03
-74.31
-77.30
(58.35)***
(41.86)*
(34.11)***
(35.15)***
NNMa = single nearest neighbour matching with replacement,
common support, and caliper (0.06); NNMb = five nearest neighbour
matching with replacement, common support, and caliper (0.06);
KBMc = kernel based matching with band width 0.06, common
support, and caliper (0.06); KBMd = kernel based matching with band
width 0.03, common support, and caliper (0.06); The observations on
common support were 420 for adopters and 868 for non-adopters
irrespective of the matching methods used; ***, and * is significant
at 1 and 10%, respectively. Bootstrapped standard errors are In
parentheses.
Source: Authors’ calculation.
from ETB 74 (USD 9) to ETB 128 (USD 16) per ha9,10 with
and without crop choice variables (see Table 4). Although
this may not seem like a lot of money to people in developed
countries, the numbers are quite significant in the context
of highland Ethiopia. Ethiopia’s gross domestic product per
capita in 2001 was only about USD 120, and the average net
value of crop income per hectare in our sample was ETB
587 (USD 73), indicating that the crop income “loss” was in
the 13-22% range.
Table 5 gives the result of the Rosenbaum bounds
sensitivity analysis. We increased the level of hidden bias
(gamma, G; see Rosenbaum, 2002) until the inference
about the adoption effect changed. The result shows that
the estimated adoption effect is not very sensitive to
unobserved selection bias. The negative adoption effect
remains significantly negative even if we allow the treated
and non-treated groups to differ by as much as 60-80% in
terms of unobserved characteristics. The critical value of G,
at which point we would have to question our conclusion
9
1 USD = 8 ETB during the survey period.
We also checked these results using Ordinary Least Square (OLS)
exogenous and endogenous switching regression adjustment estimators,
where we ran separate regressions for adopters and non-adopters, using
a matched sub-sample of adopters and non-adopters obtained from
the single nearest neighbour matching estimator. Despite its strong
distributional assumption and exclusion restrictions problems, using
endogenous switching regression we reached the same qualitative
conclusion, namely, that adoption has a negative and significant impact
on crop income. The average adoption effect (ATT) is ETB 125 (with
standard error of 15.55) and 125 (with standard error of 20.07) with
and without crop choice variables, respectively. We assume that the nonlinearity of the selection regression serves as the exclusion restriction.
The results are similar using exogenous switching regressions. The
predicted values used to estimate the average adoption effects from
switching regressions are calculated at the observed regressor values for
each observation.
10
Table 5. Estimation of Rosenbaum bounds to check the sensitivity of
results to hidden bias
Level of hidden
bias (G)
Without crop
choice variables
1
1.05
1.10
1.15
1.20
1.25
1.30
1.35
1.40
1.45
1.50
1.60
1.70
1.80
1.90
With crop
choice variables
Significance level
<0.001
<0.001
<0.001
<0.001
<0.001
<0.001
<0.001
<0.001
<0.001
<0.001
<0.001
<0.001
0
0
0
0.002
<0.001
<0.001
<0.001
<0.001
<0.001
<0.001
<0.001
<0.001
<0.001
<0.001
<0.001
<0.001
<0.001
0
Source: Authors’ calculation.
of a negative effect of fanya juu terraces, starts from G =
1.7-1.9, indicating that the unobserved covariate would
have to increase the odds of adoption by 70-90% or more to
change the significant adoption effect. This is a large value
since we included important variables that affect both the
adoption decision and the outcome variable. Based on this
result, we can conclude that the average adoption effect
estimates in Table 4 are a pure effect of fanya juu terraces
adoption.
6. Conclusions
We estimated the causal effect on net crop income from
adoption of fanya juu terraces in a high-rainfall village of
the Ethiopian highlands. Propensity score matching was
used to estimate the gains from adoption. This method does
not require ad hoc assumptions about the functional form
of impacts and exclusion restrictions, it only eliminates
selection bias on observable differences between adopters
and non-adopters.
Our empirical analysis shows that adoption of fanya juu
terraces, despite its large labour inputs and many years
of implementation, significantly reduces household net
crop income. Fanya juu terraces have the potential to
reduce net crop income in the range of ETB 74-128 per
hectare.
The highlands of Ethiopia are an unfortunately good
example of a very critical situation shared by many of the
poorest people of the world. They live on marginal lands,
with very low productivity and eroding soil capital. For
decades, interventions have been designed to alleviate this
© 2011 The Authors. Natural Resources Forum © 2011 United Nations
Menale Kassie, Stein Holden, Gunnar Köhlin and Randy Bluffstone / Natural Resources Forum 35 (2011) 89–99
critical situation through labour-intensive soil and water
conserving structures. Fanya juu terracing is among the
most labour intensive of these. In more recent years,
increasing attention has been given to the necessity that
these structures should not only conserve soil but also
increase the profitability of the agriculture for the poor
farmers. Without such a win-win outcome, the structures
will either not be adopted, or will be dismantled and/or
miss the objective of improving the livelihood of the
farmers. SWC technologies must therefore be promoted
carefully, with specific attention given to the fragile
circumstances of farming households. This study
contributes to a growing literature that shows that the
choice of SWC structure needs to be carefully matched to
local agro-ecological conditions. Specifically, there is a
risk that water-conserving structures, such as fanya juu,
do not perform well in high-rainfall areas. Although
a recommendation that SWC intervention should take
agro-ecological conditions into consideration might seem
obvious, there are unfortunately many cases where
interventions have not followed this rule in the past.
References
Ali, A., Abdulai, A., 2010. The adoption of genetically modified cotton
and poverty reduction in Pakistan. Journal of Agricultural Economics,
61: 175–192.
Amede, T., Belachew, T., Geta, E., 2001. Reversing the degradation
of arable land in the Ethiopian Highlands. International Institute
for Environment and Development Managing Africa’s Soils Paper
no. 23. International Institute for Environment and Development,
London.
Angrist, J.D., Krueger, A.B., 2001. Instrumental variables and the search
for identification: From supply and demand to natural experiments.
Journal of Economic Perspectives, 15(4): 69–85.
Barrett, C.B., Lynam, J., Place, F., Reardon, T. and Aboud, A.A.,
2002.Towards improved natural resource management in African
agriculture. In Barrett, C.B., Place, F. and Aboud, A. (eds.), Natural
Resource Management in African Agriculture: Undersigning and
Improving Current Practices, CABI Publishing, Wallingford, UK,
pp. 287–96.
Bekele, W., 2005. Stochastic dominance analysis of soil and water
conservation in subsistence crop production in the Eastern Ethiopian
highlands: the case of Hunde-Lafto area. Environmental Research
Economics, 32(4): 533–550.
Benin, S., 2006. Policies and programs affecting land management
practices, input use and productivity in the Highlands of Amhara
Region, Ethiopia. In Pender, J., Place, F. and Ehui, S. (eds.), Strategies
for Sustainable Land Management in the East African Highlands,
International Food Policy Research Institute, Washington, DC.
Bluffstone, R.A., Yesuf, M., Bushie, B., D. Damite, D., 2007. Rural
livelihoods, poverty, and the Millennium Development Goals:
Evidence from Ethiopian survey data. Unpublished working paper.
Environmental Economics Policy Forum for Ethiopia/Ethiopian
Development Research Institute, Addis Ababa, Ethiopia.
Byiringiro, F., Reardon, T., 1996. Farm productivity in Rwanda: Effects of
farm size, erosion, and soil conservation investments. Agricultural
Economics, 15: 127–36.
Caliendo, M., Kopeinig, S., 2008. Some practical guidance for the
implementation of propensity score matching. Journal of Economic
Surveys, 22: 31–72.
© 2011 The Authors. Natural Resources Forum © 2011 United Nations
97
Central Statistical Authority of Ethiopia, 1995. Agricultural sample survey
for 1994–95: Report in area and production for major crops, Statistical
Bulletin, 132(1): 1.
Dehejia, H.R., Wahba, S., 2002. Propensity score matching methods for
non-experimental causal studies. Review of Economics and Statistics,
84(1): 151–61.
Food and Agricultural Organization, 2002. New Partnership for Africa’s
Development: Comprehensive Africa Agriculture Development
Programme. FAO, Rome.
Grunder, M., Herweg, K., 1991a. Soil conservation research project
(SCRP): Eighth progress report. University of Berne, Switzerland,
in association with the Ministry of Agriculture, Addis Ababa,
Ethiopia.
Grunder, M., Herweg, K., 1991b. Soil conservation research project
(SCRP): Seventh progress report. University of Berne, Switzerland,
in association with the Ministry of Agriculture, Addis Ababa,
Ethiopia.
Heckman, J., Navarro-Lozano, S., 2004. Using matching, instrumental
variables and control functions to estimate economic choice models.
Review of Economics and Statistics, 86(1): 30–57.
Heckman, J., Vytlacil, E., 2007. Econometric evaluation of social
programs, part 2 of using the marginal treatment effect to organize
alternative economic estimators to evaluate social programs and to
forecast their effects in new environments. In Heckman, J. and Leamer,
E. (eds.), Handbook of Econometrics, Vol. 6, Elsevier Science,
Amsterdam.
Heckman, J., Ichimura, H., Smith, J. and Todd, P., 1998. Characterizing
selection bias using experimental data. Econometrica, 66(5):
1017–1098.
Herweg, K., 1993. Problems of acceptance and adoption of soil
conservation in Ethiopia. Tropics Applied Resource Management, 3:
391–411.
Jackson, L.A., Scherr, J.S., 1995. Non-degrading land use strategies
for tropical hillsides. 2020 Brief no. 27. International Food Policy
Research Institute, Washington, DC.
Jalan, J., Ravallion, M., 2003. Does piped water reduce diarrhea for
children in rural India? Journal of Econometrics, 112(1): 153–173.
Kassie, M., Zikahli, P., Pender, J., Kohlin, G., 2010. The economics of
sustainable land management practices in the Ethiopian Highlands.
Journal of Agricultural Economics, 61(3): 605–627.
Kassie, M., Pender, J., Yesuf, M., Kohlin, G., Bluffstone, R., Mulugeta,
E., 2008. Estimating returns to soil conservation adoption in the
northern Ethiopian highlands. Agricultural Economics, 38: 213–
232.
Krüger, H.J., 1994. The development of farmer friendly conservation
measures. Ethiopian Soil Conservation News (Addis Ababa), 15:
14–18.
Lee, W.S., 2008. Propensity score matching and variations on the
balancing test. In Third conference on policy evaluation, ZEW,
Mannheim, Germany, 27–28 October.
Leuven, E., Sianesi, S.B., 2003. PSMATCH2: Stata module to perform
full Mahalanobis and propensity score matching, common support
graphing, and covariate imbalance testing. Available online at: http://
ideas.repec.org/c/boc/bocode/s432001.html.
Lufumpa, L.C., 2005. The poverty-environment nexus in Africa. African
Development Review, 17(3): 366–81.
Mail & Guardian Online, 2006. More bodies found in flood-hit Ethiopia.
10 August 2006. Available online at: http://www.mg.co.za/search/
Search2007.aspx?keywords=More%20bodies%20found%20in%20
flood-hit%20Ethiopia. Accessed: 4 December 2007.
Ministry of Finance and Economic Development of Ethiopia (MOFED),
2002. Development and Poverty Profile of Ethiopia. MOFED, Welfare
Monitoring Unit, Addis Ababa, Ethiopia.
Ministry of Finance and Economic Development of Ethiopia (MOFED),
2006. Ethiopia—Building on Progress: A Plan for Accelerated and
Sustained Development to End Poverty (PASDEP). MOFED, Addis
Ababa, Ethiopia.
98
Menale Kassie, Stein Holden, Gunnar Köhlin and Randy Bluffstone / Natural Resources Forum 35 (2011) 89–99
Nyangena, W., Köhlin, G., 2008. Estimating returns to soil and water
conservation investments: an application to crop yield in Kenya, EfD
EfD Discussion Paper 09-13, a joint publication of Environment for the
Development Initiative and Resources for the Future (www.rff.org),
Washington DC, May 2008.
Okoba, B.O., Tenge, A.J., Sterk, G., Stroosnider, L., 2007. Participatory
soil and water conservation planning using an erosion mapping tool in
the Central Highlands of Kenya. Land Degradation & Development,
18(3): 303–319.
Pagiola, S., 1999. The global environmental benefits of land degradation
control on agricultural land. World Bank Environment Paper no. 16.
World Bank, Washington, DC.
Pender, J., Gebremedhin, B., 2007. Determinants of agricultural and land
management practices and impacts on crop production and household
income in the Highlands of Tigray, Ethiopia. Journal of African
Economies, 17: 395–450.
Pender, J.P., Kerr, J.M., 1998. Determinants of farmer’s indigenous soil
and water conservation investments in semi-arid India. Agricultural
Economics, 19: 113–25.
Rosenbaum, P., 2002. Observational Studies. Springer, New York.
Rosenbaum, P.R., Rubin, D.B., 1985.Constructing a control group using
multivariate matched sampling methods that incorporate the
propensity score. The American Statistician, 39: 33–38.
Rosenbaum, P.R., Rubin, D.B., 1983. The central role of the propensity
score in observational studies for causal effects. Biometrika, 70(1):
41–55.
Scherr, J.S., Yadav, S., 1997. Land degradation in the developing
countries: Issue and policy options for 2020. 2020 Brief no.44. IFPRI,
Washington, DC.
Shiferaw, B., Okello, J., Reddy, R., 2009. Challenges of adoption and
adaptation of land and water management options in smallholder
agriculture: Synthesis of lesson and experience. In Wani, S.P., John, R.,
and Oweis, T. (eds.), Rain-fed agriculture: Unlocking the potential.
Comprehensive assessment of water management in agriculture series,
CAB International, Wallingford, UK. 325 pp.
Shiferaw, B., Holden, T.S., 1998. Resource degradation and adoption
of land conservation technologies in the Highlands of Ethiopia: a case
study of Andit Tid, North-Shewa. Agricultural Economics, 21: 53–67.
Shively, G.E., 1999. Risks and returns from soil conservation: Evidence
from low-income farms in the Philippines. Environmental Monitoring
and Assessment, 62: 55–69.
Shively, G.E., 1998. Impact of contour hedgerows on upland maize yields
in the Philippines. Agroforestry Systems, 39: 59–71.
Sianesi, B., 2004. An evaluation of the Swedish system of active labour
market programmes in the 1990s. Review of Economics and Statistics,
86(1): 133–155.
Smith, J., Todd, J., 2005. Does matching overcome LaLonde’s critique
of non-experimental estimators? Journal of Econometrics, 125(1–2):
305–353.
Stocking, M., Abel, N., 1989. Labour costs: a critical element in soil
conservation. Paper presented at the 6th International Soil Conservation
Conference, Addis Ababa, Ethiopia, 6–8 November 1989.
Tadesse, M., Belay, K., 2004. Factors influencing adoption of soil
conservation measures in Southern Ethiopia: the case of Gununo area.
Journal of Agriculture and Rural Development in the Tropics and
Subtropics, 105(1): 49–62.
Tiffen, M., Michael, M., Francis, G., 1994. More People, Less Erosion:
Environmental Recovery in Kenya. John Wiley & Sons, Chichester.
World Bank, 2007. Ethiopia: Accelerating equitable growth. Country
Economic Memorandum, Poverty Reduction and Economic
Management Unit, Africa Region, World Bank, Washington, DC.
World Bank, 2009. Ethiopia: Climate risk factsheet. World Bank,
Washington, DC. Available online at: http://siteresources.worldbank.
org/INTAFRICA/Resources/Ethiopia_Country_Note.pdf. Accessed
December 2009.
World Food Programme, 2005. Report on the cost-benefit analysis
and impact evaluation of soil and water conservation and forestry
measures. Managing environmental resources to enable transitions
to more sustainable livelihoods (MERET) project, World Food
Programme, Addis Ababa, Ethiopia, February 2005.
Zeleke, G. (2006). Cost benefit framework for pro-SLM decision-making
process: Ethiopian case study, frameworks for quantifying the
biophysical processes of land degradation. Presentation at World Bank
workshop, May 2, 2006, Addis Ababa, Ethiopia.
Appendix
Propensity Score Matching Methods
In contrast to the Heckman and IV methods, matching
models assume that conditioning on observable variables
eliminates sample selection bias (Heckman and Navarro,
2004). PSM constructs a statistical comparison group by
matching every individual observation of adopters (plots
treated with fanya juu) with an observation with similar
characteristics from the group of non-adopters (plots not
treated with fanya juu). In essence, matching models create
the conditions of an experiment in which adopters and
non-adopters are randomly assigned, allowing for the
identification of a causal link between technology choice
and outcome variables.11 The seminal explanation of the
PSM method is by Rosenbaum and Rubin (1983), and its
strengths and weaknesses are elaborated, for example,
by Dehejia and Wahba (2002), Heckman et al. (1998),
Caliendo and Kopeinig (2008), and Smith and Todd (2005).
Propensity score matching is a two-step procedure. First,
a probability model for adoption of fanya juu terraces is
estimated to calculate the probability (or propensity scores)
of adoption for each observation. In the second step, each
adopter is matched to a non-adopter with similar propensity
score values, in order to estimate the average treatment
effect for the treated (ATT). Several matching methods
have been developed to match adopters with non-adopters
of similar propensity scores. Asymptotically, all matching
methods should yield the same results. However, in
practice, there are trade-offs in terms of bias and efficiency
with each method (Caliendo and Kopeinig, 2008). Here,
we use nearest neighbour matching (NNM) and kernelbased matching (KBM). The basic approach is to
numerically search for “neighbours” of non-adopters that
have a propensity score that is very close to the propensity
score of the adopters.
The main purpose of the propensity score estimation is
to balance the observed distribution of covariates across
the groups of adopters and non-adopters (Lee, 2008). The
balancing test is normally required after matching to
ascertain whether the differences in the covariates in the
two groups in the matched sample have been eliminated,
in which case, the matched comparison group can be
considered a plausible counterfactual (Ali and Abdulai,
11
We took adoption of fanya juu terraces as the technology choice
(treatment variable), whereas the net value of crop income per hectare (net
of the cost of fertilizer, seed) was the outcome of interest.
© 2011 The Authors. Natural Resources Forum © 2011 United Nations
Menale Kassie, Stein Holden, Gunnar Köhlin and Randy Bluffstone / Natural Resources Forum 35 (2011) 89–99
2010). Although several versions of balancing tests exist in
the literature, the most widely used is the mean absolute
standardized bias (MASB) between adopters and nonadopters suggested by Rosenbaum and Rubin (1985), in
which they recommend that a standardized difference of
greater than 20% should be considered too large and an
indicator that the matching process has failed. Additionally,
Sianesi (2004) proposed a comparison of the pseudo R2
and p-values of the likelihood ratio test of the joint
insignificance of all the regressors obtained from the logit
analysis before and after matching the samples. After
matching, there should be no systematic differences
in the distribution of covariates between the two groups.
© 2011 The Authors. Natural Resources Forum © 2011 United Nations
99
As a result, the pseudo-R2 should be lower and the joint
significance of covariates should be rejected (or the
p-values of the likelihood ratio should be insignificant).
If there are unobserved variables that simultaneously
affect the adoption decision and the outcome variables,
a selection or hidden bias problem might arise to which
matching estimators are not robust (Rosenbaum, 2002).
We checked the sensitivity of the estimated average
adoption effects (ATT) to hidden bias, using the
Rosenbaum (2002) bounds test. This test suggests how
great an effect unobservables would have to have in
order to reverse the findings based on matching on
observables.