Overview of the Organized Symposium # OS 06-02: Symposia 6
29th International Conference of Agricultural Economists
Milan, Italy, August 9-14, 2015
Title: Improving the methods of measuring varietal adoption by farmers in developing
countries: Recent experience with the use of DNA fingerprinting and implications for
tracking adoption and assessing impacts
Theme: Adoption of Improved Varieties Using Innovative Methods
Session Organizers: Mywish K. Maredia ([email protected]), Michigan State
University, and Byron Reyes ([email protected]), International Center for Tropical
Agriculture (CIAT)
Abstract:
Identifying and measuring the area under improved varieties and assessing
varietal turnover plays a central role in varietal adoption and impact
assessments. These studies have mostly relied on farmers’ responses in
household surveys to estimate these indicators. This method of ‘farmer
elicitation’ to estimate varietal adoption can be fairly accurate when the
varietal turnover is high and the seed system is well-functioning.
However, when the formal seed system is non-existent or ineffective, and
farmers mostly rely on harvested grain as the main source of planting
material, the reliability of estimating varietal adoption using farmer or
expert elicitation method can be challenging. This symposium brings
together researchers who have used the DNA-fingerprinting method for
varietal identification. It provides a forum for exchange of ideas and
sharing new insights on the challenges and potential of using this
innovative method for estimating varietal adoption and increasing the
accuracy of results of impact assessments.
JEL Codes: C81 Methodology for Collecting, Estimating, and Organizing
Microeconomic Data, Data Access, C83 Survey Methods, Sampling
Methods, O3 Technological Change; Research and Development
1
1. Background and Rationale
Since the pioneering research by Griliches on assessing the impact of hybrid corn
adoption in the U.S. almost six decades ago, the interest in measuring the impacts of
adoption of improved technology by farmers has expanded to include a gamut of
agricultural technologies in both developed and developing country settings. Among the
most widely assessed agricultural technologies in the developing country context is the
adoption of improved varieties. These assessments have consistently reported that
adoption of improved varieties and rapid varietal turnover increases productivity, income
and other measures of welfare of farm households.
Central to these assessments is the identification of improved varieties, measuring
the area under those varieties, and assessing varietal turnover. Most varietal adoption and
impact assessment studies in the past have relied on farmers’ responses in household
level surveys to estimate these indicators. This method of ‘farmer elicitation’ to estimate
varietal adoption can be fairly accurate in a setting where farmers are mostly planting
seeds freshly purchased or acquired from the formal seed market as certified or truthfully
labeled seed, and the seed system is well-functioning and effective in monitoring the
quality and genetic identity of varieties being sold by the seed vendors. However, in
settings where the formal seed system is non-existent or ineffective, and farmers mostly
rely on harvested grain (either from their own farms or acquired from other farmers or
purchased from the market) as the main source of planting material, the reliability of
estimating varietal adoption using this method is challenging. By implication, it also
makes the results of impact assessments based on those adoption estimates questionable.
The challenges stem from several confounding factors. These include farmers’
inability to identify varieties by names, the inconsistency in the names of the varieties as
identified by the farmers and what is in the variety registration list (i.e., varieties may
have locally adapted names), and the loss of genetic identity due to cross-pollination
when seeds are recycled several seasons. DNA fingerprinting, which is routinely used by
plant breeders and is becoming widely available and affordable, offers a reliable method
to address these challenges and to accurately identify varieties grown by farmers. The use
of this method can thus increase the accuracy and credibility in the interpretation of
results of economic analysis based on household surveys that estimate the causal link
2
between the adoption of improved varieties and the impact on crop productivity and
income.
Despite the advantages, there are several issues related to sampling, logistics, and
cost-effectiveness of using this innovative method that need to be investigated and
addressed before DNA fingerprinting method becomes routine for tracking varietal
adoption and assessing the impact at the farm level. This proposed symposium proposes
to bring together economists who have recently used the DNA fingerprinting method for
varietal identification. The symposium will provide a forum for exchange of ideas and
sharing new insights on the challenges and potential of using this innovative method for
estimating varietal adoption and varietal turnover, and increasing the accuracy of
estimates of impacts of varietal adoption by farmers.
2. Symposium Format
The symposium will feature four short paper presentations preceded by
introductory remarks by the symposium organizers. The presenters will share their recent
experiences of using the DNA fingerprinting methods to estimate adoption rates of rice
varieties in Bolivia, wheat and maize varieties in Ethiopia, cassava varieties in Ghana,
and bean varieties in Zambia. Approximately one-third of the time will be left for
discussion and obtaining feedback from attendees. Given the novelty of this method, it is
expected that symposium attendees would engage in discussing the methodology, its
cost-effectiveness, and the implications of the results on the extent to which there are
estimation errors in existing varietal adoption data. This should provide good feedback
that could be used to advance this methodology for tracking varietal adoption and in
future technology impact studies that rely on farm household survey data.
3. Presenters and their Bios (in alphabetical order)
Ricardo Labarta <[email protected]>
Ricardo Labarta is a Senior Scientist and the Impact Assessment Research Leader at the
International Center for Tropical Agriculture (CIAT). He has many years of experience
working on impact assessment of agricultural technologies and natural resource
management. His economic research has focused in biotechnology, integrated crop
3
management, experimental economics, agricultural extension, public-private partnerships
and natural resource economics. He has developed his professional career working in
many countries of Africa and Latin America and has previously worked for the
International Potato Center (CIP) and World Agroforestry Center (ICRAF). Ricardo
Labarta holds a PhD in Environmental and Natural Resource Economics, and a MSc in
Agricultural Economics, both from Michigan State University.
Mywish K. Maredia <[email protected]>
Mywish Maredia has worked extensively in the area of impact assessment and the
economics of agricultural science and technology. She is currently a Professor,
International Development, in the Department of Agricultural, Food and Resource
Economics at Michigan State University. Her research focuses on the evaluation and
impact assessment of international development interventions. She has served as the
Deputy Director of the USAID funded Dry Grain Pulses CRSP from 2000-09, and as a
member of the Standing Panel on Impact Assessment of the CGIAR’s Science Council
from 2006-11. She was the recipient of the Outstanding Ph.D. Dissertation Award in
1994 from the American Agricultural Economics Association.
Byron Reyes <[email protected]>
Byron Reyes is currently an Agricultural Economist at the International Center for
Tropical Agriculture (CIAT) based in Nicaragua. His main research area is impact
assessment of agricultural research and development. He has research experience in Latin
America (Costa Rica, Ecuador, El Salvador, Guatemala, Honduras and Nicaragua) and
Africa (Angola, Burkina Faso, Ghana, Mozambique and Zambia) and has experience in
household level survey and impact evaluation design, data collection, complex household
survey data analysis and testing alternative methods to estimate adoption of improved
varieties.
Chilot Yirga Tizale <[email protected]>
Chilot Yirga is a socio-economist at the Ethiopian Institute of Agricultural Research,
Addis Ababa. He has expertise in Agricultural development, climate change,
4
environmental science, and sustainable development. He has led many household survey
projects and authored several publications based on his research on assessing the impact
of the adoption of agricultural technologies by farmers in Ethiopia.
Greg Traxler <[email protected]>
Greg Traxler is a Senior Lecturer in the Evans School of Public Affairs at the University
of Washington. Prior to that he was a Senior Program Officer at the Bill & Melinda Gates
Foundation, and Professor in the Department of Agricultural Economics at Auburn
University. His teaching and research focuses on research policy and the impacts of
agricultural technology in the U.S. and in developing nations.
4. Summaries of Papers to be Presented
Paper 1
Using DNA Fingerprinting to Estimate the Bias of Farm Survey Identification of the
Diffusion of Improved Crop Varieties in Ethiopia.
Presenter and author(s): Chilot Yirga Tizale or Greg Traxler
Co-authors: Mariana Kim, and Dawit Alemu
All existing evidence on the adoption of improved crop varieties has been
collected either by expert elicitation or through farm surveys. The reliability of these
approaches has never been verified, leaving the bias and standard errors of existing
diffusion estimates unknown. This study will report the results of a nationally
representative survey of 2,000 maize and 2,000 wheat farmers in Ethiopia conducted
during the 2014/15 growing season. Plant samples taken from farmers’ fields were
compared to breeders’ seed reference materials using DNA‐fingerprinting to generate
precise estimates of the area under each variety released in the country. The DNA
fingerprinting estimates were compared to estimates derived using farmer variety
identification. Analysis of the relative productivity of each variety was also conducted
using crop‐cut estimated yields and information on crop management practices. Survey
information on seed sources and seed handling was used to assess the impact of seed
management on yield and variety performance.
5
The study intended to evaluate the merit of using DNA fingerprinting for tracking
crop varietal use by smallholder farmers in Ethiopia. The primary objective was to
ascertain whether DNA provides a superior alternative to other approaches for tracking
varietal diffusion. Specifically, the study aimed to determine whether DNA fingerprinting
is accurate, cost-effective, and feasible. The main study questions are:

Are the data developed through fingerprinting accurate and specific enough to
provide a useful account of varietal diffusion?

Can DNA fingerprinting be done at a cost that can be incorporated and scaled
nationally?

Can the benefits of the study outweigh the required investments in building the
operational, logistical, and technological capacity of Ethiopian agencies and actors?
Preliminary results confirm widespread use of improved wheat and maize
varieties in Ethiopia, but that farmer recall data underestimates the diffusion levels of
improved varieties. This phenomenon is more pronounced in the case of wheat than for
maize. Preliminary estimates of adoption levels of improved wheat varieties based on the
farmer recall information is 62% compared to 96% from the DNA fingerprinting
approach. In the case of maize, estimates based on farmer recall data indicate 56%
adoption rate for improved varieties compared to 61 % from the DNA finger printing.
Genetic fingerprinting appears to be a technically feasible method for tracking varietal
diffusion and that generates more precise estimates than farmer recall data. All adoption
estimates will be updated in June-July 2015.
The relatively high levels of mismatch between what the farmers reported and the
actual genetic material in their fields warrants further investigation. Only 9% of wheat
farmers correctly identified the variety that they were using; in the case of maize, this
figure was higher but still only 30%. Starting from the premise that farmers have no
incentive to hide or provide misleading information on varieties in their fields, it is
plausible to posit that the majority of the respondents reported what they knew; it would
be desirable to find out if there are any incentives for farmers to misreport to hide their
use of de-listed varieties. Understanding the cause and source of distortion or information
gap could be achieved by examining the existing farmer information networks including
6
the sources, nature and medium of communicating information on improved technologies
to farmers.
For Ethiopia’s extension and seed delivery systems the results suggest that their
primary clients do not have adequate information on maize and wheat varieties in their
fields. Likewise, given that the seed demand assessment is based on information gathered
by frontline extension staff interviews with farmers, the seed demand projections that
inform foundation seed production are inaccurate, leading to a mismatch between seed
supply and demand. For the research system, these findings imply that that previously
adoption estimates are likely underestimates.
There is potential for wider application of the DNA fingerprinting technique in
Ethiopia to balance seed supply and demand. DNA analysis could also enhance the
capacity of the seed regulatory authorities in Ethiopia by helping to resolve some of the
seed quality disputes arising from reported contamination. The identification of one
wheat variety with two different names (registered in Variety Release Registry) among
the reference materials in the present pilot study confirms the need for a more rigorous
check before a variety is released.
Generating more accurate estimates of adoption levels using genetic
fingerprinting approach requires additional inputs in terms of human capital and
operational costs, compared to the traditional farm household surveys and use of
secondary data. A typical data household survey data collection costs range between 45
to 60 USD per respondent. DNA fingerprinting is estimated to add 15 to 20 USD per
respondent. These are rough estimates based on variable cost components of the pilot as
opposed to an estimation of the full research costs.
Paper 2
Assessing Impacts of the adoption of modern rice varieties using DNA
fingerprinting to identify varieties in farmer fields: A case study in Bolivia
Presenter and author: Ricardo A. Labarta
Co-authors: Jose M. Martinez, Diana C. Lopera, Carolina Gonzalez, Constanza
Quintero, Gerardo Gallego, Juana Viruez and Roger Taboada
7
Documenting the adoption of agricultural technologies has a long history in
agricultural economics. Perhaps the most studied topic has been the adoption of improved
crop varieties which has resulted in large evidence of the level of adoption of different
improved crop varieties and about the determinants that favor or constraint the adoption
of these varieties. In most of these studies researchers have used survey instruments and
relied on farmers self-reported variety names to estimate varietal adoption and the
determinants of this adoption.
In more recent studies, there has been an attempt to verify whether the varieties
reported by farmers are actually the varieties that they are growing on their crop plots. In
spite that many adoption studies have included crop experts and used photographs and
morphological descriptors to verify the names and the origin of the varieties reported by
farmers, many of these recent studies have found difficulties in verifying the crop
varieties being grown by farmers and have failed to identify important number of
varieties in farmers’ fields (Walker et al 2014, Larochelle et al 2013). This has created
some uncertainties about the level of adoption of some crop varieties and the validity of
some studies aiming to estimate the determinants of these varieties.
DNA fingerprinting has been proposed as a means that could help to improve
crop varietal identification and especially in recent times that is becoming affordable for
large samples. This study tests the use of fingerprinting in the identification of rice
varieties and compares it with the use of varietal identification self-reported by rice
growers in Bolivia. It then provides an estimation of the level of adoption of modern
improved varieties and allow to study the determinants of the adoption of these modern
varieties. The study then compares the results of the adoption rates and estimation of
adoption determinants using farmers self-reported varietal identification
This study was designed to first estimate the level of adoption of modern
improved rice varieties in Bolivia and then the determinants of the adoption of these
modern varieties. It has two components and the first one collected household and plot
level data from rice growers of the departments of Santa Cruz, Beni and Cochabamba
that represents 97% of the total rice production in Bolivia. This field level data included
the registration of all varieties planted by rice growers interviewed and the acreage of
each of them.
8
The analysis reported in this paper is based on a sample of 298 households located
in 56 different communities that were randomly selected. For the 298 farmers that had
planting material at the time of the interview, we first ask farmers to identify all rice
varieties that they have grown in the last growing season and to estimate the area under
each rice variety. We then requested farmers in the same interview to provide a small
quantity of seed (10-20 grains) that were immediately inserted in a small paper bag and
labelled with information about the household and the location of the collection. All the
rice seed samples collected from Bolivia were shipped to CIAT headquarters in Cali,
Colombia for DNA extraction and analysis using Single Nucleotide Polymorphisms
(SNPs) that is based on the fluidigm genotyping. As a second component of this study,
the DNA fingerprinting method served as an alternative varietal identification of all
planting material found on the ground.
Our Results indicate that using DNA fingerprinting varietal identification the
adoption rate of modern varieties is estimated in 44.96% compared with the 41.58% that
is estimated when using farmers’ self-identification of rice varieties. Thus, using only
farmers’ self-identification of varieties may lead to an underestimation of the adoption of
modern rice varieties of almost 3.5 percentage points. The difference in estimates for old
varieties reaches only 2.25 percentage points. It seems that all unidentified varieties by
farmers where later confirmed as modern varieties by the genetic analysis. But behind
these relatively modest changes in the estimated adoption rate there are larger mismatches in varieties that were identified by farmers as modern or old and those that were
identified by those categories by the DNA fingerprinting analysis. Our analysis found
that 8.38% of the varieties that were reported by farmers as old varieties turned out to be
modern varieties. Likewise, 3.23% of the varieties that were identified as modern
varieties by farmers where really old varieties according to the DNA fingerprinting
identification.
Regarding the estimation of the determinants of the adoption of modern rice
varieties, our estimations found mixed results. On one hand both definitions of modern
variety indicates that there is a strong and significant effect of the distance of San Juan
the Yapacani (the main center of dissemination of rice technologies in Bolivia) and the
effect of being located in the Beni department. While the farther the distance from San
9
Juan de Yapacani reduces considerably the probability of adopting modern rice varieties,
being located in the Beni department increases significantly the probability of adoption of
modern varieties. This can be explained by the large demand of new seed that is being
experimented by Beni that is a new frontier of the rice production.
On the other hand both definitions of modern varieties also differ in other
determinants of the adoption of these modern varieties. While an adoption using farmers
self-reported varietal identification shows a significant and positive effect of years of
education of household head and farm size, an adoption using DNA fingerprinting results
suggests a significant but negative effect of household head age.
Our findings confirm the importance of using DNA fingerprinting as an
alternative and necessary method to identify varieties. Although rice is a fairly
homogenous crop with very few varieties released in Latin America, relying only in
farmers (small holders) self-identification of the varieties that they are growing may lead
to wrong conclusions. In the case of rice it seems to be an underestimation of the
adoption of modern varieties and different results to understand the determinants of this
adoption. However, more research would be needed in order to further analyze the
implications of a different identification of rice varieties using DNA fingerprinting and
the challenges faced in the application of this advance method on adoption studies.
Paper 3
Testing the effectiveness of different approaches of collecting variety-specific adoption
data against the benchmark of DNA fingerprinting: The case of cassava in Ghana
Presenter and author: Byron Reyes
Co-authors: Mywish Maredia, Joe Manu, Awere Dankyi, Peter Kulakow, Ismail Rabbi,
Elizabeth Parkes, and Tahirou Abdoulaye
Assessing technology adoption and its impact has expanded since the pioneering
work of Griliches. Among the most widely assessed agricultural technologies in the
developing country context is the adoption of improved varieties. Central to these
assessments is the identification of improved varieties, measuring the area under those
varieties, and assessing varietal turnover. Most varietal adoption and impact assessment
studies in the past have relied on farmers’ responses in household level surveys to
10
estimate these indicators (although there are other alternatives-each with its pros and
cons- also used such as secondary data, expert opinions, etc). The accuracy of the
indicators thus depends on how well the farmer identifies each variety grown, which in
turn depends on them knowing the name of the variety which in many cases changes
from one source to another (i.e., there are no consistency in the names for the same
genetic material or variety).
Because of this, it is of utmost importance to estimate adoption of varieties using
techniques that can provide more accurate information. A technique that is becoming
more popular among economists includes testing DNA from plant tissue to separate
individuals with different genetic characteristics, commonly called DNA fingerprinting.
Under the Strengthening Impact Assessment in the CGIAR (SIAC) project, several pilot
studies were implemented to explore and understand practical challenges of using DNA
fingerprinting as a method of varietal identification as part of a farmer survey, test the
effectiveness of different methods of varietal identification against a benchmark (i.e.,
DNA fingerprinting), and to come up with lessons learned and recommendations on
methods that can be scaled up.
This paper presents the results from the pilot study implemented in 2013 in Ghana
for cassava (Manihot esculenta). Cassava was selected for one of the pilot studies based
on its importance in the diet and the interest from national and CGIAR centers to
collaborate in this study, which required a multi-disciplinary team of researchers. The
study was conducted in the Brong Ahafo, Ashanti and Eastern regions, which account for
61% of cassava production. A total of 500 households across 100 villages (five farmers
per village) were sampled using a multistage cluster sampling method, where districts,
villages and farmers were randomly chosen. However, the realized sample was of 495
households. The survey was implemented in October-November 2013 and was
coordinated by a research team led by the cassava breeder from the Ghana Crops
Research Institute (CRI) and a socioeconomist from the Agriculture Innovation Consult
(AIC). The survey included a household interview, a field visit to one cassava plot (the
plot with the most varieties), and obtaining tissue samples from cassava plants for DNA
extraction and analysis. Each survey team consisted of three people with different
responsibilities: an enumerator responsible for completing the household modules, a
11
cassava expert responsible for completing the field module, and a DNA sampling expert
responsible for collecting, labeling and storing the plant tissue samples as per an
established protocol. A total of 855 samples were collected for DNA fingerprinting.
Among the methods for variety identification tested were A) farmers’ responses
about name (A1) and type of variety (A2), B) asking farmers to indicate the
morphological characteristics of the plants from observing pictures, information that was
later used for identification of varieties, C) cassava experts visiting a cassava field and
recording observations on varietal characteristics which was later used for identification
(C1) and also identifying the variety based on these observed characteristics (C2), D)
taking pictures of characteristics of the plants during the field visit for latter identification
by experts, and E) DNA fingerprinting from leaf tissue taken during the field visit. The
estimates of adoption rates obtained from methods A-D were compared to varietal
identification obtained from DNA fingerprinting (method E), the benchmark.
The information collected using methods B and C1 was used to generate a unique
code because each of the eleven morphological characteristic evaluated had a value (0-9)
that when combined, allowed creating a unique code, which was compared to a similar
code generated from known morphological characteristic of the varieties released in the
country (contained in a ‘library’ created for this purpose). Method D required assembling
a panel of crop experts familiar with the varieties grown in the study area, including
breeders and technicians from different institutions, to look at the pictures to identify
varieties. Method E required first establishing a reference library of DNA fingerprints,
and then applying the same or a sub-set of markers used to establish the reference library
to genotype the samples collected.
The results suggest that there may be considerable differences between the
estimates of adoption rates obtained by these methods, compared to fingerprinting results.
About 180 variety names were reported by farmers, being Debor and Ankra the most
common names. When asked the type of varieties grown (method A2), farmers reported
that most were local varieties (87%), followed by not knowing the type of variety (7%)
and growing improved varieties (6%). However, only 20 of the 51 names given by
farmers when reporting growing an improved variety (IV) matched with the name of an
IV. Several farmers also (mistakenly) gave names of IVs when reporting growing a local
12
variety. During the field visit, cassava experts could not identify the name of a variety for
299 observations but they reported (method C2) that most varieties were local (88%),
followed by not knowing the type of variety (7%) and improved varieties (5%). In
contrast, experts looking at photos (method D) reported that most varieties were local
(70%), followed by improved varieties (16%) and not knowing the type of variety (14%).
Results from method B are still being analyzed and are not presented in this paper. The
results from method E suggest that some of the released improved varieties are
genetically identical, many released varieties are mixtures or hybrids, and that library
accessions representing both released and local varieties (or landraces) fall under the
same cluster groups. The latter represents a challenge because farmer samples that fall
under these cluster groups could be classified either as an IV or a local variety.
Due to the ambiguity of the DNA results regarding the cluster groups, two
scenarios were analyzed: a liberal scenario where farmer samples that fall in any of these
cluster groups were assumed to be IVs, and a conservative scenario where they are
assumed to be local varieties. The DNA results suggest that adoption of IVs ranged from
4% (conservative scenario) to 31% (liberal scenario).
When comparing methods A1, A2, C1, C2, and D to the benchmark (method E),
in the conservative scenario where the DNA results suggest 4% adoption rate, adoption
rates estimated from methods A1 and D greatly differ (1% and 15%, respectively) from
the truth. However, estimations of adoption rates from methods A2, C1 and C2 are close
to the truth, being method C2 the closest. In the liberal scenario where the DNA results
suggest 31% adoption rate, adoption rates estimated from all other methods considerably
underestimate adoption rates (the highest adoption rate came from method D--15%).
These results suggest that no method stands out to be the most effective on all
measures, that methods based on farmer elicitation and field observations by experts
provide closest estimates under the conservative scenario but with high error rate, no
method come close to the truth in adoption estimates in the liberal scenario, identifying
cassava varieties accurately by name when hundreds of names exist is a challenge across
all methods, and adoption estimates by experts are substantially higher than other
methods and have much higher false positives.
13
Further, when there is a diversity of names by which farmers call their varieties,
the traditional method of farmer elicitation give an underestimation of adoption of
improved varieties. Thus, the current method most commonly used (i.e., farmer
elicitation) may not be an accurate method for measuring varietal turnover and
assessment of type II (new IVs replacing old IVs) benefits of plant breeding research.
Also, farmers and experts are better able to give an aggregate assessment of the adoption
of improved varieties as a category than by name. Finally, none of the alternative and
non-traditional methods tested emerged as most effective and their scalability remains
questionable on the grounds of cost-effectiveness.
Paper 4
Testing the effectiveness of different approaches of collecting variety-specific adoption
data against the benchmark of DNA fingerprinting: The case of beans in Zambia
Presenter and author: Mywish Maredia
Co-authors: Byron Reyes, Enid Katungi, Petan Hamazakaza, Kennedy Muimui, Bodo
Raatz and Clare Mukankusi
Rationale and objective
Varietal adoption based on household surveys has mostly relied on farmers’
response to varietal identification. This method can give biased estimates if farmers are
unable to give the correct name or give names that do not match with the improved
variety list. To tackle these potential problems requires time intensive data collection
such as including follow-up questions in the survey instrument, visiting the field to
observe plant characteristics, or collecting sample materials (i.e., photos, seeds/plant
tissues) from the farmers for later verification by experts. Each of these approaches has
implications on the cost of data collection and the accuracy with which they can correctly
identify a variety.
DNA fingerprinting offers a reliable method to accurately identify varieties grown
by farmers. The use of this method can: a) Increase the accuracy and credibility in the
interpretation of results of economic analysis based on household surveys that estimate
the causal link between the adoption of improved varieties and the impact on crop
14
productivity and income; and b) Serve as a benchmark against which to compare the
effectiveness of other potential methods for scaling up.
However, despite the advantages, DNA fingerprinting has not been used widely
for tracking varietal adoption. Questions related to sampling, logistics, and costeffectiveness of using this innovative method remains to be explored. This study reports
the results of a pilot study conducted in Zambia to: 1) Explore and understand some of
the practical challenges of using DNA fingerprinting as a method of varietal
identification as part of a farmer survey; 2) To test the effectiveness of different methods
of varietal identification against the benchmark of DNA fingerprinting; and 3) To come
up with ‘lessons learned’ and recommendations on methods / approaches that can be
scaled up.
Methodology
Four methods of tracking varietal adoption using farm household surveys were
evaluated against the benchmark of DNA fingerprinting. These methods can be grouped
into two types—farmer elicitation methods (methods A-B) and expert elicitation methods
(methods C-D). These methods include:
Method A: As part of the survey instrument, farmers were asked to provide the
name(s) and type (improved vs. local) of varieties planted in the current planting
season (for cassava) or the last completed season (for beans). This method of
eliciting the name of the variety was also implemented with vendors selling dry
beans in two local markets in the study region.
Method B: This method involved showing the farmer seed samples representing
different varieties and asking him/her to identify the seed sample that matched the
varieties grown on their farm.
Method C: This method consisted of taking photographs of harvested seeds and
later using these pictures for varietal identification by a panel of experts.
Method D: Consisted for collecting seed samples of varieties grown by the
farmer for later identification by a panel of experts.
15
For both methods C and D, the expert panel consisted of breeders and extension staff
from the study districts. After showing the photos or seed samples, a consensus name and
type of variety (improved, local, mix) was recorded for each sample.
The study was conducted in Muchinga and Northern provinces of Zambia based
on the importance of beans (Phaseolus vulgaris) and prior seed dissemination efforts. A
total of seven districts were purposively selected which together represent 59% of the
total bean area in Zambia. Data were collected from a sample of about 400 farmers across
67 villages (sample size mostly determined based on the available budget). The survey
was implemented between August-September 2013.
DNA fingerprinting methodology used to establish the benchmark involved, first
establishing a reference library of DNA fingerprints, and then collecting samples (plant
tissues or seeds) during the farm surveys and genotyping them using the same or a subset of markers used to establish the reference library. Towards this goal, 13 accessions
specific to Zambia (including 11 released varieties and two landrace Kabulengeti market
classes) and 723 accessions from the East/Southern Africa region (that were genotyped as
part of another project by CIAT) were included in the reference library as the
‘background’ materials to compare the samples collected from farm surveys. The farmer
samples were genotyped using 66 assays/markers selected as a sub-set of ~800 SNPs
used for the reference library. The 66 SNP assays were made up of 4 groups, each of
which has more or less the same power to differentiate released varieties from each other
and from the background genotypes.
Results
About 16% of bean samples collected from farmers were identified as sharing
identical DNA fingerprints with four released varieties in Zambia. A majority of these
belonged to the variety group kabulengeti, which was an improved landrace released by
ZARI in 2007. As against this benchmark, effectiveness of different methods was
evaluated using four different measures: Aggregate outcome, Accuracy of name and type,
type I error (false negative) and type II error (false positive).
Results indicate that there is no one method that stands out to be most effective
across all measures. In terms of outcome, at the aggregate level, the estimates of data
16
points classified as improved varieties by experts either based on seed photos (18%) or
seed samples (15%) were closest to the estimates using DNA fingerprinting (16%). The
farmer elicitation method A1 (i.e., asking farmers the name of the variety and matching
that with the name in the released variety list) gave the lowest estimate of varietal
adoption (4%). Identification of varieties based on farmers’ self-reported assessment of
growing an improved variety (13%) was substantially higher than the identification of
improved variety based on the self-reported names (4%) and closer to the benchmark of
16% estimated using the DNA fingerprinting method. In Method C, where farmers were
shown the seed samples of released varieties and asked to match which one their planted
seed resembled, farmers substantially over reported the adoption of varieties (71%).
All the methods had low accuracy rate and high type II error rates when the
outcome for each data point is compared against the DNA fingerprinting result. Only 9%
of data points were correctly matched with the variety name in Method A1 (i.e., farmer
elicitation of variety name) and 16% matched by variety type in Method A2. The
accuracy rate in expert elicitation methods C and D was 27% and 30%, respectively. In
other words, in more than 70% of observations, an improved variety was incorrectly
identified by experts as a local variety or by an incorrect name in Methods C and D. this
is considered that type II error. In terms of type I error, a local variety was incorrectly
identified as an improved variety for 3%, 13%, 67%, 16% and 12% cases under Methods
A1, A2, B, C and D, respectively.
Implications and need for further research
The results of this study have many implications for the methodology of varietal
identification used in the past and present. For example, this study has shown that when
there is a diversity of names by which farmers call their varieties, the traditional method
of farmer elicitation will give an underestimate of adoption of improved varieties by
names. Thus the current gold standard of eliciting varietal adoption from farmer surveys
may not be an accurate method for measuring varietal turnover and assessments of type II
benefits of plant breeding research (i.e., benefits from varietal replacement). On the other
hand, showing the seed samples to elicit farmers’ response on variety specific adoption is
prone to overestimate adoption of improved varieties if only improved varieties are
17
included. This was a limitation of this study where Method B only included seed samples
of improved varieties. More studies are needed to test whether the upward bias of this
method can be reduced by including some popular landraces in the visual samples.
Results also indicate that farmers and experts are better able to give an aggregate
assessment of the adoption of improved varieties as a category than variety specific
adoption. However, all the methods evaluated are prone to both type I and type II errors
which has implications on the accuracy of any adoption analysis conducted using such
data at the farmer level.
None of the alternative and non-traditional methods tested emerged as most
effective on all measures of effectiveness; although, methods that involved experts’
opinion and interpretation were generally closer to the benchmark estimates. However,
given the time and logistics of implementing these methods, the scalability of some of
these methods remains questionable on the grounds of ‘cost-effectiveness’ and feasibility.
This study has shown that molecular markers (SNPs) technology is a useful tool
for determining the genetic identity of varieties grown by farmers. It provides a ‘true
picture’ of what is in farmers’ fields. However, the potential for scaling up this method as
part of household surveys will depend on several factors, such as: a) the logistics of
collecting, tracking, storing and transporting the samples from farmers’ fields to a lab
facility to get high quality DNA; b) the cost of DNA fingerprinting which includes—
establishing the reference library, DNA extraction, and genotyping service. In this study
the estimated cost per data point was ~$30, which can add substantial costs to the overall
cost of doing household surveys in developing countries; and c) capacity to do high
volume DNA fingerprinting within the country or easy access to such capacity
internationally (i.e., no government restrictions on the shipment of plant tissues or DNA
samples to other countries for analysis).
Given these challenges, the use of DNA fingerprinting as part of large scale
representative household surveys may be long way from becoming routine. Potential
ways to reduce the cost and to make the logistics more manageable would be to use DNA
fingerprinting as a method of validation on a random sub-sample of households rather
than all the households. More studies on different crops and country settings are needed
to generate an experience base and derive generalizable conclusions.
18