Hyperspectral imaging system for disease scanning on banana
plants
Daniel Ochoa1 , Juan Cevallos2 , German Vargas2 , Ronald Criollo2 , Dennis Romero2 , Rodrigo
Castro2 and Oswaldo Bayona2
1,2 Escuela
Superior Politécnica del Litoral, ESPOL, Campus Gustavo Galindo Km 30.5 Vı́a
Perimetral, P.O. Box 09-01-5863, Guayaquil, Ecuador
1. ABSTRACT
Black Sigatoka (BS) is a banana plant disease caused by the fungus Mycosphaerella fijiensis. BS symptoms
can be observed at late infection stages. By that time, BS has probably spread to other plants. In this paper,
we present our current work on building an hyper-spectral (HS) imaging system aimed at in-vivo detection of
BS pre-symptomatic responses in banana leaves. The proposed imaging system comprises a motorized stage, a
high-sensitivity VIS-NIR camera and an optical spectrograph. To capture images of the banana leaf, the stage’s
speed and camera’s frame rate must be computed to reduce motion blur and to obtain the same resolution along
both spatial dimensions of the resulting HS cube. Our continuous leaf scanning approach allows imaging leaves
of arbitrary length with minimum frame loss. Once the images are captured, a denoising step is performed to
improve HS image quality and spectral profile extraction.
Keywords: Hyperspectral Imaging, Black Sigatoka
2. INTRODUCTION
In 2015, Ecuador sold 300 million banana boxes produced by both large and small scale growers. Black Sigatoka
(BS) is a constant threat to banana farmers worldwide. It can cause yield losses of 30% to 50%, particularly in
the small farms.1 Detection of BS is difficult, once visible symptoms appear in the leaves, the whole crop may be
already compromised. Clearly, early detection of BS is needed to prevent the disease spread and reduce damage
to crop production.2
HS imagery is relatively recent and has not been widely applied in plant pathology. Most works were focused
on measuring the crop damage using satellite images3 and more recently using cameras mounted on unmanned
aerial vehicles (UAV).4 Typically, the aim of HS imaging in remote sensing is the discrimination of healthy
and unhealthy plants, those that show advanced infection levels i.e. reduction in plant’s total leaf area and
concentration of chlorophyll. A comprehensive summary of HS imaging approaches in remote sensing can be
found elsewhere.5
The detection of a specific plant pathogens and the discrimination between diseased levels can be investigated
at lab scale. When plants are exposed to pathogens they activate complex defense responses. The detection of
subtle changes in light reflectance at early infection stages requires controlled conditions. In practice, greenhouse
plants are imaged in-vivo at higher spatial and spectral resolutions under controlled temperature and illumination
conditions. At the early stages, plants react to the presence of a pathogen with physiological mechanisms
such as the reduction of the photosynthesis rate, which induces an increase of fluorescence and heat emission
at the leaf blade6 and then a reduction of leaf plant chlorophyll content due to necrotic or chlorotic lesions
which are translated into reflectance variations at the VIS and NIR regions of the spectrum.7 Recently, HS
imaging techniques to detect and monitor pathogen related diseases in leaves has been investigated in controlled
environments conditions for sugar beet,8 wheat9 and tomato10 plants. To the best of our knowledge, there is not
any reported work on HS imaging to detect pathogen diseases that affect banana plants.
In this paper, we present our current work on building a HS imaging system aimed at in-vivo detection of
BS symptomatic responses in banana leaves. The proposed imaging system comprises a motorized stage, a highsensitivity VIS-NIR camera and an optical spectrograph. The following sections explains the technical details
[email protected]
Sensing for Agriculture and Food Quality and Safety VIII, edited by Moon S. Kim, Kuanglin Chao, Bryan A. Chin,
Proc. of SPIE Vol. 9864, 98640M · © 2016 SPIE · CCC code: 0277-786X/16/$18 · doi: 10.1117/12.2224242
Proc. of SPIE Vol. 9864 98640M-1
of this system. The first section describes the design criteria and elements of the system. Then, preliminary
imaging results obtained from leaves at different BS disease stages are presented. The final section gives the
main conclusions and future directions.
3. IMAGING SYSTEM
3.1 Design
In-vivo acquisition of HS images of banana leaves presents several challenges. First, banana leaves are relative big
compared to model plants such as Arabidopsis thaliana or Sugarbeet. Therefore, the proposed imaging system
employed the line scanning (push-broom) method. This technique is particularly suitable for our experiments
as banana leaves have different sizes. Given that a banana plant can be relatively heavy it was decided to move
the imaging device while keeping the sample leaf fixed, instead of moving the plant. Also, as early symptoms of
BS appear on the leaf’s blade underside, the imaging device is located below the plant with the camera pointing
upwards. An advantage of this configuration is that a simple mechanical system with low-power motors can
be used to move the camera. However, careful setting of the imaging device’s frame rate and motor’s speed is
required to obtain images without distortion.
Figure 1. Left. Hyperspectral imaging system Right. BS infected leaf.
The proposed HS acquisition system is shown in Figure 1. Images are captured using a high-resolution 12-bit
monochrome CCD camera (labeled as B) with high infrared sensitivity (1500M-GE Thorlabs) attached to an
spectrograph (Specim Inspector V10, labeled as C) with a spectral range between 400 to 1000 nm. and nominal
spectral resolution of 4.55 nm. These elements are mounted on a moving slider with a computer controlled
motorized stage (WS70 Excitron), labeled as A in the figure. The run length of the slider is 25 cm. with a step
resolution of 0.5 mm. As banana leaves can obstruct each other, it was necessary to hold the sample leaf apart
while scanning. A leaf holder with a grid pattern was built for that purpose, labeled as D in the figure. The
illumination setup uses two 50 W. halogen lamps, labeled as E in the figure, located below the leaf to avoid
shadows of the grid pattern. Using lamps with higher power caused permanent damage to the leaf. Additional
visible and thermal cameras, labeled as F and G respectively, were also included in the system to be used for
annotated ground truth data generation and thermal monitoring respectively.
A given leaf must be imaged during 3 months which is the period needed for the BS infection to reach the
last stage. Hence, we will require to match the same leaf’s patch at different times. This implies a careful
Proc. of SPIE Vol. 9864 98640M-2
alignment of the leaf before each scanning, a difficult problem considering that a leaf has few image features. We
employed a cross-hair laser fixed to structure to align the leaf vein and the leaf holder. For spectral calibration,
we derived the spectral curve from known peaks of the emission spectrum of argon (Ar) and mercury (Hg) lamps.
A common issue in this kind of imaging system is noise, in particular at wavelengths near the ultraviolet and
infrared regions of the spectrum. To estimate image noise levels we measured the standard deviation σ of dark
reference images captured at different exposure times. Results are summarized in table 1. Based on these results
the camera’s exposure time was set to 200 ms. as it provides an adequate trade-off between image contrast and
noise. Finally, the f-number was set to f/7.
Exposure time (ms)
100
200
300
400
σ
3.50
3.58
3.74
3.93
Mean FPS
10
5
3
2
Scanning time (s)
32
65
98
128
Slider speed (mm/s)
5.6
2.8
1.8
1.3
Table 1. Spatial calibration parameters
For spatial calibration, the scanning area and the working distance were estimated from the optical parameters
of the spectrograph and lens. An important property of the resulting HS image is its spatial resolution. The
image’s pixel should correspond to the light reflected from the smallest possible square section of a given leaf.
In the horizontal direction, the spatial resolution depends on the CCD sensor resolution (1392 pixels) and in the
vertical direction it depends on the number of frames to be acquired and the speed of the slider. Ideally, the
number of frames should be selected to have the same the spatial resolution in both directions.
In practice, we found that mechanical and optical hardware constraints prevented to image squared leaf
regions. However, we also noted that the CCD sensor binning could be used to reduce the differences in pixel
spatial vertical and horizontal resolutions. By changing the binning factor to 4 in the horizontal direction, we
managed to obtain HS images in which each pixel corresponds to an square section of 0.5 mm. For our leaf
holder, which has a 16 x 16 cm of effective scan area, 320 frames must be captured at 200 ms exposure time.
To obtain this number of frames the slider speed must be set accordingly. Table 1, shows the mean fps rate
and slider speed values computed for a fixed run-length (16 cm). Additionally, camera binning in the vertical
direction was set to 2 to coincide with the spectral resolution of the spectrograph. The camera controller library
allows to configure the exposure time within the [0,1000] seconds range and the pixel binning [1,24]. The slider
speed and run length can be configured using the slider controller software library.
Imaging software latencies can cause frame lost, this is an issue as the system should capture the same number
of frames for each scan session. The main software module of our system synchronizes the slider movement and
the frame acquisition. Each of these routines runs as a high-priority task in a different CPU. If enough RAM is
available the complete frame sequence can be stored in memory before disk writing but this is not always the case.
Several frame buffering and writing strategies were tested. Having a circular buffer with two concurrent threads
(one for reading and one for disk writing) outperformed alternative approaches such as sequential read-write
operations, see table (Table 2). Finallly, since not all the cameras have support for the same operating system,
socket communication was used for synchronization of the thermal and RGB cameras which run on Linux OS .
Approach
Single-thread
Multi-thread
Min (ms)
16.7
16.3
Max (ms)
300.1
18.4
average (ms)
159.3
17.4
variance (ms2 )
7.3 × 10−3
1.3 × 10−4
Table 2. Frame acquisition and storage statistics
4. RESULTS AND DISCUSSION
Regarding the calibration procedure, figure 2 shows images captured using different exposure times. As noted
before, the higher decrease in sensor noise occurs at 200 ms. Moreover, higher exposure times tend to saturate
the CCD sensor and increase the scanning time. To use higher exposure times a simple solution is to close the
iris further but doing so greatly reduce contrast in images corresponding to the NIR region. As expected, motion
Proc. of SPIE Vol. 9864 98640M-3
blur degradations appear more notably in the scanning (vertical) direction. In our calibration experiments, we
found that motion blur can be reduced by enabling pixel binning in such direction. Also, pixel binning has the
effect of reducing background noise in our test subject (grid pattern). Figure 3 depicts two test images with
and without binning. In the former, the shadows along the grid lines mostly disappear. A disadvantage of this
approach is the reduction of the leaf scanned area proportionally to the binning factor. To further reduce noise
a low pass Gaussian filter was used. The filter’s parameters were estimated from a dark reference located at the
bottom of the leaf holder. The filtered image can be observed in figure 3 as well.
Figure 2. Sample banana leaf images at different exposure times. From left to right: 100,200,300,400 ms.
Since the slider uses a belt mechanism, it requires some time until reaching a near constant speed of 2.8
mm/s, the average time was estimated in 3.57 s, that corresponded to a distance of 10 mm. After this time has
passed, the image acquisition routine starts. While a belt is not the most precise mechanical system to use, a
variation of +/- 1.5 frames was measured in our tests. This yields a variation of the same number of lines in the
resulting HS image cube. The image resolution obtained by the system was within the limits biology researchers
set for visual analysis. Once calibrated, the proposed system can generate per leaf an HS cube consisting of 520
images of 348x320 pixels with a spatial resolution of 0.5 mm.
Figure 3. On the left, effect of binning on grid lines. Scanned areas of 13x13 mm and 54x54 mm respectively. On the
right, denoising results at 435 nm.
Shape, size and color of the leaves’ lesions define each stage. Each stage gives an idea of the relative progression
of the necrosis process around the infection point. The proposed scanning system was tested on a small set of
leaves that showed each of the 6 BS infection stages, see figure 4. As expected, at early BS stages (3 weeks
after inoculation) there is a decrease of chlorophyll production which expresses as round spots. One of the most
challenging problems will be to discriminate BS related changes against those caused by other stress sources
which can depict similar behaviors. As the BS progresses infected leaf spots become elongated and darker until
most of the leaf is necrotic.
5. CONCLUSIONS
In this paper, we have discussed our work in banana leaf HS image acquisition system. Despite some hardware
constrains our system has proven useful to extract HS image cubes and allows the visualization of BS related
lesions at several wavelengths in the VIS and NIR spectrum in an in-vivo setting. The inclusion of thermal
information is the next step in the construction of our system before it becomes fully operational. Also, further
improvements in image quality can be achieved by using advanced denoising technique that consider the local
Proc. of SPIE Vol. 9864 98640M-4
HI-A[11h'
2
`-;TAIT
TAr I
17A.1;17. (.
'0111111
:i111111111
Figure 4. ROI of images of banana leaves: one healthy and the others into the six stages of BS at different wavelengths.
image content. Finally, a new annotation software is under development to generate training data for automatic
classification of BS lesions.
6. ACKNOWLEDGEMENTS
This project was supported by VLIR-UOS, the center for Computer Vision and Robotics and the center for
Biotechnology at ESPOL
REFERENCES
1. J. Orluchukwu and M. Ogburia, “Introgression of bsr Gene from Wild 2n Accessions and Derivative Hybrids
to Cultivated 3n Landraces of Plantains (Musa Sp.),” International Journal of Plant Research 4(1), pp. 1–4,
2014.
2. L. Yuan, Y. Huang, R. W. Loraamm, C. Nie, J. Wang, and J. Zhang, “Spectral analysis of winter wheat
leaves for detection and differentiation of diseases and insects,” Field Crops Research 156, pp. 199–207, feb
2014.
3. M. Mirik, R. J. Ansley, K. Steddom, D. Jones, C. Rush, G. Michels, and N. Elliott, “Remote Distinction of
A Noxious Weed (Musk Thistle: CarduusNutans) Using Airborne Hyperspectral Imagery and the Support
Vector Machine Classifier,” Remote Sensing 5, pp. 612–630, jan 2013.
4. S. Candiago, F. Remondino, M. De Giglio, M. Dubbini, and M. Gattelli, “Evaluating Multispectral Images and Vegetation Indices for Precision Farming Applications from UAV Images,” Remote Sensing 7(4),
pp. 4026–4047, 2015.
5. F. Martinelli, R. Scalenghe, S. Davino, S. Panno, G. Scuderi, P. Ruisi, P. Villa, D. Stroppiana, M. Boschetti,
L. R. Goulart, C. E. Davis, and A. M. Dandekar, “Advanced methods of plant disease detection. A review,”
Agronomy for Sustainable Development 35, pp. 1–25, sep 2014.
6. J. S. West, C. Bravo, R. Oberti, D. Lemaire, D. Moshou, and H. A. McCartney, “The Potential of Optical
Canopy Measurement for Targeted Control of Field Crop Diseases,” Annual Review of Phytopathology 41(1),
pp. 593–614, 2003. PMID: 12730386.
7. M. Wahabzada, A.-K. Mahlein, C. Bauckhage, U. Steiner, E.-C. Oerke, and K. Kersting, “Metro Maps of
Plant Disease Dynamics—Automated Mining of Differences Using Hyperspectral Images,” PLOS ONE 10,
p. e0116902, jan 2015.
Proc. of SPIE Vol. 9864 98640M-5
8. A.-K. Mahlein, Detection, identification, and quantification of fungal diseases of sugar beet leaves using
imaging and non-imaging hyperspectral techniques. PhD thesis, Universitäts-und Landesbibliothek Bonn,
2011.
9. D. Ashourloo, M. Mobasheri, and A. Huete, “Evaluating the Effect of Different Wheat Rust Disease Symptoms on Vegetation Indices Using Hyperspectral Measurements,” Remote Sensing 6, pp. 5107–5123, jun
2014.
10. J. LU, D. CUI, and H. JIANG, “Discrimination of Tomato Yellow Leaf Curl Disease using Hyperspectral
Imaging,” in 2013 Kansas City Missouri, July 21 - July 24, 2013, American Society of Agricultural and
Biological Engineers (ASABE), 2013.
Proc. of SPIE Vol. 9864 98640M-6