Grape Detection in Vineyards
Final project by
Kobi Ruham, 031588007
[email protected]
Eli Izhak, 062840335
[email protected]
Ideas and goals
Vineyards are complex natural environments where accurate detection of fruit can
facilitate numerous operations such as targeted spraying and selective picking.
Suppose a robot moves along a vineyard aisle and while doing so it takes images of
the vines with the camera positioned roughly perpendicularly to the vine (Fig. 1).
Figure 1 - Camera position in a vineyard aisle.
These images might show any of the ground, the vine, and it's supporting structure,
the foliage, and grape clusters, as is shown in the typical image in Fig. 2.
Figure 2 - A typical image seen from the aisle.
The goal of the project is to develop a system that detects grape clusters in vineyard
images taken under the conditions specified above (i.e., from within the aisle and
toward the vine). By detection it is meant that the program should be able to properly
label every pixel that belongs to a grape cluster while labeling every other pixel as
non-grape.
Course of action
We try to find what make the grape different in the vineyards typical image.
The following is all the different we manage to find:
 The grape has shape which is the most similar to cycle but not symmetric, which
means that for one grape we can find more then one cycle.
 The radius of the cycle is limited – this assumption is true only if the distance
between the camera and the grape is limited (have minimum and maximum).
The distance can be calculate and insert to the program as an input.
 Grape in the image is most likely to be found together with other grapes, which
means that when a cycle in the image was found we what at least few more
cycle close to it to be found too.
 The color of the grapes in every image is different due to the light and shadow
and the type of the grape can be different (green or black). But in the typical
images (which was given) it can be found that the color of the grapes is not the
dominant color in the image and so if we remove the dominant color from the
image (make it black) few grapes will be remove but the image will be easier to
work with.
For each of the following characteristic we test several methods in order to get to the
best results:
 In order to filter parts from the given image we examined the following
methods:
o Kmean – we use the kmean algorithm for segmentation according to the
RGB color (We test different number of segments) and then remove from
the image the largest segment. This method chooses the first pixels
randomly and so it can take long time for segmentation to be found.
o We test the RGB color of the grape in different image and get the grapes
RBG range, then we use this range and remove form any image all the
pixel which are not in this range. This method also removes grapes which
their color is not in the range (usually very bright or dark grapes).
o Gray color histogram – we first change the given image to gray color then
we calculate the color histogram and then remove from the image the most
popular colors.
 We use Hough transform for cycle detection in order to find all the cycles in the
image. Different kind of threshold (minimum number of votes for each cycle)
was declare in order to find the best result:
o Filter all the edges where the gradient size is very big due to the color filter
(edge between black and other color) – this method found as not useful
most of the time.
o The threshold was examined on the number of votes for each cycle
(different center and radius).
o Sum all the votes for nearby cycles and radius and examined threshold on
it. The idea is that the calculation of Hough transform yields double
numbers which are cast to integers (location in the image are integers)
therefore pixels on the same cycles can vote for different but nearby
cycles.
Implementation
Between all the methods which was described in the last section we will detailed in
this section the implementation which yields the best results for most of the images.
The following are the steps for getting the goal – all this steps are implemented in the
main function - DetectGrapes(I):
Step 1 - Pixels filtering
First we run the kmean algorithm (using the kmean MATLAB function) on the RGB
image and then we paint the biggest segment we got from the kmean with black. The
last action was to paint with black all the pixels which their R' G or B is above 200 or
below 60 (We try different combination of the methods which detailed above but this
one return the best results). Figure 3 is the result from step 1 for the image in figure 2.
Figure 3 –the image in figure 2 after step 1
Step 2 - Gray image
Change the color scale of the image from RGB to gray by using MATLAB function
rgb2gray. The next steps are easy to implements on gray image. Figure 4 is the result
after step 1 and 2 for the image in figure 2.
Step 3 – Edge detection
We use gradient calculation in order to find all the edges in the image. The reason for
using gradient is because the next step of Hough transform for cycle detection needs
the orientation of the edge which is given by the gradient.
Figure 4 –the image in figure 2 after step 1 and 2
Step 4 - Hough transform for cycle detection
In this step we implement the Hough transform for cycle detection: every edge pixel
in the image votes for all the cycles which tangent to it from both sides (according to
the orientation which is given by gradient). The radius of the tangent cycle is bound
between 2 and 15.
The votes array was implements as 3-dimension array where every pixel in the image
(x,y) has array of counters – votes counter for each radius.
Step 5 - Cycles filtering
The last step found many cycles in the given image, part of them are grape but most
of them are not. We implement several methods in order to filter the all the cycles
which are not represents grapes:

Sum all the votes for cycles with difference of one in the radius (i.e. sum all
the votes of cycles in location (x,y) and radius 3,4,5). We first try to sum all
the votes for cycles with different of one in the location and radius (i.e. sum all
the votes of cycles (x-1,y-1,r-1), (x,y-1,r-1),……(x+1,y+1,r+1)) but we got
poor results.

Filter all the cycles which the number of votes (sum of votes) for them is
below 10 (this threshold was defined after several testing).

Keep only the cycle which have at least 35 other cycles around them, the black
color in figure 5 is the area where other cycle is looking for, the red spot is the
tested cycle center.
Step 6 - Cycles Painting
Go over all the cycles which were not filtered and draw then on new black image.
This will be the image which returns from the function.
7 pixels
7 pixels
4 pixels
4 pixels
Figure 5 – area for searching nearby cycles.
Results
The following is few typical images and the result getting from the grape detection
implementation as detailed above, for each of the images the result is given on the
original image and in new image where the white pixel is what detect from the
DetectGrapes function as a grape and black pixel is not a grape:
Figure 6 – input image (up), result image (down)
Figure 7 – input image (up), result image (down)
Figure 8 – input image (up), result image (down)
Figure 9 – input image (up), image after step 1 (center), result image (down)
Figure 10 – input image (up), image after step 1 (center), result image (down)
Conclusions
The main method which was used to detect the grape was the Hough transform for
cycle detection but the typical image is very detailed and so the Hough transform
itself doesn’t give any good result.
In order to use Hough transform so it will return satisfied result parts from the image
must be remove. But the decision which part of the image to remove is very difficult
to implement. In the suggest implementation two method where used in order to
decide which part from the image need to be remove (the method detailed in the
implementation section). We also try many combination of different method but any
combination return good result for only part of the typical images. In the result section
above it can be notice that the result for the images in figure 6, 7 and 8 are very good
but still not perfect, On the other hand the result for the images in figure 9 and 10 are
not satisfied at all, the reason for that is the color filter (step 1) which eliminate from
the image all the grape because their color is very dark or bright (out of the scale
which was determine by the method for keeping).
In order to get better result other method for filtering need to be develop or few
parameter need to be entered before start detect the grape, like what is the most
common color of the grape, what is the distance of the camera from the objects and
other parameters.
In case no other details are given we find it very difficult to find grapes in all the
typical images.