‫ גישה‬:‫חיזוי אחוז האיסוף של רובוט חקלאי‬
‫הסתברותית‬
Predicting the Harvest Percentage
of Robotic Melon Harvesters: A
Probabilistic Approach
M. P. Mann1, D. Rubinstein1, I. Shmulevich1 and B. Zion2
Faculty of Civil and Environmental Engineering
(1)
(2)
ARO
Volcani Center
State of Israel / Ministry of Agriculture and Rural Development
INTRODUCTION
• Harvesting robots are developed to replace
manual labor by automation
• Melons are chosen as fruit for prototype robot
under development:
– The locations are known
beforehand
– Two dimensional
– Few obstacles
– Only picking-up required
Maximum robotic harvest problem
• The robot advancing along row of melons must select the
optimal sequence of melons in order to pick up as many
melons as possible - combinatorial optimization problem
• Orienteering - a game where competitors visit checkpoints
and must collect as many points as possible before time is up
• Problem is NP hard; no exact polynomial algorithm is known
• Time windows – each node (melon) may be visited within a
given time frame
vt
Objective
• Analytically predict the maximum ratio of melons
that can be successfully harvested given the
robot parameters, actuator capabilities,
dimensions and melon distribution
Motivation
• Empirical - the ratio of melons harvested by the
robot in each simulation are nearly equal for
given parameters
• Time saving - simulations are time consuming
and useful only for the parameters chosen
• Design tool – formula for the harvest ratio can be
used to determine which robot configuration and
velocity yields maximum harvest
• Scientific – discovering the principles that explain
the data is what science is all about!!
Motivation (cont’d.)
Harvest ratio of robotic melon harvester
120
2 melons/m^2
Harvest ratio (%)
100
3 melons/m^2
80
60
40
20
0
0
0.2
0.4
0.6
Platform Velocity (m/s)
0.8
1
Simplified model
• Model the robotic harvesting with uniform
picking cycle time and manipulator
displacement
• Model the workspace with one conveyor –
“fold” the field in half
Original workspace
Begin folding workspace in half…
Equivalent workspace
“All models are
wrong, but some
are useful”
(George E.P. Box
1919-2013)
Harvest ratio of simplified model
• Average invariance - manipulator does not move
sideways relative to platform, only average updown displacement yavg
• One melon picked per cycle of time 2T(yavg)
• 2T(yavg)*ρvtW melons passed over per cycle
1
average harvest ratio =
2T (yavg )vtW
ρ – density
T(x) – time of
traversal for
distance x
vt – platform
speed
W – width of
platform
Average y -Displacement
Actual and estimated harvest
percentage for field of density 2
melons/m^2 and platform velocity 20
cm/s
Harvest Percentage (%)
• Only one unknown
parameter – yavg – needed
to determine robot harvest
ratio
• The longer the field, the
more accurate the equation
• How do we determine yavg
for randomly distributed
melons?
• Need for some background
in probability…
100
90
80
70
60
50
40
30
20
10
0
Actual Harvest Percentage
Estimated Harvest Percentage
0
10
20
30
Field length (m)
40
Uniform Distribution
• What are the chances that
any given melon will be in
sector i?
• Any one of m sectors is
equally likely with
probability of p = 1/m
n
number of melons

area of sector
A
area of field
A

number of sectors
 1

A m
probability of success
m
p
Binomial distribution
• What are the chances that
k melons are located in a
sector of area ∆?
n k
nk
f (k ; n, p)    p 1  p 
k 
k
n  

 
1 

A
k  A 
nk
• As n→∞, the binomial
distribution becomes the…
Area
Δ
Poisson distribution
• What are the chances that k
melons are located in a
sector of area ∆ - in an
infinite length field with
average density ρ?
e  ( )k
f (k , ) 
k!
• ρΔ is the average number of
melons in an area Δ
• k = 0,1,2,…∞
• Chances that at least one
melon is located in ∆ = 1-e-ρΔ
Time rings
• For discrete picking
time, platform speed is
irrelevant – picking
occurs at “snapshot”
• Cartesian manipulator
– time of traversal
T(x,y) = max(T(x),T(y))
• LT – time rings
infinitesimally thick
containing all points of
equal traversal time T
Geometric Distribution
• What is the likelihood that we
need k trials before getting
one success if each trial has
success rate p?
P(k)=p(1-p)k-1
• For robotic melon harvester,
how many LT rings of equal
area Δ must the manipulator
pass until it reaches a ring
containing at least one
melon?
• As Δ→0, p→ρΔ
Average nearest neighbor
• For maximum harvest, manipulator must reach nearest
ring containing a melon – geometric distribution of
nearest rings
• Problem equivalent to average nearest neighbor (ring)
LTavg
• Important discovery - LTavg encloses an area containing
on the average one melon:
LTavg




 LT :   dxdy  1


T ( x , y )T


LTavg
• For T(x)=T(y), y = 1/2√ρ, and we have an equation to
describe the expected harvest ratio:
harvest ratio =
1
2T (0.5 0.5 )vt L
Average Nearest Neighbor – empirical
support
Havest Percentage and Average
Y Heights
100
1
0.9
Calculated Y
Height
Harvest
Percentage
Average Y
Height
80
70
60
50
0.8
0.7
0.6
0.5
40
0.4
30
0.3
20
0.2
10
0.1
0
0
0
0.5
Platform Velocity (m/s)
1
Displacement (m)
90
Harvest percentage (%)
• Invariance - despite
declining harvest ratio
for higher platform
velocities, average y
displacement is
constant
• →yavg is independent of
platform velocity
beyond critical velocity
• Why does yavg increase
below critical velocity…
Intersecting picking zones
• High velocities - LTavg do not intersect - melon
distribution independent at each picking
• Low velocities – LTavg intersect – expected number of
melons within LTavg lowered because of previous picking
• Manipulator must reach further to cover one melon –
higher yavg
Random Process
• Each stage i of picking
defined by state {XPi,XQi}
describing number of
melons in Pi and Qi
• Number of melons left
in Pi depends on
whether the stage i-1
picked from Pi or not
• Probabilistic model –
likelihood of picking in
overlapping area
depends on ratio of
densities:
Ai - event that stage i-1 picks from Lti
Āi - event that stage i-1 does not pick from LTi
P(Ai) – probability of event Ai
P(Ai|B) – probability of event Ai given B is true
P(Āi) – probability of event Āi=1- P(Ai)


P(Ai | X Pi 1 , XQi 1 ) 


P(Ai | X Pi 1 , XQi 1 ) 
XQi1
X Pi 1  XQi 1
X Pi1
X Pi 1  XQi 1
Total probability
• Total probability – likelihood of stage i having
{XPi,XQi} melons depends on both possible events
P(XPi )  P(XPi )P(Ai )  P(XPi  1)P(Ai )
• Likelihood of Ai depends on all possible previous
P(A )   P  A | X , X P  X , X 
states:
i
XPi 1 , XQi 1
i
Qi 1
  
• XQi is independent of XPi, hence:
P(Ai ) 

XPi 1 , XQi 1
 
Pi 1
Pi 1
Qi 1
P Ai | X Pi1 , XQi1 P X Pi1 , XQi1



XPi1
XQi1
P  XPi , XQi   P  XQi   
P(XPi ) 
P(XPi  1) P X Pi1 , XQi1


XPi1  XQi1
XPi 1 , XQi 1  X Pi 1  XQi 1



Markov chain
• Probability of present state depends only on
previous state:
P  XPi , XQi    P  XPi , XQi |XP , XQ P XP , XQ 
i 1
XPi 1 , XQi 1
• Written in matrix form as

i 1
i 1
i 1

P  XPi , XQi   P XP , XQ  A 
i 1
i 1

 
 
• [A] is K*L × K*L transition matrix with entries


Akl  P XPi , XQi l | XPi1 , XQi1
hence


k

P  XPi , XQi   P XP , XQ  A 
0
0

 
 
i
Infinite length Markov chain
• Stationary distribution: as i→∞:
T
P  XPi , XQi 


(
A
)

 i 
where π(AT) is the left eigenvector of A corresponding
to eigenvalue of 1 regardless of the initial state {XP0,XQ0}.
• This yields expectation value of number of melons
within picking area
XP  XQ   k (P ,Q)  XP  XQ 
XP XQ
• yavg that produces an expectation value of 1 is the
correct average y displacement
Infinite length Markov chain verification
Average Y Heights of
Robotic Manipulator
Harvest ratio (%)
1
Average Y Height - 2
melons/m^2
Average Y Height - 3
melons/m^2
Estimated Y Height - 2
melons/m^2
Estimated Y Height - 3
melons/m^2
0.9
0.8
0.7
0.6
0.5
Harvest ratio of robotic melon
harvester
100
2 melons/m^2
80
3 melons/m^2
2 melons/m^2 - estimated
60
3 melons/m^2 - estimated
40
20
0.4
0
0.3
0
0.2
0.1
0
0
0.5
Critical velocity
1
0.2
0.4
0.6
0.8
Platform Velocity (m/s)
1
Example 1 – selecting the best
actuator
• Given a selection of
three robots, which
one has the highest
harvest percentage
at any speed?
y
x
– Robot 1: max x. acc. = 1 m/s^2, max y. acc. = 8 m/s^2
– Robot 2: max x. acc. = 2 m/s^2, max y. acc. = 5 m/s^2
– Robot 3: max x. acc. = 3 m/s^2, max y. acc. = 3 m/s^2
Example 1 – selecting the best
actuator - results
• At low speeds and
high speeds, the
robots are nearly
equally efficient
• At speed 30 cm/s,
Robot #2 is the most
efficient
• Conclusion: The most
efficient robot
depends on how fast
it advances
Example 2 – determining platform
velocity for variable density
• Row of melons 36 meters long with varied densities of
3, 2, and 1 melon per square meter
• What is the fastest speed robot can travel and still pick
80% of the melons
• Speeds computed inversely from Markov chain:
3 melons/m2
Density (melons/m2)
3
2
1
Speed (cm/s)
17
22
40
2 melons/m2
1 melons/m2
Results
Density (melons/m2)
3
2
1
Total
Number of Melons
54
36
18
110
Number Harvested
44
31
14
89
Harvest Percentage
81.5%
86.1%
77.8%
80.9%
Conclusions
• Robot harvest percentage nearly constant for field that
is long enough
• Using the Simplified Model, percentage determined
completely by average y displacement
• Average displacement equal to that of norm that
encloses an average of one melon
• Expected number of melons within norm is determined
by an infinite length Markov chain
• Probabilistic method can be used for hardware
selection and optimal platform velocity determination