CMRT’09
ISPRS Workshop
O. Barinova, R. Shapovalov, S. Sudakov,
A. Velizhev, A. Konushin
EFFICIENT ROAD MAPPING VIA
INTERACTIVE IMAGE
SEGMENTATION
Presenter: Alexander Velizhev
Introduction
• Roadway monitoring systems are
widely-used for supervising road
pavement surface and repair
planning
Problem statement
• Analysis road pavement only by
video sequences
Problem statement (2)
• Object types:
– Lane marking
– Road patches and defects
• Solution requirements:
– High object detection rate
– Maximum automation
Problem statement (3)
Source image
Expected result
Problem details
• Some real examples
Our algorithm outline
1. Video rectification
2. Image preprocessing
3. Image segmentation
Automatic
offline
stage
4. Features calculation
5. Interactive classification
Interactive
online
stage
Video rectification
Video
rectification
Image
preprocessing
Image
segmentation
Features
calculation
Interactive
classification
• Using of raw video has severe
drawbacks:
– Objects are represented with different
spatial resolution on the same frame
– Projective distortions
– Elongated objects exceed the bounds of
single frame
Video rectification (2)
Video
rectification
Image
preprocessing
Image
segmentation
Features
calculation
Interactive
classification
• Video frames are converted to
orthogonal projection and stitched to
each other
Image preprocessing
Video
rectification
Image
preprocessing
Image
segmentation
Features
calculation
Interactive
classification
Source
image
Retinex
Contrast
transform adjustment
Bilateral
filter
Image segmentation
Video
rectification
Image
preprocessing
Image
segmentation
Features
calculation
Interactive
classification
• Main goal is representing all objects
of interest as different segments
• We use the hierarchical version of
mean shift algorithm
Features calculation
Video
rectification
Image
preprocessing
Image
segmentation
Features
calculation
Interactive
classification
• More than 100 various features are
used for classification of segments
• Feature types:
– Colour statistics (colour variance, Lab
components’ percentiles, ... )
– Shape statistics (elongation, orientation,
area, …)
– Difference with neighborhood of the
segments
Interactive classification
Video
rectification
Image
preprocessing
Image
segmentation
Features
calculation
Interactive
classification
Interactive classification (2)
Video
rectification
Start
Image
preprocessing
User manually marks
object segments
Learning of cascade of
classifiers
Image
segmentation
Automatic classification of
the next road part
Features
calculation
Interactive
classification
End
User corrects
classification results
Cascade of classifiers
• Cascade of classifiers corresponds
image segmentation levels
• We descend a hierarchy from large to
small segments and reject segments
that do not contain pixels of objects
of interest
• Classifier training uses the data
passed to a corresponding cascade
layer by preceding version of cascade
Why do we use the cascade?
• To solve a problem of unbalanced
classes
• To speed-up classification
Online learning
• We introduce an online version of the
random forest algorithm
• Special class costing
• The algorithm’s code is a part of our
open source “GML Balanced On-line
Learning Toolkit ”
– http://research.graphicon.ru/machine-learning/gmlbalanced-on-line-learning-toolkit-2.html
Why do we use online learning?
• We don’t need to store all training
database in memory
• Short learning time
• User actions immediately impact on
the classification results
How to measure system
efficiency?
• We are modeling “ideal” user actions
to measure the efficiency of the
interactive classification
• Efficiency criterion:
– a minimal number of mouse’s clicks for
making correct classification
Results
Source image
Segmented
image
Analysis
result
Results (2)
Interactive classification
Manual classification
Clicks
Image part
Results (3)
Error,
%
Image part
Summary
• We present a tool for efficient
interactive mapping of road defects
and lane marking
• Intensive use of computer vision
methods on different stages of our
data processing workflow increases
usability of the tool
Weak points
• Image segmentation errors can
degrade classifier and true object
bounds cannot be extracted
• Algorithm is not robust to user
mistakes
Future work
• Ultimate goal:
Development of the universal
semantic segmentation system which
can be used for object extraction
from large class of images
• Nearest plan:
Improving the quality of image
segmentation by integration colour
and range data
CMRT’09
ISPRS Workshop
Efficient road mapping via
interactive image segmentation
R. Shapovalov
[email protected]
A. Konushin
[email protected]
O. Barinova
[email protected]
S. Sudakov
[email protected]
A. Velizhev
[email protected]