Underwater Optical Mapping
Rafael Garcia and Nuno Gracias
Computer Vision and Robotics Group
University of Girona, 17071 Girona, Spain
Tel. +34 972 419 812
[email protected], [email protected]
Abstract ─ This work presents ongoing work at the University of Girona towards development and application
of vision-based seafloor survey methodologies, including
large area 2D mosaicing (>1sqkm), monocular-based
3D mosaicing, and stereo seafloor modeling. The
developed tools set a first step towards using
Autonomous Underwater Vehicles for detecting and
documenting the temporal variations associated with the
active processes operating at these sites.
Keywords ─ mosaicing, mapping, underwater optical
imaging, 3D reconstruction.
1. INTRODUCTION
Robot mapping has greatly advanced in the last
few years as a tool for environmental monitoring
and seafloor characterization. Seafloor imagery is
routinely acquired during near-bottom mapping
surveys conducted with both remotely operated
vehicles (ROVs) and autonomous underwater
vehicles (AUVs). Deep-sea hydrothermal fields or
shallow-water coral reef communities are, for
instance, two scenarios that have long been the
target of such studies. Imagery is useful to
characterize the nature and distribution of
geological features and biological communities,
extract ecological indicators, and to provide a
permanent visual record of the seafloor condition.
However, imaging studies often yield large
numbers of images (several tens of thousands,
especially in deep-sea cruises) that are frequently
underutilized largely because of the difficulties
inherent in processing and visualizing large data
sets [1]. Moreover, light suffers from a rapid and
nonlinear attenuation underwater that affects the
acquired images.
In this paper we describe ongoing work at the
University of Girona towards development and
application of vision-based seafloor survey
methodologies, including large area 2D mosaicing
(>1sqkm), monocular-based 3D mosaicing, and
stereo seafloor modeling. The developed tools set
a first step towards detecting and documenting the
temporal variations associated with the active
processes operating at these sites.
2. OPTICAL MAPPING
A large number of underwater studies rely on
visual inspection, frequently using optical maps
when the area to be explored is larger than what
can be covered by a single image. Optical mapping
can be divided into two large groups: (a) twodimensional mosaicing and (b) three-dimensional
mapping. Two-dimensional image mosaicing deals
with the process of combining the information
from multiple images of the same area, to create a
single representation with extended field of view
[2,3] In order to construct ocean floor photomosaics, the individual images forming the mosaic
are usually obtained by setting a camera on an
underwater vehicle [4]. The camera looks down
and the acquired images cover a small area of the
ocean floor. The automatic detection of a number
of points in one image, and their correspondences
in another, allows the estimation of the motion
between the images [5]. Using the motion
estimations, the images are aligned and merged,
forming composite view of the surveyed area. This
approach needs to face additional challenges such
as non-uniform illumination, light scattering and
moving objects. All these effects cause strong
visual artifacts when several images are stitched
together to form a photo-mosaic. For this reason,
once we have a globally aligned mosaic, image
blending techniques are required to obtain a
seamless
mosaic.
Conventional
blending
techniques used in terrestrial imagery are not
always adequate in the underwater context due to
light attenuation, suspended particles (producing
light scattering), strong parallax and frequent
moving elements, which are typical in underwater
imagery.
Fig. 1 illustrates the result of blending a composite
mosaic into a seamless high-resolution image to
provide a meaningful representation of the
seafloor.
Although very valuable, there are cases in which
the information provided by 2D mosaics is not
sufficient for comprehensive understanding of the
region of interest. Studies on the evolution of coral
reefs in biology and ecology, hydrothermal vents
100m
1m
3m
Fig. 1. Sample 2D photo-mosaic of the MOMAR08 cruise in the mid-Atlantic Ridge. The mosaic represents 3 full days of survey
with a Remotely Operated Vehicle and covers an area of nearly one square kilometer. After global alignment of all the
images, the mosaic has been blended to obtain a high-resolution seamless picture of the seafloor.
and lava formations in geology or underwater
archeological sites, among others, heavily benefit
from three-dimensional (3D) maps.
Side-scanning sonars and acoustic cameras offer a
solution to 3D seafloor imaging. Although the
precision of sonars has improved significantly [6],
sonars do not provide photometric information
required to obtain textured 3D models. The
combined use of sonars and cameras [7,8], to
provide optical texture has so far been very
limited, mainly due to the differences between the
effective ranges of the two sensors. Side-scanning
sonars are employed at altitudes exceeding 10 m,
which generally surpasses the range of cameras
due to light scattering and absorption.
A more flexible approach is to use optical data to
obtain both photometric and geometric information
of the seafloor. This can be achieved by estimating
the scene geometry using disparity information
either between multiple calibrated cameras, known
as stereovision [9], or by a single moving camera
[10]. When a single camera is used, the 3D
reconstruction problem becomes more complex as
it requires determining the 3D camera motion for
each frame [11]. This problem, referred to as
structure from motion (SFM), has largely been
studied by the computer vision community [12].
Oriented toward underwater imaging, [13]
proposed a SLAM approach using navigation
priors. This approach targets applications that
require maintaining the vehicle position and
velocity and their associated uncertainty. More
oriented toward 3D mapping, [14] proposed a SFM
framework that deals with large sequences by
independently processing local submaps and
registering them using global alignment
techniques. While accurate, this approach has
somewhat limited applications as it uses navigation
priors for submap generation. More recently, our
group [15] proposed a SFM-based large scale 3D
mapping approach that increases the flexibility of
the data acquisition process and the mapping
accuracy using camera pose registration
techniques. Figure 2 illustrates the result of our
approach to create accurate three-dimensional
textured models of the seafloor using monocular
video sequences. The method takes into account
the geometry of the scene through a 3D vertex
selection mechanism which results in a reduction
in the complexity of the final 3D model, with
minimal loss of precision [15]. The technique does
not require any navigation priors, and data can be
acquired by submersible robots or divers using
either video or still cameras, under various lighting
conditions. Moreover, as the estimates of the
camera position do not depend directly on the
previous estimations, it is more robust to
misregistration problems and scene occlusions
than most state-of-the-art approaches.
Although impressive technical progress is being
achieved in underwater mapping technology, it has
so far mainly concentrated in the use of a few
sensor modalities (for example optical cameras and
position data) or in increasing the autonomy of
autonomous vehicles. There is a clear comparative
lack of effort on the joint processing and
integration of subsea data, focused on end-user
scientist needs. This project is directed towards
this goal.
Despite of the extensive work in 2D and 3D
optical mapping of the ocean floor, we still face a
series of open issues for the efficient deployment
of underwater imaging systems, mainly due to the
intrinsic challenges of the medium [16]: light
attenuation, backward and forward scattering and
need for artificial illumination. These effects
degrade the quality of the acquired images,
resulting in image blur, very low contrast, reduced
visibility range, and loss of color content. Thus,
underwater optical imaging remains a challenging
problem.
3. CONCLUSIONS
We have presented a powerful set of processing
tools to obtain a highly detailed maps of
underwater interest areas, surveyed using either a
stereo system or a monocular camera as the main
sensor. The proposed methodologies do not
assume any kind of a priori geometry on the
reconstructed structure, so they are able to
accurately reconstruct complex shapes over
relatively large areas. Within this context, our
approach is well-suited for conducting temporal
studies through repeated surveys of the same area,
enabling the detection of changes in these
environments.
The developments proposed in this paper will
ultimately provide techniques and tools to a broad
scientific community (biologists, ecologists,
geologists and geophysicists), and that will allow
advancements that are possible with existing data
and processing tools. Furthermore, these tools will
open the door to numerous non-scientific applications ranging from industry (offshore operations,
Figure 2. Estimation of the 3D relief of the seafloor using a single calibrated camera. The model has
approximately 240,000 vertices and covers an area of 12 × 19 m.
drilling, mining) to the public domain (environmental impact and remediation, infrastructure
construction, public works, risk assessment and
education.), and will close the gap that exists in
our knowledge of the seafloor with respect to the
subaerial Earth’s surface.
REFERENCES
[1]
[2]
[3]
[4]
[5]
[6]
[7]
J. Escartin, R. Garcia, O. Delaunoy, N. Gracias, A. Elibol, X.
Cufí, L. Neumann, D.J.Fornari, S.E. Humphris, J. Renard,
2008, Globally aligned photomosaic of the Lucky Strike
hydrothermal vent field (Mid-Atlantic Ridge, 37º18.5'N):
Release of georeferenced data, mosaic construction, and
viewing software, Geochemistry, Geophysics and
Geosystems, vol. 9, no. 12, pp. 1-17
Irani, M., P. Anandan, S. Hsu, 1995. Mosaic based
representations of video sequences and their applications. In
Proc. of the 5th IEEE International Conference on Computer
Vision, pp. 605-611, Cambridge, Massachusetts.
Garcia, R., J. Batlle, X. Cufí, J. Amat, 2001. Positioning
an Underwater Vehicle through Image Mosaicking. IEEE
Int. Conf. on Robotics and Automation, vol. 3, pp. 2779–
2784, Seoul, Rep. of Korea.
Eustice, R., H. Singh, J. Leonard, M. Walter, R. Ballard,
2005. Visually Navigating the RMS Titanic with SLAM
Information Filters, In Proceedings of Robotics: Science
and Systems (RSS) pp. 57 – 64, Cambridge, MA, USA.
Negahdaripour, S., C. Barufaldi, A. Khamene, 2006.
Integrated system for robust 6 DOF positioning utilizing new
closed-form visual motion estimation methods over planar
terrains. IEEE J. Oceanic Engineering
Ruiz, I. T., de Raucourt, S., Petillot, Y., 2004. Concurrent
Mapping and Localization Using Sidescan Sonar, IEEE J.
of Oceanic Engineering, vol. 29, no. 2, pp. 442-456.
I. Mahon and S. Williams, 2004. "Slam using natural
features in an underwater environment,” in IEEE Control,
Automation, Robotics and Vision Conference, vol. 3, pp
2076–2081.
[8]
[9]
[10]
[11]
[12]
[13]
[14]
[15]
[16]
S. Negahdaripour, H. Sekkati, H. Pirsiavash, 2009. “Optiacoustic stereo imaging: On system calibration and 3-D
target reconstruction," IEEE Trans. Image Processing,
IEEE Trans. Image Processing, Vol 18(6) pp. 1203-1214.
Negahdaripour, S., H. Madjidi, 2003. Stereovision imaging
on submersible platforms for 3-d mapping of benthic habitats
and sea floor structures. IEEE J. Ocean. Eng, 28, pp. 625650.
T. Nicosevici and R. Garcia, 2008. "On-line robust 3D
Mapping using structure from motion cues," MTS/IEEE
Techno-Ocean Conference (Oceans'08) Kobe.
Y. Ma, S. Soatto, J. Kosecka, S. Shankar Sastry, 2003. An
Invitation to 3-D Vision: From Images to Geometric
Models. SpringerVerlag, vol. 26.
A. W. Fitzgibbon, A. Zisserman, 1998. Automatic Camera
Recovery for Closed or Open Image Sequences. European
Conference on Computer Vision, pp. 311-326.
R. M. Eustice, H. Singh, J. J. Leonard, 2006. Exactly
Sparse Delayed-State Filters for View-Based SLAM,
IEEE Transactions on Robotics, vol. 22, no. 6, pp. 11001114.
O. Pizarro and R. Eustice and H. Singh, 2004. Large area 3D
reconstructions from underwater surveys. MTS/IEEE
OCEANS, vol. 2, pp. 678-687.
T. Nicosevici, N. Gracias, S. Negahdaripour, R. Garcia,
2009. "Efficient three-dimensional Scene Modeling and
Mosaicing," Journal of Field Robotics, vol. 26, no. 10, pp.
759-788.
W. S. Pegau, D. Gray, J. R. V. Zaneveld, 1997. Absorption
and attenuation of visible and near-infrared light in water:
dependence on temperature and salinity. no.36, pp.60356046.
Acknowledgement: This work has been funded in part
by MICINN under grant CTM2010-15216/MAR, and in
part by the EU under grants FP7-ICT-2009-248497 and
FP7- IEF-2009-253322. N. Gracias has been funded by
the Ramon y Cajal programme.