MOVES Student Thesis Proposal SAMPLE 2009-07-09
MEMORANDUM
Date: 11-Dec-08
From: LT John Smith
Section(s) 399-073
To:
Program Officer, MOVES
Via:
(1) Thesis Advisor: Mathias Kölsch, Ph.D.
(2) Chair, MOVES Academic Committee: Mathias Kolsch
Subj: Thesis Report NUMBER 1
Encl: (1) MOVES Thesis Proposal
1. Tentative Title of Proposed Thesis: Statistical Models for Detecting Objects in Vertical Plane
Forward Looking Sonar
2. General Area of Proposed Thesis Research: Computer Vision; Software Engineering
3. Enclosure (1) is the Thesis Proposal with a milestone plan (dates/events) for research and
thesis completion.
4. I expect that my thesis will be unclassified. If classified, I have read Chapter V of
NAVPGSCOLINST 5510.2, and the NPS Research Admin web page
(http://www.nps.edu/research/research1.html) concerning Classified Thesis.
5. I reviewed the Institutional Review Board (IRB) web page concerning the use of humans in
research (http://www.nps.edu/research/IRB.htm). I am aware that if I use humans as subjects
I must forward an IRB application via my thesis advisor before any data collection can begin
as outlined in NPGSINST 3900.4.
6. I anticipate the following travel or other extraordinary requirements: OCEANS '08
MTS/IEEE in Quebec City, 15-18 SEP 2008; Multiple trips on NPS and Monterey Bay
Aquarium Research Institute (MBARI) research vessels; Use of NPS and possibly MBARI
Autonomous Underwater Vehicle(s) with forward looking sonar
7. Proposed Co-Advisor: Doug Horner, Ph.D.
8. Co-Advisor Signature: ________________________________________________________
Student Signature
Second Student Signature <if joint thesis>
1. Approved and forwarded:
Thesis Advisor
Date
Chair, MOVES Academic Committee
Date
Program Officer – MOVES
Date
3. Approved and forwarded:
4. Approved and retained:
DRAFT -- SAMPLE MOVES Thesis Proposal as of Jun 2009
MOVES THESIS PROPOSAL
A. General Information
1. Name: LT John Smith
2. Email: [email protected]
3. Curriculum: MOVES (399)
4. Thesis Advisor: Mathias Kölsch
5. Co-Advisor: Doug Horner
6. Second Reader: None
7. Chair, MOVES Academic Committee: Mathias Kolsch
8. Academic Associate (if another department involved): Mathias Kölsch
9. Date of Graduation: 27 March 2009
B. Area of Research
This thesis will develop two dimensional statistical models using computer vision techniques to
extract features in forward looking sonar images. A REMUS AUV with a Blue View 900
Forward Looking Sonar will be used for this application. There are two types of common
features in the sonar images. Examples of them are shown in Figure 1. The first and most
common is the ground. The second are image artifacts cause by acoustic sensors such as the
Acoustic Doppler Current Profiler (ADCP) / Doppler Velocity Log (DVL), acoustic modem and
sidescan sonar. These features can then be extracted from the image leaving behind possible
obstacles. The use of 2D models to describe the features will be compared to the current
technique used at NPS for detecting obstacles in forward looking sonar images.
Figure 1. Top left image is the vertical view of the forward looking sonar image with the ground
visible. Top right is an image of the ground and an approaching obstacle. Bottom left and right
show examples of artifact caused by acoustic sensors.
Page 1
DRAFT -- SAMPLE MOVES Thesis Proposal as of Jun 2009
C. Research Questions
1. Can a 2d model of the ground improve on the current 1d line model of the ground? What
inherent resolution does it need to have, and how many parameters?
2. Are environments different enough to require separate training data? What is the
difference between features extracted when the training data set of the same environment
is used compared to a dataset of varying environments?
3. How will processing time compare between the proposed multi dimensional model
method and the current method on a standard 1.8GHz pc/104 single board computer?
D. Discussion
Autonomous Underwater Vehicles (AUVs) perform vital roles for the NAVY in surveying and
mine detection. Since they are not tethered to the surface they can accomplish these missions
more quickly and stealthier than other tools. When performing a mission to collect side scan data
the AUV will try to hold a 10:1 swath width to altitude ratio. The side scan sonar on the Hydroid
REMUS 100 vehicle has a range of 30 meters so the vehicle tries to maintain an altitude of 3
meters. Altitude is measured with the ADCP / DVL, however, since the ADCP/DVL is facing
straight down and updates the altitude at 2Hz. the vehicle can only react to a slope change that is
less than 45°. If the vehicle is operating in an environment where objects are protruding from the
ground there is an increased chance of a collision.
The Autonomous Vehicles Lab at NPS has made progress in AUV obstacle avoidance using data
collected with the forward looking sonar. The current technique used on NPS AUVs is to model
the ground as a straight line and to apply a Hough transform to locate it. Once the ground has
been located a search area is defined above the ground. The search area is used to restrict the
search to a place where a valid object is likely to be and to filter out the artifact caused by the
ADCP/DVL. The OpenCV library cvBlobsLib is applied to the search area to create a set of
possible objects. A second frame is taken and the above steps repeated. Using the state
information from the vehicles the possible objects from the first set are compared to those of the
second. If any in the second set are within a certain range of the predicted location then that
object is marked as seen twice. This process is repeated continuously. If a possible object is
detected more than a threshold number of times it is marked as an obstacle and a reactive
obstacle avoidance flight plan is started.
This technique has the benefit of being very fast and being simple to implement. On a vehicle
with limited endurance, efficiency and power consumption are critical. The down side to the
technique is that there is a large amount of tuning required for each environment and multiple
frames are needed to identify an obstacle. Another limitation is the success of the entire search
process is dependent on finding the ground.
The suggested technique is to extract obstacles from the forward looking sonar image using
multidimensional models to describe common features. With these common features identified
the remaining features would then be obstacles. By more descriptive models these features can
also be identified in varying environments, without the need of excess tuning and reduction in
the search area. However these benefits need to be weighed against the increased power
consumption that is likely to happen.
Page 2
DRAFT -- SAMPLE MOVES Thesis Proposal as of Jun 2009
E. Scope of the Thesis
The thesis will be limited to developing models of common features in the sonar image. This
technique will then be tested using an onboard simulator in the lab to see if it can properly detect
obstacles. During this time the system will be benchmarked to compare the performance of the
multidimensional model technique to the previous one. The final step will be to demonstrate the
models ability to detect obstacles in the field. This demonstration will be during a side scan sonar
mission in Monterey Bay.
F. Methodology
This thesis will be conducted over three steps.

The first step will use MATLAB to develop models to describe the features and
techniques for locating them in the image. The models will be derived from the training
data of an existing data set from a variety of environments including a shale bed in
Monterey CA, a sandy sea floor from the Gulf Coast of Florida and San Diego CA, and a
muddy bottom from Charles River in Boston Harbor. Specific models for each
environment will be derived from a sample of data from that environments data set. A
generic model will be derived from a sample of images from all the environments.

The second step will be to convert the MATLAB code into c code that will run on the
REMUS secondary controller. Using the onboard simulator in REMUS a mission will be
run and the new technique will be tested and modified as needed. During the simulation
both techniques will be benchmarked on CPU usage, power consumption, and measured
number of false positives and missed objects. These tests will be run in each of the
environments using the model specific for that environment and the generic model.

The final step will be to validate this technique in a real world scenario. The results from
the simulation stage will dictate which models will be used for the in field demonstration.
The demonstration will be conducted at Fisherman Flats in Monterey Bay. This already
surveyed area is relatively flat except for a 4 meter tall outflow pipe running through it.
For the mission the REMUS will maintain an altitude of 3 meters and collect side scan
data of the area. The demonstration will be successful if the vehicle can perform the side
scan mission without running aground.
G. Chapter Outline
The following are the tentative chapter headings:
I.
Introduction
II.
Background
A. AUVs in the NAVY
B. REMUS
1. Details of Vehicle
2. Details of Secondary Controller
3. Details of Blue View Forward Looking Sonar
C. Computer Vision Background
Page 3
DRAFT -- SAMPLE MOVES Thesis Proposal as of Jun 2009
1. Techniques Previously Used in Sonar Images.
a. Hough Transform
b. Active Shape Models
III.
Methodology
A. Models
IV.
Experiments
A. Data
B. Simulation
C. Field Testing
V.
Discussion
A. Results
VI.
Conclusion and Recommendations
VII. Appendices
VIII. Bibliography
H. Schedule
1. Literature Review
June 1 2008
2. Develop Different Models and Processing Techniques
July 14 2008
3. Convert from MATLAB to C
August 29 2008
4. Simulation
4.1
Test plan
September 5 2008
4.2
Benchmark current technique
September 11 2008
4.3
Benchmark statistical model
September 31 2008
4.4
End simulation
September 31 2008
5. In Field Testing
5.1
Test plan
October 4 2008
5.2
End field testing
October 15 2008
6. Draft Thesis Completed
December 2009
I. Benefits of Study
Currently AUVs do not posses a robust way to react to obstacles in more dynamic terrains such
as a harbor. This thesis will look into a way to process data gathered from an AUVs forward
looking sonar in order for the vehicle to react to a possible collision. Future work can use this
technique for map building and localization and possibly replace the need of the ADCP/DVL for
altitude and velocity measurements.
Page 4
DRAFT -- SAMPLE MOVES Thesis Proposal as of Jun 2009
J. Anticipated Travel/Funding Requirements
Travel: Attend OCEANS '08 MTS/IEEE in Quebec City 15-18 SEP 2008
Funding: Require multiple trips on NPS and Monterey Bay Aquarium Research Institute research
vessels in Monterey Bay and off the local coastal areas for data collection and testing.
K. Preliminary Bibliography
Bab-Hadiashar, A. and Sutter, D., Data Segmentation and Model Selection for computer Vision,
Springer, 2000.
Paragios, N., and others, Handbook of Mathematical Models in Computer Vision, Springer,
2006.
Horner, D., Yakimenko, O. “Recent Developments for an obstacle Avoidance System for a
Small AUV,” paper presented at IFAC Conference on Control Applications in Marine Systems,
Bol island Brac, Croatia, ## September 2007.
Lane, D.M., Trucco, E., “Embedded Sonar & Video Processing for AUV Applications,” paper
presented at the 2000 Offshore Technology Conference, Houston, Texas, 1-4 May 2000.
Schneiderman, H., “Learning Statistical Structures for Object Detection,” Computer Analysis of
Images and Patterns, Groningen, The Netherlands, 25-27 August 2003.
Cuschieri, J.; Negahdaripour, S., “Use of forward scan sonar images for positioning and
navigation by an AUV,” OCEANS '98 Conference Proceedings , volume 2, p.752-756, 28
September-1 October 1998.
Reed, S., Petillot, Y., Bell, J., “Model-Based Approach to the Detection and Classification of
Mines in Sidescan Sonar,” Applied Optics, volume 43, issue 2, p. 237-246, January 2004.
Petillot, Y., Tena Ruiz, I., Lane, D.M., “Underwater vehicle obstacle avoidance and path
planning using a multi-beam forward looking sonar,” IEEE Journal of Oceanic Engineering,
volume 26, issue 2, p. 240-251, Apr 2001.
Clark, D., Ruiz, I.T., Petillot, Y., Bell, J., “Particle PHD filter multiple target tracking in sonar
image,” IEEE Transactions on Aerospace and Electronic Systems, volume 43, issue 1, p. 409416, January 2007.
Dolbec, M., “Velocity Estimation Using Forward Looking Sonar,” M.S. Thesis Naval
Postgraduate School, Monterey, CA, March 2007.
Horner, D.P, Healy, A.J., Kragelund, S.P., “AUV Experiments in Obstacle Avoidence,”
Proceedings of MTS/IEEE OCEANS 2005, volume 2, p. 1464-1470, 19-23 September 2005.
Page 5