Scientific Systems Company, Inc
Presentation at Meeting No. 96
Aerospace Control and Guidance Systems Committee
Harbour Town Resorts
Hilton Head, S. Carolina
19-21 October, 2005
By
Raman K. Mehra
Phone: 781-933-5355
Email: [email protected]
Scientific Systems
1
VISTA: Visual Threat Awareness
for Unmanned Air Vehicles
Scientific Systems
VISTA Overview
Problem: Situational awareness is
required for “nap of the earth” UAV
autonomy
Goal: Research, design and flight
test a system for real time, visual
collision obstacle detection
Proposed solution: Real time
stereo + Perceptual organization +
Image segmentation + Region
tracking = VISTA (Visual Threat
Awareness) system
Scientific Systems
3
Challenges of UAV visual obstacle detection
• 3D Reconstruction: Difficulty of computing 3D scene structure
from two or more 2D images
• Performance tradeoffs
– Missed detections and false alarms vs. available
computation
– Scene resolution vs. available computation
• Stereo vision: correspondence errors due to low contrast,
occlusions, specular reflections, foreshortening, periodic
features, threshold choice, minimum distance violation
– Traditional block matching stereo is too noisy for practical
use in general!
Scientific Systems
4
VISTA Achievements
Obstacle Map: Green = Nearest Collision Obstacle, Yellow = Obstacle
Disparity Map: Light = Close, Dark = Far, White = Undefined
•
First application of 640x480@23Hz stereo in UAV flight
•
Real time (3-5Hz) perception algorithm accelerated by Sarnoff Corp’s Acadia I vision processor for
improved performance over traditional stereo
•
Proof of concept visual obstacle detection and avoidance in two flight experiments on Georgia
Tech’s GT-Max helicopter against rural obstacles
Scientific Systems
5
VISTA Architecture
•
Stereo: Large baseline (50cm) binocular stereo for range measurement
•
Foveation: Compression appropriate for collision detection
•
Segmentation: Measurement fusion, Grouping for obstacle hypotheses
•
Detection: False alarm rejection, Obstacle tracking
Scientific Systems
6
Segmentation examples
Synthetic Sphere
Synthetic Sign
Building
Hallway
Large Tree
Small Tree
Scientific Systems
7
Flight experiment 1.1.10: Detection video
Obstacle Map: Green = Nearest Collision
Obstacle, Yellow = Tracked obstacles
Scientific Systems
Disparity Map: Light = Close, Dark = Far,
White = Undefined
8
Flight experiment 2: Detection and avoidance
Flight 2.1.6
Flight 2.2.5
Avoidance Triggered
Avoidance Triggered
Collision
Keepout
Replanning
Volume
Radius
Radius
10 ft/s
60m/5m
40 ft
65 ft
7ft @ 65ft
10 ft/s
60m/5m
40 ft
82 ft
15ft @ 82ft
Flight
Obstacle
Time
Speed
2.1.6
“Sign”
11:52 am
2.2.5
“Sign”
3:23 pm
Scientific Systems
Detection Error @
Obstacle Distance
9
Flight experiment 2.1.6: Ground video
Scientific Systems
10
Flight experiment 2.1.6: Detection video
Obstacle Map: Green = Nearest Collision
Obstacle, Yellow = Tracked obstacles
Scientific Systems
Disparity Map: Light = Close, Dark = Far,
White = Undefined
11
VISTA Summary
We have demonstrated proof of concept for a real time
visual collision detection system in a rural environment.
Major Accomplishments:
•
First application of 640x480@23Hz stereo hardware in UAV flight
•
Real time (3-5Hz) algorithm that improves collision obstacle detection performance
over traditional stereo only
•
Proof of concept in two flight experiments against real obstacles.
Scientific Systems
12