FPGA-based Platform for
Real-Time Stereo Vision
Sergiy Zhelnakov, Pil Woo (Peter) Chun,
Valeri Kirischian
Supervisor: Dr. Lev Kirischian
Reconfigurable Embedded Systems Lab
Ryerson University, Toronto, ON
Motivation
 Areas of application:
– Real-time Stereo-Vision Systems
– Telematic Systems: Remote Control of
Manipulators in Hazardous Areas
– Virtual Reality Systems and Simulators
– UAV Navigation Systems
– Telemedicine
– Surveillance / Security Systems
Objectives
 Development of the Run-Time Reconfigurable
Platform for implementation, testing and realtime verification of algorithms for stereo-vision
stereo-image recognition, and visualization of
3D images
 Implementation and test of real-time stereovideo processing algorithms (e.g. Edge
Detection in moving objects)
Specification
 Functional specification:
– The system performs: image capture from
two color cameras, stereo video
visualization with shutter glasses, image
processing (edge detection)
 Technical specification
– Input: video data with
 spatial resolution 640 x 480 pixels
 frame rate: 30 fps
 color, 8-bit resolution
– Output: standard SVGA
 resolution: 640 x 480 pixels
 frame rate: not less than 60 fps
Fastest
process:
40nS per
pixel output
Platform Components and Links
Stereo-Image Capturing &
Video Pre-processing Module
Run-time Reconfigurable
Multi-Stream Video Processor
Video Processing Module
Multi-Channel
Post-processing
Video-output
Module
Shutter Glasses
CRT Monitor
LCD Projector
Implemented Algorithms
 Color matching: (Bayer pattern) – at 2x30
frames/sec and 640x480 resolution
Implemented Algorithms
 Color matching
Implemented Algorithms
 Edge detection (Robert Cross) background
512 pixels
Max (|d-a|, |b-c|) ,where
480 pixels
a
b
c
d
Implemented Algorithms
 Edge detection (Robert Cross)
512 pixels
Block RAM 2/1
(previous row
data)
Block RAM 1/2
(current row
data)
Temporary
Storage
Calculation =
a1
b1
c1
d1
e1
f1
………………………
x1
y1
z1
X2
Y2
Z2
512 pixels
A2
B2
C2
D2
E2
F2
……………………………
………
P
C-b1
P : Previous camera data
C : Current camera data
camera DATA
C
or
P-a1
Platform Assembly and
Implementation Results
 Implementation results
– Image capture and
visualization on the
FPGA based
Reconfigurable
Functional Unit (RFU)
(1st stage - XCV50E;
2nd - XC2V1000)
– Edge detection
algorithm (Robert
Cross) implemented
Summary
 The Run-Time Reconfigurable Platform was
developed for different Stereo-Vision applications.
 All components of the Platform were tested by
implementation of real-time stereo-image capture,
image processing and visualization on stereovideo output display system
 Perspectives of the platform development: stereopanoramic vision system