Visual Information Processing
Human Perception V.S. Machine
Perception

Human perception:
pictorial information improvement for
human interpretation

Machine perception:
scene data processing for machine
understanding.
Visual Information Processing
 Why processing visual information:
- Better perception for visualization
- Medical imaging
- Video images (e.g., television commercials to feature films)
- Storage and transmission
- Application for machine intelligence, robotics, multimedia, graphics and
human computer interaction technology
 Topics in this course:
- Image processing fundamentals
- Image analysis
- Motion image processing (video)
- Object recognition
- Application with state-of-the-art techniques
Basic Concepts
 Computer Vision
- simulate the human visual system, not only “see” the world, but also
“understand” the world (emulate human vision – analysis and understanding)
 Image Processing
- pre-process the image for better “see” or “understand”
 Pattern Recognition
- classify and recognize both image content and some other statistic data.
 Computer Graphics
- create or synthesize a virtual image
 Artificial Intelligence
- emulate human intelligence
Digital Image Processing
Development
 Digital image processing:
- Low-level: Primitive operations (e.g., contrast enhancement,
sharpening in Photoshop and Photo-Stacker);
Image->image
- Mid-level: Image segmentation, classification;
image->attributes (e.g., edges, objects).
- High-level: Ensemble of recognized objects
(vision: make it understood)
 1920’s - Digitized newspaper picture transmitted through submarine cable
(London<->New York)
- 5 distinct brightness level -> 15 levels
Digital Image Processing
Development (Cont’d)
 1960’s - Images from space probe (distortion correction – image
transform)
 1970’s - Computerized Tomography (CT) (a ring of detectors collect the xrays to represent a slice)
 1980’s and later
- Computer image processing in industry, biomedical area, military
recognition, satellite imagery for weather and environment.
- Development of signal processing.
Digital Image Processing
Development (Cont’d)
left: Original image
right: Processed image
Image
Representation
 Image (Monochrome image / color image)
-- f(x,y) - two-dimensional light intensity function
-- (x,y) denote spatial coordinates;
-- the value of f at any point (x, y) is proportional to the brightness
(or gray level or gray scale) of the image at that point.
 Example:
o
y
Digital Image
Representation
 Digital image
-- an image f(x,y) that has been discretized both in the image
coordinates and in brightness
-- A matrix – the elements of digital array are called
pixels (picture elements, image elements, pels)
-- In computer programming: 2D array
-- Size: width - number of pixels horizontally
height – number of pixels vertically
 Example:
Digital Image Processing System
Image Display
Hardcopy
scene
Computer
Specialized Image
Processing Hardware
(digitizer, ALU
Arithmetic logic unit)
Image Sensors
(optical to electronic)
Image Storage
Image Processing
Software
Digital Image Processing
Fundamentals
Color Image
Processing
Multiresolution
Processing
Image
Compression
Image
Restoration
Image
Enhancement
Image
Acquisition
Morphological
Processing
Image
Segmentation
Knowledge
Base
Representation
& Description
Object
Recognition
Still Image vs.
Motion image
 Still Image
 JPEG, JPEG2000
 All the fundamental processing
 Image synthesis
 Motion Image
 Motion analysis and detection
 Video processing and transmission (H.261, H.263, H.264, MPEG1, 2, 4, 7,
21)
Illusion
http://www.michaelbach.de/ot/fcs_thompson-thatcher/index.html
http://members.tripod.com/~RBHcognitions/thatc1.htm
http://www.wjh.harvard.edu/~lombrozo/home/illusions/thatcher.html
Image Analysis
- Computer Vision
 Human Vision
- 70% information from visual perception
- 30% information from sound, touch, taste, smell, ……
 Computer Vision
- object detection (edge, region, texture, color,…)
- camera calibration
- 2D to 3D (shape from shading, shape from texture,
shape from motion, …)
Virtual Image
- Image Synthesis
 Real Image and Virtual Image:
 Image analysis: real image
- using computer to understand the real world
 Image Synthesis: virtual image
- using computer to create a virtual world (computer graphics)
Evaluation
 Subjective and Objective Evaluation:
 Subjective: No better way to judge the quality of an image than human vision
- rating
 Objective: pixel-by-pixel comparison
- mean square errors measurement
Recent Development
1.
Characteristics:

Better quality, Fast processing, Accurate Detection, and More understanding

Architecture: Parallel algorithm

Robotics and Active vision

Face and Gesture Recognition

Document Image Analysis

Texture Analysis

Motion tracking and Analysis

Color image analysis

Image Segmentation and Feature Extraction

3D Reconstruction
Relevant areas
Computer Graphics
 Image Processing
 Multimedia
 Human Computer
Interaction/Interface

Applications
Biomedical area (CT images)
 Military recognition
 Satellite imagery for weather and
environment
 Motion video (MPEG video)
 Still image (JPEG)
 TV and Film making

Stereo images as seen through LCD
Shutterglasses
Image Processing
Programming and tools

MS Windows: Visual C++

Unix, Linux, Irix: C and C++

Intel Image Processing Library.

Image Vision Library

Image format information (BMP, JPEG, TIF,…)
In this Class
 You will learn:
Image Processing & Computer Vision: Basics and application
 You will do:
Programming Assignments, Course Project (proposal, class
presentation, project report)
 Class attendance is required.
References
 Text books:
(1) Digital Image Processing (R. Gonzales)
(2) Computer Vision (L. G. Shapiro)
 Journals:
IEEE Transactions on Pattern Analysis and Machine Intelligence
IEEE Transactions on Image Processing
Computer vision, Graphics and Image Processing
Pattern Recognition
 Conferences:
IEEE International Conf. on Computer Vision and Pattern Recognition
IEEE International Conference on Computer Vision
IEEE International Conference on Pattern Recognition
IEEE International Conference on Image Processing