IMAGE RETRIEVAL WITH
RELEVANCE FEEDBACK
Hayati CAM
Ozge CAVUS
Image Retrieval With Relevant Feedback
Hayati Cam & Ozge Cavus
Outline

Question:

What is Content Based Image Retrieval?

Recent Work on CBIR

Our Approach

Evaluation

Summary
Image Retrieval With Relevant Feedback
Hayati Cam & Ozge Cavus
CBIR

Large quantities of multimedia data is used in
archives

Traditional way: Using keywords in IR(Image
Retrieval)

Problems:



Annotation is very difficult
Keywords may be insufficient to represent the contents
of the images
Keywords are user dependent
Image Retrieval With Relevant Feedback
Hayati Cam & Ozge Cavus
CBIR
Image Retrieval With Relevant Feedback
Hayati Cam & Ozge Cavus
Recent Work

Extracting global low-level features
(texture or color) from images

Problem: limited in capability of deriving
higher semantic meanings of the images
Image Retrieval With Relevant Feedback
Hayati Cam & Ozge Cavus
Recent Work

Partitioning images into nonoverlapping
grid cell

Problem: Grids are not meaningful regions
Image Retrieval With Relevant Feedback
Hayati Cam & Ozge Cavus
Our Approach
Image Retrieval With Relevant Feedback
Hayati Cam & Ozge Cavus
Our Approach

Image Segmentation

Codebook Construction

Image Representation by using Posterior
Class Probability Values

Content Based Image Retrieval with
Relevance Feedback
Image Retrieval With Relevant Feedback
Hayati Cam & Ozge Cavus
Dataset

TRECVID 2005 dataset

29832 video shots

Contain approximately 20 different classes

exp: mountain, seaside, urban, sports …
Image Retrieval With Relevant Feedback
Hayati Cam & Ozge Cavus
Image Segmentation
Image Retrieval With Relevant Feedback
Hayati Cam & Ozge Cavus
Image Segmentation

Cluster the RGB color values of the pixels
by k-means
Image Retrieval With Relevant Feedback
Hayati Cam & Ozge Cavus
Image Segmentation

Smooth the regions by combined classifier
approach
Image Retrieval With Relevant Feedback
Hayati Cam & Ozge Cavus
Codebook Construction
Image Retrieval With Relevant Feedback
Hayati Cam & Ozge Cavus
Image Representation

Calculate region k=1000 bins histograms
for each image
Image Retrieval With Relevant Feedback
Hayati Cam & Ozge Cavus
Image Representation
Image Retrieval With Relevant Feedback
Hayati Cam & Ozge Cavus
Image Representation
Image Retrieval With Relevant Feedback
Hayati Cam & Ozge Cavus
Relevance Feedback

At the first iteration images are ranked by
distances to the query image

After each iteration user labels the images
as relevant and irrelevant

The new result are retrieved according to
the user feedback
Image Retrieval With Relevant Feedback
Hayati Cam & Ozge Cavus
Content Based Image Retrieval
Image Retrieval With Relevant Feedback
Hayati Cam & Ozge Cavus
Relevance Feedback
Assign a weight value w to each class
probability value
 The weights are assigned uniformly in the
first iteration.

Image Retrieval With Relevant Feedback
Hayati Cam & Ozge Cavus
Relevance Feedback

Given two images:

Distances between the corresponding
probability terms are computed


di
= distance between the ith probability values of
two images where i=1, …, c
These distances are combined as
 d = ∑ w d
i i
Image Retrieval With Relevant Feedback
Hayati Cam & Ozge Cavus
Relevance Feedback

Given the positive and negative examples,
for a probability term being significant for
a particular query:

Distances for the corresponding probability
values for relevant images must usually be
similar (hence, a small variance),

Distances between the probability values for
relevant images and irrelevant images must
usually be different (hence, a large variance).
Image Retrieval With Relevant Feedback
Hayati Cam & Ozge Cavus
Relevance Feedback

Weights are computed as:
std(distances of ith probability term
between relevant and irrelevant
images)
Wi =
std(distances of ith probability term
between relevant images)
Image Retrieval With Relevant Feedback
Hayati Cam & Ozge Cavus
Evaluation

Yao’s formula for cluster validation

ntr > nt

Why do we need this?

Better Clustering -> Better Probability Values ->
Better Retrieval
Image Retrieval With Relevant Feedback
Hayati Cam & Ozge Cavus
Evaluation

Precision-Recall
Image Retrieval With Relevant Feedback
Hayati Cam & Ozge Cavus
Summary

Steps of Our Approach




Image Segmentation
Codebook Construction
Image Representation by probabilities
CBIR with Relevance Feedback
Image Retrieval With Relevant Feedback
Hayati Cam & Ozge Cavus
THANK YOU
Image Retrieval With Relevant Feedback
Hayati Cam & Ozge Cavus