FRIP: A Region-Based Image
Retrieval Tool Using Automatic
Image Segmentation and
Stepwise Boolean AND Matching
IEEE TRANSACTIONS ON MULTIMEDIA, VOL.
7, NO. 1, FEBRUARY 2005, pp. 105-113
ByoungChul Ko and Hyeran Byun
Reporter: Jen-Bang Feng
Outline




Image Retrieval
Content-Based Image Retrieval
The Proposed Scheme
Experimental Results
2
Image Retrieval
Image
DB
Image retrieval
scheme
Features
Features
Features
Features
Features
Features
Features
Compare
Query
Image
Searching
Results
Image retrieval
scheme
Feature
3
Content-Based Image
Retrieval

From text-based retrieval scheme

WWW search engine

Query-by-image in early 90’s

From global to local (region)

Region Of Interest
4
The Proposed Scheme
1.
Image Segmentation


2.
Feature extraction




3.
Two-Level Segmentation Using Adaptive Circular Filter and
Bayes’ Theorem
Iterative Level Using Region Labeling and Iterative Region
Merging
Color
Texture
Normalized Area
Shape and Location
Stepwise Similarity Matching
5
Two-Level Segmentation Using
Adaptive Circular Filter and Bayes’
Theorem

Image
(RGB)
Adaptive Circular Filter
Image
(CIE Lab)
Smoothed
Image
(CIE Lab)
Remove
middle frequency
Color
histogram
Separate regions
by circular filters
Regions
Regions
Regions
Regions
6
Two-Level Segmentation Using
Adaptive Circular Filter and Bayes’
Theorem
a is similar to c in color
but a is closer to b than c
Example of circular filtering process
7
Two-Level Segmentation Using
Adaptive Circular Filter and Bayes’
Theorem
P C M | c x , y  
PCM P c x , y | C M 
PC M P c x , y | CM   P C M P c x , y | C M 
if P C M | c x , y   0.5, then c x , y  M C
else c x , y  c xy
Three circular filters
3x3, 7x7, 11x11
CM: the most frequently observed histogram bins
CM: other bins
cx,y: center value of CM
MC: the major class color
8
Two-Level Segmentation Using
Adaptive Circular Filter and Bayes’
Theorem
division according to the edge distribution
Segmentation result
Selected filter, 3x3, 7x7, 11x11
Final segmented image
9
Iterative Level Using Region Labeling
and Iterative Region Merging
Image
(RGB)
Image
(CIE Lab)
Smoothed
Image
(CIE Lab)
Color
histogram
Remove
middle frequency
Separate regions
by circular filters
Regions
Regions
Regions
Regions
Regions
Regions
Merge regions
10
Iterative Level Using Region Labeling
and Iterative Region Merging
For the N neighbor regions
If
 R
N

i
i
i

R

R

R

R

R
L
mL
a
ma
b
mb  T
i
Then merge the regions
If the number of regions is larger than 30
Then increase the threshold
and repeat the circular filter
11
Feature extraction
Color


Average AL, Aa, Ab

Variance VL, Va, Vb

Color distance of Q and T
d
C
Q ,T

   QC  TC   QC  TC
C VL,Va,Vb
 C  Al , Aa, Ab

 2

12
Feature extraction

Texture

Biorthogonal wavelet frame (BWF)

The X-Y directional amplitude Xd, Yd

The distance in texture
d
T
Q ,T
Yd Q
Yd T


Xd Q XdT
13
Feature extraction

Normalized Area
NPQ =
(Size of the region) / (Size of the image)
d
NArea
Q ,T
 NPQ  NPT
14
Feature extraction

Shape and Location

The global geometric shape feature

eccentricity


Estimate the bounding rectangle for each
segmented region
For the major axis Rmax and minor axis Rmin
E
RQ min
RQ max

RT min
RT max
15
Feature extraction

Shape and Location

The local geometric shape feature

MRS (modified radius-based shape signature)

invariant under shape’s scaling, rotation, and
translation
16
Feature extraction

Shape and Location

The local geometric shape feature

MRS (modified radius-based shape signature)

Extracts 12 radius distance values
d QMRS
,T  min d clockwise , d counterclockwise 
1
dC 
N
 1




i 1  N

N
N

j 1
Q j  Qi 


T j  Ti 

2
17
Stepwise Similarity Matching
 p
i
i

Sim X , Y      w j  D j q j , t j
i 1  j 1
r






18
Experimental Results
query: flower
best case
19
Experimental Results
query: ship
worst case
20
21
22