Hierarchical Segmentation of
Polarimetric SAR Images
Jean-Marie Beaulieu
Computer Science Department
Laval University
Ridha Touzi
Canada Centre for Remote Sensing
Natural Resources Canada
Hierarchical Segmentation of
Polarimetric SAR Images
•Hierarchical Image Segmentation
•As a maximum likelihood estimation problem
•Segmentation of polarimetric images
•Segment sizes – shape constraints
•Results
Image Segmentation
is the division of
the image plane
into regions
S1
S3
S4
S2
S6
S5
Two basic questions:
1- What kind of regions do we want ?
• Homogeneous regions
• Segment similarity
2- How can we obtain them ?
• Algorithm design
HIERARCHICAL SEGMENTATION
BY STEP-WISE OPTIMISATION
A hierarchical segmentation begins with an initial partition
P0 (with N segments) and then sequentially merges these
segments.
level n+1
level n
level n-1
Segment tree
STEP-WISE OPTIMISATION

A criterion, corresponding to a measure of segment similarity,
is used to define which segments to merge.

At each iteration, an optimization process finds the two most
similar segments and merges them.

This can be represented by a segment tree, one node per
iteration, where only the two most similar segments are
merged.
Sequence of segment merges.
SEGMENTATION AS
MAXIMUM LIKELIHOOD ESTIMATION
1) need a partition of the image
P  sk , sk  i  I
2) need statistical parameters
  s ,
sP
3) need an image probability model
p( xi | s )
xi are conditionally independent
S1
S3
S4
S2
S6
S5
X  xi ,
the likelihood of   s ,
Given an image
is
iI
S1
P
L(, P | X )  p( X | , P)
S4
S2
S6
L(, P | X )   p( xi | s (i ) )
iI
S3
S5
P
The segmentation problem is to find the partition
that maximizes the likelihood.
Global search – too many possible partitions.
 s is derived from statistics calculated over a segment s.
The maximum likelihood increases with the number of segments
p( X | , P)
k
number of segments
Can't find the optimum partition with k segments, Pk
Too many, except for P1 and Pnxn.
Hierarchical segmentation
 get Pk from Pk+1 by merging 2 segments.
Stepwise optimization
• examine each adjacent
segment pair
• merge the pair that
minimizes the criterion
Merging criterion:
merge the 2 segments producing the smallest
decrease of the maximum likelihood
(stepwise optimization)
p( X | , P)
k
number of segments
Sub-optimum within hierarchical merging framework.
Log likelihood form


ln  L(, P | X )   ln   p( xi | s (i ) )    ln  p( xi | s (i ) ) 
 iI
 iI
Summation inside region
  ln  p( xi | s )   
sP
is
LML( s )
sP
Criterion  cost of merging 2 segments
  LML( si )  LML( s j )  LML( si  s j )

 ln  p( x | s )    ln  p( x | s )   
i
xsi
minimize

xs j
j
xsi  s j

ln p( x | si  s j )

POLARIMETRIC SAR IMAGE
Multi-channel image – 3 complex elements
 hh 
x   hv 
 
 vv 
Complex gaussian pdf
each element has
a zero mean circular
gaussian distribution
( is the covariance matrix)

1
p( x | )  3 exp  x* 1 x
 

x* is the complex conjugate transpose of x
The best maximum likelihood estimate of  is
the covariance calculated over the region (segment)
ˆ  C  1
ns
*
x
x

ns is the number of pixels
in segment s
xs
  hh hh*
1
C    hv hh*
n
  vv hh*

*
hh
hv

*
hv
hv

*
vv
hv

*

hh
vv

*
 hv vv 
*

vv
vv


LML for a region s is
 1
LML( s )   ln  p( x | Cs )    ln  3
exp  x*Cs1 x
 C
xs
xs
s


    ln 3  ln Cs  x*Cs1 x 
xs
 ns ln 3  ns ln Cs   x*Cs1 x
xs
 ns ln Cs  ns ln 3  3ns
constant term for the whole image




The variation produced by merging 2 segments is
  LML( si )  LML( s j )  LML( si  s j )
 nsi ln Csi  nsj ln Csj  (nsi  nsj )ln Csi  sj
Hierarchical segmentation:
at each iteration, merge the 2 segments
that minimize the stepwise criterion Ci,j
Ci , j  (nsi  nsj )ln Csi sj  nsi ln Csi  nsj ln Csj
SEGMENTATION BY HYPOTHESIS TESTING
Test the similarity of segment covariances Ci = Cj = C
- merge segment with same covariance
Use the difference of determinant logarithms as a test statistic

Ci , j  K ( nsi  nsj )ln Csi  sj  nsi ln Csi  nsj ln Csj

With the scaling factor K, the statistic is approximately
distributed as a chi-squared variable with 6 degrees of freedom
as nsi and nsj become large.
K  1  1312 (1/ nsi  1/ nsj  1/(nsi  nsj ))
Segmentation by hypothesis testing
Two hypothesis
H0: segments are similar
H1: segments are different
Under H1
Under H0
Distributions of
the statistic d
under H0 and H1
0
Type II
Two types of errors
Type I: not merging similar segments
Type II: merging different segments
Type I
 = Prob( Type I errors )
 = Prob( Type II errors )
Under H1
Under H0


0
Select the threshold to minimise  or ,
but not both simultaneously
In hierarchical segmentation, type II errors
(merging different segments) can not be corrected,
while type I errors can be corrected later on.
Under H0

Under H1

0
The distribution of H1 and  are unknown.
Reduce  by increasing .
Sequential testing:
 will be reduced as segment sizes increase.
1+2+…  minimum( 1, 2, … )
1+2+…  maximum( 1, 2, … )
test # 2
Si 2
Sj 2
test # 1
Sh1
Si 1
Sj 1
Sk 1
Stepwise criterion
Find and merge the segment pair (i, j)
that minimizes Vi,j (= 1 -  ).
Under H0

Under H1
vi,j
0
Vi,j = Prob( d  di,j ; H0 )
(= 1 -  ).
Amplitude values
|hh|
|hv|
|vv|
|hh| / |hv| / |vv|
80 pixels / cell
Correlation – module (0 – 1)
hh vv*
vv hv*
hh hv*
hh vv*/ hh hv*/ vv hv*
Correlation – phase (-180o – 180o)
hh vv*
vv hv*
hh hv*
hh vv*/ hh hv*/ vv hv*
Amplitude image
1000 segments
500 segments
200 segments
100 segments
Amplitude image
5 pixels / cell
CRITERION FOR SMALL SEGMENTS
The determinant |C| is null for small segments
  hh hh*
1
C    hv hh*
n
  vv hh*

*
hh
hv

*
hv
hv

*
vv
hv

*

hh
vv

*
hv
vv


*
 vv vv 
Reduce covariance matrix model for small segments
  hh hh *
1
0

n
  vv hh *

0
*
hv
hv

  hh hh *
1
0

n

0

0
 hv hv*
0
0
*

hh
vv


0

*
 vv vv 



 vv vv* 
0
0
Gradual transition between models
1.0
*

 * 



*
* * *
* * *


* * *
weight
10
segment size
SEGMENT SHAPE CRITERIA
High speckle noise
 first merges produce ill formed segments
•Bonding box – perimeter
Cp
•Bonding box – area
Ca
•Contour length
Cl
New criteria
contour
i, j
C
C
polar
i, j
 Cp  Ca  Cl
2
Bonding box – perimeter
Cp 
perimeter of Si  S j
perimeter of bonding box
Bonding box – area
area of bonding box
Ca 
area of Si  S j
Contour length
Lc  length of common part of contours
Lex i  length of exclusive part for Si
 Lex i
Cl  Min 
,
 Lc
Lex j 

Lc 
Lex i
Lex j
Lc
1000 segments – low resolution
1000 segments
500 segments
200 segments
100 segments
CONCLUSION
•Hierarchical segmentation produces good results
•Criterion should be adapted to the application
•Good polarimetic criterion
•The first merges should be done correctly