A Multivariate Statistical Model for Multiple
Images Acquired by Homogeneous or
Heterogeneous Sensors
Jorge Prendes1,2 , Marie Chabert1,3 , Frédéric Pascal2 ,
Alain Giros4 , Jean-Yves Tourneret1,3
1
3
TéSA Laboratory, 2 Supélec - SONDRA,
University of Toulouse, 4 CNES (French Space Agency)
ICASSP 2014
Introduction
Image Model
Similarity Measure
Results
Conclusions
Outline
1 Introduction
2 Image Model
3 Similarity Measure
4 Results
5 Conclusions
J. Prendes
TéSA – Supélec-SONDRA – INP/ENSEEIHT – CNES
A Multivariate Statistical Model for Multiple Images Acquired by Homogeneous or Heterogeneous Sensors
2 / 23
Introduction
Image Model
Similarity Measure
Results
Conclusions
Introduction
Motivation: Change detection on remote sensing images
Monitor urban/rural area evolution
Detect new constructions
Track changes in agricultural areas
Track urban growth
Coordinate efforts after natural disasters
Volcano eruptions
Floodings
Earthquakes
Improve the analysis of remote sensing images
Find new objects
Different type of sensors: Optical, SAR, Hyperspectral, etc.
Joint analysis of heterogeneous sensors!
J. Prendes
TéSA – Supélec-SONDRA – INP/ENSEEIHT – CNES
A Multivariate Statistical Model for Multiple Images Acquired by Homogeneous or Heterogeneous Sensors
3 / 23
Introduction
Image Model
Similarity Measure
Results
Conclusions
Introduction
Change Detection Framework
Images
Sliding Window: W
Sliding window W
Similarity measure on W
Optical
SAR
Threshold
Statistical Similarity Measures
Dependency between pixel
intensities
Correlation Coefficient
Linear dependency, Fails on homogeneous areas
Mutual Information
WOpt
WSAR
Decision
Similarity Measure
d = f (WOpt , WSAR )
H0 : Absence of change
H1 : Presence of change
H0
d≷τ
H1
Result
Using several
windows
...
Requires pdf estimation, Fails on homogeneous areas
Objective: Similarity measure for homogeneous and heterogeneous
sensors based on a statistical model
J. Prendes
TéSA – Supélec-SONDRA – INP/ENSEEIHT – CNES
A Multivariate Statistical Model for Multiple Images Acquired by Homogeneous or Heterogeneous Sensors
4 / 23
Introduction
Image Model
Similarity Measure
Image Model – Optical image
Results
Conclusions
for Homogeneous Regions
Optical Sensor
Affected by additive
Gaussian noise
IOpt = TOpt (P) + νN (0,σ2 )
IOpt |P ∼ N TOpt (P), σ 2
where
TOpt (P) is how an object with physical
properties P would be ideally seen by an
optical sensor
σ 2 is associated with the noise variance
10
5
0
0
IOpt
1
Histogram of the normalized image
J. Prendes
TéSA – Supélec-SONDRA – INP/ENSEEIHT – CNES
A Multivariate Statistical Model for Multiple Images Acquired by Homogeneous or Heterogeneous Sensors
5 / 23
Introduction
Image Model
Similarity Measure
Image Model – SAR Image
Results
Conclusions
for Homogeneous Regions
Radar Sensor
Affected by multiplicative
speckle noise (with gamma
distribution)
ISAR = TSAR (P) × νΓ(L, 1 )
L
TSAR (P)
ISAR |P ∼ Γ L,
L
4
2
where
TSAR (P) is how an object with physical
properties P would be ideally seen by a SAR
sensor
L is the number of looks of the SAR sensor
J. Prendes
0
0
ISAR
1
Histogram of the normalized image
TéSA – Supélec-SONDRA – INP/ENSEEIHT – CNES
A Multivariate Statistical Model for Multiple Images Acquired by Homogeneous or Heterogeneous Sensors
6 / 23
Introduction
Image Model
Similarity Measure
Image Model – Generic Image
Results
Conclusions
for Homogeneous Regions
Generic Model: Sensor S
IS |P = fS [TS (P), νS ]
Optical Image
SAR Image
IOpt = TOpt (P) + νN (0,σ2 )
ISAR = TSAR (P) × νΓ(L, 1 )
TOpt (P) = µP
TSAR (P) = αP × θP
J. Prendes
L
TéSA – Supélec-SONDRA – INP/ENSEEIHT – CNES
A Multivariate Statistical Model for Multiple Images Acquired by Homogeneous or Heterogeneous Sensors
7 / 23
Introduction
Image Model
Similarity Measure
Image Model – Joint Distribution
Results
Conclusions
for Homogeneous Regions
Independence assumption
for the sensor noises
p(IS1 , IS2 |P) =
p(IS1 |P) × p(IS2 |P)
Conclusion
Statistical dependency
(CC, MI) is not always an
appropriate similarity
measure
J. Prendes
ISAR
1
0
0
IOpt
1
TéSA – Supélec-SONDRA – INP/ENSEEIHT – CNES
A Multivariate Statistical Model for Multiple Images Acquired by Homogeneous or Heterogeneous Sensors
8 / 23
Introduction
Image Model
Similarity Measure
Results
Conclusions
Image Model – Heterogeneous Regions
Sliding window W
Usually includes a finite
number of objects, K
Different values of P for
each object
p(IS1 , IS2 |W ) =
K
X
wk p(IS1 , IS2 |Pk )
k=1
1
ISAR
Pr(P = Pk |W ) = wk
0
0
IOpt
1
Mixture distribution!
J. Prendes
TéSA – Supélec-SONDRA – INP/ENSEEIHT – CNES
A Multivariate Statistical Model for Multiple Images Acquired by Homogeneous or Heterogeneous Sensors
9 / 23
Introduction
Image Model
Similarity Measure
Results
Conclusions
Image Model – Mixture Distribution
Mixture Distribution
p(IS1 , IS2 |W ) =
K
X
wk p(IS1 , IS2 |Pk )
k=1
Parameter Estimation
Expectation Maximization
Iteratively Algorithm
(i)
Estimate class prob. πn,k
(i)
Maximize parameters θk
Repeat
Selection of the number of classes [1]
[1] M. A. T. Figueiredo and A. K. Jain, ”Unsupervised learning of finite mixture models,” IEEE Trans. Pattern Anal. Mach. Intell., vol.
24, no. 3, pp. 381–396, March 2002.
J. Prendes
TéSA – Supélec-SONDRA – INP/ENSEEIHT – CNES
A Multivariate Statistical Model for Multiple Images Acquired by Homogeneous or Heterogeneous Sensors
10 / 23
Introduction
Image Model
Similarity Measure
Results
Conclusions
Similarity Measure – Introduction
Related to P
Can be used to derive
[TS1 (P), TS2 (P), . . . ] for
each object
Example:
TOpt (Pk ) = µk
TSAR (Pk ) = L × θk
0
0
0
J. Prendes
IOpt
1
1
P2
TSAR (P )
Mixture distribution
Parameter Estimates
ISAR
1
P3
P4
P1
0
TOpt (P )
1
TéSA – Supélec-SONDRA – INP/ENSEEIHT – CNES
A Multivariate Statistical Model for Multiple Images Acquired by Homogeneous or Heterogeneous Sensors
11 / 23
Introduction
Image Model
Similarity Measure
Results
Conclusions
Similarity Measure – Manifold
Main assumption
For each unchanged window,
v (P) = [TS1 (P), TS2 (P), . . . ] can be
considered as a point on a manifold
Several
unchanged windows
...
Manifold
D: Number of combined channels
J. Prendes
0.3
TSAR (P )
Describes the joint behavior of the
different images
Belongs to a D-dimensional space
0
0
TOpt (P )
1
TéSA – Supélec-SONDRA – INP/ENSEEIHT – CNES
A Multivariate Statistical Model for Multiple Images Acquired by Homogeneous or Heterogeneous Sensors
12 / 23
Introduction
Image Model
Similarity Measure
Results
Conclusions
Similarity Measure – Manifold
Unchanged regions
Changed regions
Pixels belong to the same
object
Pixels belong to different
objects
P is the same for both
images
P changes from one image
to another
0
J. Prendes
TSAR (P )
0.3
TSAR (P )
0.3
0
TOpt (P )
1
0
0
TOpt (P )
1
TéSA – Supélec-SONDRA – INP/ENSEEIHT – CNES
A Multivariate Statistical Model for Multiple Images Acquired by Homogeneous or Heterogeneous Sensors
13 / 23
Introduction
Image Model
Similarity Measure
Results
Conclusions
Similarity Measure – Manifold
Distance measure between
Optical and SAR images
H0 : Absence of change
PDF of v (P)
H1 : Presence of change
Good distance measure
Learned using training data
from unchanged images
Learning strategies
Histogram
Parzen windows
Mixture models
J. Prendes
K
X
k=1
where
H0
bk pbT (b
w
v W ,k ) ≷ τ
H1
bk is the estimated wk
w
b
v W ,k is the estimated vector v for the k-th
component of the window W
pbT is the estimated density of v (P)
τ is an application dependent threshold
TéSA – Supélec-SONDRA – INP/ENSEEIHT – CNES
A Multivariate Statistical Model for Multiple Images Acquired by Homogeneous or Heterogeneous Sensors
14 / 23
Introduction
Image Model
Similarity Measure
Results
Conclusions
Similarity Measure – Summary
Mixture
WOpt
WSAR
Sliding Window: W
h
TbS1 (P1 ), TbS2 (P1 )
...
i
µ
b4 , σ
b42 , b
k4 , α
b4
h
vbP4 :
TbS1 (P4 ), TbS2 (P4 )
i
0.3
0
Manifold Estimation
0.3
TSAR (P )
...
µ
b1 , σ
b12 , b
k1 , α
b1
vbP1 :
θb4 :
TSAR (P )
θb1 :
Using several
windows
...
0
0
TOpt (P )
0
TOpt (P )
1
1
Manifold Samples
TS2 (P )
0.3
0
J. Prendes
P2
P3 P4
P1
0 TS1 (P ) 1
TéSA – Supélec-SONDRA – INP/ENSEEIHT – CNES
A Multivariate Statistical Model for Multiple Images Acquired by Homogeneous or Heterogeneous Sensors
15 / 23
Introduction
Image Model
Similarity Measure
Results
Conclusions
Results – Synthetic Optical and SAR Images
Mutual
Information
Synthetic optical image
Correlation
Coefficient
Proposed Method
Synthetic SAR image
PD
1
Proposed
Correlation
Mutual Inf.
0
0
PFA
1
Performance – ROC
Change mask
J. Prendes
TéSA – Supélec-SONDRA – INP/ENSEEIHT – CNES
A Multivariate Statistical Model for Multiple Images Acquired by Homogeneous or Heterogeneous Sensors
16 / 23
Introduction
Image Model
Similarity Measure
Results
Conclusions
Results – Real Optical and SAR Images
Mutual
Information
Conditional
Copulas [1]
1
1
0
[1] G. Mercier, G. Moser, and S. B. Serpico, “Conditional copulas
for change detection in heterogeneous remote sensing images,”
IEEE Trans. Geosci. and Remote Sensing, vol. 46, no. 5, pp.
1428–1441, May 2008.
J. Prendes
TSAR (P )
Change mask
PD
Optical image SAR image during
before the
the flooding
flooding
Proposed Method
Proposed
Copulas
Correlation
Mutual Inf.
0
PFA
Performance – ROC
1
0
0
TOpt (P )
1
Manifold Projection
TéSA – Supélec-SONDRA – INP/ENSEEIHT – CNES
A Multivariate Statistical Model for Multiple Images Acquired by Homogeneous or Heterogeneous Sensors
17 / 23
Introduction
Image Model
Similarity Measure
Results
Conclusions
Results – Pléiades Images
Pléiades – May 2012
Pléiades – Sept. 2013
Change Map
1
0
Change mask
PD
TPleiades (P )
1
0
TPleiades (P )
1
Proposed
Correlation
Mutual Inf.
0
0
PFA
1
Manifold Projection
Performance – ROC
Special thanks to CNES for providing the Pléiades images
J. Prendes
TéSA – Supélec-SONDRA – INP/ENSEEIHT – CNES
A Multivariate Statistical Model for Multiple Images Acquired by Homogeneous or Heterogeneous Sensors
18 / 23
Introduction
Image Model
Similarity Measure
Results
Conclusions
Results – Pléiades and Google Earth Images
1
1
PD
0
Change Mask
J. Prendes
Change Map
Google Earth – July 2013
TGoogle (P )
Pléiades – May 2012
Proposed
Correlation
Mutual Inf.
0
TPleiades (P )
Manifold Projection
1
0
0
PFA
1
Performance – ROC
TéSA – Supélec-SONDRA – INP/ENSEEIHT – CNES
A Multivariate Statistical Model for Multiple Images Acquired by Homogeneous or Heterogeneous Sensors
19 / 23
Introduction
Image Model
Similarity Measure
Results
Conclusions
Results
Homogeneous images
Heterogeneous images
Pléiades – Pléiades
Pléiades – Google Earth
1
PD
PD
1
Proposed
Correlation
Mutual Inf.
0
J. Prendes
0
PFA
Proposed
Correlation
Mutual Inf.
1
0
0
PFA
1
CC and MI
Similar performance
CC
Reduced Performance
Proposed method
Improved performance
Proposed method and MI
Performance not affected
TéSA – Supélec-SONDRA – INP/ENSEEIHT – CNES
A Multivariate Statistical Model for Multiple Images Acquired by Homogeneous or Heterogeneous Sensors
20 / 23
Introduction
Image Model
Similarity Measure
Results
Conclusions
Conclusions and Future Work
Conclusions
New statistical model to describe multi-channel images
Analyze the joint behavior of the channels to detect changes,
in contrast with channel by channel analysis
New similarity measure showing encouraging results for
homogeneous and heterogeneous sensors
Interesting for many applications
Change detection
Registration
Segmentation
Classification
J. Prendes
TéSA – Supélec-SONDRA – INP/ENSEEIHT – CNES
A Multivariate Statistical Model for Multiple Images Acquired by Homogeneous or Heterogeneous Sensors
21 / 23
Introduction
Image Model
Similarity Measure
Results
Conclusions
Conclusions and Future Work
Future Work
Model validation on larger datasets.
Include priors on the sensor parameters: Bayesian methods
Study the method performance for different image features
Texture coefficients: Haralick, Gabor, QMF
Wavelet coefficients
Gradients
J. Prendes
TéSA – Supélec-SONDRA – INP/ENSEEIHT – CNES
A Multivariate Statistical Model for Multiple Images Acquired by Homogeneous or Heterogeneous Sensors
22 / 23
Introduction
Image Model
Similarity Measure
Results
Conclusions
Thank you for your attention
Jorge Prendes
[email protected]
J. Prendes
TéSA – Supélec-SONDRA – INP/ENSEEIHT – CNES
A Multivariate Statistical Model for Multiple Images Acquired by Homogeneous or Heterogeneous Sensors
23 / 23