Inverse problems in
earth observation
and cartography
Shape modelling via
higher-order active
contours and phase fields
Ian Jermyn
Josiane Zerubia
Marie Rochery
Peter Horvath
Ting Peng
Aymen El Ghoul
2
Overview

Problem: entity extraction from (remote sensing) images.


Modelling prior shape knowledge: higher-order active
contours (HOACs).



Two examples: networks (roads) and circles (tree crowns).
Difficulties.
Phase fields:




Need for prior ‘shape’ knowledge.
What are they and why use them?
Phase field HOACs.
Two examples: networks (roads) and circles (tree crowns).
Future.
3
Problem: entity extraction

Ubiquitous in image processing and computer
vision:

Find in the image the region occupied by particular
entities.

E.g. for remote sensing: road network, tree crowns,…
4
Problem formulation

Calculate a MAP estimate of the region:
^ = arg max P(RjI ; K )
R
R
P(RjI ; K ) / P(I jR; K )P(RjK )

In practice, minimize an energy:
E (R; I ) = ¡ ln P(I jR; K ) ¡ ln P(RjK )
= E I (I ; R) + E G (R) + const
^ = argmin E(R; I )
R
R
5
Building EG: active contours
A region is represented by
its boundary, R = [], the
‘contour’.
 Standard prior energies:



Length of R and area of R.
Single integrals over the
region boundary:



Short range dependencies.
Boundary smoothness.
Insufficient prior knowledge.
S1
R2

R
R
6
Difficulties
(Remote sensing) images are
complex.
 Regions of interest distinguished by
their shape.



But topology can be non-trivial, and
unknown a priori.
Strong prior information about the
region needed, without constraining
the topology.
Building a better EG:
higher-order active
contours

Introduce prior knowledge via long-range
dependencies between tuples of points.
° (t 0); °_(t 0)
° (t); °_(t)
Dependent

How? Multiple integrals over the contour.
E.g. Euclidean invariant two-point term:
ZZ
E (° ) = ¡
dt dt 0 °_(t) ¢_
° (t 0) ª (j° (t) ¡ ° (t 0)j)

7
8
Prior for networks
E G (R) = ¸ L (@R) + ®A(R)
ZZ
¯
¡
dt dt 0 °_(t) ¢_
° (t 0) ª (j° (t) ¡ ° (t 0)j)
2

(r)
Gradient descent with this energy.

A perturbed circle evolves towards a structure
composed of arms joining at junctions.
r
9
Prior for a gas of circles

The same energy EG can model a
‘gas of circles’ for certain parameter
ranges.


Which ranges?
A circle should be a stable
configuration of the energy (local
minimum).

Stability analysis.
10
Phase diagrams
Circle
Bar
11
Optimization problem
Minimize E(R, I) = EI(I, R) + EG(R).
 Algorithm: gradient descent using distance
function level sets.
 But the gradient of the HOAC term is non-local,
and requires:




The extraction of the contour;
Many integrations around the contour;
Velocity extension.
12
Example I: HOAC results
13
Example I: HOAC results
14
Example II: HOAC results
15
Example II: HOAC results
16
Problems with HOACs

Modelling:


Space of regions is complicated to express in the
contour representation.
Probabilistic formulation is difficult.


Algorithm:




Parameter and model learning are hampered.
Not enough topological freedom.
Gradient descent is complex to implement for
higher-order terms.
Slow.
Solution: ‘phase fields’.
17
Phase fields
Phase fields are a level set representation (z()
= {x : (x) > z}), but the functions, , are
unconstrained.
 How do we know we are modelling regions?

½
Z
E 0 ( Á) =
ÁR = arg
-
d2 x
min
Á: ³ (Á)= R
R
R = 1
R = -1
D
1 4 1 2
r Á ¢r Á + ¸ ( Á ¡ Á )
2
4
2
E 0 (R)
¾
18
Relation to active contours

One can show that
E 0 (ÁR ) ' ¸ C L(@R)

R is a minimum for fixed R.


Thus gradient descent with E0 mimics
gradient descent with L: ‘valley
following’.
Can also add odd potential term to
mimic:
E 0 (ÁR ) ' ¸ C L (@R) + ®C A(R)
R1
R2
R3
19
Why use them?
Complex topologies are easily represented.
 Representation space is linear:




 can be expressed, e.g., in wavelet basis for
multiscale analysis of shape.
Probabilistic formulation (relatively) simple.
Gradient descent is based solely on the PDE
arising from the energy functional:


No reinitialization or ad hoc regularization.
Implementation is simple and the algorithm is fast.
20
Why use them?

Neutral initialization:



No initial region.
No bias towards “interior” or
“exterior”.
Greater topological freedom:

Can change number of
connected components and
number of handles without
splitting or wrapping.
+
21
How to write HOACs as
phase fields?

Use that rR is zero except near R, where it is
proportional to the normal vector.
¯C
E Q (R) = ¡
2
¯
E N L (Á) = ¡
2

ZZ
dt dt 0 °_(t) ¢G C (° (t); ° (t 0)) ¢_
° (t 0)
ZZ
d2 x d2 x 0 r Á(x) ¢G (x; x 0) ¢r Á(x 0)
-
2
One can show that
E N L (ÁR ; ¯; G) ' E Q (R; ¯C ; G)
22
Phase fields : likelihood
energies EI
r : normal vector to the contour.
 r¢r : boundary indicator.
 (1 § )/2 : characteristic function of the region
(+) or its complement (-).
 Using these elements, one can construct the
equivalents of active contour and HOAC
likelihoods.

23
Optimization problem

Minimize
E (R; I ) = E I (I ; R) + E G (R)
= E I (I ; R) + E 0 (R) + E N L (R)

Algorithm: gradient descent, but…
24
Major advantage of phase
fields for HOAC energies

Whereas, due to the multiple integrals, HOAC
terms require


Contour extraction, contour integrations, and force
extension,
Phase field HOACs require only a
convolution:
Z
±E N L
= ¯
d2 x 0r 2 G( x ¡ x 0) Á( x 0)
±Á( x)
-
25
Phase field HOACs: results
26
Phase field HOACs: VHR
image results
27
Phase field HOACs: tree
crown results
28
Phase field HOACs: results
29
Future

New prior models



Directed and rectilinear networks (rivers, big
cities…).
‘Gas of rectangles’;
Controlled perturbed circle.
Multiscale models;
 New algorithms (multiscale, stochastic,…);
 Parameter estimation;
 Higher-dimensions;
…

30
Thank you
31
Stability analysis
"
#
Z 2¼
¯C
2¼
F 00 dp
2
0
"
#
Z 2¼
¯C
+ a0 ¸ C 2¼+ ®C 2¼r 0 ¡
4¼
F 10 dp
2
0
¸ C 2¼r 0 + ®C ¼r 02 ¡
E C;g( ° 0 + ±° ) =
+
X
"
jak j 2 ¸ C ¼r 0 k 2 + ®C ¼
k
( Ã
¡
¯C
4¼
2
2
Ã
Z 2¼
0
+ k 2i r 0
Ã
F 20 dp +
Z 2¼
0
Z 2¼
+ k 2 r 02
0
Z 2¼
0
!
F 21 ei r 0 kpdp
!
F 23 ei r 0 kpdp
! ) #
F 24 ei r 0 kpdp
32
HOACs: EI

Linear term that favours large gradients normal
to contour.
Z
E I ;1 ( ° ) =

S1
dt °_( t ) £ r I ( ° ( t ) )
Quadratic term that favours pairs of points with
tangents and image gradients parallel or antigradients
parallel.
ZZ
E I ;2 (° ) = ¡
dt dt 0 °_ ¢_
° 0 (r I ¢r I 0) ª (j° (t) ¡ ° (t 0)j)
tangent vectors
weighting function
34
Nonlinearity in the model, not
the representation: example

Probability distribution P on R2.



P pushes forward to Q on S1.
If P is strongly peaked at r0 then
Q() ' P(r0, ).
Gradient descent with –ln(P) on R2
mimics gradient descent with -ln(Q)
on S1 (‘valley following’).
P ( r ; µ) = d2 x e¡ E ( r ;µ)
4
2
r
r
£ exp ¡ ( 4 ¡ 2 )
35
Turing stability

To avoid decay of interior and exterior, we
require stability of functions § = § 1:


2E/2 must be positive definite (E = E0 + ENL).
For prior terms, this is diagonal in the Fourier
basis:
n
o
±2 E
0
2
^
(Á
)
=
±(k;
k
)
k
(D
¡
¯
G(k))
+ 2(¸ ¨ ®)
§
0
±Á(k )±Á(k)
Gives condition on parameters.
 Better result would be existence and
uniqueness of R for ‘any’ R.
