Registration I
NBE-E4010 Medical Image Analysis
Lecture VII, Oct. 27th 2015
Mika Pollari, [email protected]
Goals and Contents for the Lecture
• Why registration is needed – applications
• Registration Framework
• Transformations
• Rigid point-based registration
• How do we get the point-sets (extrinsic marks, intrinsic landmarks, intrinsic
segmentation based).
• Closed form solution and Iterative Closest Point (ICP) method.
• When can we apply closed-form solution and when we need ICP or other
iterative approach.
• Accuracy in Point-based registration
Why do we need registration
• Registration problems can be divided into
• Intrasubject Registration Problems
• Intersubject Registration Problems
• Atlas based Registration Problems
Examples of Intra-Subject Registration
• Combine information from different imaging modalities.
• Determine changes that occur over time e.g. treatment outcome
verification.
• Treatment Planning e.g. radiation dose calculation (attenuation
coeffisients from CT and the tumour from MR).
An Example of Intra-Subject Multimodal
Registration
"CT-PET" by User:MBq - Own work. Licensed under CC BY-SA 3.0 via Commons https://commons.wikimedia.org/wiki/File:CT-PET.jpg#/media/File:CT-PET.jpg
An example of RFA treatment planning
Registration Framework
Cost Function
Similarity Metric
Fixed Image
Moving Image
Image
Interpolator
Cost Function
Optimizer
Transform
Resample Image
Input:
Today our input is two point sets. How do we generate these point
sets?
Extrinsic Markers
Image from: izimed.com
Image from: Toennis: Guide to Medical Image Analysis
Extrinsic Markers
1) Non-invasive; skin markers (finducials), moulds, dental adapters etc…
2) Invasive: screws (finducials); stereotactic frames
Extrinsic Markers Properties
• Markers should be visible in used imaging modalities
• Markers should stay in precise place between imaging session
• Some Problems:
•
•
•
•
Need should be recognized well in advance
Invasiveness screws, stereotactic frames
Motion (skin markers)
Sterility (surgical applications)
Examples of Intrinsic Landmarkers
Kenneth L. Weiss et al. AJNR Am J
Neuroradiol 2003;24:922-929
Toennis: Guide to Medical Image Analysis
Intrinsic Landmarks
1. Anatomical i.e. salient and accurately locatable points of the anatomy (typically) annotated by user.
2. Geometrical i.e. points at the locus of the optimum of some geometric property, e.g. local curvature
extrema, corners detected in an automatic fashion (Recal lecture 5)
Segmentation Based Registration
• Segmentation can be used to aid registration. We can segment the
object that we want to register from both images and use extracted
surface.
• From Surface we can use
• All Points, all vertex points.
• Point subset such as geometric features extracted from the surface: salient
points (e.g. high curvature points); curves (e.g. ridges).
Features extracted from the surface
• Points, all vertex points.
• Geometric features extracted from the surface: salient points (e.g.
high curvature points); curves (e.g. ridges).
Transformations
J. B. A. Maintz and M. A. Viergever: A survey of medical image registration,
Medical Image Analysis (1998) volume 2, number 1, pp 1–36
Rigid Transformation
Rigid Transformation preserved the shape and size of the object – only
the position (rotated, translated) is changed.Lines map to lines and the
angle between lines is preserved
𝑦𝑦𝑖𝑖 = 𝑅𝑅𝑥𝑥𝑖𝑖 + 𝑡𝑡;
𝑅𝑅 𝑟𝑟𝑟𝑟𝑟𝑟𝑟𝑟𝑟𝑟𝑟𝑟𝑟𝑟𝑟𝑟 𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚,
𝑡𝑡 𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡 𝑣𝑣𝑣𝑣𝑣𝑣𝑣𝑣𝑣𝑣𝑣𝑣
Rotation matrix
Rotation matrix can be
represented as a combination
of three rotation matrices
𝑟𝑟
𝑟𝑟
𝑟𝑟
1
2
3
x-axis rotation, angle 𝛼𝛼1
y-axis rotation, angle 𝛼𝛼2
z-axis rotation, angle 𝛼𝛼3
Transformation Properties
• Transformations are not commutative: 𝑇𝑇2 °𝑇𝑇1 𝑥𝑥 ≠ 𝑇𝑇1 °𝑇𝑇2 (𝑥𝑥)
• Also inside the single transformation the order cannot be changes for
instance in Rigid case: 𝑅𝑅𝑥𝑥𝑖𝑖 + 𝑡𝑡 ≠ 𝑅𝑅 𝑥𝑥𝑖𝑖 + 𝑡𝑡 ; 𝑅𝑅𝑥𝑥 𝑅𝑅𝑦𝑦 𝑅𝑅𝑧𝑧 ≠ 𝑅𝑅𝑦𝑦 𝑅𝑅𝑥𝑥 𝑅𝑅𝑧𝑧
General Matrix Transformation
• General Transformation using single Matrix:
General Affine Transformation
• A coordinate transformation of the form:
x’ = axx x + axy y + axz z + bx ,
y’ = ayx x + ayy y + ayz z + by ,
z’ = azx x + azy y + azz z + bz ,
 x'   a xx
  
 y '   a yx
 z'  =  a
   zx
 w  0
  
a xy
a yy
a zy
0
a xz
a yz
a zz
0
bx  x 
 
b y  y 
bz  z 
 
1  1 
Properties of affine transformation:
– translation, scaling, shearing, rotation (or any combination of them) are
examples affine transformations.
– Lines and planes are preserved.
– parallelism of lines and planes are also preserved, but not angles and
length.
19
Transformation Domain
• Global (the most common case), the same transformation is applied
for each voxel in the image
• Local, we apply different transformations for different regions of the
image
Similarity – Cost
The used similarity (inverse of cost function) is dependent on the
feature which is driving the registration. Today, we are considering only
points thus the cost that we want to minimize is
min Ξ2 = 𝑥𝑥𝐵𝐵 − 𝑅𝑅𝑥𝑥𝐴𝐴 + 𝑡𝑡
2
Cost Function Optimizer
• Typically cost function is minimized (=similarity maximized) using
iterative optimization approach. Today, we can also found noniterative (closed-form solution) for the simplest kind of registration
problem. “Optimizers” considered today:
1. Closed-form solution (simple case)
2. Iterative Closest Point method
A closed-form solution for point pairs
(Arun, IEEE Trans PAMI, 1987)
We assume that point pairs are known. The method has four steps:
1. Translate pointset into the centre of mass coordinate system
2. Separate translations and rotations
3. Calculate Rotation matrix using SVD-approach
4. Solve the translation vector
Arun: step 1 translate to center of mass
coordinates
𝐵𝐵 = 1/𝑁𝑁 ∑𝑁𝑁 𝑥𝑥 𝐵𝐵
• 𝑥𝑥𝑐𝑐𝑐𝑐
𝑖𝑖=1 𝑖𝑖
𝐴𝐴 = 1/𝑁𝑁 ∑𝑁𝑁 𝑥𝑥 𝐴𝐴
• 𝑥𝑥𝑐𝑐𝑐𝑐
𝑖𝑖=1 𝑖𝑖
𝐵𝐵
• 𝑝𝑝𝑖𝑖𝐵𝐵 = 𝑥𝑥𝑖𝑖𝐵𝐵 − 𝑥𝑥𝑐𝑐𝑐𝑐
𝐴𝐴
• 𝑝𝑝𝑖𝑖𝐴𝐴 = 𝑥𝑥𝑖𝑖𝐴𝐴 − 𝑥𝑥𝑐𝑐𝑐𝑐
Separate translation and rotation
𝑁𝑁
𝐶𝐶𝑓𝑓 = �
𝑖𝑖=1
=
=
𝑥𝑥𝑖𝑖𝐴𝐴
∑𝑁𝑁
𝑖𝑖=1
∑𝑁𝑁
𝑖𝑖=1
−
𝑝𝑝𝑖𝑖𝐴𝐴
𝑝𝑝𝑖𝑖𝐴𝐴
𝑅𝑅𝑥𝑥𝑖𝑖𝐵𝐵
+
−
𝐴𝐴
𝑥𝑥𝑐𝑐𝑐𝑐
+ 𝑡𝑡
−
2
𝑅𝑅𝑝𝑝𝑖𝑖𝐵𝐵
𝐵𝐵 2
𝑅𝑅𝑝𝑝𝑖𝑖 ,
−
𝐵𝐵
𝑅𝑅𝑥𝑥𝑐𝑐𝑐𝑐
− 𝑡𝑡
2
𝐴𝐴 = 𝑅𝑅𝑥𝑥 𝐵𝐵 + 𝑡𝑡
as 𝑥𝑥𝑐𝑐𝑐𝑐
𝑐𝑐𝑐𝑐
Solve the Rotation
• Using singular value decomposition
• 𝐻𝐻 =
𝐵𝐵
∑𝑁𝑁
𝑝𝑝
𝑖𝑖=1 𝑖𝑖
𝑈𝑈ΛV T
𝐴𝐴 𝑇𝑇
𝑝𝑝𝑖𝑖
• 𝐻𝐻 =
• 𝑋𝑋 = 𝑉𝑉𝑈𝑈 𝑇𝑇
• If 𝑑𝑑𝑑𝑑𝑑𝑑 𝑋𝑋 = 1, 𝑡𝑡𝑡𝑡𝑡𝑡𝑡 𝑅𝑅 = 𝑋𝑋
• If 𝑑𝑑𝑑𝑑𝑑𝑑 𝑋𝑋 = −1, 𝑡𝑡𝑡𝑡𝑡𝑡𝑡 𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠 𝑖𝑖𝑖𝑖 𝑎𝑎𝑎𝑎 𝑟𝑟𝑟𝑟𝑟𝑟𝑟𝑟𝑟𝑟𝑟𝑟𝑟𝑟𝑟𝑟𝑟𝑟𝑟𝑟
Solve the translation
𝐴𝐴 − 𝑅𝑅𝑥𝑥 𝐵𝐵
𝑡𝑡 = 𝑥𝑥𝑐𝑐𝑐𝑐
𝑐𝑐𝑐𝑐
Iterative Closest Point (ICP) method
The number of points differ #𝑋𝑋 𝐴𝐴 ≠ #𝑋𝑋 𝐵𝐵 or we do not know the point
correspondence.
Transformation: rigid-body transformation
𝑑𝑑 𝑥𝑥𝐴𝐴 , RkxB + t k = min 𝑑𝑑(𝑥𝑥𝐴𝐴 , 𝑅𝑅𝑘𝑘 𝑥𝑥𝐵𝐵 + 𝑡𝑡𝑘𝑘 )
𝑥𝑥𝐵𝐵 ∈𝑋𝑋𝐵𝐵
We have now selected for each point from set A a corresponding point
in set B which minimize the distance to reference point after applying
the rigid-transformation with current parameter estimates.
Now we have equal number of points and point correspondence
We can use close-form solutions to get better estimates for next
iteration (k+1)
Point-Based Registration Accuracy (rigid)
• Point-based rigid registration is most often used in registering brain
images or in different brain navigation systems. It is vital to
understand how, registration accuracy depends on the number and
configuration of the chosen landmarks.
• Point-based registration for rigid-body objects can be approximated.
Finducial Registration Error (FRE)
• The RMS error for all landmarks that were used in determing the
transformation (How well the least squares fit succeeded).
• 𝑒𝑒𝑖𝑖 =
• 𝐹𝐹𝐹𝐹𝐹𝐹 =
𝑥𝑥𝑖𝑖𝐴𝐴
−
𝐵𝐵 2
𝑥𝑥𝑖𝑖
1 𝑁𝑁
∑𝑖𝑖=1 𝑒𝑒𝑖𝑖2
𝑁𝑁
+
𝑦𝑦𝑖𝑖𝐴𝐴
−
𝐵𝐵 2
𝑦𝑦𝑖𝑖
+
𝑧𝑧𝑖𝑖𝐴𝐴
−
𝐵𝐵 2
𝑧𝑧𝑖𝑖
Finducial Localization Error (FLE)
• The error what we make when localizing the landmark from the
image
•
𝐹𝐹𝐿𝐿𝐸𝐸 2
=
𝐹𝐹𝐹𝐹𝐸𝐸 2
∗
𝑁𝑁
𝑁𝑁−2
Target Registration Error (TRE)
The error for any interesting object point
𝑇𝑇𝑇𝑇𝑇𝑇 ≈
𝐹𝐹𝐹𝐹𝐸𝐸 2
𝑁𝑁
1
1 𝑑𝑑𝑥𝑥2
+
3 𝑓𝑓𝑥𝑥2
+
2
𝑑𝑑𝑦𝑦
𝑓𝑓𝑦𝑦2
+
𝑑𝑑𝑧𝑧2
𝑓𝑓𝑧𝑧2
,
𝑊𝑊𝑊𝑊𝑊𝑊𝑊𝑊𝑊 𝑑𝑑{𝑥𝑥,𝑦𝑦,𝑧𝑧} ; 𝑓𝑓 𝑥𝑥,𝑦𝑦,𝑧𝑧 𝑎𝑎𝑎𝑎𝑎𝑎 𝑡𝑡𝑡𝑡𝑡 𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡 𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝 𝑑𝑑 𝑎𝑎𝑎𝑎𝑎𝑎 𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚 𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝 𝑓𝑓 𝑅𝑅𝑅𝑅𝑅𝑅
𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑𝑑 𝑓𝑓𝑓𝑓𝑓𝑓𝑓𝑓 𝑡𝑡𝑡𝑡𝑡 𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚 𝑎𝑎𝑎𝑎𝑎𝑎𝑎𝑎 𝑜𝑜𝑜𝑜 𝑡𝑡𝑡𝑡𝑡 𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚 𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝
Registration error depends on the number of the used markers and
configuration of those markers.
Summary
• Applications
• Registration Framework
• Rigid Transformation
• Rigid point-based registration
• How do we get the point-sets (extrinsic marks, intrinsic landmarks, intrinsic
segmentation based).
• Closed form solution and Iterative Closest Point (ICP) method.
• When can we apply closed-form solution and when we need ICP or other
iterative approach.
• Accuracy in point-based registration