e FAIR: Fl e xi b l Algorithms for Image Registration Jan Modersitzki Computing And Software, McMaster University 1280 Main Street West, Hamilton On, L8S 4K1, Canada [email protected] August 13, 2008 Under Construction c Jan Modersitzki 2 FAIR [August 13, 2008, 13:12] Contents 1 2 3 Introduction 1.1 Image Registration . . . . . . 1.2 Brief Outline . . . . . . . . . . 1.3 Literature and Links . . . . . 1.3.1 Image Registration 1.3.2 Optimization . . . 1.3.3 Matlab . . . . . 1.3.4 Other Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 13 14 15 15 15 16 16 FAIRConcepts 2.1 FAIR Theory . . . . . . . . . . . . . . . . . . . . 2.1.1 Images and Transformations . . . . 2.1.2 Distances and Regularization . . . . 2.2 FAIR Numerics . . . . . . . . . . . . . . . . . . 2.2.1 Discretize then Optimize . . . . . . 2.2.2 A Family of Nested Approximations 2.2.3 Optimization Techniques . . . . . . 2.3 FAIR Matlab . . . . . . . . . . . . . . . . . . . 2.3.1 Comments on Comments . . . . . . 2.3.2 Notation and Conventions . . . . . . 2.3.3 Coordinate System . . . . . . . . . . 2.3.4 Arguments, Parameter, and Defaults 2.3.5 Overwriting Default Parameter . . . 2.3.6 Matlab’s “@” Constructor . . . . . 2.3.7 FAIR Administration . . . . . . . . . 2.3.8 Memory versus Clarity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 17 18 20 20 20 21 21 21 21 21 22 22 23 23 23 24 Image Interpolation 3.1 Cells, Grids, and Numbering . . . . . . . . . . . . . . . . . . . 3.2 Next Neighbor Interpolation . . . . . . . . . . . . . . . . . . . 3.3 Linear Interpolation . . . . . . . . . . . . . . . . . . . . . . . . 3.3.1 Linear Interpolation for One Dimensional Data . . 3.3.2 Linear Interpolation for Higher Dimensional Data 3.4 Spline Interpolation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 25 29 29 29 30 31 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . c Jan Modersitzki 3.5 3.6 3.7 3.8 3.9 4 5 6 4 FAIR 3.4.1 Spline Interpolation for One Dimensional Data . . 3.4.2 Spline Interpolation for Higher Dimensional Data Derivatives of Interpolation Schemes . . . . . . . . . . . . . . 3.5.1 Derivative of the Interpolant . . . . . . . . . . . . 3.5.2 Derivative of Multivariate Interpolant . . . . . . . 3.5.3 Testing the Implementation of a Derivative . . . . Multiscale Spline Interpolation . . . . . . . . . . . . . . . . . . 3.6.1 Multiscale Interpolation in One Dimension . . . . 3.6.2 Truncating High Frequencies . . . . . . . . . . . . 3.6.3 Multiscale Interpolation in Higher Dimensions . . Multiresolution of Data . . . . . . . . . . . . . . . . . . . . . . Summarizing the Interpolation Toolbox . . . . . . . . . . . . . Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 34 35 35 36 36 37 38 40 41 42 43 45 Transforming Images by Parameterized Transformations 4.1 Translations . . . . . . . . . . . . . . . . . . . . . . . . . 4.2 Affine Linear Transformations . . . . . . . . . . . . . . . 4.3 Rigid Transformations . . . . . . . . . . . . . . . . . . . 4.4 Rotations About the Domain Center . . . . . . . . . . . 4.5 Spline Based Transformations . . . . . . . . . . . . . . . 4.6 More Bizarre Transformations . . . . . . . . . . . . . . . 4.7 Derivatives of Parameterized Transformations . . . . . . 4.8 Summarizing the Parameterized Transformations . . . . 4.9 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 49 50 51 52 52 54 54 56 57 Landmark Based Registration 5.1 Affine Linear Landmark Based Registration 5.2 Quadratic Landmark Based Registration . . 5.3 Thin-Plate-Spline Registration . . . . . . . . 5.4 Summarizing Landmark Based Registration 5.5 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59 60 61 63 64 66 Parametric Image Registration 6.1 Discretizing an Integral . . . . . . . . . . . . . . . . . . . . 6.2 Sum of Squared Differences . . . . . . . . . . . . . . . . . . 6.2.1 The Continuous SSD . . . . . . . . . . . . . . . 6.2.2 The Discretized SSD . . . . . . . . . . . . . . . 6.2.3 SSD and Parametric Transformations . . . . . 6.3 Numerical Optimization of Parametric Image Registrations 6.3.1 The Objective Function . . . . . . . . . . . . . 6.3.2 A Gauß-Newton Scheme . . . . . . . . . . . . . 6.3.3 PIR Examples . . . . . . . . . . . . . . . . . . 6.4 Experiments on Fixed Levels . . . . . . . . . . . . . . . . . 6.4.1 Rotations about the Center of the Domain . . 6.4.2 Rigid Transformations . . . . . . . . . . . . . . 6.5 Regularized Parametric Image Registration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67 68 70 70 71 72 74 75 77 79 82 83 83 84 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . [August 13, 2008, 13:12] c Jan Modersitzki FAIR 6.6 . . . . . . . . . . . . . . . . . . . . . . . . . 89 90 91 91 92 Distance Measures 7.1 Sum of Squared Differences . . . . . . . . . . . . . . . . . 7.1.1 SSD and Forces . . . . . . . . . . . . . . . . 7.1.2 Discretized SSD . . . . . . . . . . . . . . . . 7.2 Cross-Correlation . . . . . . . . . . . . . . . . . . . . . . 7.2.1 Continuous Normalized Cross-Correlation . . 7.2.2 Discretized Normalized Cross-Correlation . . 7.3 Mutual Information . . . . . . . . . . . . . . . . . . . . . 7.3.1 Estimating the Joint Density, Principles . . . 7.3.2 Estimating the Joint Density of Two Images 7.3.3 Mutual Information . . . . . . . . . . . . . . 7.3.4 Discretizing Mutual Information . . . . . . . 7.4 Normalized Gradient Field . . . . . . . . . . . . . . . . . 7.4.1 Continuous Normalized Gradient Field . . . 7.4.2 Discretized Normalized Gradient Field . . . . 7.5 Derivatives of Distance Measures . . . . . . . . . . . . . 7.6 Summarizing the Distance Measures Toolbox . . . . . . . 7.7 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95 95 96 97 97 97 99 100 102 105 106 106 107 107 109 110 111 116 Regularizer 8.1 Ill-posedness . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.2 L2 -Norm Based Regularizer . . . . . . . . . . . . . . . . . . 8.3 Discretizing L2 -Norm Based Regularizer . . . . . . . . . . . 8.3.1 Discretizing First Order Derivatives . . . . . . . 8.3.2 The Discretized Diffusion and Elastic Operators 8.3.3 The Discretized Curvature Operator . . . . . . . 8.3.4 The Discretized L2 -Norm Based Regularizer . . 8.4 Matrix Free Operations . . . . . . . . . . . . . . . . . . . . . 8.4.1 Matrix Free Elastic Operator . . . . . . . . . . . 8.4.2 Matrix Free Curvature Operator . . . . . . . . . 8.5 Summarizing the Regularization . . . . . . . . . . . . . . . . 8.6 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117 118 120 122 122 126 126 128 128 129 129 130 131 6.7 6.8 7 8 Multilevel Based Optimization . . . . . . . . . . . . 6.6.1 SSD and Rigid Transformations . . . . 6.6.2 SSD and Affine Linear Transformations Summarizing Parametric Image Registrations . . . Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 Numerical Methods for Non-Parametric Image Registration 133 9.1 The NPIR Objective Function . . . . . . . . . . . . . . . . . . . 133 9.1.1 Staggered Grids and Image Interpolation . . . . . . 134 9.1.2 Matrix Free Operations . . . . . . . . . . . . . . . . 135 10 Discussion and Examples [August 13, 2008, 13:12] 137 5 c Jan Modersitzki 11 12 13 14 15 6 General Tools 11.1 Get Cell Centered Grids . . . . . . . . . 11.2 Visualize 2D Data . . . . . . . . . . . . . 11.3 Checking Derivatives . . . . . . . . . . . 11.4 Set-up Data from an Ultrasound Image . 11.5 Set-up the Hand Example . . . . . . . . 11.6 Set-up the HNSP Example . . . . . . . . 11.7 Set-up the PET/CT Example . . . . . . 11.8 Setting Persistent Options . . . . . . . . 11.9 Creating Multilevel Data Representation FAIR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139 139 140 140 141 142 143 144 145 147 Image Interpolation Tools 12.1 Linear Interpolation in 1D using Matlab 12.2 Linear Interpolation in 2D using Matlab 12.3 Linear Interpolation in 3D using Matlab 12.4 Linear Interpolation in 1D, final . . . . . . 12.5 Linear Interpolation in 2D, final . . . . . . 12.6 Linear Interpolation in 3D, final . . . . . . 12.7 “Mother” Spline Function . . . . . . . . . . 12.8 Spline Interpolation in 1D, final . . . . . . 12.9 Spline Interpolation in 2D, final . . . . . . 12.10 Spline Interpolation in 3D, final . . . . . . 12.11 Computing the Spline Coefficients . . . . . 12.12 The Interpolator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149 149 149 150 150 151 151 152 152 153 154 155 156 Parameterization Tools 13.1 Translations in 2D . . . . . . . . . . . . . . 13.2 Affine Linear Transformations in 2D . . . . 13.3 Rigid Transformations in 2D . . . . . . . . 13.4 Rotations around the domain center in 2D 13.5 Spline Based Transformations in 2D . . . . 13.6 Affine Linear Transformations in 3D . . . . 13.7 Rigid Transformations in 3D . . . . . . . . 13.8 Generic Parameteric Transformation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157 157 157 158 158 159 160 160 161 Landmark Registration Tools 14.1 Plot Landmark Registration Results . . . 14.2 Affine Linear Landmark Registration . . 14.3 Quadratic Landmark Registration . . . . 14.4 Get Thin-Plate-Spline Coefficients . . . . 14.5 Evaluate Thin-Plate-Splines . . . . . . . 14.6 Thin-Plate-Spline Registration . . . . . . 14.7 Landmark Registration . . . . . . . . . . 14.8 Computing and Evaluating TPS Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163 163 163 164 164 164 165 165 165 Parametric Image Registration . . . . . . . . 167 [August 13, 2008, 13:12] c Jan Modersitzki FAIR 15.1 15.2 15.3 15.4 15.5 16 Objective Function . . . . . . . . . . . . . . . . . . Regularization . . . . . . . . . . . . . . . . . . . . . Armijo Line Search . . . . . . . . . . . . . . . . . . Gauß-Newton Based Parametric Image Registration MLPIR: Multilevel Parametric Image Registration . Distance Measures 16.1 Sum of Squared Differences . . 16.2 Normalized Cross Correlation 16.3 Joint Density Estimator . . . 16.4 Mutual Information . . . . . . 16.5 Normalized Gradient Fields . 16.6 FAIR Distance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167 168 169 170 172 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175 175 175 176 177 179 180 17 Regularization 181 17.1 The Elastic Matrix . . . . . . . . . . . . . . . . . . . . . . . . . 183 17.2 The Curvature Matrix . . . . . . . . . . . . . . . . . . . . . . . 184 17.3 FAIR Regularizer . . . . . . . . . . . . . . . . . . . . . . . . . . 186 18 Non-Parametric Image Registration 18.1 Prolongation Operator . . . . . . 18.2 Staggered to Cell-center Operator 18.3 Objective Function . . . . . . . . 18.4 Staggered to Cell-Center Operator [August 13, 2008, 13:12] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187 187 189 191 192 7 c Jan Modersitzki 8 FAIR [August 13, 2008, 13:12] FAIR Listing FAIR FAIR FAIR FAIR FAIR FAIR FAIR FAIR FAIR FAIR FAIR FAIR FAIR FAIR FAIR FAIR FAIR FAIR 1: Flexible Algorithms for Image Registration . . . . . . . . . 2: Interpolation Toolbox . . . . . . . . . . . . . . . . . . . . . 3: Parametric Transformations . . . . . . . . . . . . . . . . . . 4: Landmark Based Registration . . . . . . . . . . . . . . . . . 5: Thin-Plate-Spline Interpolation . . . . . . . . . . . . . . . . 6: Thin-Plate-Spline Approximation . . . . . . . . . . . . . . . 7: Sum of Squared Difference Distance Measure (SSD) . . . . . 8: Discretized Sum of Squared Difference (SSD) . . . . . . . . 9: Objective Function for Parametric Image Registration . . . 10: Parametric Image Registration (PIR) . . . . . . . . . . . . 11: The normalized cross-correlation distance measure (NCC) 12: Parzen-Window Estimator . . . . . . . . . . . . . . . . . . 13: The mutual information distance measure (MI) . . . . . . 14: The normalized gradient field distance measure (NGF) . . 15: Distance Measure Toolbox . . . . . . . . . . . . . . . . . . 16: L2 -Norm Based Continuous Regularizer . . . . . . . . . . . 17: Regularizer Toolbox . . . . . . . . . . . . . . . . . . . . . . 18: Objective Function for Non-Parametric Image Registration 9 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 45 56 60 63 64 70 72 77 79 99 105 106 109 111 120 130 134 c Jan Modersitzki 10 FAIR [August 13, 2008, 13:12] List of Figures 1.1 Modified slices from CT scans of a human knee. . . . . . . . . . . 14 2.1 2.2 Visualizations of an image and a transformation. . . . . . . . . . . Transforming images. . . . . . . . . . . . . . . . . . . . . . . . . . 18 19 3.1 3.2 3.3 3.4 3.5 3.6 3.7 3.8 3.9 3.10 3.11 3.13 3.14 3.12 Discretization of a 1D domain Ω = (0, ω) ⊂ R. . . . . . . Discretization of a 2D domain Ω = (0, ω1 ) × (0, ω2 ) ⊂ R2 . Next neighbor interpolation in 1D (dashed line). . . . . . Linear interpolation in 1D. . . . . . . . . . . . . . . . . . Linear interpolation in 2D. . . . . . . . . . . . . . . . . . “Mother” spline b = b0 . . . . . . . . . . . . . . . . . . . . Spline interpolation in 1D. . . . . . . . . . . . . . . . . . Spline interpolation in 2D. . . . . . . . . . . . . . . . . . A typical result of checkDerivative. . . . . . . . . . . . Spline approximation in 1D. . . . . . . . . . . . . . . . . Oscillations in spline interpolation. . . . . . . . . . . . . . Multiresolution of an ultrasound image. . . . . . . . . . . Interpolation on duty. . . . . . . . . . . . . . . . . . . . . Spline approximations in 2D. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 27 29 30 31 32 33 35 37 39 40 43 44 48 4.1 4.2 Translation, rigid, and affine linear transformations. . . . . . . . . Spline based transformation and more. . . . . . . . . . . . . . . . 51 53 5.1 5.2 5.3 Reference and template with corresponding landmarks. . . . . . . Linear and quadratic landmark based registrations. . . . . . . . . Thin-Plate-Spline registration with various θ’s. . . . . . . . . . . . 60 62 65 6.1 6.2 6.3 6.4 6.5 6.6 6.7 6.8 Midpoint quadrature rule in 1D. . . . . . . Quadrature on duty. . . . . . . . . . . . . . SSD versus rotations, coarse. . . . . . . . . SSD versus rotations, fine. . . . . . . . . . SSD versus translations. . . . . . . . . . . . Plots from Parametric Image Registration. PIR for SSD and rotations, m = (32, 16). . PIR for SSD and rotations, m = (256, 128). 68 71 73 73 74 80 84 84 11 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . c Jan Modersitzki 12 FAIR 6.9 6.10 6.11 6.12 6.13 6.14 6.15 6.16 6.17 6.18 6.19 PIR iteration histories for SSD and rotations. . . . . . . . . . PIR for SSD and rigid transformations, m = (32, 16). . . . . PIR for SSD and rigid transformations, m = (256, 128). . . . PIR iteration histories for SSD and rigid transformations. . . PIR results for SSD and spline transformation. . . . . . . . . Regularized PIR for SSD and spline transformations. . . . . Multilevel representation of data and images. . . . . . . . . . MLPIR iteration history for SSD and rigid transformations . MLPIR results for SSD and rigid transformations . . . . . . MLPIR iteration history for SSD and affine transformations . MLPIR results for SSD and affine transformations . . . . . . 7.1 7.2 7.3 7.4 7.5 7.6 7.7 7.8 7.9 7.10 Force field of SSD. . . . . . . . . . . . . . . . . . . . . . . . . . . . 96 Distance Measures for mon-modal images. . . . . . . . . . . . . . . 98 T1/T2 weighted MRI’s of a head. . . . . . . . . . . . . . . . . . . 99 Distance Measures for multi-modal images. . . . . . . . . . . . . . 100 Histograms of discretizations from n samples of a univariate function.103 Spline based Parzen-Window kernel k(·, σ). . . . . . . . . . . . . . 104 Parzen-Window density estimators. . . . . . . . . . . . . . . . . . 105 NGF for MRI’s of a head. . . . . . . . . . . . . . . . . . . . . . . . 108 Distance Measures for PET/CT images. . . . . . . . . . . . . . . . 113 Distance Measures for PET/CT images. . . . . . . . . . . . . . . . 114 8.1 8.2 8.3 8.4 Two images showing squares with texture. . . . . Cell centered finite difference approximation of a Staggered Grids in 2D. . . . . . . . . . . . . . . Staggered Grids in 3D. . . . . . . . . . . . . . . . . . . . . derivative. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85 85 86 86 87 88 89 90 90 91 92 119 122 123 124 [August 13, 2008, 13:12] List of Tables 4.1 A more efficient implementation of the affine linear transformation using a persistent variable Q. . . . . . . . . . . . . . . . . . . . . . 55 6.1 6.2 6.3 6.4 Implementations of the rotation and translation examples. Paramatric Image Registration on a fixed level. . . . . . . . Parametric Image Registration Results. . . . . . . . . . . . Parametric Spline Registration. . . . . . . . . . . . . . . . . 75 80 81 87 7.1 Calling differentdistance measures. . . . . . . . . . . . . . . . . . . 112 8.1 8.2 8.3 8.4 Staggered grid based discrete derivatives. . . . . . Staggered grid based discrete derivative operators. Discretized elastic operator in matrix form. . . . . Discretized curvature operator in matrix form. . . 13 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125 125 127 127
© Copyright 2026 Paperzz