Prof. Samir Kumar Bandyopadhyay, Int. J. Comp. Tech. Appl., Vol 1 (1), 16-23
An Approach for registration method to find corresponding mass lesions in
temporal mammogram pairs
Prof. Samir Kumar Bandyopadhyay
Dept. of Computer Science & Engineering,
University of Calcutta,
92 A.P.C. Road, Kolkata – 700009
Email: [email protected]
Abstract
Radiologists generally use multiple mammographic views to detect and characterize suspicious
regions. When radiologists discover a suspicious lesion in one view, they try to find a corresponding lesion in
the other views. Views from different projections, typically cranio caudal (CC) and medio lateral oblique
(MLO) views, allow for a better realization of the lesion. Most current computer aided detection (CAD) systems
differ considerably from radiologists in the way they use multiple views. These systems do not combine
information from available views but instead analyse each view separately. Given the positive effect of multiview systems on radiologists' performance we expect that fusion of information from different views will
improve CAD systems as well.
Such multi-view CAD programs require regional registration methods to find corresponding regions
in all available views. In this paper we concentrate on developing such a method for corresponding mass lesions
in prior and In other words, starting from a current image containing a mass lesion, the method aims at locating
the same mass lesion in the prior image.
The method was tested on a set of 412 cancer cases. In each case a malignant mass, architectural
distortion or asymmetry was annotated. In 92% of these cases the candidate mass detections by
CAD included the cancer regions in both views. It was found that in 82% of the cases a correct link between the
true. Positive regions in both views could be established by our method.
Key words: Multiple View, Computer-Aided Detection, masses, mammography
Introduction
Most of the development of
Computer-Aided
Detection
(CAD)
systems has been based on the analysis of
single views. To decrease the number of
missed cancer cases during breast cancer
screening, CAD systems have been
developed. These systems are intended to
aid the radiologist by prompting suspicious
regions. To decrease the number of false
positives and to improve consistency, there
is a lot of interest to develop CAD
techniques that use multiple view
information (temporal, bilateral, or two
views of the same breast). Radiologists in
breast cancer screening are trained to use
comparisons of the left and right breast to
identify suspicious asymmetric densities.
Views from previous screening rounds are
used to detect developing densities. It is
also known that screening with two
mammographic
views,
mediolateral
oblique (MLO) and craniocaudal (CC),
improves the detection accuracy of
abnormalities in the breast, which can be
16
Prof. Samir Kumar Bandyopadhyay, Int. J. Comp. Tech. Appl., Vol 1 (1), 16-23
explained by the fact that two projections
allow a better estimation of the conspicuity
of lesions and may reveal lesions hidden
by glandular tissue in one of the
projections.
Breast cancer is the most common
malignancy among women in Western
society, with incidence rates continuing
their upward trend, increasing by 70% cent
since 1971, and by 15% in the 10 years to
2000 in England. It is the most common
cause of cancer death in women. There are
large between-individual differences in
size and asymmetry of breasts and this
could be indicative of differences in
developmental stability, and possibly
disease predisposition.
Breast cancer is the most common
malignancy among women in Western
society, with incidence rates continuing
their upward trend, increasing by 70% cent
since 1971, and by 15% in the 10 years to
2000 in England. It is the most common
cause of cancer death in women. There are
large between-individual differences in
size and asymmetry of breasts and this
could be indicative of differences in
developmental stability, and possibly
disease predisposition.
The contrast enhancement phase is
done using the Contrast Limited Adaptive
Histogram
Equalization
(CLAHE)
technique, which is a special case of the
histogram equalization technique [7] that
functions adaptively on the image to be
enhanced. The pixel's intensity is thus
transformed to a value within the display
range proportional to the pixel intensity's
rank in the local intensity histogram.
CLAHE [8] is a refinement of Adaptive
Histogram Equalization (AHE) where the
enhancement calculation is modified by
imposing a user-defined maximum, i.e. clip
level, to height of the local histogram and
thus
on
the
maximum
contrast
enhancement factor. The enhancement is
there by reduced in very uniform areas of
the
image,
which
prevent
over
enhancement of noise and reduces the
edge-shadowing effect of unlimited AHE
[9-11].
The CLAHE method seeks to
reduce the noise and edge-shadowing
effect produced in homogeneous areas and
was originally developed for medical
imaging [12]. This method has been used
for enhancement to remove the noise and
reduces the edge-shadowing effect in the
pre-processing of digital mammogram
[12].
The CLAHE operates on small
regions in the image called tiles rather than
the entire image. Each tile’s contrast is
enhanced, so that the histogram of the
output region approximately matches the
uniform
distribution
or
Rayleigh
distribution or exponential distribution.
Distribution is the desired histogram shape
for the image tiles. The neighbouring tiles
are then combined using bilinear
interpolation to eliminate artificially
induced
boundaries.
The
contrast,
especially in homogeneous areas, can be
limited to avoid amplifying any noise and
reduce edge-shadowing effect that might
be present in the image;
Review Works
Lau et al (1991) aligned breast
areas using three control points on the
boundary for guiding the alignment
process. These control points were the
nipple point and two corner points between
the breast boundary and the chest wall. The
alignment of the left and right breast areas
was achieved in four steps: rotation of one
mammographic image to match the breast
orientations, alignment of both images to
the point Cm, transformation of one image
to obtain ΘL = ΘR and compensation for
possible differences in size and shape [1].
Yin et al. (1994) extracted breast
border and nipple position as landmarks for
the alignment of left and right breast. They
identified breast border by a four-point
17
Prof. Samir Kumar Bandyopadhyay, Int. J. Comp. Tech. Appl., Vol 1 (1), 16-23
connectivity tracker and nipple position
was identified based on the presence of a
thicker skin line and greater subcutaneous
parenchymal opacity around the nipple
position.
Image
registration
was
accomplished in two steps: determining a
constrained
correspondence
between
locations along the borders of the two
breasts and matching the established
corresponding points of the right breast
image with those of the left breast image
using the least-square methods that
involved translation and rotation [2].
Méndez et al. (1998) also used
nipple and breast border as a reference
point for the alignment of the breasts. The
coordinates of the detected nipples of both
images determined the displacement and
the coordinates of the points along the
detected breast borders were used to
determine the angle of rotation to achieve
correspondence. The left breast image was
rotated according to the angle of rotation so
that the left breast border matched the right
breast border [3].
Stamatakis et al. (1996) aligned left
and right breast using a single reference, a
point of maximum curvature on the breast
curve. They normalized the images to
minimize differences in illumination
between images before comparison [3].
Wirth et al. (1999) proposed a
method, which accounts for some of the
non- rigid-body characteristics associated
with breasts. They selected corresponding
control points between left and right
mammographic image from the breast
contour and according to these control
points they applied a multiquadric radial
basis function (RBF) to transform right
mammographic image to match relative the
left mammographic image [4].
Good et al. (2003) applied techniques for
automatically making the appropriate local
image corrections, based on the variations
in tissue thickness during breast
compression. Afterwards they used a
nonlinear geometrical transformation,
which geometrically deformed the images
to match a semicircular template [6].
Georgsson (2003) proposed method for
bilateral registration based on anatomical
features and assumptions of how the
female breast is deformed under
compression. He established an anatomical
coordinate system defined with pectoral
muscle and nipple. The coordinate system
was used to divide breast into two boxes
containing upper and lower part of the
breast. He independently scaled each of
these boxes to match the bilateral
counterpart in size and transformed the
skin lines of the two breasts to match each
other [5].
Proposed Method
The initial detection step results in
a number of suspect image locations. Each
of the detected peaks is used as seed point
for region segmentation, based on dynamic
programming. For each region, features
are calculated that describe the position of
a region in the breast for instance, the
distance to the pectoral and the skin,
region size, contrast, texture, compactness,
and acutance measures.
For every region in the MLO view a
search area is defined in the CC view
based on the distance to the nipple. For
every candidate region in the search area
in the CC view, features are calculated that
compare both regions Radiologists use the
distance to the nipple to correlate a lesion
in the MLO and CC view. It is generally
believed that this distance remains fairly
constant. Therefore, we use this distance to
define an annular search area in the CC
view. For all points within the search area
in the CC view, the distance to the nipple
is comparable to the distance between the
candidate region and the nipple in the
MLO view.
To set the search area width, we
used an annotated database containing 373
MLO/CC image pairs with a mass lesion
that is visible in both views. For a lesion in
the MLO view the corresponding lesion in
the CC view is within the search area if
18
Prof. Samir Kumar Bandyopadhyay, Int. J. Comp. Tech. Appl., Vol 1 (1), 16-23
the radial distances in both views do not
deviate too much, i.e., the difference in
radial distance to the nipple is less than
half the search area width. It presents the
percentage of lesions in the CC view that
is within the search area for varying width.
Based on this, we set the width of the
search area for all cases to 48 mm. The
nipple location was roughly estimated
using a simple approach in which we
assumed that the nipple is the point on the
skin contour with the largest distance to
the chest or the pectoral muscle for the
MLO views.
Our regional registration method
comprises three steps. First, we align both
images. Second, we denote for each mass
lesion on the current view a search area on
the prior view in which the mass lesion is
most likely to be located. In the third step
we combine three registration measures to
determine the best location inside the
search area. Finally, we choose this
location as estimate for the centre of the
prior mass lesion.
More specifically, in the third step
we apply the following three registration
measures. The first measure represents the
likelihood that a mass is present, i.e. the
mass likelihood. As second measure we
use Pearson's correlation coefficient that
measures the similarity between the mass
on the current view and a candidate region
for the corresponding mass on the prior
view. We evaluate the different template
shapes on the performance of this
correlation measure and select the best
performing shape as our second
registration measure. The last measure is a
distance criterion that gives preference to
locations near an initial estimate.
Pertaining to the running time of the
method, we also provide a fast variant of
this registration method in which the
measures are applied sequentially.
We compare the performance of this
method with methods that use only one
registration measure. For this purpose we
use a dataset consisting of 389 temporal
mammogram pairs all containing a visible
mass on both prior and current views.
Finally,
we
investigate
possible
shortcomings of each method by
comparing the registration performance on
different subsets including benign and
malignant masses, and masses that are
subtle.
In the process of Binary Homogeneity
Enhancement Algorithm (BHEA) for
digital mammogram each row of
mammogram image is treated as an array
of data. First step of the process is to
determine middle position of array. Value
of middle position of array is subtracted
from all the element of array starting from
first position to the last position of array
and check with the maximum difference
threshold (MDT) value, which is constant
threshold determine by observation. If
result of any subtraction is greater than the
MDT, the array will be divided into two
equal subsets along middle position and
the first and last positions of the two
subsets will be pushed to stack. Otherwise,
the value of middle position will be
propagated to all other position after
modifying value using uniform colour
quantization technique in colour space
breaking in eight level scales. The process
will be continued, popping the start and
end position subset array from the stack
and repeat the aforesaid process. The
process will be continued until the stack is
empty.
Registration Methods
The registration method based on
mass likelihood selects the location with
the highest mass likelihood value as
estimate for the mass location on the prior
image. To this end we assign all locations
inside the search area a mass likelihood
value based on the outcome of a pixel level
mass detection algorithm. This algorithm
calculates at each location inside the breast
19
Prof. Samir Kumar Bandyopadhyay, Int. J. Comp. Tech. Appl., Vol 1 (1), 16-23
area two features for the detection of
stellate lesions and two features for the
detection of focal masses. Below we will
shortly describe these features of features
to detect stellate lesions. This spiculation
features are based on the idea that stellate
lesions show a patterns of lines directed
toward the centre pixel of a lesion. We
first estimate the line orientation at each
location inside the image using directional
second order Gaussian derivatives. We
then denote a neighbourhood around each
pixel in the image. We call this pixel the
centre pixel. To estimate whether a
spiculated lesion is present we derive the
following two features from the map of
line
orientations
inside
this
neighbourhood. The first feature is a
normalized measure for the fraction of
pixels with a line orientation directed
toward the centre pixel. The second feature
calculates to what extent the pixels with a
line orientation toward the centre pixel are
equally distributed in all directions.
Features to detect focal mass
lesions for the detection of masses we use
a similar approach as for the detection of
spicules. Instead of determining the line
orientations we now calculate the gradient
orientation at each location in the image. If
a mass is present pixels in a
neighbourhood of the centre pixel will
have a gradient orientation toward the
centre. Otherwise a random gradient
direction will be found. We derive the
following two features from the calculated
gradient orientations. The first feature is a
normalized measure of the fraction of
template Image of the current mass the
current mass template| and a candidate
region on the prior image. We design
different templates for the registration
method based on correlation: an inner
mass template, an outer mass template, and
three extended templates. These templates
cover different parts of the underlying
mass lesion and its surrounding tissue.
Figure1, figure2, figure 3 and figure4
illustrate original mammogram pair before
contrast enhancement. Mammogram pair
and after contrast enhancement
For all templates we first determine the
contour of the current mass with a
segmentation algorithm based on dynamic
programming. The last registration method
combines the mass likelihood with the
correlation measure and a distance
criterion.
pixels with an intensity gradient pointing
toward the centre pixel. The second feature
calculates whether these pixels occur in all
directions of the centre pixel.
Mass likelihood and mass features
are used as input for a 3-layer feedforward neural network trained on known
abnormalities. Then we use the classifier
output as mass likelihood. Registration
methods based on gray scale correlation
calculate the pixel correlation between a
FIGURE 2
.
FIGURE 1
FIGURE 3
20
Prof. Samir Kumar Bandyopadhyay, Int. J. Comp. Tech. Appl., Vol 1 (1), 16-23
FIGURE 4
FIGURE 5 Original image
FIGURE 6 Area isolation
asymmetry between the breast boundaries.
In almost all cases the said control points
are oversimplified, so, the error levels of
are quite high. In our research, we have
tried to standardize the control point more
accurately in the mediolateral oblique
(MLO) images. First, two distances are
measured from the images; first one is the
height posterior-to-anterior, measured
perpendicular to pectoral muscle from
anterior muscle border or edge to most
anterior portion of breast typically but not
necessarily nipple-areola complex. Second
one is the smallest distance of axillary
concavity region, posterior-to-anterior and
parallel to the previous one, measured
perpendicular to pectoral muscle to most
anterior portion of breast border. In second
step, a line is plotted superolateral-toinferomedial from the middle point of the
second distance measured to extreme point
of the inframarry fold. Now joining the
most anterior point of breast height to
middle point of second one and extreme
point of the inframarry fold respectively
forms a triangle. It is measured that it is the
largest triangle to be drawn within the edge
of a breast border. In last step, a
perpendicular
anterior-to-posterior
is
plotted from the circumcenter of the
triangle to most posterior edge of the
image. The intersection point on the
posterior edge is used as the control point
in our research work. To detect the bilateral
asymmetries of the breast boundaries, the
left and horizontally flipped right
mammogram images is fused on the basis
of the said control point. Using the said
process, the intersection, deference and
union of breast boundaries are determined
clearly which is used to measure level of
bilateral asymmetry of breast.
FIGURE 6 Border isolation
In most of researches some breast
landmarks are used as control points to
match the breasts and compare the bilateral
21
Prof. Samir Kumar Bandyopadhyay, Int. J. Comp. Tech. Appl., Vol 1 (1), 16-23
lateral oblique (MLO) view of bilateral
pairs of mammogram images is used as a
test case. Figure 5-8 shows the results.
Conclusions
FIGURE 7 To determine the
circumcenter of the left and horizontally
flipped right mammogram images.
In this paper we present a method to link
potentially suspicious mass regions
detected by a Computer-Aided Detection
scheme in mediolateral oblique and
craniocaudal mammographic views of the
breast. For all possible combinations of
mass candidate regions, a number of
features are determined. These features
include the difference in the radial distance
from the candidate regions to the nipple,
the gray scale correlation between both
regions, and the mass likelihood of the
regions determined by the single view
CAD scheme.
References
FIGURE 8 The fused the left and
horizontally flipped right mammogram
images on the basis of control point.
Experimental Results
The mammogram images used in
this experiment are taken from the mini
mammography database of MIAS [33].
The original MIAS Database (digitized at
50 micron pixel edge) has been reduced to
200-micron pixel edge and clipped/padded
so that every image is 1024 pixels x 1024
pixels. All images are held as 8-bit gray
level scale images with 256 different gray
levels (0-255) and physically in portable
gray map (.pgm) format. The list is
arranged in pairs of mammograms, where
each pair represents the left and right
breast of a single patient. In our
experiment we have consider all types of
breast tissues i.e. Fatty, Fatty-glandular,
Dense-glandular and the abnormalities like
calcification,
well-defined
or
circumscribed masses, speculated masses
and other ill-defined masses. Fifty, medio-
[1] T.K. Lau, W.F. Bischof, "Automated
Detection of Breast Tumors Using the
Asymmetry Approach", Computers and
Biomedical Research, Vol. 24, No. 3, 1991,
pp. 273-295
[2] F.F. Yin, M.L. Giger, K. Doi, C.J.
Vyborny, R.A. Schmidt, "Computerized
Detection
of
Masses
in
Digital
Mammograms: Automated Alignment of
Breast Images and its effect on BilateralSubtraction Technique", Medical Physics,
Vol. 21, No. 3, March 1994, pp. 445- 452
[3] A.J. Méndez, P.G. Tahoces, M.J.
Lado, M. Souto, J.J. Vidal, "ComputerAided Diagnosis: Automatic Detection of
Malignant
Masses
in
Digitized
Mammograms", Medical Physics, Vol. 25,
No. 6, June 1998, pp. 957-964
[4] E.A. Stamatakis, I.W. Ricketts, A.Y.
Cairns, T. Walker, P.E. Preece, "Detecting
Abnormalities on Mammograms by
Bilateral Comparison", IEE Colloquium on
Digital Mammography, London, UK,
March 1996, pp. 12/1-12/4
22
Prof. Samir Kumar Bandyopadhyay, Int. J. Comp. Tech. Appl., Vol 1 (1), 16-23
[5] M.A.Wirth, C.Choi, A. Jennings, "A
Nonrigid-Body Approach to Matching
Mammograms", Seventh International
Conference on Image Processing and Its
Applications, Manchester, UK, July 1999,
pp. 484-488
[6] F. Georgsson, "Differential analysis
of bilateral mammograms", International
Journal on Pattern Recognition and
Artificial Intelligence, Vol. 17, July 2003,
pp. 1207-1226
[7] W.F. Good, X.H. Wang, G. Maitz,
"Featurebased
Differences
Between
Mammograms", Medical Imaging 2003:
Image Processing, Proceedings of SPIE,
Vol. 5032, 2003, pp. 919-929.
[8] Gonzalez, R.C., Woods,
(1992), Digital image processing.
R.E.
[9] Pizer SM, Psychovisual issues in
the display of medical images, in Hoehne
KH (ed), Pictoral Information systems in
Medicine, Berlin, Germany, SpringerVerlag, 1985, PP 211-234.
[10] Pisano ED, et al, “Contrast Limited
Adaptive Histogram Equalization Image
Processing to Improve the Detection of
Simulated
Spiculations
in
Dense
Mammograms”, Journal of Digital
Imaging, Publisher Springer New York,
Issue Volume 11, Number 4, 1998, pp 193200.
[11] Wanga X, Wong BS, Guan TC,
“Image enhancement for radiography
inspection”, International Conference on
Experimental Mechanics, 2004, pp 462468.
[12] Ball JE, “Digital mammogram
spiculated mass detection and spicule
segmentation
using
level
sets”,
Proceedings of the 29th Annual
International Conference of the IEEE
EMBS, 2007, pp. 4979-4984.
23