654
IEEE TRANSACTIONS ON
APPLICATIONS
Optimal velocity filters are useful and improved replacements for any processing scheme that uses trace
summation. The advantages of optimal velocity filters
over summing methods are due to undistorted signal
estimates, sharper rejection rates outside the signal
region, and increased rejection of coherent noise.
Early applications of velocity filters in processing
record cross sections were disappointing largely because
the filters used were invariant in time and space. However, by using dip search techniques and time and space
variant filters [8], velocity filters are useful in improving the signal-to-noise ratio on record sections.
ACKNOWLEDGMENT
The two-dimensional spectrum in Fig. 2 and the
application of the velocity filter in Fig. 3 were taken
from a study made by Dr. J. N. Galbraith, Jr., and
Techniques
COMPUTERS, VOL. c-21, NO.
7,
JULY 1972
Dr. W. Keckler of Mlobil Research and Development
Corporation.
REFERENCES
[1] R. A. Peterson, W. R. Fillipone, and F. B. Coker, "The synthesis
of seismograms from well log data," Geophysics, vol. 20, no. 3,
1955.
[2] F. J. McDonal, F. A. Angona, R. L. Mills, R. L. Sengbush, R. G.
VanNostrand, and J. E. White, "Attenuation of compressional
and shear waves in Pierre shale," Geophysics, vol. 23, no. 3, 1958.
[3] J. P. Fail and G. Grau, "Les filters en eventail," Geophys. Prospect., vol. 11, 1963.
[4] P. Embree, J. B. Burg, and M. M. Backus, "Wide band filteringthe pie-slice process," Geophysics, vol. 28, no. 6, 1963.
[5] M. R. Foster, R. L. Sengbush, and R. J. Watson, "Design of suboptimum filter systems for muilti-trace seismic data processing,"
Geophys. Prospect., vol. 12, no. 2, 1964.
[6] M. B. Dobrin, A. L. Ingalls, and J. A. Long, "Velocity and frequency filtering of seismic data using laser light," Geophysics,
vol. 30, no. 6, 1965.
[7] R. L. Sengbush and M. R. Foster, "Optimum multichannel
velocity filters," Geophysics, vol. 33, no. 1, 1968.
[8] W. Letton, III, and A. M. Bush, "Time varying velocity filters,"
Abstracts, 39th Annu. Int. Meeting of SEG (Calgary, Canada),
Sept. 14-18, 1969.
ror Change
Detection
ROBERT L. LILLESTRAND
Abstract-The problem of change detection presents itself for
imaging systems that view the same scene repeatedly. Current research programs based on the processing of side-looking radar imagery show that spatial alignment of the various parts of the image
must be highly accurate if noise in the difference picture is to be reduced to acceptably low levels. Typically, the spatial alignment accuracy must be better than one-fourth of the diameter of the smallest
resolvable feature in the imagery, and this often requires several
hundred degrees of freedom in the performance of the map warp for
images that are of the order of 107 picture cells (pixels) in size. Gray
scale rectification of conjugate sampling points is less difficult, requiring typically only 10 to 20 percent as many degrees of freedom.
Point by point adjustment for differences in mean transparency and
contrast is employed. Recently developed equipment provides a continuous pipeline processing capability. With this equipment, each
picture element of the second image is transformed with four degrees
of freedom (two spatial and two gray scale). The digital correlator is
capable of processing 4X105 six-bit picture elements per second
when used in conjunction with a CDC 1700 computer.
Index Terms-Digital correlator, geometric image distortion,
image superposition, side-looking radar, subregion correlation,
transparency rectification.
Manuscript received November 16, 1971; revised March 6, 1972.
This work was supported by the U. S. Air Force and by Control Data
Corporation, Minneapolis, Minn. A preliminary version of this paper
was presented at the IEEE, UMC, Two-Dimensional Digital Signal
Processing Conference, Columbia, Mo., October 6-8, 1971.
The author is with the Research Division, Control Data Corporation, Minneapolis, Minn. 55440.
SYSTEM DEFINITION
A T THE present time most efforts at the detection
of changes in images of the same scene involve
nianual comparisons. Usually these are done with
some form of image superposition equipment that provides alternate viewing of one image and then the other
in quick succession. Rather than expose the human
viewer to all of the information contained in both
images, significant increases in the quantity and quality
of his work can be achieved by presenting only the
changes. Typically, in the case of side-looking radar,
increases in the speed of change detection ranging from
101 to 103 result from the use of automatic techniques,
the higher level being typical of urban scenes that contain a high concentration of features that have large
radar cross sections.
In one of the early papers on change detection, given
by Rosenfeld [1], the basic problems are defined. This
paper includes a consideration of various possible coefficients of correlation as measures of the quality of
image registration as well as the description of steps
necessary for the implenmentation of an automatic
change detection system. Shepard [2] discussed the
need for an automatic system that detects changes between two sets of aerial photographs and points out
655
LILLESTRAND: CHANGE DETECTION
(ON-LINE
FILM A
DATA BASE)
}
MEOR
SCANNER
}&|
DETETIO
DUAL FILM
TRANSPORT
A
ISPLAY
SPECIAL-PURPOSE DIGITAL COMPUTER
*
I
T
~1~~_
GENERAL-PURPOSE DIGITAL COMPUTER
l
Fig. 1. System block diagram.
performs such tasks as shadow suppression and thresholding.
For an image that is 1024 pixels wide, one image
frame moves past the viewer every 2.5 s when correlating 4X105 pixels. By adding additional computing
equipmient, the image is modularly expandable in steps
of 210 pixels, witlh an upper limit of about 214 pixels
imposed by the resolution of the scanner. For this
image widtl, the expanded systenm can correlate up to
6.4 million six-bit pixels per second.
The autonmatic digital change detection may be divided into the following steps: 1) subregion correlation;
2) spatial rectification; 3) transparency rectification;
4) change presentations. This is the order of presentation for the succeeding sections of the present paper.
The image research work conducted in our laboratory
involves a C,DC 6600 computer that is used to perform
these functions when new imagery is received, during
which time the algorithms are developed and the processing parameters are selected. Typically, for types of
imagery not previously-processed, this may involve 100
to 200 h of image processing. At this stage, no particular
effort is made to achieve a high processing speed, but
rather, emphasis is placed on the achievement of the desired processing result through tlhe use of experimental
methods. Having completed this work, a special-purpose system of the type shown in Fig. 1 is constructed
(in this case, for SLR imagery). This latter equipmlent
is a pipeline processor with a high degree of internal
types of imagery.
The digital processor is capable of correlating 4X 10 parallelity and is capable of processing inmagery at
six-bit pixels per second. Using appropriate conversions speeds that are between one and two orders of miagnito reduce the processing load of the special-purpose tude faster than the 6600 processing used in the initial
digital comnputer to equivalent six-bit adds, a value of development of the algorithms.
about 40 X106 six-bit adds per second is obtained.
METHOD OF CORRELATION
The correlated output of the B channel (BW,T) is comThe method of correlation cbnsists of dividing the
pared with the A channel gray scale value in the change
detection matrix to provide whatever transformiation image into a series of small subregions arid searcling for
function is required prior to display. This equipment maxinmum values of the correlation coefficiept by dis-
that manual procedures are too slow. Several methods
for automating the process of change detection are
given by Paolantonio 13]. Kawamura 14] describes a
system used to automatically detect meaningful chlanges
in photographic data for purposes of city planning. In
this paper he considers the problems of both change
detection and pattern recognition. Typical geonmetric
image distortions derived from sensor-related and external errors are described in the paper by Bernstein
and Silverman [51. They also describe a general class of
sequential algorithms for image registration.
The block diagram of a higlh-speed change detection
system constructed at Control Data Corporation is
shown in Fig. 1. Since the SLR imagery is continuous,
rather than appearing a frame at a time, as is the case
for most photography, a dual film continuous transport
optical scanner is used for the A/D conversion. A small
general-purpose computer (CDC 1700) is paired witlh a
special-purpose lhardwired computer for the data processing. This latter equipment is a multiple strip (up to
16 channels) digital correlator that has an auxiliary
digital memory that serves as an input buffer and picture cell interpolator. By means of the buffer memory,
the requirement for the precise servo control of one of
the film scanner beams relative to the other is eliminated. The small general-purpose computer slhown in
Fig. 1 contains about 30 alterable parameters and this
makes it possible to optimize the processing for various
656
placing one subregion x and y relative to its conjugate.
The fraction of the total image area that these subregions cover, as well as the number of subregions that
are used, depend upon the complexity of the spatial
warp of the imagery and the distribution of features,
but typical values range from 5 to 50 percent. The
number of pixels contained in one subregion area can
be varied from (25)2 to (27)2.
In the case of the continuous processor shown in
Fig. 1, a series of more or less parallel correlation tracks
are employed. As shown in Fig. 2, these tracks creep
along thie film strip in the direction of film transport.
The interpolated location of the maximum value of the
correlation coefficient is computed each time the subregion advances by a one-pixel step along the track and
a syntlhetic image of conjugate points of film B is computed. This is illustrated in Fig. 3. The location of the
mnaximum of the correlation coefficient can be predicted
with a hiigh degree of accuracy since the subregions
creep along the track incrementally. Resultantly, not
more than nine trial subregion positions need to be
used to compute the location of the maximum of the
correlation coefficient. Since the correct conjugate point
on film B will not, in general, correspond withl any
specific point sanmpled by the scanner, an interpolated
value based on the mieasured transparency of the four
pixels surrounding it is used.
Should one or more subregions encounter a featureless area, as would be the case for tracks 4, 5, and 6 of
Fig. 3, a cross-track linkage has been built into the
servo system. This "elastic harness" permits tracks that
are highly correlated with the data base imagery to
carry along adjacent strips for wlhich the values of the
correlation coefficient are low. This feature is desirable
at the startup of the image tracking since lock-on for
any one track propagates laterally as the film is transported, eventually resulting in lock-on over the full
width of the inmagery.
In general, the assessmient of the quality of subregion
correlation for the operation of the cross-track linkage
requires nmore than just a hiigh value of the correlation
coefficient. For example, if thle features contained in a
given subregion consist largely of parallel edges, a
highly elliptical shape will be obtained for the contours
of the correlation coefficient shown in Fig. 3. In this
case, even thotughi the correlation coefficient is large,
there will b)e almost no subregion positional constraint
in the direction of the senimuajor axis of the ellipse.
Because of this problem, tests Imlust be mlade for both
the mnagnittude of the correlatioin coefficient at the central maximlumii and for the milaniner in wlhiclh it varies iIn
the neighborhood of this m11axiimum11Ii. In the case of sidelooking radar iimagerv, the ratio of senimilajor to seimiiminor axes seldonm exceeds 3: 1 since thiere are usuallv a
significant IlLml)er of poinlt symmlnietric feattures.
A second order servo is used to sl-vethe film B track
to its filmii A conjugate. The valuies assigned to the feedback loop paramleters are derived fromii sinmuilations on
IEEE TRANSACTIONS ON COMPUTERS, JULY
FILM B/
O
TRACK I-.4j.
2,
S
FILM A
j-.
-----
---
1972
DIGITAL
WIDTH OF
IMAGERY
CORRELATIOI
..F#'
SUBREGION
r-
IMAGERYj
6. [[II?
ti
~ ~ ~ AREA
~~~~~~~~~FEATURELESS
Fig. 2. Film strip correlation showing individual tracks.
TRIAL POSITION FOR CENTER OF
SUBREGION CORRELATION APERTURE
IN SEARCH FOR MAXIMUM VALUE
OF CORRELATION COEFFICIENT, r.
+
.754
r=
z(Pi
-
P) (Q
-Q)
FhIEQ,-i)
rP
SPACING BETWEEN
INDIVIDUAL TRIAL
POSITIONS IS ONE
IMAGE SAMPLE STEP.
Fig. 3. Initerpolation between image sample cells to determine
location of maximum of correlation coefficient.
the 6600 conmputer and depend on the pattern of statistical and systematic deviations of the conjugate subregions for the type of imagery in question.
SPATIAI RECTIFICATION
Experience gained during the past three years in the
digital processing of SLR imagery lhas repeatedly shown
the inmportance of precise image registration if low noise,
levels in the difference image are to be achieved. There
are many possible reasons for correlating imagery, and
stable lock-on at the principal maximum of the correlation coefficient can often be achieved acceptably witlhout enmploying the elaborate techniques that we lhave
found necessary for clhange detection.
Fig. 4 illustrates thiree teclhniques that lhave been
used for the imlplemiientation of tlle map warp so as to
aclhieve the required precision in image registration. For
imnagerv consisting of, say 106 pixels, a 16X16 array of
correlation subregions niglht typically be used in studying the miiap warp problem. We then use a least square
curve-fitting procedure for bivariate polynonmials of
increasing order and thein exanine the size of the
residuals. Thlis is slhown as JfFIT in Fig. 5 wAvhere an 8 X 16
subregion array was used. Thle remiiaining 8 X 16 sub-
657
LILLESTRAND: CHANGE DETECTION
P
BEFORE
POLYNOMIAL
Ax
=
AY-
P-i
I
Y
'°0 '
WHERE P IS THE
ORDER OF THE
a,,. xi vi.
POLYNOMIA
0
bl,
=
xl
=
QUADRALATERAL
-
STRIP PROCESSOR
_l_l-_ A _.
Fig. 4.
10
FIT POINTS
INT POINTS
Co
-J
-J
LU
x
d
x
o....
0
x
0
x ..
x
0
x
0
x
Cd
LL
cc
2f
0
/
x
0
0-
o
.
x
iIalNT /
4
/
n
/°TOT'
/
uLJ
a
CO
2
0
I
3
6
I1 St
1
I
'
'
42
30
20
NUMBER OF PARAMETERS
12
2nd
3rd
4th
5th
I
I
56
72
6th
'
7th
ORDER OF POLYNOMIAL
Fig. 5. Residuals after spatial rectification as a function of
order of approximating polynomial.
form a checkerboard pattern with the others
used to chieck the quality of the fit in terms of
the interpolation problem. The rms residual of these
intermediate points is designated O-INT.
The three parameter solution consists of Ax, Ay, and
AO and this forms the reference case for the study of
spatial rectification since this is indicative of the superposition accuracy that can be achieved by merely sliding the two images relative to one another. By adding
one more parameter, a difference in the scale of the
two images can be corrected, vhereas if three more
parameters are added to the reference case, one can correct for anamorphosis, in which the differing scales are
at right angles to one another, but at some unknown
orientation relative to the x-y axes.
regions
and are
As one would expect when using increasingly higher
polno
ia
approximations, the values of a-FIT
order polynomial
approach zero. This advantage is lost, however, because
of the poor interpolatory properties of the higher order
polynomials, as is shown by the increase in the value of
INT. The value of a-TOT, which is tlhe rnms error for the
total collection of poinlts, thus shows a broad minimum
with the best -values obtained for this particular type of
imagery for the third- and fourth-order solutions.
Beyond fifth order, the check with intermediate
points becomes so bad that a different map warp
scheme must be employed. Experience with the quadrilaterals shown in Fig. 4, in wlhich each corner corresponds to the center of a correlation subregion, shows
improved performance for the 16 X 16 array, both in
terms of the value of the correlation coefficient for the
overall image and in terms of the noise in the difference
image. A logical extension of the quadrilateral technique involves the use of bicubic spline fitting to remove
the discontinuity in the first derivative of the map
warp at the corners of the quadrilaterals. While this is
consistent with physical intuition about the nature of
the warp, this technique has not proved necessary in the
spatial rectification of SLR imagery since a nearly
effect can be produced by simply using more
equivalent
./
quadrilaterals.
The strip processor shown at the lower right portion
of Fig. 4 resembles the quadrilateral solution, except
that the interpolation between the strips is recomputed
each time the tracker subregion is advanced by one
pixel.
Experimerital data such as that shown in Fig. 5 indicates that the map warp is often highly nonlinear. This,
coupled with the nonuniform distribution of features in
the various subregions, suggests that iterative processing may be worthy of investigation. When the feature
content is low, the size of the correlation subregions
must be large. If this condition occurs in combination
with a large map warp, the* images being processed are
*
*
map warp processing.
to benfit ffrom iterative
likely
TRANSPARENCY RECTIFICATION
If the six-bit gray scale values for conjugate pixels
are plotted against one another, as shown on the diagrams on the left of Fig. 6, systematic differences in
mean transparency and contrast will usually be found.
To the extent that the gray scale values of conjugate
points are similar, the values will lie along a 450 regression line. Deviations from this line are a measure of
nonrepeatability of the two spatially rectified images,
and a-i is the rms deviation of the individual pixels in a
direction perpendicular to the rectified regression line.
The pixels along (1) represent a feature added to film B
that is embedded in a uniformly low background level
of film A; pixels along (2) represent this same case except that the background is strong.
Since the corrections for mean transparency and contrast vary from one part of the inmage to another, the
IEEE TRANSACTIONS ON COMPUTERS, JULY
1972
AJ- CHANGE IN MEAN
TRANSPARENCY
log T,
AY
-
CHANGE IN
CONTRAST
log T,
log T.
Fig. 6. Transparency rectification of imagery.
digital processor must be capable of varying these cor-
rections over the area of the image, as shown by the
contours at the right of Fig. 6. Thus the combined
spatial and transparency rectifications require that each
pixel of film B be modified witlh four degrees of freedom:
Ax, Ay, Ajiu, A-y. Althouglh some of the imagery processed
has shown evidence of nonlinearities in the, scatter diagram of Fig. 6, the values normally obtained for al
are sufficiently large to make the use of a inore coImlplicated nonlinear regression line of questionable advantage.
From the point of view of change detection, the individual pixels may be classified as shown in Fig. 7.
Computer programs have been developed that automatically plot data in this form for various subregions
or for an entire picture. This type of feature classification diagram is normlally made only after comiipleting
the spatial and transparency rectifications.
Points falling on different regions of a scatter diagram
of this type represent differing feature classifications.
Typically, the diagonal threslhold bounds are set at
values that are three to four times oal. Features that
are added to film B fall in the upper left-hand corner,
while features that have been removed from film B fall
in the lower right-lhand corner. Shadows are regions of
ho radar return and correspond to transparent areas on
the film. By studying diagrams of the type shOwn in
Fig. 7, shadow suppression threslholds can be established to avoid the creation of changes in the difference
picture that are the result of shadows of varying lengtlh.
A feature classification diagram for a subregion consisting of about 104 pixels is shown in Fig. 8. Spatial
and transparency rectification were performed prior
to making this plot. The distribution of gray scale
values is bimodal, as can be seen from the camelback
shape. That is, there is a preponderance of the film
area that is either very light or very dark. This particular subregion contains no significant feature changes
or shadows.
Large values of the correlation coefficient are obtained
either because the value of al is small or because the
scene contrast is large, as is the case for the data shown
63
log T,
TRANSPARENT
.
.t-
-.4
PHOTO-DENSITY
OF FILM STRIP
OPAQUE
-
Fig. 7. Feature classification diagram.
Fig. 8. Example of feature classification diagram.
in Fig. 8. When evaluating the result obtained for the
superposition of an individual subregion as compared
with others on the same image, a large value of the correlation coefficient should not be used as the sole criterion of image repeatability and it is the value of ai
that is used to automatically set tihe thresholds for
change detection.
ILLUSTRATIVE IMAGERY
There are many ways of displaying the results of
automatic digital change detection, particularly if color
LILLESTRAND: CHANGE DETECTION
659
Fig. 9. Data base imagery. Features added are enhanced.
Fig. 10. Data base imagery. Features removed are enhanced.
displays are available. The two black and white renditions given in Figs. 9 and 10 are for an image that consists of about 10 pixels, each of which is encoded to
six bits of gray scale information. A digitally generated
presentation of the data base imagery (film A) is shown
in the background. In the case of Fig. 9, the features
that have been added to film B are shown in enhanced
form. For Fig. 10, the features that have been removed
are shown in enhanced form. (Some of the contrast in
the original imagery has been lost in the offset printing,
and consequently the changes as shown are somewhat
more difficult to detect than is normally the case.) When
making the original digital images, the background
scene was reduced to one-fourth of its original value,
thus providing the viewer with some contextual information to aid in the evaluation of the changes that
have taken place.
ACKNOWLEDGMENT
This paper has presented a very abbreviated description of image processing techniques being developed
at Control Data Corporation, Minneapolis, Minn., and
contributions were made by many people on the staff
of the Research Division.
REFERENCES
[1] A. Rosenfeld, "Automatic detection of changes in reconnaissance
data," in Proc. 5th Conv. Mil. Electron., 1961, pp. 492-499.
[2] J. R. Shepard, "A concept of change detection," Photogrammetr.
Eng., vol. 30, pp. 648-651, July 1964.
[3] A. Paolantonio, "Difference measurements in automatic photointerpretation of surveillance maps," Inform. Display, vol. 6,
pp. 41-44, Mar./Apr. 1969.
[4] D. G. Kawamura, "Automatic recognition of changes in urban
development from aerial photographs," IEEE Trans. Syst., Man,
Cybern., vol. SMC-1, pp. 230-239, July 1971.
[5] R. Bernstein and H. Silverman, "Digital techniques for earth
resource image data processing," presented at the 8th Annu.
AIAA Meeting, AIAA Paper 71-978, Oct. 25-28, 1971.