Rotation Invariant Local Phase Quantization
for Blur Insensitive Texture Analysis
Ville Ojansivu, Esa Rahtu and Janne Heikkilä
Machine Vision Group, University of Oulu, PO Box 4500, 90014, Finland
{vpo,erahtu,jth}@ee.oulu.fi
Abstract
This paper introduces a rotation invariant extension
to the blur insensitive local phase quantization texture
descriptor. The new method consists of two stages, the
first of which estimates the local characteristic orientation, and the second one extracts a binary descriptor vector. Both steps of the algorithm apply the phase
of the locally computed Fourier transform coefficients,
which can be shown to be insensitive to centrally symmetric image blurring. The new descriptors are assessed in comparison with the well known texture descriptors, local binary patterns (LBP) and Gabor filtering. The results illustrate that the proposed method has
superior performance in those cases where the image
contains blur and is slightly better even with sharp images.
1. Introduction
Natural surfaces usually exhibit some repetitive intensity variations or patterns that are in general referred
to as texture. Analysis of texture information is important in machine vision, and it has numerous applications, including surface inspection, medical image analysis, and remote sensing [6]. In some applications, image degradation may limit the applicability of the texture information. One class of degradation is blur, due
to motion, out of focus, or atmospheric turbulence. Because image deblurring is very difficult and introduces
new artifacts, it is desirable that the texture analysis
could be done in such a way that it is insensitive to blur.
If the orientation of the textures changes, also rotation
invariance is a desired property.
The focus of this paper is on the blur and rotation
insensitive texture classification. There are not many
texture analysis methods that claim to be insensitive to
blurring. One method, although not invariant to rotation, based on short term Fourier transform (STFT),
978-1-4244-2175-6/08/$25.00 ©2008 IEEE
called local phase quantization (LPQ), was proposed in
[5]. Another blur robust descriptor based on color constancy was proposed in [7]. Also, blur invariant moments or modified Fourier phase [1] could be used in
principle, but they are mainly targeted at global object
recognition, not local texture analysis.
In this paper, we propose a new blur and rotation insensitive texture descriptor for gray scale images. The
approach is based on the application of STFT. The advantage in STFT is that the phase of the low frequency
coefficients is insensitive to centrally symmetric blur,
which is commonly encountered in real images; for example, in the case of out of focus, motion and atmospheric turbulence.
Local frequency analysis has been used for texture
analysis also previously. One of the best known methods uses Gabor filters and is based on the magnitude
information [2]. Phase information has been used in
[8] and histograms together with spectral information
in [9]. Nevertheless, blur sensitivity has not been considered as a design criterion in these operators.
2. LPQ texture descriptor
In this section, we shortly present the LPQ method.
For a comprehensive discussion, one can refer to [5],
where also a decorrelation scheme is presented.
Spatial blurring is represented by a convolution between the image intensity and a point spread function
(PSF). In the frequency domain, this results in a multiplication G = F ·H, where G, F and H are the discrete
Fourier transforms (DFT) of the blurred image, original
image, and the PSF respectively. Further considering
only the phase of the spectrum the relation turns into a
sum 6 G = 6 F + 6 H. When the PSF of the blur is
centrally symmetric its Fourier transform H is always
real valued i.e. 6 H ∈ {0, π}. Furthermore, the shape
of H for a regular PSF is close to a Gaussian or a sincfunction ensuring that at least the low frequency values
of H are positive. At these frequencies, 6 H = 0 causing 6 F to be a blur invariant property. Because LPQ
uses finite size 2-D discrete STFT computed locally,
this invariance is in part disturbed but is still pertinent.
In LPQ, the phase is examined in local neighborhoods Nx at each pixel position x = [x1 , x2 ]T of the
image f (x). These local spectra are computed using a
discrete STFT defined by
X
T
F (u, x) =
f (y)wR (y − x)e−j2πu y , (1)
y
where u is the frequency, and w(x) is a window function defining the neighborhood Nx . In the case of
regular LPQ, wR is a NR -by-NR rectangle given as
wR (x) = 1 if |x1 |, |x2 | < NR /2 and 0 otherwise.
The local Fourier coefficients are computed at four
frequency points u1 = [a, 0]T , u2 = [0, a]T , u3 =
[a, a]T , and u4 = [a, −a]T , where a is a sufficiently
small scalar to satisfy H(ui ) > 0. For each pixel position this results in a vector
F(x) = [F (u1 , x), F (u2 , x), F (u3 , x), F (u4 , x)].
(2)
The phase information in the Fourier coefficients is
recorded by observing the signs of the real and imaginary parts of each component in F(x). This is done by
using a simple scalar quantization
1 , if gj ≥ 0
qj =
(3)
0, otherwise,
where gj is the j-th component of the vector G(x) =
[Re{F(x)}, Im{F(x)}].
The resulting eight binary coefficients qj are represented as integer
between 0-255 using coding
P8 valuesj−1
fLP Q (x) =
. Finally, a histogram of
j=1 qj 2
these values from all positions is composed, and used
as a 256-dimensional feature vector in classification.
3. Rotation invariant LPQ
The rotation invariant local phase quantization (RILPQ) method is composed of two stages: characteristic orientation estimation and directed descriptor extraction. We start by introducing the first of these and
present two alternative approaches to it.
3.1. Estimation of characteristic orientation
Let Rθ be a 2-D rotation matrix corresponding to
angle θ, and let f (x)′ = f (R−1
θ x) denote the rotated
image. It is a known fact that the Fourier transform of f ′
is simply the Fourier transform of f rotated by Rθ . The
1
1
0
0
−1
−1
0
pi/2
pi 3pi/2 2pi
0
pi/2
(a)
pi 3pi/2 2pi
(b)
Figure 1. Example of C(x) (solid), C(x) >
0 (dotted), and sine approx. (dashed). (a)
Typical case (98%), (b) Rare case (2 %).
same will apply to the circular local neighborhoods Nx ,
whose position will in addition change to x′ = Rθ x.
Taking advantage of this phenomenon, we estimate
the coefficients (1) on a circle of radius r at frequencies vi = r[cos(φi ) sin(φi )]T , where φi = 2πi/M ,
and i = 0, . . . , M − 1. We further replace the rectangular window wR in (1) with a circular Gaussian one
defined as wG (x) = 1/(2πσ 2 )exp{−(x21 +x22 )/(2σ 2 )}
if |x1 |, |x2 | < NG /2 and 0 otherwise.
Based on the above arguments, in the
case of rotation the resulting vector V(x) =
[F (v0 , x), . . . , F (vM−1 , x)] will be relocated to
x′ , and it undergoes a circular shift corresponding to
the rotation angle θ, within the discretization accuracy
2π/M . Note also that due to separability, V(x) is
efficiently evaluated for all image positions x using
simply 1-D convolutions for the rows and columns
successively.
In order to also achieve blur insensitivity, we consider only the phase of V(x). This is performed by
observing the signs of the imaginary part C(x) =
Im{V(x)} similarly to (3) for G(x).
The characteristic orientation is extracted from the
quantized coefficients using a complex moment as
b(x) =
M−1
X
ci ejφi ,
(4)
i=0
where ci is the i-th quantized component of C(x). The
characteristic orientation is defined for each pixel position as ξ(x) = 6 b(x).
Now for a neighborhood Nx from f ′ , the characteristic orientation ξ(x)′ ≈ ξ(R−1
θ x) + θ, where ξ(y) is
the characteristic orientation of Ny from f . The approximation in the equation is due to the discretization.
The direct evaluation of C(x) requires frequency coefficients to be computed at M points. In the experiments with real textures, we however noticed that in
most cases (approx. 98%) the shape of C(x) had a form
of one period of sinusoid shown in Figure 1(a). Because
of this, the direct approach is perhaps not the most efficient way of evaluating the characteristic orientation.
Hence, we consider also an approximation scheme,
where the form of C(x) is interpolated using a cosine
Ĉ = A(x) cos (φ + τ (x)), with φ ∈ [0, 2π], and the
parameters A(x) and τ (x) are estimated using only two
sample points [F (v0 , x), F (vM/4−1 , x)]. The characteristic orientation is then derived as the position of the
maximum value of Ĉ(x) i.e. ξ(x) = −τ (x), which corresponds to the definition of the complex moment (4).
Some error to the approximation scheme is caused
by those C(x) that do not have the aforementioned
shape. However, most of these still have a sinusoidal
form for just more than one period as shown in Figure
1(b). In most of these cases, the approximation still produces adequately stable orientation.
3.2. Computation of oriented LPQ
In the second stage of the method, we extract the binary descriptor vector. The procedure here is similar
to the original LPQ, but the neighborhood at each location is rotated to the direction of the characteristic orientation. The operation can be formulated by defining
oriented frequency coefficients as
X
−j2πuT R−1
y
ξ(x) .
Fξ (u, x) =
f (y)wR (R−1
ξ(x) (y−x))e
y
For the rotated image f ′ this becomes
X
−j2πuT R−1
y
ξ(x)′
Fξ (u, x)′ =
f (y)′ wR (R−1
ξ(x)′ (y − x))e
y
=
X
f (t)wR (R−1
(t − R−1
θ x))e
ξ(R−1 x)
t
= Fξ (u, R−1
θ x),
−j2πuT R−1
ξ(R
t
−1
x)
θ
θ
(5)
where we have used the fact that Rφ+γ = Rφ Rγ and
substituted t = R−1
θ y. For RI-LPQ, we define Fξ (x)
by replacing the F (ui , x) in (2) with the oriented coefficients Fξ (ui , x). We then proceed similarly to in
LPQ, only using Fξ (x) instead of F(x), to produce
256-dimensional descriptor vector.
Equation (5) shows that the rotation of f only relocates the coefficients Fξ (u, x), which does not affect
the histogram constructed later in the process. Hence
due to the same arguments as for LPQ the resulting features are insensitive to both rotation and blur.
Furthermore, we quantized the orientations in
Fξ (u, x) to K possible values, and precomputed the
corresponding set of window functions and complex exponentials. In this way, the computational complexity
became close to that of the standard LPQ.
Figure 2. Examples of the textures: (left)
sharp and (right) blurred (blur radius is
one pixel).
4. Experiments
We performed three experiments to show the applicability of the proposed method for classification of
blurred as well as sharp textures. As test material, we
used the three applicable test suites of the Outex texture image database 1 [3]. For comparison, we performed the experiments also using two well known texture classification methods, namely the local binary pattern (LBP) descriptor 2 [4], and the Gabor filter bank
based method 3 [2]. The LBP has also a rotation invariant version which was taken into the experiments.
According to [3], the Gabor method had the best performance in the Outex TC 00000 test and the rotation
invariant LBPriu2
P,R for the other two, Outex TC 00010
and Outex TC 00012 containing rotated textures.
In the experiments, we used a 3-nearest neighbor
classifier, which was trained and tested using the appropriate image sets defined in the Outex tests. The blur
was generated by convolving the test images with a circular PSF before the classification. Figure 2 contains an
example of a sharp and a blurred texture.
For the RI-LPQ method, we used two approaches for
the characteristic orientation estimation, as described in
Section 3. These two variants are referred to as RI-LPQ
and RI-LPQa respectively. The parameter values for
both were r = 1/5, NG = 5, M = 36, σ = 2, NR = 9,
a = 1/9, and K = 36. The comparison methods LBP
and standard LPQ are computed with standard parameters in 7-by-7 neighborhoods, as explained in [5].
In the first experiment, we used the test suite Outex TC 00000, which contains 480 images from 24
classes without rotation. The purpose here is to
compare the rotation invariant methods, RI-LPQ and
LBPriu2
P,R , to non-invariant ones. The results with different blur sizes are shown in Figure 3(a). It seems that
all of the methods manage well when no blur is present,
but as the blur strength increases the performance of all
1 http://www.outex.oulu.fi/
2 http://www.ee.oulu.fi/mvg/page/lbp
matlab/
3 http://vision.ece.ucsb.edu/texture/software/
80
60
40
20
0
0.25 0.5 0.75
1
Circular blur radius [pixels]
(a) Blur
100
Classification accuracy [%]
Classification accuracy [%]
Classification accuracy [%]
100
80
60
40
20
0
0.25 0.5 0.75
1
Circular blur radius [pixels]
(b) Blur and rotation
100
80
60
40
20
0
0.25 0.5 0.75
1
Circular blur radius [pixels]
(c) Blur, rotation and diff. illumin.
Figure 3. Classifications results for test suites (a) Outex TC 00000 (blur), (b) Outex TC 00010 (rotation and blur) and (c) Outex TC 00012 (Rotation, illumination changes and blur)
other than LPQ, RI-LPQ, and RI-LPQa start to decrease
soon. It can be also noticed that standard LPQ performs
slightly better than RI-LPQ when no rotation is present,
but the difference is rather small. On the other hand,
comparing the rotation invariant LBP to standard LBP,
the performance difference is significant already with
small blur sizes.
In the second and third experiment, we used test
suites Outex TC 00010 and Outex TC 00012, which
contain 4320 and 9120 images from 24 classes with
rotation. The latter contains also images with different illuminations. The results for these experiments,
are shown in Figures 3(b) and 3(c), respectively. As
can be seen, both RI-LPQ methods are insensitive to
blur, while the performance of the LBPriu2
P,R method decreases rapidly when the blur strength increases. RILPQ achieves also better or similar results for sharp images in both experiments. The results for RI-LPQ seem
to be better for a less blurred image, but RI-LPQa performs equally well with high blur radiuses. The nonrotation invariant methods achieved only slightly over
50 % accuracy even without any blur.
5. Conclusions
In this paper, we proposed a new rotation and blur
insensitive texture analysis method. The approach introduced uses the local low frequency Fourier phase information in two stages: first to estimate the local characteristic orientations, and then to compute the directed
binary descriptors. Both steps are performed in a way
that they are insensitive to centrally symmetric blur. Because only phase information is used, the method is also
invariant to uniform illumination changes.
In the experiments, the new method was shown to
outperform LBP and Gabor descriptors, with both sharp
and blurred textures. Also in the comparison to the
standard LPQ, the new method performed clearly better in the case of rotated patterns, and with comparable
accuracy when no rotation was present. The proposed
method can be also implemented in such a way that the
computation load is close to that of the standard LPQ.
References
[1] J. Flusser and T. Suk. Degraded image analysis: An invariant approach. IEEE Trans. Pattern Anal. Machine Intell., 20(6):590–603, 1998.
[2] B. S. Manjunathi and W. Y. Ma. Texture features for
browsing and retrieval of image data. IEEE Trans. Pattern Anal. Machine Intell., 18(8):837–842, 1996.
[3] T. Ojala, T. Mäenpää, M. Pietikäinen, J. Viertola,
J. Kyllönen, and S. Huovinen. Outex - new framework
for empirical evaluation of texture analysis algorithms. In
Proc. 16th Int. Conf. on Pattern Recognition (ICPR’02),
pages 701–706, 2002.
[4] T. Ojala, M. Pietikäinen, and T. Mäenpää. Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans. Pattern Anal.
Machine Intell., 24(7):971–987, 2002.
[5] V. Ojansivu and J. Heikkilä. Blur insensitive texture classification using local phase quantization. In Proc. Int.
Conf. on Image and Signal Processing (ICISP’08), pages
236–243, 2008.
[6] M. Tuceryan and A. K. Jain. Texture analysis. In C. H.
Chen, L. F. Pau, and P. S. P. Wang, editors, The Handbook
of Pattern Recognition and Computer Vision, pages 207–
248. World Scientific Publishing Co., 1998.
[7] J. van de Weijer and C. Schmid. Blur robust and color
constant image decription. In Proc. IEEE Int. Conf. on
Image Processing (ICIP’06), pages 993–996, 2006.
[8] A. P. Vo, S. Oraintara, and T. T. Nguyen. Using phase and
magnitude information of the complex directional filter
bank for texture image retrieval. In Proc. IEEE Int. Conf.
on Image Processing (ICIP’07), pages 61–64, 2007.
[9] L. Xiuwen and W. DeLiang. Texture classification using spectral histograms. IEEE Trans. Image Processing,
12(6):661–670, 2003.