Lecture outline
„
Texture
„
What is texture?
Texture descriptors
… First
order
order, GLCM
… Fourier, autocorrelation
… Higher order, GLRLM
… Second
INF 3300/4300 Lecture 9
Asbjørn Berge
„
Analyzing texture for different scales
INF 3300 / 4300 Autumn 2005
Meet Spot, the invisible dalmatian
INF 3300 / 4300 Autumn 2005
What is texture?
„
Intuitively obvious, but no precise definition exists
„
Texture consists of texture primitives, texels
…
…
„
…
INF 3300 / 4300 Autumn 2005
Texel what?
Textures are highly scale dependent, a region can have
textures in multiple scales
Uses for texture analysis
„
„
„
„
INF 3300 / 4300 Autumn 2005
tone, intensity properties of texels
structure, spatial relationships of texels
INF 3300 / 4300 Autumn 2005
„
Note that you can define texels in any image scale, and that the best image
scale for analysis is problem dependent.
a contiguous set of pixels with some tonal and/or regional
property
Texture can be characterized by
…
„
”fine, coarse, grained, smooth” etc.
Segment an image into regions
with the same texture, i.e. as a
complement to greylevel or color
Recognize or classify objects
based on their texture
Find edges in an image, i.e.
where the texture changes
”shape from texture”
object detection, compression,
synthesis
INF 3300 / 4300 Autumn 2005
1
A simple approach for texture
description
„
„
„
„
To be able to find changes in texture in an
image, a simple strategy is to perform texture
measurements on a sliding window
Most texture features can be summed up as
scalars, so we can assign features to each of the
image pixels corresponding to window centers
For each pixel, we now have a description of the
”texture” in its neighborhood
Beware of image edges, edge artifacts will occur
”Texture” – description of region
characteristics
„
„
Remember: we estimate local properties (features) to be able to
isolate regions which are similar in an image (segmentation), and
possibly later identify these regions (classification), usually with the
final goal of object description.
One can describe the ”texture” of a region by:
…
„
„
…
…
„
smoothness, roughness, regularity, orientation...
Problem: we want the local properties to be as ”local” as possible
Large region or window ⇒
Precise estimate of features
Imprecise estimate of location
Small window ⇒
…
…
Precise estimate of location
Imprecise estimate of features
INF 3300 / 4300 Autumn 2005
Texture description is scale
dependent
What is our goal for texture description in the image? Scale impacts the
choice of texels, or vice versa. The curtain can be described as a repetition
of single threads, a configuration of meshes or as a repetition of folds.
INF 3300 / 4300 Autumn 2005
Statistical texture description
„
Describe texture in a region by a vector of
statistics (feature vector)
… First
order statistics from the (texel) greylevel
intensity histogram
… Second order statistics, describing relation
between pairs of texels
… Higher order statistics, quantize region in runs
of texels
INF 3300 / 4300 Autumn 2005
First order statistics from greylevel
histogram
„
Mean (hardly a useful feature)
„
Variance (a more credible feature, measure region "roughness")
„
Skewness (are the texel intesities usually darker/lighter than
average?)
„
Kurtosis (how "uniform" is the greylevel distribution?)
INF 3300 / 4300 Autumn 2005
INF 3300 / 4300 Autumn 2005
First order statistics from greylevel
histogram
„
Entropy (how normal/nonnormal is the
greylevel distribution?)
„
Energy (how much intensity variation is
there in the region?)
INF 3300 / 4300 Autumn 2005
2
Using variance estimates
Using variance estimates
Variance, σ2, is directly a measure of
”roughness”
„ A measure of ”smoothness” is
„
„
“Coefficient of variation”
…
where σw and μw is dependent on the window, w centered in (x,y)
cv is scale invariant
…
but not location invariant
…
alternatives
…
…R
is close to 0 for homogenous areas
… R tends to 1 as σ2, ”roughness”, increase
„
„
use median instead of mean
interpercentil-distance instead of standard deviation
INF 3300 / 4300 Autumn 2005
Skewness and kurtosis
„
„
As mentioned, the higher 1. order moments can also be
used for texture description
Recall: n'th moment (from the histogram) is
INF 3300 / 4300 Autumn 2005
Entropy and energy
„
…
…
„
„
…
„
„
Skewness
"departure from symmetry", sort of measures if there is a "wider"
range of either darker than/lighter than average
Kurtosis
…
"distribution flatness", measures if the different graylevels have
almost equal chance of occuring and is invariant for i' = Ai + B,
note also that γ4 = 0.0 for gaussian distributions with μ = 0.0
INF 3300 / 4300 Autumn 2005
1. order statistics discussion
„
„
„
1. order statistics can separate two regions even if
μ1 = μ2 , as long as σ21 ≠ σ22
The statistics of a pixel (x, y) is found in a local window
Problems:
…
…
…
…
Edges around objects are exaggerated
Solution: use adaptive windows
1. order statistics does not describe geometry or context
Solution:
„
„
Calculate 1. order statistics with different resolutions, and obtain
indirect information about 2. and higher order statistics
Simply use 2. order statistics
INF 3300 / 4300 Autumn 2005
Entropy
Histogram uniformity measure
We see that 0 · H · log G
H is also the lower limit for mean codeword lenght
(bits/pixel) in lossless, noncontextual compression
Energy
…
…
…
…
…
Histogram non-uniformity measure
If all P(i) are similar we have,
If just one graylevel is present (homogenous region),
This means that G-1 · E · 1
INF 3300 / 4300 Autumn 2005
1. order description example
„
Brickwall texture
…
…
…
…
…
„
Mean 79.13
Variance 42.73
CV 0.08
Skew 1.37
Kurtosis 5.93
Granite texture
…
…
…
…
…
Mean 157.08
Variance 96.9573
CV 0.06
Skew -0.73
Kurtosis 3.25
INF 3300 / 4300 Autumn 2005
3
Second order statistics
„
„
Correlation as a function of distance
Spatial frequencies / edge frequency
…
…
„
„
Autocorrelation
…
Gray Level Coocurrence Matrices (GLCM)
„
Transform to fourier domain and integrate over rings or wedges
Gabor filters combine estimate of orientation and frequency
Co-Occurence matrices
…
…
Intensity change ”histograms” as a function of distance and
direction
By far the most popular texture description method due to its
simplicity
Matrix element P(m,n) in a GLCM is 2. order probability of changing
from graylevel m to n when moving distance d in the direction θ of
the image
From a M × N image with G graylevels, and f(m,n) is the intensity.
Then
, where
and
„
Alternative notation, dependent on direction and distance, P(i,j | d, θ)
INF 3300 / 4300 Autumn 2005
GLCM
„
INF 3300 / 4300 Autumn 2005
GLCM
Matrix size = number of graylevels
„
„
Usually a good idea to reduce the number of (d,θ) variations evaluated
Simple pairwise relations:
…
…
„
„
„
„
The matrix must have an ``average occupancy level'' that is large enough:
…
…
„
P(d,00) = Pt(d,1800) , P(d,450) = Pt(d,2250)
P(d,900) = Pt(d,2700) , P(d,1350) = Pt(d,3150)
Isotropic matrix by averaging P(θ), θ ∈ {0o, 45o, 90o, 135o}
There is a lot of scalar texture descriptors T(d,θ)
Mean, MT(d), range, RT(d), and variance, V2T(d), can be used as features
Reduce number of graylevels (Less precise description if the texture has low
contrast)
Increase window size (Errors due to changes in texture)
Heuristically, 16 graylevels is usually sufficient and the window should be 30
× 30 - 50 × 50 pixels.
INF 3300 / 4300 Autumn 2005
Isotropic GLCM example
INF 3300 / 4300 Autumn 2005
How to use the GLCM
„
„
Can be used directly to measure statistical distance
between textures
Usually, used by extracting secondary features
…
…
…
…
„
„
INF 3300 / 4300 Autumn 2005
Haralick et al and Conners et al.
Features are usually strongly correlated, and using more than 45 simultaneously is not advisable
Need to evaluate several distances d
Optimal set of features is problem dependent
It is also advisable to preprocess by histogram
equalization to remove effect of absolute gray level
Usually, we want to make the features ”rotation” invariant
by using the isotropic GLCM
INF 3300 / 4300 Autumn 2005
4
GLCM Features
„
Angular Second Moment,
…
„
„
„
Correlation is a measure of gray level linear dependence
between the pixels at the specified positions relative to each
other.
…
„
IDM is also influenced by the homogeneity of the image.
Because of the weighting factor (1+(i-j)2)-1 IDM will get small
contributions from inhomogeneous areas (i ≠ j). The result is a
low IDM value for inhomogeneous images, and a relatively
higher value for homogeneous images.
Sum of Squares, Variance,
…
INF 3300 / 4300 Autumn 2005
This measure of contrast or local intensity variation will favour
contributions from P(i,j) away from the diagonal, i.e. i ≠ j
Inverse Difference Moment,
Inhomogeneous scenes have low first order entropy, while a
homogeneous scene has a high entropy.
Correlation,
…
Contrast,
…
Entropy,
…
„
ASM is a measure of homogeneity of an image. A homogeneous
scene will contain only a few gray levels, giving a GLCM with
only a few but relatively high values of P(i,j). Thus, the sum of
squares will be high.
GLCM Features
This feature puts relatively high weights on the elements that
differ from the average value of P(i,j)
INF 3300 / 4300 Autumn 2005
GLCM Features
Sum and difference histograms
Sum Average,
„ Sum Entropy,
„ Difference Entropy,
„ Inertia,
„ Cluster Shade,
„ Cluster Prominence,
„
„
„
„
INF 3300 / 4300 Autumn 2005
Sum and difference histograms
„
„
GLCM features can be derived from Ps and Pd
Example:
…
Fourier analysis
„
„
„
…
Contrast from GLCM
„
„
„
Some of the features mentioned earlier is derived from the
histograms
…
…
…
…
Sum Average,
Sum Entropy,
Difference Entropy,
Inverse Difference moment,
INF 3300 / 4300 Autumn 2005
The number of possible values of sum histogram is 2G-1, whereas
for difference histogram it is G-1
INF 3300 / 4300 Autumn 2005
„
Contrast from Pd
Sum and differences define the principal axes in a 2. order
probability density function
Define normalized sum and difference histograms for a domain D
„
„
The fourier spectra give direction and frequency for periodic or near
periodic 2D patterns
Local FFT in windows
Texture with a dominating direction will have peaks in the spectra
along a line orthogonal to the texture orientation
High frequency = fine texture = peaks in the spectra far from the
origin
Thus it is possible to separate fine and coarse spectra
The spread in image frequencies = width of the peak
Isotropic textures with a defined frequency can be seen as rings in
the spectra
Scalar features can be extracted by integration over rings, wedges
or from results of Gabor filtering
INF 3300 / 4300 Autumn 2005
5
Fourier analysis example
Autocorrelation
„
Autocorrelation can be loosely defined as
„
and is calculated over window W × W, with shift (i,j)
„
We can assume that a coarse texture will have higher correlation
ρf(i,j) for a given shift (i,j) than a fine texture.
If the texture consists of large texels, then ρ(i,j) will decay slowly.
Small texels implies rapid decay of ρ(i,j)
Texture coarseness can be measured by measuring the spread in
ρf(i,j)
„
„
„
INF 3300 / 4300 Autumn 2005
Autocorrelation
„
INF 3300 / 4300 Autumn 2005
Autocorrelation example
A general measure of spread is
where
„
Note the symmetry
INF 3300 / 4300 Autumn 2005
INF 3300 / 4300 Autumn 2005
Higher order statistics
Gray Level Run Length
Several authors argue that 2. order is
sufficient
„ Higher order methods include
„
„
level runlength matrices - ”histograms”
of greylevel run lengths in different directions
… Laws’ texture masks - masks resulting from
combinations of 0,1,2 derivatives
„
A ``run'' = contigous (8-neighbor) pixels on a line with the same
graylevel.
``Run length'' = The number of pixels in a run
``Run length value'' = number of times one observe a specific run in
a (sub) image
Each element p(i,j|θ) in a Gray Level Run Length Matrix (GLRLM)
define the number of runs of graylevel i, with length j, in a specific
direction θ.
Let P(i,j|θ) be elements in the normalized GLRLM, i.e
„
where S is the total number of runs in the image.
„
„
„
… Gray
INF 3300 / 4300 Autumn 2005
INF 3300 / 4300 Autumn 2005
6
GLRLM - an example
GLRLM - cookbook
„
„
„
Histogram equalize image
Reduce the number of graylevels by
quantization. Need to try several values of G
Let P(i,j|θ) = element in GLRLM
…G
= number of graylevels in the image
= the longest run in the image
… N = The number of pixels in the image
…R
„
„
An image window (top) and its Gray Level Run Length
Matrix p(i,j|θ=0o) (left) and p(i,j|θ=45o) (right).
INF 3300 / 4300 Autumn 2005
GLRLM - scalar features
„
Short Runs Emphasis
„
By dividing each element in P(i,j) by the square of its length (j), we
emphasize short runs.
Long Runs Emphasis
We can find several scalar features from our G ×
R GLRL matrix.
INF 3300 / 4300 Autumn 2005
GLRLM - scalar features
„
Gray Level Non-uniformity
„
High run length values will dominate.GLN will have its lowest value if
all graylevels are equiprobable (1/G · GLN · 1)
Run Length Non-uniformity
By multiplying each ``run length value'' by the square of its length,
long runs are weighted more.
RLN will have its lowest value if run lengths are equiprobable. (1/R
· RLN · 1)
INF 3300 / 4300 Autumn 2005
GLRLM - scalar features
„
Run Percentage
„
Ratio between the total number of
observed runs to the number of pixels in
the image (1/N · RP · 1)
INF 3300 / 4300 Autumn 2005
INF 3300 / 4300 Autumn 2005
GLRLM - scalar features
„
Low Gray level Runs Emphasis
„
High Gray level Runs Emphasis
„
Low Gray Level Emphasis (LGRE) and High Gray Level Emphasis
(HGRE) is introduced to separate textures that have equal SRE or
LRE, but non-equal distributions of graylevels in the runs.
INF 3300 / 4300 Autumn 2005
7
Simplifying GLRLM
„
Let
„
represent the number of runs of length j, while
„
is the number of runs with graylevel i
Let S be the total number of runs in the image:
„
Simplifying GLRLM
INF 3300 / 4300 Autumn 2005
Simplifying GLRLM
„
INF 3300 / 4300 Autumn 2005
Simplifying GLRLM
The expressions for the GLRLM features
become:
„
Note that all features can be calculated without
a 2-D GLRL matrix.
INF 3300 / 4300 Autumn 2005
Laws' texture energy measures
„
„
„
„
Based on convolution
Uses 3×3 or 5×5 separable masks that is
symmetric or antisymmetric.
From the results of convolution find the standard
deviation or average of absolute values over a
larger window (e.g. 15×15)
This is a measure of texture energy in some
direction depending on the mask chosen
INF 3300 / 4300 Autumn 2005
INF 3300 / 4300 Autumn 2005
Laws' texture energy measures
„
„
„
„
Might want to use masks that are null sum
(to avoid introducing "energy" into the
system)
5 × 5 masks can be found by convolving a
vertical 5 vector with a horizontal 5-vector :
Each 5-vector is a result of convolution of
two 3-vectors:
A 5 × 5 convolution can be done by:
…
…
…
two 3 × 3 convolutions
one 5 × 1 followed by a 1 × 5 convolution
two 3 × 1 and two 1 × 3 convolution
L5
=
[
1
4
6
4
1
]
E5
=
[
-1
-2
0
2
1
]
S5
=
[
-1
0
2
0
-1
]
R5
=
[
1
-4
6
-4
1
]
L3
=
[
1
2
1
]
E3
=
[
-1
-0
1
]
S3
=
[
-1
2
-1
]
INF 3300 / 4300 Autumn 2005
8
"Finding rippling texture"
R5R5 example
Multiscale texture analysis
„
„
„
„
INF 3300 / 4300 Autumn 2005
We have ascertained that texture description is scale
dependent
By representing the image in a pyramid structre we can
describe image texture on different scales
A discrete wavelet-transform will give four new images
with reduced resolution sll, slh, shl, and shh
where a,b,c,d are four neighbors in the original image
INF 3300 / 4300 Autumn 2005
Multiscale texture analysis
„
„
„
By repeating this decomposition on sll, the result is a hierarchical pyramid structure for different
resolutions, both in a the lowpass image and the other subbands
The image can be represented in multiple resolutions and multi-scale GLCM (for example) can be
calculated
Furthermore, each of the three other subbands can be viewed as edge information for that scale
INF 3300 / 4300 Autumn 2005
9