ITE Trans. on MTA Vol. 3, No. 4, pp. 251-257 (2015)
Copyright © 2015 by ITE Transactions on Media Technology and Applications (MTA)
Paper
Smooth Gradation Reconstruction from False Contours
in Quantized Natural Image
Akira Mizuno † , Masayuki Ikebe †
Abstract
We propose a bit-depth expansion (BDE) method targeting natural images. In the analog part of an imaging
system, signal intensity fluctuations occur due to noise (e.g. thermal noise in the image sensor). After that, in the digital
part, the intensities are rounded off to limited levels. The latter process, which is quantization, increases the intensity of
fluctuation errors caused by stochastic resonance. These errors are viewed as false contour artifacts in the gradation region.
Our goal was to obtain the original signal from the quantized noisy signal. We formulated a probabilistic model based on this
quantization process, and successfully reconstructed smooth gradations from noisy contours. Subjective evaluation by voting
clarified that the output image has higher quality.
Key words: Quantization, bit-depth expansion, gray level reconstruction, false contours
maximize both accuracy and smoothness as much as
possible. Some previous methods work well on images
with no noise (e.g., computer graphics). Unfortunately,
however, none of them can work on natural images.
Previous work’s premise is that no noise is added to
the original signal. In this case, the image quantization
process is formulated as
1. Introduction
Bit-depth expansion (BDE) is desired in high dynamic range (HDR) imaging. Prior research on an HDR
display system1) has shown that more than 10 bits are
necessary for the number of quantization levels to cover
the desired range without quantization artifacts.
When displaying a relatively low bit-depth (LBD)
image on a monitor, the brightness gets discontinuous
values. Histogram equalization or other image enhancement methods may generate a sparse histogram. These
discontinuities of brightness create an artifact called a
false contour, appearing in low spatial frequency or low
contrast regions such as the sky, sea, or skin. The false
contour, which does not exist in the original scene, seriously degrades the image quality. Therefore, a method
for reconstructing the original smooth gradation by interpolating the false contour is required. Coming up
with an optimized reconstruction method is currently
the most important issue in BDE.
Many attempts have been made to address this need
in previous work. Low-pass filtering methods2)˜9) have
been proposed to smooth the contour region. Ditheringbased methods7)8)10)11) have been shown to reduce the
visibility of undesirable artifacts. Flooding-based methods12)˜14) can convert the 2D extrapolation problem into
a 1D interpolation. Optimization-based methods14)15)
y = Q(x),
(1)
where x = [x1 , x2 , · · · , xn ]T ∈ Rn is an original signal, y ∈ Zn is a quantized signal, and Q is a quantizer
function defined as rounding off fractions in this paper.
Eq.(1) requires that the pixel values of an estimated
signal is in per-pixel quantization bins:
x∗i ∈ [yi , yi + 1) ,
(2)
where x∗i is the estimated value of xi (1 <
=i<
= n).
We use a more accurate formulation because natural
images are inseparable from noise:
y = Q(x + ξ),
(3)
where ξ ∈ Rn denotes the noise. Eq.(3) only states that
the fluctuating signal is in the quantization bins:
x∗i + ξi ∈ [yi , yi + 1) .
(4)
It does not decide the value range of estimated signal
x∗ .
Figure 1 shows example results of BDE using two different models, Eq.(1) and Eq.(3). The quantized signal
(green) given from the fluctuating signal (cyan) is input for a BDE method, and then the dequantized signal
Received Fabruary 6, 2015; Revised
May 9, 2015; Accepted; May
16,
Recieved
; Revised
Accepted
2015
†Graduate
† GraduateSchool
SchoolofofInformation
InformationScience
Scienceand
andTechnology,
Technology,Hokkaido
Hokkaido
University
University
(Sapporo,
Japan)
(Kita 14,
Nishi 9, Kita-ku, Sapporo, 060–0814, Japan)
251
ITE Trans. on MTA Vol. 3, No. 4 (2015)
8
6
Intensity level
this section. A brief description of the graph Fourier
transform and an application for BDE are provided.
2. 1 Graph Fourier transform
The graph Fourier transform is a technique of graph
signal processing. It enables analyzing signals on graphs
in the graph spectral domains. We can use this to measure the smoothness of an image.
Now we define a graph G = {V, E, W} that consists
of a set of vertices V (i.e., pixels of an image), a set of
edges E, and an adjacency matrix W ∈ Rn×n . W store
edge weights between one pixel and another. If there
is an edge e = (i, j) connecting vertices vi and vj , the
entry Wi,j represents the weight of the edge; otherwise,
Wi,j = 0.
The graph Laplacian matrix L ∈ Rn×n is defined as
L = D − W, where the degree matrix D ∈ Rn×n is a
diagonal matrix satisfying Di,i = j Wi,j . The graph
Fourier transform is defined as
Fluctuating signal x + ξ
Quantized signal y = Q(x + ξ)
Dequantized signal
7
5
4
3
2
1
0
−1
8
Fluctuating signal x + ξ
Quantized signal y = Q(x + ξ)
Dequantized signal
7
Intensity level
6
5
4
3
2
1
0
−1
Spatial position
(a) Dequantization results
,QWHQVLW\OHYHO
λ = vT Lv,
where v = [v1 , v2 , . . . , vn ]T ∈ Rn is a graph signal.
When the graph signal v is smooth, the scalar λ takes
a small value. Thus, the graph Fourier transform can
be used to measure the smoothness of a graph signal.
Refer to (Shuman et al., 201316) ) for more details.
2. 2 BDE via the maximum a posteriori
(MAP) estimation
One particular BDE method estimates the original
image by using a graph Fourier transform and MAP
estimation15) . To find the most probable x from an observed y, they use a MAP solution:
6SDWLDOSRVLWLRQ
(b) Zoom-in. The blue result can not stray from quantization bins
(gray).
Fig. 1
(5)
Example results of BDE methods based on different models Eq.(1) (blue signal) and Eq.(3) (red
signal)
(blue or red) is obtained. The upper of Figure 1(a) is
generated by one of flooding–based method12) , and the
lower by our method described later in this paper. The
result of Eq.(1) fails to reconstruct smooth gradations
because the dequantized signal must be in quantization bins. The allowed range for dequantized signal,
the quantization bins denoted as gray in Figure 1(b), is
raised and collapsed incessantly at noisy area. Therefore, to achieve a continuous function, the dequantized
signal is compelled to be flat by taking lower bounds of
the raised bins and upper of the collapsed bins.
Because the result of Eq.(3) has smooth gradations,
it is better. We explain the BDE method using Eq.(3)
in this paper.
x∗ = arg maxPr (y|x)Pr (x),
(6)
where Pr (y|x) is a conditional probability of y under
the condition x and Pr (x) is a prior probability of x.
Their optimization problem is formulated as (see also
their paper15) ):
minxA σl2 xTA Mp xA + σq2 ||xA − fA ||22 ,
s.t.|xAi − yA i | < 0.5, ∀i, i xAi = 0,
(7)
where σl , σq are parameters to decide the balance of
terms, xA is an AC component of x, M is a graph Laplacian matrix designed for their problem, p is a positive
integer, and fA is the fidelity. The problem Eq.(7) is
iteratively solved. The fidelity fA is initialized as yA ,
which is an AC component of observed y, at first itera∗(k+1)
is obtained
tion. After that, (k + 1)–th solution xA
by solving Eq.(7) with fidelity updated to (k)–th solu∗(k)
tion: fA = xA . The graph Fourier transform is used
2. Image processing using spectral graph
theory
We describe related work needed for our method in
252
Paper » Smooth Gradation Reconstruction from False Contours in Quantized Natural Image
as a smoothness term.
/ )(3(d
/ )(3d
3. Formulation
/ )(3x
r7
where Ni is a set of 4 nearest neighbors of the i–th
pixel, and κ is a threshold parameter. The adjacency
matrix W stores information about the connections in
a graph. In the image domain, Wi,j = 1 denotes that
pixels yi and yj are in the same region.
Using the graph Fourier transform, we define a prior
probability Pr (x) as
xT Lx
,
(9)
Pr (x) = exp −
2σs2
r7 Ld
?7
Fig. 2
Function shape of Eq.(11)
range, we define the probability of yi as
bi (xi )
Pr b (yi |xi ) = exp −
,
2σb2
⎧
⎪
⎪
(y
− x i )2
⎪
⎨ min
bi (xi ) = (ymax − xi + 1)2
⎪
⎪
⎪
⎩0
where σs is a parameter to adjust the smoothness.
3. 2 Likelihood
We guess the distribution of noise ξ as a normal distribution in our method. Thus, the probability distribution of x̃i = xi + ξi generated from xi with added a
stochastic variable ξi , Pr (x̃i |xi ), can be formulated as
a common normal distribution function:
(x̃i − xi )2
1
,
(10)
exp −
Pr (x̃i |xi ) = 2σg2
2πσg2
(12)
(yi = ymin ) ,
(yi = ymax ) ,
otherwise.
(13)
The likelihood that Pr (y|x) is calculated by using
Eq.(11) and Eq.(12) is as follows:
Pr (y|x) =
where σg is the variance of the distribution of ξi . From
the definition of the quantization bin (Eq.(4)), yi is generated when x̃i is in range [yi , yi + 1). Therefore the
probability of yi with given xi , Pr l (yi |xi ), can be calculated by accumulating Pr (x̃i |xi ) for all possible patterns of x̃i as follows:
yi +1
Pr l (yi |xi ) =
Pr (x̃i |xi )dx̃i
yi
⎛ ⎞
yi − x i + 1
yi − x i ⎠
1⎝
erf
=
−erf ,
2
2σ 2
2σ 2
g
/ )d
3UREDELOLW\h„q wr7 /?7 D
In this section, we formulate the MAP estimation
based on the model Eq.(3).
3. 1 Prior probability
In this paper, we define a edge weight Wi,j of quantized image y as follows:
⎧
⎨1 (j ∈ N ∧ |y − y | < κ) ,
i
i
j =
(8)
Wi,j =
⎩0 otherwise,
n
Pr b (yi |xi )Pr l (yi |xi ).
(14)
i=1
The probability Pr b (yi |xi ) works as a bias to shift the
peak of probability Pr l (yi |xi ).
3. 3 Optimization
We define the following optimization problem by
combining Eq.(6), Eq.(9), and Eq.(14):
F =
g
(11)
n
1 1 T
x
Lx
+
bi (xi )
2σs2
2σb2 i=1
z
−
where erf(z) = √2π 0 exp(−t2 )dt is called the error
function. Figure 2 shows the function shapes of Eq.(11).
Here is another formulation for a special case. When
yi is a maximum (ymax ) or minimum (ymin ) value of
its possible range, we can empirically estimate that the
signal may be saturated. At the boundary of the signal
n
log Pr l (yi |xi ),
(15)
i=1
x∗ = arg min F.
(16)
x
The partial derivatives of objective function F are as
follows:
253
ITE Trans. on MTA Vol. 3, No. 4 (2015)
)OXFWXDWLQJVLJQDO
4XDQWL]HGVLJQDO
5HVXOWRIORZSDVVILOWHULQJ
,QWHQVLW\OHYHO
6SDWLDOSRVLWLRQ
Fig. 3
Table 1
(a) For artificial images. Voters choose an image most similar to
displayed correct one.
Simulation of low-pass filtering
Visual quality evaluation by voting
Percentage of votes (%)
Wan 201213) Wan 201415) Proposed
Image
Artificial I
12.9
6.5
80.6
Artificial II
22.6
16.1
61.3
Natural I
12.9
6.5
80.6
Natural II
6.5
3.2
90.3
Natural III
38.7
9.7
51.6
(b) For natural images. Voters choose his or her favorite image
without seeing original.
Fig. 4
peak signal to noise ratio (PSNR) and voting by persons. Note that the PSNR values of Figure 6 are not
valid because a ground-truth signal cannot be provided
for natural images.
Two types of the conventional methods are compared
with our method: the flooding–based one13) and the
optimization–based one15) . The reason why the low–
pass based methods are excluded is that they are not
suitable to reconstruct smooth gradation from false contours. Figure 3 shows the simulation result of decontouring using a low-pass filter. When the flat region is
in the input signal, the contour effect remains in the result. The low–pass filtering can remove high–frequency
effects in quantized signal, but can not remove flat region because flat signal is completely low–frequency signal. The filter can only convert the step function (quantized signal in Figure 3) to the smoothed step function
(result signal in Figure 3). Actually, we want to obtain a smooth slope–like function from a step function.
Therefore, the low–pass based methods fails to reconstruct smooth gradation from large contours typically
appearing in the sky, sea or skin.
4. 1 Experimental results
Figure 5 shows the results for artificial data. The
original image was generated with additional noise in
our computer. Then, the image was quantized and inputted into each method. The conventional methods
cannot reconstruct smooth gradation because of noise,
but our method can.
(Grayscale 4 → 8 bits, Number of samples = 31)
∂F
1
1
Pr l (yi |xi )
,
= 2 [Lx]i + 2 bi (xi ) −
∂xi
σs
2σb
Pr l (yi |xi )
bi (xi )
⎧
⎪
⎪
−2(ymin − xi )
⎪
⎨
=
−2(ymax − xi + 1)
⎪
⎪
⎪
⎩0
Pr l (yi |xi ) = 1
2πσg2
Voting page samples (screenshots)
(17)
(yi = ymin ) ,
(yi = ymax ) ,
otherwise,
(18)
(yi − xi + 1)2
− exp −
2σg2
(yi − xi )2
+ exp −
,
2σg2
(19)
where [Lx]i is an i–th element of the vector Lx. The
problem can be solved with the L–BFGS–B quasiNewton method17)˜19) .
4. Experimentation
We compared the results of the conventional methods and our method in Figure 5, Figure 6 and Table 1.
The outputs were estimated from a quantized image.
We evaluated their image qualities by calculating the
254
Paper » Smooth Gradation Reconstruction from False Contours in Quantized Natural Image
(a) Ground-truth (left) and quantized image after adding noises
(right) of “Artificial I”
(a) Original (left) and quantized image (right) of “Natural I”
(b) Wan et al. (2012)13) (PSNR:41.9 dB)
(b) Wan et al. (2012)13) (PSNR:36.4 dB)
(c) Wan et al. (2014)15) (PSNR:34.1 dB)
(c) Wan et al. (2014)15) (PSNR:35.2 dB)
(d) Our method (PSNR:46.1 dB): κ = 1, σs = 0.01, σg = 0.1 and
σb = 0.5
Fig. 5
Results and absolute error maps of BDE
(grayscale 4 → 8 bits) for an artificially generated data. For (b)–(d), the left column shows
results and the right error maps against the
ground–truth. The ranges of all error maps are
amplified (×16) for visibility.
(d) Our method (highest PSNR) (PSNR:36.7 dB): κ = 1, σs =
1, σg = 0.1 and σb = 10000
Figure 6 shows the results for a natural image. Our
results selected two as the highest PSNR image and
the subjective best. In the sky of the input image, false
contour artifacts are evident. Our method sufficiently
removed the artifacts (Figure 6(e)). This demonstrates
that it can reconstruct smooth gradations efficiently.
Table 1 shows the visual quality evaluations by persons. The voters are presented three images which are
outputs of Wan et al. (2012)13) , Wan et al. (2014)15)
and our method (subjective best). And then, they
choose one’s best image.
The voting was held on a specially prepared web site.
Voters access to the site and look pages like Figure 4
with digital monitor. There are a simple question and 3
links to output images (See Figure 4). The order of the
links are randomly shuffled for each voter. For artificial images, the ground-truth image is opened to voters,
(e) Our method (subjective best) (PSNR:33.2 dB): κ = 1, σs =
0.05, σg = 0.1 and σb = 10000
Fig. 6
Results and absolute error maps of
(grayscale 4 → 8 bits) for natural image
BDE
and they choose an image most similar to it. On the
other hand, voting for natural images, original image is
not opened. The voters can only see 3 output images,
and they have to decide which one is a most quality
photograph (See appendix section A for more details
about voting environments). It is blind test because
the voters are not able to know any identities about
the images. As a result, most of the voters chose our
method in every case.
255
ITE Trans. on MTA Vol. 3, No. 4 (2015)
σs
Output
Error map
PSNR (dB)
0.01
33.5
0.1
35.8
1
38.5
10
36.7
Original image
Input
Fig. 7
Results with different σs values for “Natural III”: κ = 1, σg = 0.1 and σb = 10
you can obtain a higher PSNR result by setting a larger
σs (like Figure 6(d)).
4. 2 Parameters settings
Four parameters κ, σs , σg and σb should be set for
our method. The parameter κ is the threshold parameter to separate different regions in an image. For 4 → 8
bits BDE, it is set as κ = 1 normally because we assume
that the contour effects are constructed only with fluctuations of the least significant bit. The parameter σs
decides the magnitude of the first term in Eq.(16). The
parameter σg should be set as the actual noise variance
if possible. The parameter σb should be small when
signal clipping occurs in the bright or dark area.
The smoothness of the output is mainly controlled
by σs . Figure 7 shows the results of our method with
different σs values. The determination of the σs value
depends on whether or not gradation smoothness is important for image quality. In this paper, the subjective
best images (like Figure 6(e)) are generated by setting
small σs to smooth false contours. If you think that accuracy is more important than gradation smoothness,
5. Conclusion
We proposed a method for improving BDE quality
for natural images. We formulated an inverse quantization problem via MAP estimation focusing on noise
distribution. The smoothness of images was measured
using the graph Fourier transform. Our method can
recover smooth gradations from false contour artifacts.
In addition, we examined what is important for the
image quality. We showed that the higher PSNR did
not mean the higher image quality. The images with
smooth gradations were rated as high quality by persons. Therefore, our method can obtain the high quality
output.
References
1) H. Seetzen, W. Heidrich, W. Stuerzlinger, G. Ward, L. Whitehead, M. Trentacoste, A. Ghosh and A. Vorozcovs: “High dy-
256
Paper » Smooth Gradation Reconstruction from False Contours in Quantized Natural Image
2)
3)
4)
5)
6)
7)
8)
9)
10)
11)
12)
13)
14)
15)
16)
17)
18)
19)
app.Table 1
namic range display systems”, ACM SIGGRAPH 2004 Papers,
SIGGRAPH ’04, New York, NY, USA, ACM, pp. 760–768 (2004)
A. Taguchi and J. Nishiyama: “Bit-length expansion by inverse quantization process”, Signal Processing Conference (EUSIPCO), 2012 Proceedings of the 20th European, pp. 1543 –
1547(Aug. 2012)
C. H. Liu, O. Au, P. Wong, M. Kung and S. C. Chao: “Bit-depth
expansion by adaptive filter”, IEEE International Symposium on
Circuits and Systems, 2008. ISCAS 2008, pp. 496 –499(May 2008)
S. J. Daly and X. Feng: “Decontouring: prevention and removal
of false contour artifacts”, Proc. SPIE, Human Vision and Electronic Imaging IX, Vol. 5292, San Jose, CA, SPIE, pp. 130–
149(June 2004)
H.-R. Choi, J. Lee, R.-H. Park and J.-S. Kim: “False contour reduction using directional dilation and edge-preserving filtering”,
IEEE Transactions on Consumer Electronics, 52, 3, pp. 1099 –
1106(Aug. 2006)
M.-H. Park, J. W. Lee, R.-H. Park and J.-S. Kim: “False contour reduction using neural networks and adaptive bi-directional
smoothing”, IEEE Transactions on Consumer Electronics, 56, 2,
pp. 870 –878(May 2010)
W. Ahn and J.-S. Kim: “Flat-Region Detection and False Contour Removal in the Digital TV Display”, IEEE International
Conference on Multimedia and Expo, 2005. ICME 2005, pp. 1338
–1341(July 2005)
N. Xu and Y.-T. Kim: “A simple and effective algorithm for
false contour reduction in digital television”, 2010 Digest of Technical Papers International Conference on Consumer Electronics
(ICCE), pp. 231 –232(Jan. 2010)
J. W. Lee, B. R. Lim, R.-H. Park, J.-S. Kim and W. Ahn: “Twostage false contour detection using directional contrast and its
application to adaptive false contour reduction”, IEEE Transactions on Consumer Electronics, 52, 1, pp. 179 – 188(Feb. 2006)
C.-H. Cheng, O. Au, N.-M. Cheung, C.-H. Liu and K.-Y.
Yip: “Low color bit-depth image enhancement by contour-region
dithering”, IEEE Pacific Rim Conference on Communications,
Computers and Signal Processing, 2009. PacRim 2009, pp. 666
–670(Aug. 2009)
S. Bhagavathy, J. Llach and J. Zhai: “Multiscale Probabilistic Dithering for Suppressing Contour Artifacts in Digital Images”, IEEE Transactions on Image Processing, 18, 9, pp. 1936
–1945(Sept. 2009)
C.-H. Cheng, O. Au, C.-H. Liu and K.-Y. Yip: “Bit-depth expansion by contour region reconstruction”, IEEE International
Symposium on Circuits and Systems, 2009. ISCAS 2009, pp. 944
–947(May 2009)
P. Wan, O. C. Au, K. Tang, Y. Guo and L. Fang: “From 2D
Extrapolation to 1D Interpolation: Content Adaptive Image BitDepth Expansion”, 2012 IEEE International Conference on Multimedia and Expo (ICME), pp. 170 –175(July 2012)
P. Wan, O. Au, K. Tang and Y. Guo: “Image de-quantization via
spatially varying sparsity prior”, 2012 19th IEEE International
Conference on Image Processing (ICIP), pp. 953–956 (2012)
P. Wan, G. Cheung, D. Florencio, C. Zhang and O. Au: “Image
bit depth enhancement via maximum-a-posteriori estimation of
graph AC component”, IEEE Institute of Electrical and Electronics Engineers(Oct. 2014)
D. Shuman, S. Narang, P. Frossard, A. Ortega and P. Vandergheynst: “The emerging field of signal processing on graphs:
Extending high-dimensional data analysis to networks and other
irregular domains”, IEEE Signal Processing Magazine, 30, 3, pp.
83–98(May 2013)
R. Byrd, P. Lu, J. Nocedal and C. Zhu: “A Limited Memory
Algorithm for Bound Constrained Optimization”, SIAM Journal
on Scientific Computing, 16, 5, pp. 1190–1208(Sept. 1995)
C. Zhu, R. H. Byrd, P. Lu and J. Nocedal: “Algorithm 778: LBFGS-B: Fortran Subroutines for Large-scale Bound-constrained
Optimization”, ACM Trans. Math. Softw., 23, 4, pp. 550–
560(Dec. 1997)
J. L. Morales and J. Nocedal: “Remark on “Algorithm 778: LBFGS-B: Fortran Subroutines for Large-scale Bound Constrained
Optimization””, ACM Trans. Math. Softw., 38, 1, pp. 7:1–
7:4(Dec. 2011)
Voter’s platform
Platform name
number of voters
Windows
26
iPhone
2
Macintosh
1
Linux
1
X11
1
app.Table 2
Voter’s web browser
Browser name
number of voters
Chrome
17
Firefox
5
Internet Explorer
5
Safari
3
Android
1
app.Table 3
Voter’s monitor size
Monitor size (width×height)
number of voters
1920×1080
15
1680×1050
5
1536×864
3
1280×1024
2
1080×1920
1
1366×768
1
2560×1440
1
320×568
1
375×667
1
720×1280
1
Appendix
A. Voting environment
Voters accessed to the voting site from various environments through the internet. We did not fix machine
platforms, web browsers and display settings for simplicity. We only limited the number of accessible time
to one per voter. Here shows statistics of the voter’s
platforms (app.Table 1), web browsers (app.Table 2)
and monitor sizes (app.Table 3).
Akira Mizuno
received his B.S. and M.S.
degree in electrical engineering from Hokkaido University, Hokkaido, Japan, in 2011 and 2013, respectively. His current research includes image processing. Mr. Mizuno is a student member of the IEICE.
Masayuki Ikebe
received his B.S., M.S.
and Ph.D. degrees in electrical engineering from
Hokkaido University, Hokkaido, Japan, in 1995,
1997 and 2000, respectively. During 2000–2004,
he worked for the Electronic Device Laboratory,
Dai Nippon Printing Corporation, Akabane, Japan,
where he was engaged in the research and development of wireless communication systems and image processing systems. Presently, he is an Associate Professor of the Graduate School of Information Science and Technology at Hokkaido University. His current research includes CMOS image
sensors and analog circuits. Dr. Ikebe is a member
of the IEEE and the IEICE.
257