Uniform and textured Regions Separation in Natural images towards MPM Adaptive Denoising Noura Azzabou (1,2) Nikos Paragios (1) Frédéric Guichard(2) (1) Laboratoire MAS, Ecole Centrale de Paris, Grande Voie des Vignes, France (2) DxOLabs, 3 Rue Nationale, Boulogne Billancourt, France [email protected], [email protected], [email protected] Abstract. Natural images consist of texture, structure and smooth regions and this makes the task of filtering challenging mainly when it aims at edge and texture preservation. In this paper, we present a novel adaptive filtering technique based on a partition of the image to ”noisy smooth zones” and ”texture or edge + noise” zones. To this end, an analysis of local features is used to recover a statistical model that associates to each pixel a probability measure corresponding to a membership degree for each class. This probability function is then encoded in a new denoising process based on a MPM (Marginal Posterior Mode) estimation technique. The posterior density is computed through a non parametric density estimation method with variable kernel bandwidth that aims to adapt the denoising process to image structure. In our algorithm the selection of the bandwidth relies on a non linear function of the membership probabilities. Encouraging, experimental results demonstrate the potential of our approach. 1 Introduction In spite of the progress made in the field of image denoising, we can claim that it is still an open research problem. Traditional techniques of image enhancement and noise reduction rely on the assumption that image is homogeneous at local scale. Natural images consist of smooth and patterned regions like texture and therefore the use of such a simplistic denoising techniques deteriorate the quality of the reconstruction in regions with texture. Texture, refers to regions with repetitive patterns and structure at various scale and orientations. In this paper our goal is to propose a technique that takes into account the particularities of such patterns and design an adaptive enhancement technique able to preserve texture while removing noise. State of the art techniques in image enhancement refers to local methods, image decomposition in orthogonal spaces, partial differential equations as well as complex mathematical models. Filters and morphological operators are the most prominent local approaches [13, 19, 22, 5] and exploit homogeneity of the image 2 N. Azzabou N. Paragios F. Guichard through convolution. Global methods represent images through a set of invertible transformations of an orthogonal basis [14, 8, 12] where noise is removed through the modification of the coefficients with limited importance in the reconstruction process. Partial differential equations [23, 17, 1, 2, 20] like the heat equation, anisotropic diffusion , etc. incorporate more structure in the denoising process where the noise-free images corresponds to the steady state solution of the PDE. Last, but not least global approaches [16, 18, 11] recover the noise-free image through the lowest potential of a cost function that aims to separate image structure from noise. Presence of texture often violates the fundamental assumption considered in the image enhancement field that is local homogeneity and despite numerous provisions of the above methods, denoising of texture images is still an open problem. Separating structure from texture is the most prominent technique to deal with such limitation and has gained significant attention in the past years [4, 21, 15]. These techniques model images as a mixture of a uniform and an oscillatory component which are computed through optimization of a specifically designed cost. In spite of their performance, these methods fail sometimes to separate noise from texture. A more recent work introduced in [3] addresses such limitation but the proposed approach is highly dependent on the noise model. One can conclude that, traditional/state-of-the art techniques make the assumption that an image is a mixture of a piecewise constant and an oscillatory component is relative to the noise. It is clear that this hypothesis is not satisfied because texture is also an oscillatory pattern and many examples of natural images show that it’s sometimes difficult to distinguish between noisy regions and textured ones. An effort to address this limitation was carried out in [10] where an adaptive total variation algorithm was proposed. This algorithm selects the coefficient of the fidelity-to-data term according to the presence or not of texture. The texture characterization is only based on local variance information which often is not sufficient to separate texture from noise. In the present paper, we propose a novel denoising technique which relies in a soft pre-classification step that aims to (i) identify regions where the assumption of local smoothness is valid and (ii) patterned regions that contain texture, edges and other image details. To this end, we propose an automatic technique of image partition into two classes: ”homogeneous regions + noise” and ” (texture, edges, details) + noise”. This partition is performed through a local feature analysis and assigns to each pixel a probability measure that reflects its degree of membership to noise or texture. This classification is then integrated in a non parametric image model with variable bandwidth kernel to perform an MPM (Marginal Posterior Mode) estimation of the original image. The reminder of the paper is organized as follows; section 2 is devoted to the soft image classification. In section 3, we present our new non-parametric image model to perform image denoising. Validation and comparisons with the state of the art methods are presented in section 4. Finally, we conclude in section 5. MPM adaptive denoising 2 3 Texture and Homogeneous region classification Analysis of texture has been a long term research topic in computer vision. While most of the existing techniques aim to separate different texture patterns, we focus on a simpler problem that is separation of texture and smooth regions (with noise patterns being present). In other words, if we consider a local image patch, we want to know whether it corresponds to a texture pattern or to a uniform one altered because of noise. To this end, relative image moments (the mean is subtracted) at local scale are computed. If we assume that noise is independent from signal and independent from image position, we expect that all image patches that correspond to the noise will produce similar local descriptors, that is not the case of edges as well as textured regions. We consider a fairly simple set of feature vectors that consists of the variance, the skewness, the kurtosis and the entropy. With this set of features we can capture local behavior for each pixel and thus can discriminate between patterns that correspond to noise and the ones with important deviations that refer to texture or edges and small details. Intuitively, we expect that image patches that correspond to noise will have similar variance (the noise variance), with a low skewness value because noise have a symmetric behavior. Once local descriptors are computed, our main concern is to define a proper statistical model to interpret them towards classification. The dimension of the feature vector and the small number of present samples (number of pixels in the image) makes density estimation in this space rather impractical. In order to reduce the dimensionality of our problem, we perform a principle component analysis (PCA), towards a linear transformation of the feature space that retains the largest amount of variation within the data. Such a selection is a reasonable compromise between computing complexity and discrimination power. Thus, we decrease the dimensionality of the classification problem by considering the projection of the feature vector on the first mode of variation. Once the feature space has been determined, classification consists of assigning to each pixel of the image a probability value according to its membership to a textured pattern or a noisy patch. We recall that our observation space is the projection of the local descriptor on the first eigen vector. Now, our concern is to find a statistical model that is able to describe the distribution of the observation and provides an automatic classification tool of them. To this end, we use the gaussian mixture model which is a very popular tool to approximate a probability density function of the projected samples. Considering that three population are present in an image that are: edges, texture and smooth region, we can consider that each gaussian in the model describes a population. Let’s call O = (o1 , o2 , ..., on ) the n unlabeled observations corresponding to the feature vectors of the image pixels. We can consider in the case of three component: p(ox |Θ) = Pedge pedge (ox ) + Ptex ptex (ox ) + Psmooth psmooth (ox ) with Θ being the parameter vector and Pedge , Ptex , Psmooth are respective conditional probabilities of edges, texture and noise. Recovering the prior marginals 4 N. Azzabou N. Paragios F. Guichard (a) (b) (c) (d) Fig. 1. Example of partition of an image: (a) original image, and conditional probability function relative to (b) ”smooth component” p(ox |smooth) , (c) ”texture” p(ox |tex) and (d) ”edges” p(ox |edge) (Pedge , Ptex , Psmooth ) and the parameters of each gaussian is done according to the maximum likelihood principle and the Expectation Maximization (EM) algorithm [9]. Some results of unsupervised image partitions based on gaussian mixture model is shown in [Fig(1)]. It is important to point out the fact that we perform a fully unsupervised classification where each gaussian is representative of one image component. Thus, we need to assign a label to each gaussian component and more explicitly we want to know which one of the computed gaussian is representative of the smooth component in the image. Under the assumption that an image consists mostly of smooth regions we can use the prior density of each gaussian to assign labels to them according to the following relation Psmooth > Ptex > Pedge . Parallel to that, knowing that pixels that belong to a uniform noisy patch have similar descriptor we expect that the gaussian which represents the noise will have a weak variance compared to the others, an assumption that has been validated by experimental results. For the example shown in [Fig.(1)], the gaussian component with the smallest variance and its dominance over the other hypotheses corresponds to the column (b). It is clear that this component refers to the class describing the uniform assumption with noise. 3 Non parametric model and adaptive denoising These memberships now can be encoded in the denoising process towards adjusting the behavior of the algorithm according the pixel classification. In this section we will focus on the definition of our denoising model as well as the use of the partition step to perform adaptive denoising. 3.1 Non parametric model To introduce our denoising method, let us consider I, U and N three random variables defined on a discrete partition Ω ∈ Z2 relative to the image domain related according to: I =U +N MPM adaptive denoising 5 with N being an additive noise independent from the signal, I is the observed noisy image and U is the noise-free image. Given such a model, we consider a method based on Marginal Posterior Mode estimation. It consists in estimating the intensity of a given pixel by maximizing its conditional probability relative to the whole observed noisy image. This estimation is done in an independent manner for each image pixel. Thus, the estimate Ûx of the original observation at a given position x satisfies, Ûx = argmaxUx [p (Ux |I)] Considering that U and I are random markov fields in Ω, we can define the conditional probability accordong to the observations in the neighborhood of the pixel instead of the whole image. Thus if we note Nx , a local neighborhood of x the marginal posterior is defined as: p (Ux |I) = p (Ux |I (Nx )) = p (Ux , I (Nx )) p (I (Nx )) To perform the MPM estimation, one has to determine a model relative to the posterior. To this end, we consider a non parametric density function based on multidimensional gaussian kernels with variable bandwidth. The set of samples used to perform this estimation is extracted from the local neighborhood of the observed noisy image. 1 X p (Ux , I (Nx )) = Gy k(Ux , I(Nx )) − (Iy , I(Ny ))k2 M y∈Rx with M being the total number of observations and Rx the local neighborhood system relative to x. Gy is a multidimensional isotropic gaussian kernel with zero mean and a covariance matrix Σ = σy2 In where In is the identity matrix. The bandwidth of these kernels depends on the image content associated with the given pixel position and plays a critical role in the denoising process. The selection of this parameter using the image partition introduced in the beginning of this work will be discussed later. The posterior can now be expressed as follows: P y∈Rx Gy k(Ux , I(Nx )) − (Iy , I(Ny ))k2 p (Ux |I (Nx )) = (1) M p (I (Nx )) the maximum of this probability density function corresponds to the value that provides the optimal numerator since the denominator is constant with respect to U . This function penalizes high photometric distances between similar neighbouring pixels which reduces the amount of noise in the image. A calculus of variation and a gradient descent algorithm is used to compute an estimate of the mode of the marginal posterior probability. If we note, E the numerator of the previous expression, its gradient with respect to Ux in case of gaussian kernel is equal to: ∂ X ∂E =− Gy k(Ux , I(Nx )) − (Iy , I(Ny ))k2 ∂Ux ∂Ux y∈Rx 6 N. Azzabou N. Paragios F. Guichard X Iy − Ux ∂E Gy k(Ux , I(Nx )) − (Iy , I(Ny ))k2 = 2 ∂Ux σy y∈Rx If we introduce wxy that are weights reflecting the image content agreement between local neighborhood around x and y and defined as: wxy = Gy k(Ux , I(Nx )) − (Iy , I(Ny ))k2 The gradient energy expression becomes: X Iy − Ux ∂E wxy = ∂Ux σy2 y∈Rx According to the gradient descent algorithm the estimated intensity is updated according to: X Iy − U t x t+1 t Ux = Ux − dt wxy (2) σy2 y∈Rx X wxy X wxy Iy − Ut = Uxt − dt σy2 σy2 x y∈Rx y∈Rx Where dt is the time step and Uxt is the intensity value at time t. We point out that such an expression bears certain similarities to the well known mean shift filtering algorithm [7] where the update is proportional to the mean shift value which is defined by the distance between the weighted mean of samples using the kernel G and Uxt the center of the kernel window. P 2 y∈Rx wxy /σy Iy t t mG (Ux ) = Ux − P 2 y∈Rx wxy /σy The main difference between our approach and the mean shift filtering algorithm lies in the fact that we consider in the mean computation a local neighborhood and we don’t restrict the observation to only a pixel level. Therefore, we are able to improve the performance in the case of textured regions because the process goes beyond simple pixel-wise comparisons between pixels to be extended to the entire local neighborhood. An important parameter of the proposed denoising approach is the kernel bandwidth. A main contribution of the present paper is to consider variable bandwidth gaussian, to model the posterior probability density function. The selection of this parameter will be the focus of the following section. 3.2 Bandwidth selection The role of the bandwidth in the density approximation is to guarantee the proper use of samples when constructing the pdf. In the case of smooth regions we can assume that all samples have equal contribution on the pdf and MPM adaptive denoising 7 therefore, their bandwidth should reflect such condition. Towards decreasing the importance of variation in the kernel, we can consider the increase of their bandwidth. All the samples will inherit equal importance from such a selection within the pdf approximation process. In case of textured regions or edges such a choice will over smooth the small details. That’s why in case of sparse distribution we will use smaller bandwidth values which lead to a multi modal density function which is coherent with the fact that in natural image the local histograms in textured regions and edges are often multi modal. Such a selection relies on the implicit assumption that only a small portion of samples express the pdf. The selection of these samples is purely based on the photometric matching between the associated patches. In the other hand, using small bandwidth in case of texture and edges, enable a better selection of the neighborhood samples that will be used in the intensity estimation. To satisfy such a demand, it is more appropriate to use kernels with variable bandwidth than fixed ones in order to guarantee at the same time detail preservation and good denoising. To this end, we introduce a new function to determine the gaussian kernel bandwidth using the conditional probability values obtained after the partition step. Such a function should be monotonically decreasing with respect to the conditional probability relative to textured region or edges. One possible choice for such a function is defined as follows: psmooth (oy ) +c (3) σy = σ0 psmooth (oy ) + ptex (oy ) + pedge (oy ) we recall that oy is the observation relative to the feature vector in the position y and psmooth ,ptex ,pedge are respectively the conditional probability for oy to be in a noisy smooth region, textured region and edges. σ0 and c are parameters to be fixed according to noise level. With such a choice, we adopt for pixels that belong to smooth regions with high value of psmooth , high kernel bandwidth. For image component identified as texture or edges, psmooth tends to be close to zero and implies smaller bandwidth values 1 . 4 Experimental results and validation One can now use the theoretical framework introduced in the previous section for image enhancement. This denoising method is based on a non parametric model estimation of marginal posterior mode. This model is based on an automatic and unsupervised partition of the image on local smooth regions and textured one. Parameters that are involved in our denoising approach are the size of the two neighborhood Rx and Nx as well as the bandwidth of the kernels that are selected according to expression (3). Rx is the size of the neighborhood used for the posterior estimation. One can think that using an important size of Rx allows a better estimation, but such a choice will involve many irrelevant samples in the 1 In order to account for error in classification due to noise a morphological filtering approach is used to smooth the obtained probability map 8 N. Azzabou N. Paragios F. Guichard estimation process. Experimental validation has shown that choosing Rx = 9 × 9 gives good results while remaining computationally efficient. Nx is the size of the noisy patch around the pixel that we want to recover, in our experiments Nx is set to 7 × 7. Finally the choice of the photometric bandwidth is dependent on noise level. In case of additive gaussian noise with standard deviation σn = 20, we considered this couple of parameter (σ0 =4, c = 4). The contribution of the use a variable kernel bandwidth (MPMvar ) towards fixed one (MPMf ix ) was also evaluated through our tests. In order to evaluate the performance of our method, we have used natural images corrupted by a synthetic gaussian noise (σn =10,20) as well as digital images corrupted by real camera noise. We compared our approach to well known filtering techniques such as the bilateral filter [19], the Non Local Mean approach [6], the total variation [18] and the anisotropic filtering [17] using an edge stopping func2 tion of the type (1 + |∇I| /K 2 )−1 . The parameters of the considered methods were tuned to get a good balance between texture preserving and noise suppression as well as the highest possible PSNR value. As far as subjective criteria are concerned, we adopt the whole aspect of the image in term of noise suppression and small detail preservation. Visual comparison results of denoising [Fig.(2), (3)] show that the total variation, the anisotropic diffusion and the bilateral filter, fail to preserve small detail and image texture. Furthermore, one can observe structured noise component in the presence of texture. We explain such behavior by the local nature of these methods where comparisons and structure information are considered only on a pixel level and not at local patches. Better quality of denoising was reached using the proposed approach, since the residual images contain less image structure compared to other techniques [Fig.(3)]. When considering real digital camera noise [Fig.(4,5,6)], we noticed a better restoration using our method with variable bandwidth kernels. In [Fig.(4,5)], we noticed that in case of fixed bandwidth the texture skin is over smoothed to lead to an artificial appearance when the variable bandwidth model is able to suppress the same amount of noise in smooth regions while preserving the texture. In fact this region was identified by our classification algorithm as textured region leading thus to more adapted smoothing constraints [Fig.(4-b)]. As far as quantitative validation is concerned we used the Peak Signal to Noise Ratio criterion defined by P SN R = 10log10 2552 M SE M SE = 1 X (Ux − Ûx )2 kΩk x∈Ω where U is the noise free ideal image and Û its estimation by the denoising process. In table 1 and 2, we report experimental validation results for the different methods on a set of image with various content corrupted by additive gaussian noise. PSNR values, confirm the subjective results and show that our non parametric estimation technique outperforms prior state-of-the art methods. Nevertheless for some examples, the classification step fails to capture fine scale details and texture is considered as noise. It tends to be a binary classification leading to MPM adaptive denoising TV AD Bilateral NLmean MPMf ix MPMvar barbara 29.60 30.85 31.05 32.96 33.07 32.61 Boat 32.17 31.92 31.52 32.49 32.57 32.43 FingerPrintHouse 30.65 33.86 29.02 33.72 28.81 33.40 30.60 34.66 30.58 34.8 30.43 34.67 Lena 33.83 33.36 33.01 34.65 34.77 34.48 9 baboon 27.81 28.11 29.31 29.54 29.7 29.62 Table 1. PSNR values for denoised image (The PSNR of the image corrupted by gaussian noise of std=10 is equal to 28.11) an important variation in the choice of the kernel bandwidth and thus affecting the quality of reconstruction. (a) (b) (c) (d) (e) (f) (g) (h) Fig. 2. Results of denoising on barbara image corrupted by gaussian noise with std=10: (a) original (b) noisy image (c) total variation (d) Anisotropic filtering (e) Bilateral filter (f) Non local mean (g) MPMf ix (h)MPMvar 5 Conclusion In this paper we have proposed a new model to image denoising that exploits information of the image context. Such a method first decomposes the image domain into smooth and patterned regions. To this end, we associate to each image location a feature vector that refers to a statistical descriptor of local patches. Then, an analysis on these features provides measures of ”smoothness”. The obtained measures are then encoded in a new denoising process through an MPM estimation based on a non parametric model of the posterior density. As shown in tables (1,2) our method outperforms in all cases the existing ones. In spite of 10 N. Azzabou N. Paragios F. Guichard (a) (b) (c) (d) (e) (f) Fig. 3. Zoom on the residual of the different tested methods: (a) Total variation (b) Anisotropic filtering (c) Bilateral filter (d) Non local mean (e) MPMf ix (f)MPMvar (a) (b) (c) (d) Fig. 4. Results of our proposed denoising method on real digital camera Noise, (a) original image (b) variable bandwidth function (low intensity ( σx =2 ), high intensity (σx =4)) (c)MPMf ix denoising, (d) MPMvar denoising. (a) (b) (c) Fig. 5. Results of our proposed denoising method on real digital camera Noise, (a) original image (b)MPMf ix denoising, (c) MPMvar denoising. MPM adaptive denoising (a) (b) (c) 11 (d) Fig. 6. Results of our proposed denoising method on real digital camera Noise, (a) original image (b) variable bandwidth function (low intensity ( σx =2 ), high intensity (σx =4)), (c)MPMf ix denoising, (d) MPMvar denoising. TV AD Bilateral NLmean MPMf ix MPMvar barbara 26.18 26.45 26.75 28.78 29.18 28.9 Boat 27.72 28.06 27.82 28.92 28.84 29.11 FingerPrintHouse 26.08 28.43 24.81 29.41 24.12 29.18 26.45 30.86 26.38 31.16 26.68 31.02 Lena 28.45 29.27 29.28 31.13 31.19 31.25 baboon 25.18 23.68 24.95 25.18 25.26 25.39 Table 2. PSNR values for denoised image (The PSNR of the image corrupted by gaussian noise of std=20 is equal to 22.15) the marginal improvement of the statistical denoising process while considering the image partition, we believe that this idea is promising and a better integration of the image structure in the denoising model should be studied. In fact, more appropriate selection of the kernels as well as their bandwidth could also improve the performance of the method, and is a direction that we are willing to address in the coming future. Furthermore, better extraction of local features using higher order projections is the most prominent direction to improve the initial step of our approach. References 1. L. Alvarez, F. Guichard, P-L. Lions, and J-M. Morel. Axioms and fundamental equations of image processing. Archive for Rational Mechanics, 123:199–257, 1993. 2. G. Aubert and P. Kornprobst. Mathematical Problems in Image Processing: Partial Differential Equations and the Calculus of Variations. Springer-Verlag, 2001. 3. JF. Aujol and Chambolle A. Dual norms and image decomposition models. International Journal of Computer Vision, 63(1):85–104, 2005. 4. JF. Aujol, G. Gilboa, T. Chan, and S. Osher. Structure-texture image decomposition - modeling, algorithms, and parameter selection. International Journal of Computer Vision, 67(1):111–136, 2006. 12 N. Azzabou N. Paragios F. Guichard 5. N. Azzabou, N. Paragios, and F. Guichard. Random walks, constrained multiple hypotheses testing and image enhancement. In ECCV, pages 379–390, 2006. 6. A. Buades, B. Coll, and J-M. Morel. A non-local algorithm for image denoising. In CVPR, pages 60–65, 2005. 7. D. Comaniciu and P. Meer. Mean shift: A robust approach toward feature space analysis. IEEE Transactions on pattern analysis and machine intelligence, 15(5):603–619, 2002. 8. M. Do and M. Vetterli. Pyramidal directional filter banks and curvelets. In ICIP01, pages III: 158–161, 2001. 9. R.O. Duda and P. Hart. Pattern Classification And Scene Analysis. WileyInterscience Publication, 1973. 10. G. Gilboa, N. Sochen, and Zeevi Y.Y. Variational denoising of partly-textured images by spatially varying constraints. IEEE Transactions on Image Processing, 24(8):2281–2289, 2006. 11. R. Kimmel, R. Malladi, and N. Sochen. Image processing via the beltrami operator. In ACCV, pages 574–581, 1998. 12. E. Le Pennec and S. Mallat. Sparse geometric image representations with bandelets. IEEE Transactions on Image Processing, pages 423– 438, 2005. 13. S. Lee. Digital image smoothing and the sigma filter. CVGIP, 24(2):255–269, November 1983. 14. S. Mallat. A theory for multiscale signal decomposition: The wavelet representation. IEEE Transactions on Pattern and Machine Intelligence, pages 674–693, 1989. 15. Y. Meyer. Oscillating Patterns in Image Processing and Nonlinear Evolution Equations: The Fifteenth Dean Jacqueline B. Lewis Memorial Lectures. American Mathematical Society, Boston, MA, USA, 2001. 16. D. Mumford and J. Shah. Optimal Approximation by Piecewise Smooth Functions and Associated Variational Problems. Communications on Pure and Applied Mathematics, 42:577–685, 1989. 17. P. Perona and J. Malik. Scale space and edge detection using anisotropic diffusion. IEEE Transactions on Pattern and Machine Intelligence, 12:629–639, 1990. 18. L. Rudin, S. Osher, and E. Fatemi. Nonlinear Total Variation Based Noise Removal. Physica D, 60:259–268, 1992. 19. C. Tomasi and R. Manduchi. Bilateral filtering for gray and color images. In ICCV, pages 839–846, 1998. 20. D. Tschumperle and R. Deriche. Vector-valued image regularization with pde’s : A common framework for different applications. IEEE Transactions on Pattern Analysis and Machine Intelligence, pages 506–517, 2005. 21. L. Vese and S. Osher. Modeling Textures with Total Variation Minimization and Oscillating Patterns in Image Processing. Journal of Scientific Computing, 19:553– 572, 2003. 22. L. Vincent. Morphological grayscale reconstruction in image analysis: Applications and efficient algorithms. IEEE Transactions on Image Processing, 2:176–201, 1993. 23. J. Weickert, B.M. ter Haar Romeny, and M. Viergever. Efficient and reliable schemes for nonlinear diffusion filtering. IEEE Trans. lmage Processing, 7(2):398, 1998.
© Copyright 2026 Paperzz