Image Sampling and Quantisation Introduction to Signal and Image Processing Prof. Dr. Philippe Cattin MIAC, University of Basel March 29th, 2016 Introduction to Signal and Image Processing 1 of 46 March 29th, 2016 22.02.2016 09:17 Ph. Cattin: Image Sampling and Quantisation Contents Contents 1 Motivation Introduction and Motivation 3 Sampling Example 4 Quantisation Example 5 2 Sampling 2.1 Tessellation Tessellation 8 Tessellation Examples by M.C. Escher (1) 9 Tessellation Examples by M.C. Escher (2) 10 Tessellation Basics 11 Tessellation Claim 12 How Many Tessellations Exist with Regular Polygons? 13 Combinatorial Analysis 14 All Semi-Regular Tessellations 15 All Regular Tessellations 16 Tessellation Rules 17 Advantages of Square Tessellation 18 2.2 A Sampling Model A Sampling Model 20 The Neighbourhood Function 21 Fourier Transform of the Neighbourhood Function 22 Filtering with the Neighbourhood Function 23 Sampling of a Continuous 1D Function 24 Sampling of a Continuous 1D Function (2) 25 Sampling of a Discrete 1D Function 26 An Alternative Reasoning for Periodicity in the 27 DFT Introduction to Signal and Image Sampling of Processing Two-Dimensional 2 of 46 March 29th, 2016 Functions 28 22.02.2016 09:17 (Images) Summary Sampling Theorem 29 Aliasing Example 1 30 Aliasing Example 2 31 Aliasing Example 3 32 Remark on the Discrete Fourier Transform 33 Linear, Shift-Invariant Operators 34 Linear, Shift-Invariant Operators (2) 35 Liner, Shift-Invariant Operators (3) 36 Liner, Shift-Invariant Operators (4) 37 3 Quantisation Quantisation 39 Lloyd-Max Quantisation 40 Quantisation Example 41 Quantisation Example (2) 42 Quantisation Example (3) 43 Introduction to Signal and Image Processing 3 of 46 March 29th, 2016 22.02.2016 09:17 Motivation Introduction and Motivation (3) In order for computers to process an image, this image has to be described as a series of numbers, each of finite precision This calls for two kinds of discretisation: Sampling, and Quantisation By sampling is meant that the brightness information is only stored at a discrete number of locations. Quantisation indicates the discretisation of the brightness levels at these positions. Introduction to Signal and Image Processing 4 of 46 March 29th, 2016 22.02.2016 09:17 Ph. Cattin: Image Sampling and Quantisation Motivation Sampling Example (4) Sampling is the process of measuring the brightness information only at a discrete number of locations Fig 4.1: Hight profile of Switzerland Introduction to Signal and Image Processing 5 of 46 Fig 4.2: Sampled hight profile March 29th, 2016 22.02.2016 09:17 Ph. Cattin: Image Sampling and Quantisation Motivation Quantisation Example (5) Quantisation is the process of discretising the brightness at a finite number of positions Height map with with grey values grey values with grey values with grey values Fig 4.3: Introduction to Signal and Image Processing 6 of 46 March 29th, 2016 22.02.2016 09:17 Sampling Tessellation Tessellation (8) Definition Tessellations are patterns that cover a plane with repeating figures so there is no overlapping or empty spaces Sampling is best performed following a regular tessellation of the image: 1. Brightness is integrated over cells of same size 2. Cells should cover the whole image These cells are usually referred to as picture elements or pixels. Introduction to Signal and Image Processing 7 of 46 March 29th, 2016 22.02.2016 09:17 Ph. Cattin: Image Sampling and Quantisation Tessellation Examples by M.C. Escher (1) Tessellation (9) Fig 4.4: Sample Escher images Introduction to Signal and Image Processing 8 of 46 March 29th, 2016 22.02.2016 09:17 Ph. Cattin: Image Sampling and Quantisation Tessellation Examples by M.C. Escher (2) Tessellation (10) Fig 4.5: Sample Escher images Introduction to Signal and Image Processing 9 of 46 March 29th, 2016 22.02.2016 09:17 Ph. Cattin: Image Sampling and Quantisation Tessellation Basics Tessellation (11) Three types of tessellations with polygons exist 1. regular tessellations (using the same regular polygon) 2. semi-regular tessellations (using various regular polygons) 3. hyperbolic tessellations (they use non-regular polygons) They are formed by translating, rotating, and reflecting polygons Fig 4.6: regular Fig 4.7: semi-regular Introduction to Signal and Image Processing 10 of 46 Fig 4.8: hyperbolic March 29th, 2016 22.02.2016 09:17 Ph. Cattin: Image Sampling and Quantisation Tessellation Tessellation Claim (12) There exist only 11 possible tessellations with regular polygons that can cover the entire image Introduction to Signal and Image Processing 11 of 46 March 29th, 2016 22.02.2016 09:17 Ph. Cattin: Image Sampling and Quantisation Tessellation How Many Tessellations (13) Exist with Regular Polygons? Observation 1: Since the regular polygons in a tessellation must fill the plane at each vertex, the interior angle must be an exact divisor of Observation 2: A regular of -gon has an internal angle degrees Fig 4.9: Of the regular polygons, only triangles ( ), squares ( ), pentagons ( ), hexagons ( ), octagons ( ), decagons ( ) and dodecagons ( ) can be used for tiling around a common vertex - again because of the angle value Introduction to Signal and Image Processing 12 of 46 March 29th, 2016 22.02.2016 09:17 Ph. Cattin: Image Sampling and Quantisation Combinatorial Analysis Tessellation (14) A combinatorial analysis of these base polygons produces the following 14 solutions Regular Tessellations Semi-regular Tessellations Semi-regular Tessellations that can not be extended infinitely 4.4.4 6.6.6 3.3.4 3.6.3 3.4.6 3.3.3 4.8.8 3.12 4.6.1 3.4.4 5.5.1 Fig 4.10: Tessellations Introduction to Signal and Image Processing 13 of 46 March 29th, 2016 22.02.2016 09:17 Ph. Cattin: Image Sampling and Quantisation All Semi-Regular Tessellations Tessellation (15) Eight semi-regular tessellations exist Snub hexagonal Trihexagonal Prismatic trisquare Truncated Small Truncated square hexagonal rhombitrihexagonal Fig 4.11: Introduction to Signal and Image Processing 14 of 46 Snub square Great rhombitrihexagonal March 29th, 2016 22.02.2016 09:17 Ph. Cattin: Image Sampling and Quantisation Tessellation All Regular Tessellations (16) But only three regular tessellations exist Triangular tiling Square tiling Fig 4.12: Introduction to Signal and Image Processing 15 of 46 Hexagonal tiling March 29th, 2016 22.02.2016 09:17 Ph. Cattin: Image Sampling and Quantisation Tessellation Rules Tessellation (17) For practical applications in computer vision the tessellation has to adhere to the following rules The tessellation must tile an infinite area with no gaps or overlapping Each vertex must look the same The tiles must all be the same regular polygon This leaves us with the following three regular tessellations Regular Tessellations 4.4.4 6.6.6 Although the hexagonal tessellation offers some substantial advantages (e.g. no ambiguities in defining connectedness, closer spatial organisation as found in mammalian retinas), the square tessellation is the most commonly used. Introduction to Signal and Image Processing 16 of 46 March 29th, 2016 22.02.2016 09:17 Ph. Cattin: Image Sampling and Quantisation Advantages of Square Tessellation Tessellation (18) They directly support operations in the Cartesian coordinate frame Most algorithms (FFT, Image pyramids) are based on square tessellations The resolution is often a power of 2: e.g. 16x16, 32x32, ..., 256x256, 512x512 Introduction to Signal and Image Processing 17 of 46 March 29th, 2016 22.02.2016 09:17 A Sampling Model A Sampling Model (20) As we have seen, The intensity value attributed to a pixel corresponds to the integration of the incoming irradiance over a cell of the tessellation The cells are only located at discrete locations The sampling process can thus be modeled in a 2-step scheme: 1. Integrate brightness over regions of the pixel size, 2. Read out values only at the pixel positions. Introduction to Signal and Image Processing 18 of 46 March 29th, 2016 22.02.2016 09:17 Ph. Cattin: Image Sampling and Quantisation A Sampling Model The Neighbourhood Function (21) First a neighbourhood function has to be defined, that is 1 inside a region with the shape of a pixel/cell and 0 outside. Integrating the incoming intensity region then yields over such a Fig 4.13: Neighbourhood function for square pixels (4.1) rewriting this expression as (4.2) we recognise it as the convolution of with which can also be written as . Since is symmetric we can equally well write . Introduction to Signal and Image Processing 19 of 46 March 29th, 2016 22.02.2016 09:17 Ph. Cattin: Image Sampling and Quantisation A Sampling Model Fourier Transform of the Neighbourhood Function (22) To gain a deeper understand of the sampling model we need its Fourier Transform : Fig 4.14: , the Fourier Transform of the neighbourhood (4.3) function (notice the negative values) Because is real and even its Fourier Transform is too → the neighbourhood filter will not change the phase but only their amplitude. Since becomes negative for some some frequencies undergo a complete phase reversal (shift over see next slide). Introduction to Signal and Image Processing 20 of 46 March 29th, 2016 22.02.2016 09:17 Ph. Cattin: Image Sampling and Quantisation Filtering with the Neighbourhood Function A Sampling Model (23) As the Fourier Transform of the neighbourhood function has negative amplitudes for some frequencies, complete phase reversals can be observed for higher frequencies: Fig 4.15: Star pattern that increases its frequency towards the centre Fig 4.16: Complete phase reversals occur at higher frequencies Introduction to Signal and Image Processing 21 of 46 March 29th, 2016 22.02.2016 09:17 Ph. Cattin: Image Sampling and Quantisation A Sampling Model Sampling of a Continuous 1D Function (24) As the second step after filtering with the neighbourhood function we have to select values only at discrete pixel positions. This is modelled as a multiplication with a 1D or 2D pattern (train) of Dirac impulses at these discrete positions. Consider the real neighbourhood function filtered Suppose its Fourier Transform is band limited and thus vanishes outside the interval To obtain a sampled version of simply involves multiplying it by a sampling function , which consists of a train of Dirac impulses apart Its Fourier Transform is also a train of Dirac impulses with a distance inversely proportional to , namely apart By the convolution theorem multiplication in the image domain is equivalent to convolution in the frequency domain The transform is periodic, with period , and the individual repetitions of can overlap → aliasing!!! The centre of the overlap occurs at To avoid these problems, the sampling interval has to be selected so that , or (4.4) Introduction to Signal and Image Processing 22 of 46 March 29th, 2016 22.02.2016 09:17 Once the individual are separated a multiplication with the window function yields a completely isolated The inverse Fourier Transform then yields the original continuous function Complete recovery of a band-limited function that satisfies the above inequality is known as the WhittakerShannon Sampling Theorem Introduction to Signal and Image Processing 23 of 46 March 29th, 2016 22.02.2016 09:17 Ph. Cattin: Image Sampling and Quantisation A Sampling Model Sampling of a Continuous 1D Function (2) (25) All the frequency domain information of a band-limited function is contained in the interval If the Whittaker-Shannon Sampling Theorem or Nyquist Sampling Theorem (4.5) is not satisfied, the transform in this interval is corrupted by contributions from adjacent periods. This phenomenon is frequently referred to as aliasing. Introduction to Signal and Image Processing 24 of 46 March 29th, 2016 22.02.2016 09:17 Ph. Cattin: Image Sampling and Quantisation A Sampling Model Sampling of a Discrete 1D Function (26) The preceding example applies to functions of unlimited duration in the spatial domain. For practical examples only functions sampled over a finite region are of interest. This situation is shown graphically below Consider a real neighourhoodfunction-filtered function Suppose its Fourier Transform is band limited and thus vanishes outside the interval The sampling function fulfils the Whittaker-Shannon Theorem As the Whittaker-Shannon Sampling Theorem (aka Nyquist Criterion) is fulfilled, the are well separated and no aliasing is present The Sampling Window and its Fourier Transform has Frequency components that extend to infinity Because has frequency components that extend to infinity, the convolution of these functions introduces a distortion in the frequency domain representation of a function that has been sampled and limited to a finite region by Introduction to Signal and Image Processing 25 of 46 March 29th, 2016 22.02.2016 09:17 These considerations lead to the important conclusion that No function of finite duration can be band limited Conversely, A function that is band limited must extend from in the spatial domain to These important practical results establish fundamental limitations to the treatment of digital functions. Introduction to Signal and Image Processing 26 of 46 March 29th, 2016 22.02.2016 09:17 Ph. Cattin: Image Sampling and Quantisation A Sampling Model An Alternative Reasoning for Periodicity in the DFT (27) So far, all the results in the Fourier domain have been of a continuous nature. To obtain a discrete Fourier Transform simply requires to sample it with a train of Dirac impulses that are units apart. Consider the signals and as the results of the operation sequence on the previous slide To sample we multiply it with a train of Dirac impulses that are units apart The inverse Fourier Transform of yields , an other train of Dirac impulses with inversely spaced pulses The graph shows the result of sampling As the equivalent of a multiplication in the Fourier domain is a convolution in the spatial domain, it yields a periodic function, with period If samples of and are taken and the spacings between samples are selected so that a period in each domain is covered by uniformly spaced samples, then in the spatial domain and in the frequency domain. The latter equation is based on the periodic property of the Fourier Transform of a sampled function, with period , as shown earlier. The Sampling Theorem for discrete signals can thus be formulated as Introduction to Signal and Image Processing 27 of 46 March 29th, 2016 22.02.2016 09:17 (4.6) Introduction to Signal and Image Processing 28 of 46 March 29th, 2016 22.02.2016 09:17 Ph. Cattin: Image Sampling and Quantisation A Sampling Model Sampling of (28) Two-Dimensional Functions (Images) The preceding sampling concepts (after some modifications in notation) are directly applicable to 2D functions The sampling process for these functions can be formulated making use of a 2D train of Dirac impulses For a function , where and are continuous, a sampled function is obtained by forming the product . The equivalent operation in the Frequency domain is the convolution of and , where is a train of Dirac impulses with separation and . If is band limited it might look like shown on the right Let and represent the widths in and direction that completely enclose the band-limited function No aliasing is present if and The 2D sampling theorem can thus be formulated as (4.7) and (4.8) A periodicity analysis similar to the discrete 1D case shown previously would yield a 2D Sampling Theorem of (4.9) and Introduction to Signal and Image Processing 29 of 46 March 29th, 2016 22.02.2016 09:17 (4.10) Introduction to Signal and Image Processing 30 of 46 March 29th, 2016 22.02.2016 09:17 Ph. Cattin: Image Sampling and Quantisation A Sampling Model Summary Sampling Theorem (29) The One-Dimensional Sampling Theorem states that If the Fourier Transform of a function is zero for all Frequencies beyond , i.e. the Fourier Transform is band-limited, then the continuous function can be completely reconstructed as long as . The Two-Dimensional Sampling Theorem states that If the Fourier Transform of a function is zero for all Frequencies beyond , i.e. the Fourier Transform is band-limited, then the continuous function can be completely reconstructed as long as and . Introduction to Signal and Image Processing 31 of 46 March 29th, 2016 22.02.2016 09:17 Ph. Cattin: Image Sampling and Quantisation A Sampling Model Aliasing Example 1 (30) The input image contains regions with clearly different frequency content. Going from the centre to boundary, the frequency increases. It can be seen that once the Nyquist rate is higher than the actual (a) Original pattern sampling, aliasing occurs. (a) the 256x256 sample pattern (b) the sinc function for a sampling rate of (grey is zero, brighter is positive, and darker is negative) (c) the original pattern is sampled with (d) the reconstructed pattern. In regions where the Nyquist rate is higher strong aliasing artefacts are present (c) Sampled pattern (b) Sinc size 5 (d) Reconstruction Fig 4.17 Aliasing example Introduction to Signal and Image Processing 32 of 46 March 29th, 2016 22.02.2016 09:17 Ph. Cattin: Image Sampling and Quantisation A Sampling Model Aliasing Example 2 (31) This example shows the reconstruction of the rolling pattern for a sampling rate ( ) that is well above the Nyquist rate. (a) the 128x128 sample rolling pattern (a) Original pattern (b) Sinc of size 5 (b) the sinc function for a sampling rate of . The grey background is zero, brighter is positive, and darker is negative (c) the original pattern is sampled with (d) the reconstructed rolling pattern. The reconstruction is perfect (except for boundary (d) Reconstruction (c) Sampled effects) pattern Fig 4.18 Aliasing example 2 Introduction to Signal and Image Processing 33 of 46 March 29th, 2016 22.02.2016 09:17 Ph. Cattin: Image Sampling and Quantisation Aliasing Example 3 A Sampling Model (32) In this example the sampling rate ( ) is below the Nyquist rate. (a) the 128x128 sample rolling pattern (b) the sinc function for a sampling rate of . The grey (a) Original pattern (b) Sinc size 15 background is zero, brighter is positive, and darker is negative (c) the original pattern is sampled with (d) the reconstructed rolling pattern is no longer valid. It is interesting that not only the frequency changed, but even the orientation of the pattern. (d) Reconstruction (c) Sampled pattern Fig 4.19 Aliasing example 3 Introduction to Signal and Image Processing 34 of 46 March 29th, 2016 22.02.2016 09:17 Ph. Cattin: Image Sampling and Quantisation A Sampling Model Remark on the Discrete Fourier Transform (33) As already noted, Sampling in one domain implies periodicity in the other If both domains are discretised and thus should both the original image and its Fourier Transform be interpreted as periods of periodic signals. The discrete Fourier Transform is therefore not the Fourier Transform of the image as such, but rather of the periodic signal created by repeating the image data both horizontally and vertically Introduction to Signal and Image Processing 35 of 46 Periodically repeated image Flipped images March 29th, 2016 22.02.2016 09:17 Ph. Cattin: Image Sampling and Quantisation A Sampling Model Linear, Shift-Invariant Operators (34) Convolution theory is not only important in image acquisition but plays an important role at several other occasions. To fully benefit from the convolution theorem a little bit more background theory is required. In fact, it will be explained that Every linear, shift-invariant operation can be expressed as a convolution and vice versa. Definition: Consider a 2D system that produces output and when given inputs and respectively. The system is called linear if the output is produced when the input is The system is called shiftinvariant if the output is produced when the input is Introduction to Signal and Image Processing 36 of 46 Fig 4.20: Linear system Fig 4.21: Shift-invariant system March 29th, 2016 22.02.2016 09:17 Ph. Cattin: Image Sampling and Quantisation A Sampling Model Linear, Shift-Invariant Operators (2) (35) Suppose a process, e.g. camera with lens system, can be modeled as a linear, shift-invariant operation . As we have seen, any image can be considered as a sum of point sources (Dirac impulses). The output of for a single point source is called Point spread function (PSF) of which we denote as . Fig 4.22: Point spread function Knowledge of the PSF can be used to determine the output for Assuming shift-invariance implies that the output to such a Dirac pulse is always the same irrespective of its position. In terms of image acquisition, we assume that the light comming from a point source will be distributed over the image following a fixed spatial pattern. The projection of such a point will therefore always be blurred in the same way independent of its position in the image. Introduction to Signal and Image Processing 37 of 46 March 29th, 2016 22.02.2016 09:17 Ph. Cattin: Image Sampling and Quantisation A Sampling Model Liner, Shift-Invariant Operators (3) Let us consider an input picture linear combination of point sources (36) . It can be written as a (4.11) For the linear and shift-invariant operation we obtain (4.12) The linear, shift-invariant operation has led to a convolution operation. This is true in general and every LSI operation can be written as a convolution and vice versa. A simple variable substitution shows that the above expression can also be written as (4.13) so that (4.14) i.e. convolution is commutative (convolution is also associative). Introduction to Signal and Image Processing 38 of 46 March 29th, 2016 22.02.2016 09:17 Introduction to Signal and Image Processing 39 of 46 March 29th, 2016 22.02.2016 09:17 Ph. Cattin: Image Sampling and Quantisation A Sampling Model Liner, Shift-Invariant Operators (4) (37) Suppose we would like to process an image by first convolving with , followed by a convolution with , thus (4.15) the global operation can therefore be interpreted as applying a single (generally larger) filter . The reverse analysis might be useful too, i.e. if a filter (separable) can be decomposed as a convolution of two simpler filter efficiency can be increased by applying the smaller filters sequentially. Example The Figures on the right show a 2D Gauss kernel and a 1D Gauss kernel of size and respectively. Fig 4.23: 2D Gauss kernel It can be easily shown numerically that the kernel can be separated into two 1D kernels and thus (4.16) Fig 4.24: 1D Gauss kernel Convolving the image sequentially with the 1D kernels is computationally more efficient than convolving the entire image with the 2D kernel. Introduction to Signal and Image Processing 40 of 46 March 29th, 2016 22.02.2016 09:17 Quantisation Quantisation (39) The subjective image quality depends on (1) the number of samples and (2) the number of grey-values . Figure 4.26 shows this relation. The key point of interest is, that isopreference curves tend to become more vertical as the detail in the image increases → images with large amount of detail require fewer grey levels. Fig 4.26: Isopreference curves for the three sample images Fig 4.25: (a) Low detail face image, (b) Cameraman with mid detail, and (c) crowd with high detail content Introduction to Signal and Image Processing 41 of 46 March 29th, 2016 22.02.2016 09:17 Ph. Cattin: Image Sampling and Quantisation Quantisation Lloyd-Max Quantisation (40) In the Introduction of this Lecture we have already shortly explained the effect of using more or less quantisation levels. This part is concerned with the optimal placement of these quantisation levels Suppose we create intervals in the range of possible intensities, defined by the decision levels . Fig 4.27: Principle of the Lloyd-Max quantiser We therefore assign to all intensities in the interval the new grey level . The mean-square quantisation error between the input and output of the quantiser for a given choice of boundaries and output levels is thus (4.17) where is the probability density function for the input sample value. For a given number of output levels, we would like to determine the output levels and interval boundaries that minimise . The partial derivatives of with respect to and must thus vanish: (4.18) Introduction to Signal and Image Processing 42 of 46 March 29th, 2016 22.02.2016 09:17 For not equal to zero we obtain the Lloyd-Max Quantiser Equations (4.19) We see that the decision levels are located halfway between the output levels whilst each is the centroid of the portion of between and If the sample values occur equally frequently, the optimal quantised will spread the values and uniformly, and the Lloyd-Max Quantiser Equations can be simplified to (4.20) As can be seen from the following examples, improvement can be disputed. The main problem is, that Lloyd-Max quantisation does not take local image structure or interpretation into account. Introduction to Signal and Image Processing 43 of 46 March 29th, 2016 22.02.2016 09:17 Ph. Cattin: Image Sampling and Quantisation Quantisation Quantisation Example Original image with 256 grey values 32 equally spaced grey values (41) 32 Lloyd-max quantised grey values Fig 4.28: Quantisation example with 32 grey values Introduction to Signal and Image Processing 44 of 46 March 29th, 2016 22.02.2016 09:17 Ph. Cattin: Image Sampling and Quantisation Quantisation Quantisation Example (2) Original image with 256 grey values 16 equally spaced grey values (42) 16 Lloyd-max quantised grey values Fig 4.29: Quantisation example with 16 grey values Introduction to Signal and Image Processing 45 of 46 March 29th, 2016 22.02.2016 09:17 Ph. Cattin: Image Sampling and Quantisation Quantisation Quantisation Example (3) Original image with 256 grey values 8 equally spaced grey values (43) 8 Lloyd-max quantised grey values Fig 4.30: Quantisation example with 8 grey values Introduction to Signal and Image Processing 46 of 46 March 29th, 2016 22.02.2016 09:17
© Copyright 2026 Paperzz