3.5 Golomb Codes
Assumption: Larger the integer, lower is its probability of occurrence
Adopted (modified version)
In JPEG-LS (lossless and near lossless of continuous tone still images).
(http://hpl.hp.com/loco)
Also http://www.ece.ubc.ca/image
Unary code for positive integers
n 1’s followed by a 0
n = 4, code → 11110
n = 5, code → 111110
Golomb Code:
Split integer into two parts : 1) unary code
2) different code
Ex: integer n (n>0) (m>0)
Represent n>0, using q and r
𝑛
q = ⌊ ⌋ , r = n - qm (m = parameter)
𝑚
q = quotient, use unary code to represent q
r = remainder = 0,1,2……………..m-1
n = qm+r
In H.264 baseline, profile all syntax elements other than transform coefficients,
are coded using FLC or exponential Golomb VLC.
P.S: These notes are adopted from K. Sayood, “Introduction to data compression,” San Francisco,
CA: 3rd Edition, Morgan Kaufmann, 2006.
Page 1
If m is a power of two, use log2m bit representation of r
Otherwise use ⌈𝑙𝑜𝑔2 𝑚⌉ bit representation
Alternate:
Use ⌊𝑙𝑜𝑔2 𝑚⌋-bit representation for the first 2⌈𝑙𝑜𝑔2 𝑚⌉ -m values and ⌈𝑙𝑜𝑔2 𝑚⌉ bit
binary representation of r+2⌈𝑙𝑜𝑔2 𝑚⌉ -m for the rest of the values.
Ex. 3.5.1 Golomb code
Design Golomb code for m = 5
⌈𝑙𝑜𝑔2 5⌉ = 3, ⌊𝑙𝑜𝑔2 5⌋ = 2
2⌈𝑙𝑜𝑔2 𝑚⌉ -m = 8-5 = 3
0 00
r = 0,1,2]→[1 01
2 10
r = 3,4 → 3 bit representation of r+3
codeword for 3→
0110
codeword for 21→
1111001
21
q = ⌊ ⌋ = 4 → 11110
5
r = 1 → 01
For m = 5
𝑞
𝑟
⏞
⏞
n = 21 →11110
01
Represent quotient q by its unary code
⌈𝑙𝑜𝑔2 5⌉ = 3, ⌊𝑙𝑜𝑔2 5⌋ = 2
P.S: These notes are adopted from K. Sayood, “Introduction to data compression,” San Francisco,
CA: 3rd Edition, Morgan Kaufmann, 2006.
Page 2
𝑛
q = ⌊ ⌋ , r = n – qm
𝑚
q
0→0
1 → 10
q is represented by unary code
2 → 110
3 → 1110
4 → 11110]
3.7 Tunstall Codes
P.S: These notes are adopted from K. Sayood, “Introduction to data compression,” San Francisco,
CA: 3rd Edition, Morgan Kaufmann, 2006.
Page 3
B.P. Tunstall “ Synthesis of noisless compression codes” Ph.D. Thesis, Georgia
Tech, Sept 1967.
All codewords are of equal length. Decoding is simple. Errors in codewords do not
propagate. Each codeword represents a different number of letters.
Ex. 3.7.1 Alphabet A = {A, B}
Encode the sequence
𝐴𝐴𝐴
⏟𝐵
⏟ 𝐴𝐴𝐵
⏟ 𝐴𝐴𝐵
⏟ 𝐴𝐴𝐵
⏟ 𝐴𝐴𝐴
⏟ (use table 3.18)
00 11
01
01
01
00
1. Able to parse a source output sequence into sequences of symbols that appear in
the codebook.
2. Maximize the average number of source symbols represented by each codeword.
(use table 3.19)
𝐴𝐴𝐴
⏟𝐵
⏟ 𝐴𝐴𝐵
⏟ 𝐴𝐴𝐵
⏟ 𝐴𝐴𝐵
⏟ 𝐴𝐴𝐵
⏟ 𝐴𝐴𝐴
⏟
00 11
01
𝑛𝑜 𝑐𝑜𝑑𝑒
𝑤𝑜𝑟𝑑
Table 3.19 is not a desirable code for this sequence
P.S: These notes are adopted from K. Sayood, “Introduction to data compression,” San Francisco,
CA: 3rd Edition, Morgan Kaufmann, 2006.
Page 4
Tunstall codes
Derive n-bit Tunstall code for a source that generates iid letters from an alphabet of
size N. Number of codewords = 2n
3 bit Tunstall codes A = (A, B, C)
P(A) = 0.6
P(B) = 0.3
P(C) = 0.1
Table 3.21: Remove ‘A’ in Table 3.20, highest probability and add two letter
strings starting with ‘A’
P.S: These notes are adopted from K. Sayood, “Introduction to data compression,” San Francisco,
CA: 3rd Edition, Morgan Kaufmann, 2006.
Page 5
Table 3.22: Remove ‘AA’ in Table 3.21, highest probability and add 3 letter
strings with ‘AA’.
Next iteration
𝐴𝐴𝐴𝐴
𝐴𝐴𝐴𝐵] Codebook size = 10>8
𝐴𝐴𝐴𝐶
Go back to previous iteration
3-bit Tunstall code is table 3.22
Remove ‘B’ Add BA, BB, BC. Each stage codebook size increases by (N-1).
P.S: These notes are adopted from K. Sayood, “Introduction to data compression,” San Francisco,
CA: 3rd Edition, Morgan Kaufmann, 2006.
Page 6
Figure 3.10: Test Images
a b
c d
a) Sena, b) Sensin, c) Earth, d) Omaha
8 bit PCM, (0-255) levels
Number of pixels = (256)2 = 65536
P.S: These notes are adopted from K. Sayood, “Introduction to data compression,” San Francisco,
CA: 3rd Edition, Morgan Kaufmann, 2006.
Page 7
Code: (xn-xn-1)
0 0 0 0 0 0 0 0 0 0
xn-2 xn-1 xn
Huffman code for 8 bit PCM
(0-255) levels.
Generate Huffman code for each image and use this code to encode the image.
This is a two pass system.
Compression Ratio (CR) =
𝑇𝑜𝑡𝑎𝑙 𝑛𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑏𝑖𝑡𝑠 𝑜𝑟𝑖𝑔𝑖𝑛𝑎𝑙 𝑖𝑚𝑎𝑔𝑒
𝑇𝑜𝑡𝑎𝑙 𝑛𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑏𝑖𝑡𝑠 𝑖𝑛 𝑡ℎ𝑒 𝑒𝑛𝑐𝑜𝑑𝑒𝑑 𝑖𝑚𝑎𝑔𝑒
Includes number of bits needed to store the Huffman code
Code = 𝑥𝑛 − 𝑥̂𝑛
𝑥̂𝑛 = xn-1 , i.e, previous pel predictor
𝑥𝑛 − 𝑥̂𝑛 = prediction error, residual
x x x x x x x x x x x x x x x
x x x x x x x x x x x x x x x
(First pel in any row, assume previous pel intensity is 128 for 8 bit pixel.)
P.S: These notes are adopted from K. Sayood, “Introduction to data compression,” San Francisco,
CA: 3rd Edition, Morgan Kaufmann, 2006.
Page 8
Topics related to
Binarization, Fixed-Length Binarization, Truncated Unary Binarization, ExpGolomb Binarization, Truncated Rice Binarization, Binarization for the CU
Partitioning Mode, Binarization for the Intra Chroma Prediction Mode
and Binarization for the Inter Prediction Direction Index
See
M. Wien, "High Efficiency Video Coding : Coding tools and specification,"
chapter-10, pp. 266 - 272, Springer, 2015.
P.S: These notes are adopted from K. Sayood, “Introduction to data compression,” San Francisco,
CA: 3rd Edition, Morgan Kaufmann, 2006.
Page 9
Exponential Golomb codes [1],[2] (for data elements other than transform
coefficients – these codes are actually fixed and are also called Universal Variable
Length Codes (UVLC).
code Num
0
1
2
3
4
5
6
7
8
code
1
010
011
00100
00101
00110
00111
0001000
0001001
code Num
9
10
11
12
13
14
15
16
17
code
0001010
0001011
0001100
0001101
0001110
0001111
000010000
000010001
000010010
Golomb Codes
Some Applications
1) H.264/AVC Advanced Video Coding [3]
2) AVS China. [4]
3) JPEG-LS (lossless and near lossless) [5]
4) In Sample-based weighted prediction for lossless enhancement layer coding
(SELC) combination of truncated Rice (TR) and exponential Golomb (EG) code is
used. [6]
5) Exp - Golomb codes applied in HEVC - High Efficiency Video Coding [10] &
[11]
References
P.S: These notes are adopted from K. Sayood, “Introduction to data compression,” San Francisco,
CA: 3rd Edition, Morgan Kaufmann, 2006.
Page 10
[1] R. Yu et al, “Bit-plane Golomb coding for sources with Laplacian
distribution,” IEEE ICASSP, vol. 4, pp. 277-280, April 2003.
[2] W. Di et al, “An exp-Golomb encoder and decoder architecture for JVT/AVS”
Proc., 5th International ASIC Conference, vol.2, pp. 910-913, 2003.
[3] I. E. Richardson, “The H.264 Advanced Video Compression standard,” 2 nd
edition, Hoboken, NJ:, Wiley, 2010.
[4] L. Yu et al, “Overview of AVS video coding standard,” SP:IC, vol. 24,
pp. 263-267, April 2009.
[5] M.J. Weinberger, G. Seroussi and G. Sapiro, “The LOCO-I Lossless Image
Compression Algorithm: Principles and Standardization into JPEG-LS,
IEEE Trans. On Image Processing, vol. 9 pp. 1309-1324, Aug. 2000.
[6] A. Heindel, E. Wige and A. Kaup, “ Sample-based weighted prediction for
lossless enhancement layer coding in HEVC”, IEEE PCS 2013, Grand
Compression Challenge, 2013.
[7] T. Nguyen, et al, “Reduced-complexity Entropy Coding of Transform
Coefficient Levels Using Truncated Golomb-Rice Codes in Video Compression”,
IEEE ICIP 2011, Paper MP. L.7, Belgium, Brussels, pp. 753- 756, Sept. 2011.
[8] J. Venbrux, P.-S. Yeh and M.N. Liu, “A VLSI chip set for high speed lossless
data compression”, IEEE Trans., CSVT, vol.2, pp.381-391, Dec. 1992.
[9] J. sole et al, " Transform coefficient coding in HEVC", IEEE Transactions on
Circuits and Systems for Video Technology (CSVT), Vol. 22 (12), pp.1765 - 1777,
2012.
[10] M. Wien, "High Efficiency Video Coding : Coding tools and specification,"
Springer, 2015.
[11] G. J. Sullivan et al, "Overview of the High Efficiency Video Coding (HEVC)
standard," IEEE Trans. CSVT, Vol. 22, No.12, pp. 1649 - 1668, Dec. 2012.
P.S: These notes are adopted from K. Sayood, “Introduction to data compression,” San Francisco,
CA: 3rd Edition, Morgan Kaufmann, 2006.
Page 11
[12] X. Lian et al, "Pixel - grain prediction and k-order ueg-rice entropy coding
oriented lossless frame memory compensation for motion estimation in HEVC,"
IEEE ICIP , Quebec city, Canada, 27 - 30 Sept. 2015.
[13] V. Sze and D. Marpe, "Entropy coding in HEVC," chapter 8, see Tables 8.12
and 8.13, pp. 253 - 254, from the book, V. Sze, M. Budagavi and G. J. Sullivan,
"High Efficiency Video Coding (HEVC) : Algorithms and Architectures,"
Springer, 2014.
P.S: These notes are adopted from K. Sayood, “Introduction to data compression,” San Francisco,
CA: 3rd Edition, Morgan Kaufmann, 2006.
Page 12