1
Multimedia Video Coding & Architectures
(5LSE0), Module 05
Differential PCM (DPCM) and
Linear Predictive Coding (LPC)
Peter H.N. de With
([email protected] )
slides version 1.0
PdW / 2016
Fac. EE SPS-VCA
Multimedia Video Coding & A / 5LSE0 /
Module 05 DPCM & Linear Pred. Coding
2
5LSE0 - Mod 05
Part 1
Correlation and Prediction
PdW / 2016
Fac. EE SPS-VCA
Multimedia Video Coding & A / 5LSE0 /
Module 05 DPCM & Linear Pred. Coding
1
3
General Compression System
Sampler
Fine Quantizer
PCM Encoded Image
(166 Mb/s)
A/D CONVERTER
Compressed bitstream
Transform
Quantizer
VLC coding
(10 Mb/s)
COMPRESSION/SOURCE CODING
Channel bitstream
Error prot.
Encipher
(15 Mb/s)
CHANNEL CODING
PdW / 2016
Fac. EE SPS-VCA
Multimedia Video Coding & A / 5LSE0 /
Module 05 DPCM & Linear Pred. Coding
4
Correlation in Signals – (1)
∗ Meaningful signals are often highly predictable:
From the behavior in the past, the future signal
values can be approximately estimated
250
200
150
100
50
0
500
1000
1500
2000
2500
3000
3500
4000
4500
∗ (Linear) Predictability has something to do with the
autocorrelation function, so ...
PdW / 2016
Fac. EE SPS-VCA
Multimedia Video Coding & A / 5LSE0 /
Module 05 DPCM & Linear Pred. Coding
2
5
Correlation in Signals – (2)
∆x(n)=x(n)-x(n-1)
∗ Differential PCM works on the principle that:
Anything that can be predicted from the signal’s
past can be reconstructed by the decoder
∗ Simple example: Predict next signal value on basis
of actual one. Send only differences to the decoder
100
0
100
500
1000
PdW / 2016
Fac. EE SPS-VCA
1500
2000
2500
3000
3500
4000
4500
Multimedia Video Coding & A / 5LSE0 /
Module 05 DPCM & Linear Pred. Coding
n
Effect of Prediction on Quantization
6
∗ If the same quantizer is used on x(n) and ∆x(n),
then compare:
For x(n)
σ q2 = σ x2σ q2,prototype
For ∆x(n)
σ q2 = σ ∆2xσ q2,prototype
Quality scales with variance of “signal to be quantized”
PdW / 2016
Fac. EE SPS-VCA
Multimedia Video Coding & A / 5LSE0 /
Module 05 DPCM & Linear Pred. Coding
3
Principle of Differential PCM
7
∗ DPCM encoder consists of three steps:
– Predict the current signal value x(n) from past values
x(n-1), x(n-2), ...
– Quantize the difference signal = prediction error
– Encode, e.g. VLC the prediction difference (prediction error)
∗ DPCM Decoder reproduces same prediction of x(n)
∗ Prediction in encoder is based on quantized values
~
x ( n − 1), ~
x ( n − 2 ),...
PdW / 2016
Fac. EE SPS-VCA
Multimedia Video Coding & A / 5LSE0 /
Module 05 DPCM & Linear Pred. Coding
Diagram of DPCM Encoder & Signals
8
Signal to be encoded Prediction difference/error
Predicted signal
PdW / 2016
Fac. EE SPS-VCA
Reconstructed signal
Multimedia Video Coding & A / 5LSE0 /
Module 05 DPCM & Linear Pred. Coding
4
DPCM Decoder Diagram
9
System aspects 1: Loop
• Closed loop in encoder mimics decoder to
avoid decoding mismatch (noise inside)
Reconstructed signal
System aspects 2:
Decoder is typically simpler,
decorrelation, inv. Q, no
decision making
PdW / 2016
Fac. EE SPS-VCA
Predicted signal
Multimedia Video Coding & A / 5LSE0 /
Module 05 DPCM & Linear Pred. Coding
What Linear Predictor to Use?
10
∗ What is “best” predictor?
– Theoretically: x$( n ) = E [ x ( n )| ~x ( n − 1), ~x ( n − 2 ),..., ~x ( n − N )]
– Practically: Linear predictors only
∗ Examples:
–
–
–
–
PCM
Simple differences
Average last two samples
General linear predictor
x$( n ) = 0
x$ ( n ) = ~
x ( n − 1)
x$( n ) = h1 ~
x ( n − 1) + h2 x~( n − 2)
N
x$( n) = ∑ hk ~
x (n − k )
k =1
∗ How to evaluate “success/usefulness” of prediction
(compared to PCM): Prediction Gain
PdW / 2016
Fac. EE SPS-VCA
Multimedia Video Coding & A / 5LSE0 /
Module 05 DPCM & Linear Pred. Coding
5
11
5LSE0 - Mod 05
Part 2
Reconstruction Error, Prediction Gain
and Optimal Prediction
PdW / 2016
Fac. EE SPS-VCA
Multimedia Video Coding & A / 5LSE0 /
Module 05 DPCM & Linear Pred. Coding
Reconstruction Error in DPCM – (1)
x ( n)
∗ Reconstruction / r( n) = x( n) − ~
Coding error:
[
12
]
= x( n) − x$( n) + ∆x * ( n)
= ∆x( n) − ∆x * ( n)
= q( n)
PdW / 2016
Fac. EE SPS-VCA
Multimedia Video Coding & A / 5LSE0 /
Module 05 DPCM & Linear Pred. Coding
6
Reconstruction Error in DPCM – (2)
13
∗ Overall reconstruction error in DPCM is equal to the
quantization error of Q[∆x(n)]
∗ DPCM reconstruction error variance:
σ r2 = σ q2
∗ SNR will be determined by quantizer performance,
relative to the variance of the input signal ∆x(n)
PdW / 2016
Fac. EE SPS-VCA
Multimedia Video Coding & A / 5LSE0 /
Module 05 DPCM & Linear Pred. Coding
Rate-Distortion Model
14
∗ Model for relation bit-rate vs. distortion:
– Gaussian independent data samples
– Variance σ x2
1
σ x2
R( D) = log2
2
D
1
σ2
RPCM = log 2 x2
σq
2
∗ For PCM:
∗ For DPCM:
(bits/sample)
RDPCM
PdW / 2016
Fac. EE SPS-VCA
σ ∆2x
1
= log 2 2
σq
2
Multimedia Video Coding & A / 5LSE0 /
Module 05 DPCM & Linear Pred. Coding
7
(Coding) Gain Obtained by Prediction
15
∗ Theoretical difference in bit rate if same quantizer (with
same σq2 ) is used (same quality of encoded signal):
σ2
1
RPCM − RDPCM = log 2 2x
2
σ ∆x
Ratio of variances!
∗ Advantage of DPCM over PCM depends on the ratio of
signal variance: prediction error variance:
GP =
– Prediction gain:
PdW / 2016
Fac. EE SPS-VCA
σ x2
σ ∆2x
Multimedia Video Coding & A / 5LSE0 /
Module 05 DPCM & Linear Pred. Coding
Prediction Gain Example
Case
GP
16
Comments
σ2∆x < σ2x
GP >1
DPCM outperforms PCM
σ2∆x = σ2x
GP = 1
DPCM identical to PCM
σ2∆x > σ2x
GP < 1
DPCM worse than PCM
σ2∆x = 0
GP → ∞ Perfect prediction
Sometimes the prediction gain is given as: 1 log2 GP
2
• Practical example: Gp=20 => Gain bit rate: 2.1 bits/sample
PdW / 2016
Fac. EE SPS-VCA
Multimedia Video Coding & A / 5LSE0 /
Module 05 DPCM & Linear Pred. Coding
8
The Optimal Predictor – (1)
17
∗ Maximal benefit of DPCM is obtained if Gp is large
2
– Make variance of prediction difference σ ∆x
as small
as possible
∗ Rationale
– Make the prediction ~
x (n) as close as possible to x(n) in
quadratic sense:
σ∆x2 = E[( x( n) − ~
x ( n))2 ]
• Theoretical
2
• Practical (est.) σ∆x =
PdW / 2016
Fac. EE SPS-VCA
1
( x(n) − ~
x (n))2
∑
N all n
Multimedia Video Coding & A / 5LSE0 /
Module 05 DPCM & Linear Pred. Coding
The Optimal Predictor – (2)
18
N
x$( n) = ∑ hk x( n − k )
∗ Predictor:
k =1
∗ Minimize:
Find weights hk
σ ∆x2 = E[( x( n ) − x$( n ))2 ]
– note that the prediction takes place on original data
∗ Solution for N linear prediction coefficients hk
∂σ ∆x2
=0
∂hi
N
⇒
RX (i ) = ∑ hk RX (i − k )
i = 1,2,..., N
k =1
(Normal or Yule-Walker Equations)
PdW / 2016
Fac. EE SPS-VCA
Multimedia Video Coding & A / 5LSE0 /
Module 05 DPCM & Linear Pred. Coding
9
The Optimal Predictor – (3)
19
 R (1)   R (0)
L R ( N − 1)  h 
R (1)
 x   x
 1 
x
x

 
 
R ( 0)
R ( N − 2) h 
 Rx (2)  =  Rx (1)
x
x
2

 
 
M
O
M

 
 
R ( 0)   h 
 Rx ( N )  Rx ( N − 1) Rx ( N − 2) L
x
 N 
∗ From this set of equations the N linear prediction
coefficients can be computed.
∗ Need to know RX(k): Autocorrelation function of x(n)
PdW / 2016
Fac. EE SPS-VCA
Multimedia Video Coding & A / 5LSE0 /
Module 05 DPCM & Linear Pred. Coding
20
Estimation Autocorrelation Function – (1)
∗ Theoretical:
RX ( k ) = E[ X ( n) X ( n − k )]
∗ Practical choice:
1
RX ( k ) =
N
N
∑ X ( n) X ( n − k )
n=0
∗ Practical problems:
– Only limited amount of data (especially in adaptive versions)
– Boundary effects:
x(n)
x(n-1)
– Different “estimators” for autocorrelation function exist, with
theoretical and practical implications
PdW / 2016
Fac. EE SPS-VCA
Multimedia Video Coding & A / 5LSE0 /
Module 05 DPCM & Linear Pred. Coding
10
Estimation Autocorrelation Function – (2) 21
Overhead:
•
•
•
•
Prediction coefficients hk for the signal
Variance of prediction difference: used to scale the quantizer
Mean of signal (usually first subtracted)
Start values for decoding process (boundary values)
PdW / 2016
Fac. EE SPS-VCA
Multimedia Video Coding & A / 5LSE0 /
Module 05 DPCM & Linear Pred. Coding
22
Example of Prediction Gain (1-D)
∗ Model:
Rx ( k ) = σ x2 ρ |k |
1
Rx ( k )
∗ Predictor:
ρ
x$( n ) = h1 x( n − 1)
0.5
∗ “Set” of linear
equations:
Rx (1) = Rx ( 0)h1
0
∗ Solution:
R (1)
h1 = X
=ρ
R X ( 0)
∗ Resulting prediction
error variance: σ ∆x2 = σ x2 (1 − ρ 2 )
∗ Prediction gain:
PdW / 2016
Fac. EE SPS-VCA
GP =
1
1 − ρ2
20
10
0
10
20
10
5
ρ
0
0
0.5
1
Multimedia Video Coding & A / 5LSE0 /
Module 05 DPCM & Linear Pred. Coding
11
DPCM Artifacts – Coding noise
23
∗ Granular noise (like PCM)
∗ Slope overload:
– Prediction differences ∆x(n) too large for quantizer to handle
– Encoder cannot track rapidly changing signal values (i.e. at
signal slopes)
∗ Example:
– Encoder
x$ ( n ) = ~
x ( n − 1)
– Decoder
~
x ( n) = x$( n) + ∆x * ( n)
= x~( n − 1) + ∆x * ( n)
PdW / 2016
Fac. EE SPS-VCA
Multimedia Video Coding & A / 5LSE0 /
Module 05 DPCM & Linear Pred. Coding
DPCM Slope Overload
24
x(n)
x ( n)
Reconstruction ~
Input to quantizer ∆x( n )
*
Quantizer output ∆x ( n )
x$( n)
x~( n − 1)
x$ ( n ) = ~
x ( n − 1)
PdW / 2016
Fac. EE SPS-VCA
~
x ( n) = ~
x ( n − 1) + ∆x * ( n )
Multimedia Video Coding & A / 5LSE0 /
Module 05 DPCM & Linear Pred. Coding
12
DPCM on Images / Video
25
∗ Same principle as 1-D
∗ Definition of “Past”
and “Future” in images:
∗ Predictions:
– horizontal (scan line)
– vertical (column)
– two-dimensional
PdW / 2016
Fac. EE SPS-VCA
Multimedia Video Coding & A / 5LSE0 /
Module 05 DPCM & Linear Pred. Coding
Linear Predictor for Images
26
∗ Typical linear predictor for images:
x( m, n ) = h01 x ( m, n − 1) + h10 x( m − 1, n ) + h11 x( m − 1, n − 1)
∗ Yule-Walker for 2-D predictor: Solve for optimal
Need 2-D
prediction coefficients
correlation function!
 R(01
, )  R(0,0) R(1,−1) R(1,0) h01 

 
 
 R(1,0) =  R(1,−1) R(0,0) R(0,1) h10 
 R(11
, )   R(1,0) R(01
, ) R(0,0) h11 
PdW / 2016
Fac. EE SPS-VCA
Multimedia Video Coding & A / 5LSE0 /
Module 05 DPCM & Linear Pred. Coding
13
DPCM on Video pictures
27
∗ Usually difference between subblocks in the frames
(e.g. 16x16 pixels)
∗ Often fixed simple predictor: x$ ( q, t ) = x~( q, t − 1)
– No need to estimate prediction coefficients
PdW / 2016
Fac. EE SPS-VCA
Multimedia Video Coding & A / 5LSE0 /
Module 05 DPCM & Linear Pred. Coding
28
5LSE0 - Mod 05
Part 3
Adaptive Prediction and Systems
PdW / 2016
Fac. EE SPS-VCA
Multimedia Video Coding & A / 5LSE0 /
Module 05 DPCM & Linear Pred. Coding
14
29
Adaptive DPCM systems
250
200
150
100
50
0
0
2000
4000
6000
8000
4
1 10
j
10800
∗ Apply the same predictor everywhere is not a good
idea, because signal correlation changes over time
– Adapt predictor (and quantizer) locally
– Need to estimate autocorrelation function locally
– Incurs overhead
PdW / 2016
Fac. EE SPS-VCA
Multimedia Video Coding & A / 5LSE0 /
Module 05 DPCM & Linear Pred. Coding
30
DPCM
Forward
Adaptation
Buffer N samples
– Effective adaptation needs small segments
(e.g. speech: 32 msec, video?)
– Small segments:
• generate a lot of overhead (prediction coefficients)
• difficult to estimate autocorrelation reliably
– Make trade-off on basis of prediction gain
PdW / 2016
Fac. EE SPS-VCA
Multimedia Video Coding & A / 5LSE0 /
Module 05 DPCM & Linear Pred. Coding
15
DPCM with Backward Adaptation
Use information previous
data segment to calculate
prediction coefficients for
current segment
• Decoder can
reproduce the
calculation of the
prediction
coefficients
• Drawback:
Adaptation may
come too late
PdW / 2016
Fac. EE SPS-VCA
31
Buffer N samples
Multimedia Video Coding & A / 5LSE0 /
Module 05 DPCM & Linear Pred. Coding
Adaptation by Mode Selection
32
∗ In images, simplify the adaptation by
– only allowing to choose among few predictors
– limiting the incurred overhead (e.g. per 8x8 block)
∗ Per block, the “best” predictor is selected
– “Best” means: largest prediction gain
∗ Example: Three predictors among which to choose
~
x ( m, n ) = x$( m − 1, n )
~
x ( m, n ) = x$ ( m, n − 1)
“horizontal edge”
“vertical edge”
~
x ( m, n ) = 0.4 x$ ( m − 1, n ) + 0.4 x$ ( m, n − 1) + 0.2 x$ ( m − 1, n − 1)
no directional preference
PdW / 2016
Fac. EE SPS-VCA
Multimedia Video Coding & A / 5LSE0 /
Module 05 DPCM & Linear Pred. Coding
16
Application: Speech Production and LPC
33
∗ Human speech is produced in the vocal tract as
combination of vocal cords (in glottis) interacting
with articulators in vocal tract
– Model this as a signal processing process
∗ Human speech perception exhibit masking
– Model this as a signal processing (masking) filter
∗ Results in DPCM, like speech coding:
– LPC Vocoder
– Linear Predictive Coding (LPC)
– Analysis-by-synthesis coding (CELP)
PdW / 2016
Fac. EE SPS-VCA
Multimedia Video Coding & A / 5LSE0 /
Module 05 DPCM & Linear Pred. Coding
Application / LPC Vocoder
Speech
LPC
analysis
LPC
quantizer
34
LPC information
Voiced/Unvoiced
Pitch Analysis
“Signal” information
Model for speech generation
Voiced
LPC filter
Unvoiced
Reconstructed speech
∗ Voice Coder (VoCoder) analyzes:
– Spectral properties of speech segment
– Voice/Unvoiced segment information
– Pitch of signal
∗ Objective: Understandable speech
PdW / 2016
Fac. EE SPS-VCA
Multimedia Video Coding & A / 5LSE0 /
Module 05 DPCM & Linear Pred. Coding
17
Application / Adaptive LPC (APC Coder)
Speech
+
input
-
35
Quantizer
+
LPC filter
Pitch filter
Local decoder
Quantizer
LPC analysis
Pitch analysis
∗ Very similar to (adaptive) DPCM
∗ Objective: Waveform precise reproduction
PdW / 2016
Fac. EE SPS-VCA
Multimedia Video Coding & A / 5LSE0 /
Module 05 DPCM & Linear Pred. Coding
Application / Analysis-by-Synthesis LPC
36
Speech
LPC
analysis
Local decoder
Error minimization
Excitation
generation
Quantizer
Pitch filter
LPC filter
+
Synthesized speech
Perceptual
weighting
Perceptual error
Selected excitation
∗ Marriage between vocoder and APC/DPCM
∗ CELP: Codebook-excited linear prediction
PdW / 2016
Fac. EE SPS-VCA
Multimedia Video Coding & A / 5LSE0 /
Module 05 DPCM & Linear Pred. Coding
18
37
Standard Speech Coders using DPCM/LPC
Standard
number
Year
ITU G.711
ITU G.726
ITU G.728
ETSI
ITU G.723.1
1972
1990
1992/1994
1994
1995
Type of
Coder
Bit rate
Companded ADPCM
PCM
64 kbs
16-40 kbs
LD-CELP
VSELP
ACELP ++
16 kbs
5.6 kbs
5.3 & 6.3 kbs
Quality
Delay
Toll
0.125 msec
Toll
0.625 msec
GSM
20 msec
< Toll
30 msec
< Toll
0.125 msec
PdW / 2016
Fac. EE SPS-VCA
Multimedia Video Coding & A / 5LSE0 /
Module 05 DPCM & Linear Pred. Coding
Concluding statements
38
∗ Original DPCM system encoder is a feedback loop
– Decoder reconstructs the same signal without side information
∗ Adaptive prediction and/or quantization improves SNR
– Adaptive prediction to image structure
– Adaptive quantization for reducing noise in specific areas
∗ Model-based prediction such as synthesis enforce
decorrelation, but increase the complexity of that stage
∗ Buffering approach: trade-off complexity & performance
– Forward: accurate but expensive overhead and backward v.v.
∗ Predictive systems often embedded into larger systems
PdW / 2016
Fac. EE SPS-VCA
Multimedia Video Coding & A / 5LSE0 /
Module 05 DPCM & Linear Pred. Coding
19