8.2 The Rate-Distortion Function
Definition of a Rate-Distortion Code
Definition of a Rate-Distortion Code
All the discussions are with respect to an i.i.d. information source {Xk , k
with generic random variable X and a distortion measure d.
1}
Definition 8.8 An (n, M ) rate-distortion code is defined by an encoding function
f : X n ! {1, 2, · · · , M }
and a decoding function
g : {1, 2, · · · , M } ! X̂ n .
• Index set I
• Codewords the reproduction sequences g(1), g(2), · · · , g(M ) in X̂ n
• Codebook the set of all codewords
Definition of a Rate-Distortion Code
All the discussions are with respect to an i.i.d. information source {Xk , k
with generic random variable X and a distortion measure d.
1}
Definition 8.8 An (n, M ) rate-distortion code is defined by an encoding function
f : X n ! {1, 2, · · · , M }
and a decoding function
g : {1, 2, · · · , M } ! X̂ n .
• Index set I = {1, 2, · · · , M }
• Codewords the reproduction sequences g(1), g(2), · · · , g(M ) in X̂ n
• Codebook the set of all codewords
Definition of a Rate-Distortion Code
All the discussions are with respect to an i.i.d. information source {Xk , k
with generic random variable X and a distortion measure d.
1}
Definition 8.8 An (n, M ) rate-distortion code is defined by an encoding function
f : X n ! {1, 2, · · · , M }
and a decoding function
g : {1, 2, · · · , M } ! X̂ n .
• Index set I = {1, 2, · · · , M }
• Codewords the reproduction sequences g(1), g(2), · · · , g(M ) in X̂ n
• Codebook the set of all codewords
Definition of a Rate-Distortion Code
All the discussions are with respect to an i.i.d. information source {Xk , k
with generic random variable X and a distortion measure d.
1}
Definition 8.8 An (n, M ) rate-distortion code is defined by an encoding function
f : X n ! {1, 2, · · · , M }
and a decoding function
g : {1, 2, · · · , M } ! X̂ n .
• Index set I = {1, 2, · · · , M }
• Codewords the reproduction sequences g(1), g(2), · · · , g(M ) in X̂ n
• Codebook the set of all codewords
Definition of a Rate-Distortion Code
All the discussions are with respect to an i.i.d. information source {Xk , k
with generic random variable X and a distortion measure d.
1}
Definition 8.8 An (n, M ) rate-distortion code is defined by an encoding function
f : X n ! {1, 2, · · · , M }
and a decoding function
g : {1, 2, · · · , M } ! X̂ n .
• Index set I = {1, 2, · · · , M }
• Codewords the reproduction sequences g(1), g(2), · · · , g(M ) in X̂ n
• Codebook the set of all codewords
A Rate-Distortion Code
X
source
sequence
f (X)
Encoder
Decoder
X
reproduction
sequence
Definition 8.9 The rate of an (n, M ) rate-distortion code is n
per symbol.
1
log M in bits
Definition 8.10 A rate-distortion pair (R, D) is (asymptotically) achievable if
for any ✏ > 0, there exists for sufficiently large n an (n, M ) rate-distortion code
such that
1
log M R + ✏
n
and
Pr{d(X, X̂) > D + ✏} ✏,
where X̂ = g(f (X)).
Remark If (R, D) is achievable, then (R0 , D) and (R, D0 ) are achievable for
all R0 R and D0 D. This in turn implies that (R0 , D0 ) are achievable for all
R0 R and D0 D.
Definition 8.9 The rate of an (n, M ) rate-distortion code is n
per symbol.
1
log M in bits
Definition 8.10 A rate-distortion pair (R, D) is (asymptotically) achievable if
for any ✏ > 0, there exists for sufficiently large n an (n, M ) rate-distortion code
such that
1
log M R + ✏
n
and
Pr{d(X, X̂) > D + ✏} ✏,
where X̂ = g(f (X)).
Remark If (R, D) is achievable, then (R0 , D) and (R, D0 ) are achievable for
all R0 R and D0 D. This in turn implies that (R0 , D0 ) are achievable for all
R0 R and D0 D.
Definition 8.9 The rate of an (n, M ) rate-distortion code is n
per symbol.
1
log M in bits
Definition 8.10 A rate-distortion pair (R, D) is (asymptotically) achievable if
for any ✏ > 0, there exists for sufficiently large n an (n, M ) rate-distortion code
such that
1
log M R + ✏
n
and
Pr{d(X, X̂) > D + ✏} ✏,
where X̂ = g(f (X)).
Remark If (R, D) is achievable, then (R0 , D) and (R, D0 ) are achievable for
all R0 R and D0 D. This in turn implies that (R0 , D0 ) are achievable for all
R0 R and D0 D.
Definition 8.9 The rate of an (n, M ) rate-distortion code is n
per symbol.
1
log M in bits
Definition 8.10 A rate-distortion pair (R, D) is (asymptotically) achievable if
for any ✏ > 0, there exists for sufficiently large n an (n, M ) rate-distortion code
such that
1
log M R + ✏
n
and
Pr{d(X, X̂) > D + ✏} ✏,
where X̂ = g(f (X)).
0
0
Remark If (R, D) is achievable, then (R
___, D) and (R, D ) are achievable for
all ___
R0 R and D0 D. This in turn implies that (R0 , D0 ) are achievable for all
R0 R and D0 D.
Definition 8.9 The rate of an (n, M ) rate-distortion code is n
per symbol.
1
log M in bits
Definition 8.10 A rate-distortion pair (R, D) is (asymptotically) achievable if
for any ✏ > 0, there exists for sufficiently large n an (n, M ) rate-distortion code
such that
1
log M R + ✏
n
and
Pr{d(X, X̂) > D + ✏} ✏,
where X̂ = g(f (X)).
0
0
Remark If (R,
___ D) is achievable, then (R
___, D) and (R, D ) are achievable for
all ___
R0 ___
R and D0 D. This in turn implies that (R0 , D0 ) are achievable for all
R0 R and D0 D.
Definition 8.9 The rate of an (n, M ) rate-distortion code is n
per symbol.
1
log M in bits
Definition 8.10 A rate-distortion pair (R, D) is (asymptotically) achievable if
for any ✏ > 0, there exists for sufficiently large n an (n, M ) rate-distortion code
such that
1
log M R + ✏
n
and
Pr{d(X, X̂) > D + ✏} ✏,
where X̂ = g(f (X)).
Remark If (R, D) is achievable, then (R0 , D) and (R, ___
D0 ) are achievable for
all R0 R and ___
D0 D. This in turn implies that (R0 , D0 ) are achievable for all
R0 R and D0 D.
Definition 8.9 The rate of an (n, M ) rate-distortion code is n
per symbol.
1
log M in bits
Definition 8.10 A rate-distortion pair (R, D) is (asymptotically) achievable if
for any ✏ > 0, there exists for sufficiently large n an (n, M ) rate-distortion code
such that
1
log M R + ✏
n
and
Pr{d(X, X̂) > D + ✏} ✏,
where X̂ = g(f (X)).
Remark If (R, ___
D) is achievable, then (R0 , D) and (R, ___
D0 ) are achievable for
all R0 R and ___
D0 ___
D. This in turn implies that (R0 , D0 ) are achievable for all
R0 R and D0 D.
Definition 8.9 The rate of an (n, M ) rate-distortion code is n
per symbol.
1
log M in bits
Definition 8.10 A rate-distortion pair (R, D) is (asymptotically) achievable if
for any ✏ > 0, there exists for sufficiently large n an (n, M ) rate-distortion code
such that
1
log M R + ✏
n
and
Pr{d(X, X̂) > D + ✏} ✏,
where X̂ = g(f (X)).
Remark If (R, D) is achievable, then (R0 , D) and (R, D0 ) are achievable for
all R0 R and D0 D. This in turn implies that (R0 , D0 ) are achievable for all
R0 R and D0 D.
Definition 8.11 The rate-distortion region is the subset of <2 containing all
achievable pairs (R, D).
Theorem 8.12 The rate-distortion region is closed and convex.
Proof
• The closeness follows from the definition of the achievability of an (R, D)
pair.
• The convexity is proved by time-sharing. Specifically, for any 0 1
and ¯ = 1
, if (R(1) , D(1) ) and (R(2) , D(2) ) are achievable, then so is
(R( ) , D( ) ) = ( R(1) + ¯ R(2) , D(1) + ¯ D(2) ).
This can be seen by time-sharing between two codes, one achieving (R(1) , D(1) )
for fraction of the time, and the other one achieving (R(2) , D(2) ) for ¯
fraction of the time.
Definition 8.11 The rate-distortion region is the subset of <2 containing all
achievable pairs (R, D).
Theorem 8.12 The rate-distortion region is closed and convex.
Proof
• The closeness follows from the definition of the achievability of an (R, D)
pair.
• The convexity is proved by time-sharing. Specifically, for any 0 1
and ¯ = 1
, if (R(1) , D(1) ) and (R(2) , D(2) ) are achievable, then so is
(R( ) , D( ) ) = ( R(1) + ¯ R(2) , D(1) + ¯ D(2) ).
This can be seen by time-sharing between two codes, one achieving (R(1) , D(1) )
for fraction of the time, and the other one achieving (R(2) , D(2) ) for ¯
fraction of the time.
Definition 8.11 The rate-distortion region is the subset of <2 containing all
achievable pairs (R, D).
Theorem 8.12 The rate-distortion region is closed and convex.
Proof
• The closeness follows from the definition of the achievability of an (R, D)
pair.
• The convexity is proved by time-sharing. Specifically, for any 0 1
and ¯ = 1
, if (R(1) , D(1) ) and (R(2) , D(2) ) are achievable, then so is
(R( ) , D( ) ) = ( R(1) + ¯ R(2) , D(1) + ¯ D(2) ).
This can be seen by time-sharing between two codes, one achieving (R(1) , D(1) )
for fraction of the time, and the other one achieving (R(2) , D(2) ) for ¯
fraction of the time.
Definition 8.11 The rate-distortion region is the subset of <2 containing all
achievable pairs (R, D).
Theorem 8.12 The rate-distortion region is closed and convex.
Proof
• The closeness follows from the definition of the achievability of an (R, D)
pair.
• The convexity is proved by time-sharing. Specifically, for any 0 1
and ¯ = 1
, if (R(1) , D(1) ) and (R(2) , D(2) ) are achievable, then so is
(R( ) , D( ) ) = ( R(1) + ¯ R(2) , D(1) + ¯ D(2) ).
This can be seen by time-sharing between two codes, one achieving (R(1) , D(1) )
for fraction of the time, and the other one achieving (R(2) , D(2) ) for ¯
fraction of the time.
Definition 8.11 The rate-distortion region is the subset of <2 containing all
achievable pairs (R, D).
Theorem 8.12 The rate-distortion region is closed and convex.
Proof
• The closeness follows from the definition of the achievability of an (R, D)
pair.
• The convexity is proved by time-sharing. Specifically, for any 0 1
and ¯ = 1
, if (R(1) , D(1) ) and (R(2) , D(2) ) are achievable, then so is
(R( ) , D( ) ) = ( R(1) + ¯ R(2) , D(1) + ¯ D(2) ).
This can be seen by time-sharing between two codes, one achieving (R(1) , D(1) )
for fraction of the time, and the other one achieving (R(2) , D(2) ) for ¯
fraction of the time.
Definition 8.11 The rate-distortion region is the subset of <2 containing all
achievable pairs (R, D).
Theorem 8.12 The rate-distortion region is closed and convex.
Proof
• The closeness follows from the definition of the achievability of an (R, D)
pair.
• The convexity is proved by time-sharing. Specifically, for any 0 1
and ¯ = 1
, if (R(1) , D(1) ) and (R(2) , D(2) ) are achievable, then so is
(R( ) , D( ) ) = (__R(1) + ¯ R(2) , __D(1) + ¯ D(2) ).
This can be seen by time-sharing between two codes, one achieving (R(1) , D(1) )
for __ fraction of the time, and the other one achieving (R(2) , D(2) ) for ¯
fraction of the time.
Definition 8.11 The rate-distortion region is the subset of <2 containing all
achievable pairs (R, D).
Theorem 8.12 The rate-distortion region is closed and convex.
Proof
• The closeness follows from the definition of the achievability of an (R, D)
pair.
• The convexity is proved by time-sharing. Specifically, for any 0 1
and ¯ = 1
, if (R(1) , D(1) ) and (R(2) , D(2) ) are achievable, then so is
¯ R(2) , D(1) + __
¯ D(2) ).
(R( ) , D( ) ) = ( R(1) + __
This can be seen by time-sharing between two codes, one achieving (R(1) , D(1) )
¯
for fraction of the time, and the other one achieving (R(2) , D(2) ) for __
fraction of the time.
R
max
DDmax
D
R
The ratedistortion
region
max
DDmax
D
R
The ratedistortion
region
(R(1) , D(1) )
max
DDmax
D
R
The ratedistortion
region
(R(1) , D(1) )
(R(2) , D(2) )
max
DDmax
D
R
The ratedistortion
region
(R(1) , D(1) )
(R(2) , D(2) )
max
DDmax
D
R
The ratedistortion
region
(R(1) , D(1) )
✓
R(1) + R(2) D(1) + D(2)
,
2
2
◆
(R(2) , D(2) )
max
DDmax
D
Definition 8.13 The rate-distortion function R(D) is the minimum of all rates
R for a given distortion D such that (R, D) is achievable.
Definition 8.14 The distortion-rate function D(R) is the minimum of all
distortions D for a given rate R such that (R, D) is achievable.
Remarks
1. Most of the time we will be using R(D) instead of D(R).
2. If (R, D) is achievable, then R
R(D).
Definition 8.13 The rate-distortion function R(D) is the minimum of all rates
R for a given distortion D such that (R,___
D) is achievable.
Definition 8.14 The distortion-rate function D(R) is the minimum of all
distortions D for a given rate R such that (R, D) is achievable.
Remarks
1. Most of the time we will be using R(D) instead of D(R).
2. If (R, D) is achievable, then R
R(D).
Definition 8.13 The rate-distortion function R(D) is the minimum of all rates
R for a given distortion D such that (R, D) is achievable.
Definition 8.14 The distortion-rate function D(R) is the minimum of all
distortions D for a given rate R such that (R, D) is achievable.
Remarks
1. Most of the time we will be using R(D) instead of D(R).
2. If (R, D) is achievable, then R
R(D).
Definition 8.13 The rate-distortion function R(D) is the minimum of all rates
R for a given distortion D such that (R, D) is achievable.
Definition 8.14 The distortion-rate function D(R) is the minimum of all
distortions D for a given rate R such that (R,
___ D) is achievable.
Remarks
1. Most of the time we will be using R(D) instead of D(R).
2. If (R, D) is achievable, then R
R(D).
Definition 8.13 The rate-distortion function R(D) is the minimum of all rates
R for a given distortion D such that (R, D) is achievable.
Definition 8.14 The distortion-rate function D(R) is the minimum of all
distortions D for a given rate R such that (R, D) is achievable.
Remarks
1. Most of the time we will be using R(D) instead of D(R).
2. If (R, D) is achievable, then R
R(D).
Definition 8.13 The rate-distortion function R(D) is the minimum of all rates
R for a given distortion D such that (R, D) is achievable.
Definition 8.14 The distortion-rate function D(R) is the minimum of all
distortions D for a given rate R such that (R, D) is achievable.
Remarks
1. Most of the time we will be using R(D) instead of D(R).
2. If (R, D) is achievable, then R
R(D).
Definition 8.13 The rate-distortion function R(D) is the minimum of all rates
R for a given distortion D such that (R, D) is achievable.
Definition 8.14 The distortion-rate function D(R) is the minimum of all
distortions D for a given rate R such that (R, D) is achievable.
Remarks
1. Most of the time we will be using R(D) instead of D(R).
2. If (R, D) is achievable, then R
R(D).
R
The ratedistortion
region
DDmax
max
D
R
The ratedistortion
region
D
DDmax
max
D
R
The ratedistortion
region
R(D)
D
DDmax
max
D
R
The ratedistortion
region
R
DDmax
max
D
R
The ratedistortion
region
R
D(R)
DDmax
max
D
Theorem 8.15 The following properties hold for the rate-distortion function
R(D):
1. R(D) is non-increasing in D.
2. R(D) is convex.
3. R(D) = 0 for D
Dmax .
4. R(0) H(X).
Proof
1. Let D0
D. (R(D), D) achievable ) (R(D), D0 ) achievable.
R(D) R(D0 ) by definition of R(D0 ).
Then
2. Follows from the convexity of the rate-distortion region.
3. (0, Dmax ) is achievable ) R(Dmax ) = 0. Then R(D) = 0 for D
because R(·) is non-increasing.
Dmax
4. Since d is assumed to be normal, (H(X), 0) is achievable, and hence R(0)
H(X).
Theorem 8.15 The following properties hold for the rate-distortion function
R(D):
1. R(D) is non-increasing in D.
2. R(D) is convex.
3. R(D) = 0 for D
Dmax .
4. R(0) H(X).
Proof
1. Let D0
D. (R(D), D) achievable ) (R(D), D0 ) achievable.
R(D) R(D0 ) by definition of R(D0 ).
Then
2. Follows from the convexity of the rate-distortion region.
3. (0, Dmax ) is achievable ) R(Dmax ) = 0. Then R(D) = 0 for D
because R(·) is non-increasing.
Dmax
4. Since d is assumed to be normal, (H(X), 0) is achievable, and hence R(0)
H(X).
Theorem 8.15 The following properties hold for the rate-distortion function
R(D):
1. R(D) is non-increasing in D.
2. R(D) is convex.
3. R(D) = 0 for D
Dmax .
4. R(0) H(X).
Proof
1. Let D0
D. (R(D), D) achievable ) (R(D), D0 ) achievable.
R(D) R(D0 ) by definition of R(D0 ).
Then
2. Follows from the convexity of the rate-distortion region.
3. (0, Dmax ) is achievable ) R(Dmax ) = 0. Then R(D) = 0 for D
because R(·) is non-increasing.
Dmax
4. Since d is assumed to be normal, (H(X), 0) is achievable, and hence R(0)
H(X).
Theorem 8.15 The following properties hold for the rate-distortion function
R(D):
1. R(D) is non-increasing in D.
2. R(D) is convex.
3. R(D) = 0 for D
Dmax .
4. R(0) H(X).
Proof
1. Let D0
D. (R(D), D) achievable ) (R(D), D0 ) achievable.
R(D) R(D0 ) by definition of R(D0 ).
Then
2. Follows from the convexity of the rate-distortion region.
3. (0, Dmax ) is achievable ) R(Dmax ) = 0. Then R(D) = 0 for D
because R(·) is non-increasing.
Dmax
4. Since d is assumed to be normal, (H(X), 0) is achievable, and hence R(0)
H(X).
Theorem 8.15 The following properties hold for the rate-distortion function
R(D):
1. R(D) is non-increasing in D.
2. R(D) is convex.
3. R(D) = 0 for D
Dmax .
4. R(0) H(X).
Proof
1. Let D0
D. (R(D), D) achievable ) (R(D), D0 ) achievable.
R(D) R(D0 ) by definition of R(D0 ).
Then
2. Follows from the convexity of the rate-distortion region.
3. (0, Dmax ) is achievable ) R(Dmax ) = 0. Then R(D) = 0 for D
because R(·) is non-increasing.
Dmax
4. Since d is assumed to be normal, (H(X), 0) is achievable, and hence R(0)
H(X).
Theorem 8.15 The following properties hold for the rate-distortion function
R(D):
1. R(D) is non-increasing in D.
2. R(D) is convex.
3. R(D) = 0 for D
Dmax .
4. R(0) H(X).
Proof
1. Let D0
D. (R(D), D) achievable ) (R(D), D0 ) achievable.
R(D) R(D0 ) by definition of R(D0 ).
Then
2. Follows from the convexity of the rate-distortion region.
3. (0, Dmax ) is achievable ) R(Dmax ) = 0. Then R(D) = 0 for D
because R(·) is non-increasing.
Dmax
4. Since d is assumed to be normal, (H(X), 0) is achievable, and hence R(0)
H(X).
Theorem 8.15 The following properties hold for the rate-distortion function
R(D):
1. R(D) is non-increasing in D.
2. R(D) is convex.
3. R(D) = 0 for D
Dmax .
4. R(0) H(X).
Proof
1. Let D0
D. (R(D), D) achievable ) (R(D), D0 ) achievable.
R(D) R(D0 ) by definition of R(D0 ).
Then
2. Follows from the convexity of the rate-distortion region.
3. (0, Dmax ) is achievable ) R(Dmax ) = 0. Then R(D) = 0 for D
because R(·) is non-increasing.
Dmax
4. Since d is assumed to be normal, (H(X), 0) is achievable, and hence R(0)
H(X).
Theorem 8.15 The following properties hold for the rate-distortion function
R(D):
1. R(D) is non-increasing in D.
2. R(D) is convex.
3. R(D) = 0 for D
Dmax .
4. R(0) H(X).
Proof
0
1. Let D0
D. (R(D), D)
__ achievable ) (R(D), D ) achievable.
R(D) R(D0 ) by definition of R(D0 ).
Then
2. Follows from the convexity of the rate-distortion region.
3. (0, Dmax ) is achievable ) R(Dmax ) = 0. Then R(D) = 0 for D
because R(·) is non-increasing.
Dmax
4. Since d is assumed to be normal, (H(X), 0) is achievable, and hence R(0)
H(X).
Theorem 8.15 The following properties hold for the rate-distortion function
R(D):
1. R(D) is non-increasing in D.
2. R(D) is convex.
3. R(D) = 0 for D
Dmax .
4. R(0) H(X).
Proof
0
1. Let D0
D. (R(D), D)
__ achievable ) (R(D), D
__ ) achievable.
R(D) R(D0 ) by definition of R(D0 ).
Then
2. Follows from the convexity of the rate-distortion region.
3. (0, Dmax ) is achievable ) R(Dmax ) = 0. Then R(D) = 0 for D
because R(·) is non-increasing.
Dmax
4. Since d is assumed to be normal, (H(X), 0) is achievable, and hence R(0)
H(X).
Theorem 8.15 The following properties hold for the rate-distortion function
R(D):
1. R(D) is non-increasing in D.
2. R(D) is convex.
3. R(D) = 0 for D
Dmax .
4. R(0) H(X).
Proof
0
1. Let D0
D. (R(D), D) achievable ) (R(D),
______ D ) achievable.
R(D) R(D0 ) by definition of R(D0 ).
Then
2. Follows from the convexity of the rate-distortion region.
3. (0, Dmax ) is achievable ) R(Dmax ) = 0. Then R(D) = 0 for D
because R(·) is non-increasing.
Dmax
4. Since d is assumed to be normal, (H(X), 0) is achievable, and hence R(0)
H(X).
Theorem 8.15 The following properties hold for the rate-distortion function
R(D):
1. R(D) is non-increasing in D.
2. R(D) is convex.
3. R(D) = 0 for D
Dmax .
4. R(0) H(X).
Proof
R(D0 )
0
1. Let D0
D. (R(D), D) achievable ) (R(D),
______ D ) achievable.
R(D) R(D0 ) by definition of R(D0 ).
Then
2. Follows from the convexity of the rate-distortion region.
3. (0, Dmax ) is achievable ) R(Dmax ) = 0. Then R(D) = 0 for D
because R(·) is non-increasing.
Dmax
4. Since d is assumed to be normal, (H(X), 0) is achievable, and hence R(0)
H(X).
Theorem 8.15 The following properties hold for the rate-distortion function
R(D):
1. R(D) is non-increasing in D.
2. R(D) is convex.
3. R(D) = 0 for D
Dmax .
4. R(0) H(X).
Proof
R(D0 )
0
1. Let D0
D. (R(D), D) achievable ) (R(D),
______ D ) achievable.
R(D) R(D0 ) by definition of R(D0 ).
_________________
Then
2. Follows from the convexity of the rate-distortion region.
3. (0, Dmax ) is achievable ) R(Dmax ) = 0. Then R(D) = 0 for D
because R(·) is non-increasing.
Dmax
4. Since d is assumed to be normal, (H(X), 0) is achievable, and hence R(0)
H(X).
Theorem 8.15 The following properties hold for the rate-distortion function
R(D):
1. R(D) is non-increasing in D.
2. R(D) is convex.
3. R(D) = 0 for D
Dmax .
4. R(0) H(X).
Proof
1. Let D0
D. (R(D), D) achievable ) (R(D), D0 ) achievable.
R(D) R(D0 ) by definition of R(D0 ).
Then
2. Follows from the convexity of the rate-distortion region.
3. (0, Dmax ) is achievable ) R(Dmax ) = 0. Then R(D) = 0 for D
because R(·) is non-increasing.
Dmax
4. Since d is assumed to be normal, (H(X), 0) is achievable, and hence R(0)
H(X).
Theorem 8.15 The following properties hold for the rate-distortion function
R(D):
1. R(D) is non-increasing in D.
2. R(D) is convex.
3. R(D) = 0 for D
Dmax .
4. R(0) H(X).
Proof
1. Let D0
D. (R(D), D) achievable ) (R(D), D0 ) achievable.
R(D) R(D0 ) by definition of R(D0 ).
Then
2. Follows from the convexity of the rate-distortion region.
3. (0, Dmax ) is achievable ) R(Dmax ) = 0. Then R(D) = 0 for D
because R(·) is non-increasing.
Dmax
4. Since d is assumed to be normal, (H(X), 0) is achievable, and hence R(0)
H(X).
Theorem 8.15 The following properties hold for the rate-distortion function
R(D):
1. R(D) is non-increasing in D.
2. R(D) is convex.
3. R(D) = 0 for D
Dmax .
4. R(0) H(X).
Proof
1. Let D0
D. (R(D), D) achievable ) (R(D), D0 ) achievable.
R(D) R(D0 ) by definition of R(D0 ).
Then
2. Follows from the convexity of the rate-distortion region.
3. (0, Dmax ) is achievable ) R(Dmax ) = 0. Then R(D) = 0 for D
because R(·) is non-increasing.
Dmax
4. Since d is assumed to be normal, (H(X), 0) is achievable, and hence R(0)
H(X).
Theorem 8.15 The following properties hold for the rate-distortion function
R(D):
1. R(D) is non-increasing in D.
2. R(D) is convex.
3. R(D) = 0 for D
Dmax .
4. R(0) H(X).
Proof
1. Let D0
D. (R(D), D) achievable ) (R(D), D0 ) achievable.
R(D) R(D0 ) by definition of R(D0 ).
Then
2. Follows from the convexity of the rate-distortion region.
3. (0, Dmax ) is achievable ) R(Dmax ) = 0. Then R(D) = 0 for D
because R(·) is non-increasing.
Dmax
4. Since d is assumed to be normal, (H(X), 0) is achievable, and hence R(0)
H(X).
Theorem 8.15 The following properties hold for the rate-distortion function
R(D):
1. R(D) is non-increasing in D.
2. R(D) is convex.
3. R(D) = 0 for D
Dmax .
4. R(0) H(X).
Proof
1. Let D0
D. (R(D), D) achievable ) (R(D), D0 ) achievable.
R(D) R(D0 ) by definition of R(D0 ).
Then
2. Follows from the convexity of the rate-distortion region.
3. (0, Dmax ) is achievable ) R(Dmax ) = 0. Then R(D) = 0 for D
because R(·) is non-increasing.
Dmax
4. Since d is assumed to be normal, (H(X), 0) is achievable, and hence R(0)
H(X).
Theorem 8.15 The following properties hold for the rate-distortion function
R(D):
1. R(D) is non-increasing in D.
2. R(D) is convex.
3. R(D) = 0 for D
Dmax .
4. R(0) H(X).
Proof
1. Let D0
D. (R(D), D) achievable ) (R(D), D0 ) achievable.
R(D) R(D0 ) by definition of R(D0 ).
Then
2. Follows from the convexity of the rate-distortion region.
3. (0,
__ Dmax ) is achievable ) R(Dmax ) = 0. Then R(D) = 0 for D
because R(·) is non-increasing.
Dmax
4. Since d is assumed to be normal, (H(X), 0) is achievable, and hence R(0)
H(X).
Theorem 8.15 The following properties hold for the rate-distortion function
R(D):
1. R(D) is non-increasing in D.
2. R(D) is convex.
3. R(D) = 0 for D
Dmax .
4. R(0) H(X).
Proof
1. Let D0
D. (R(D), D) achievable ) (R(D), D0 ) achievable.
R(D) R(D0 ) by definition of R(D0 ).
Then
2. Follows from the convexity of the rate-distortion region.
R(Dmax )
3. (0,
__ Dmax ) is achievable ) R(Dmax ) = 0. Then R(D) = 0 for D
because R(·) is non-increasing.
Dmax
4. Since d is assumed to be normal, (H(X), 0) is achievable, and hence R(0)
H(X).
Theorem 8.15 The following properties hold for the rate-distortion function
R(D):
1. R(D) is non-increasing in D.
2. R(D) is convex.
3. R(D) = 0 for D
Dmax .
4. R(0) H(X).
Proof
1. Let D0
D. (R(D), D) achievable ) (R(D), D0 ) achievable.
R(D) R(D0 ) by definition of R(D0 ).
Then
2. Follows from the convexity of the rate-distortion region.
R(Dmax )
3. (0,
R(Dmax ) = 0. Then R(D) = 0 for D
__ Dmax ) is achievable ) ________________
because R(·) is non-increasing.
Dmax
4. Since d is assumed to be normal, (H(X), 0) is achievable, and hence R(0)
H(X).
Theorem 8.15 The following properties hold for the rate-distortion function
R(D):
1. R(D) is non-increasing in D.
2. R(D) is convex.
3. R(D) = 0 for D
Dmax .
4. R(0) H(X).
Proof
1. Let D0
D. (R(D), D) achievable ) (R(D), D0 ) achievable.
R(D) R(D0 ) by definition of R(D0 ).
Then
2. Follows from the convexity of the rate-distortion region.
3. (0, Dmax ) is achievable ) R(Dmax ) = 0. Then R(D) = 0 for D
because R(·) is non-increasing.
Dmax
4. Since d is assumed to be normal, (H(X), 0) is achievable, and hence R(0)
H(X).
Theorem 8.15 The following properties hold for the rate-distortion function
R(D):
1. R(D) is non-increasing in D.
2. R(D) is convex.
3. R(D) = 0 for D
Dmax .
4. R(0) H(X).
Proof
1. Let D0
D. (R(D), D) achievable ) (R(D), D0 ) achievable.
R(D) R(D0 ) by definition of R(D0 ).
Then
2. Follows from the convexity of the rate-distortion region.
3. (0, Dmax ) is achievable ) R(Dmax ) = 0. Then R(D) = 0 for D
because R(·) is non-increasing.
Dmax
4. Since d is assumed to be normal, (H(X), 0) is achievable, and hence R(0)
H(X).
Theorem 8.15 The following properties hold for the rate-distortion function
R(D):
1. R(D) is non-increasing in D.
2. R(D) is convex.
3. R(D) = 0 for D
Dmax .
4. R(0) H(X).
Proof
1. Let D0
D. (R(D), D) achievable ) (R(D), D0 ) achievable.
R(D) R(D0 ) by definition of R(D0 ).
Then
2. Follows from the convexity of the rate-distortion region.
3. (0, Dmax ) is achievable ) R(Dmax ) = 0. Then R(D) = 0 for D
because R(·) is non-increasing.
Dmax
4. Since d is assumed to be normal, (H(X), 0) is achievable, and hence R(0)
H(X).
Theorem 8.15 The following properties hold for the rate-distortion function
R(D):
1. R(D) is non-increasing in D.
2. R(D) is convex.
3. R(D) = 0 for D
Dmax .
4. R(0) H(X).
Proof
1. Let D0
D. (R(D), D) achievable ) (R(D), D0 ) achievable.
R(D) R(D0 ) by definition of R(D0 ).
Then
2. Follows from the convexity of the rate-distortion region.
3. (0, Dmax ) is achievable ) R(Dmax ) = 0. Then R(D) = 0 for D
because R(·) is non-increasing.
Dmax
4. Since d is assumed to be normal, (H(X),
______ 0) is achievable, and hence R(0)
H(X).
Theorem 8.15 The following properties hold for the rate-distortion function
R(D):
1. R(D) is non-increasing in D.
2. R(D) is convex.
3. R(D) = 0 for D
Dmax .
4. R(0) H(X).
Proof
1. Let D0
D. (R(D), D) achievable ) (R(D), D0 ) achievable.
R(D) R(D0 ) by definition of R(D0 ).
Then
2. Follows from the convexity of the rate-distortion region.
3. (0, Dmax ) is achievable ) R(Dmax ) = 0. Then R(D) = 0 for D
because R(·) is non-increasing.
R(0)
Dmax
4. Since d is assumed to be normal, (H(X),
______ 0) is achievable, and hence R(0)
H(X).
Theorem 8.15 The following properties hold for the rate-distortion function
R(D):
1. R(D) is non-increasing in D.
2. R(D) is convex.
3. R(D) = 0 for D
Dmax .
4. R(0) H(X).
Proof
1. Let D0
D. (R(D), D) achievable ) (R(D), D0 ) achievable.
R(D) R(D0 ) by definition of R(D0 ).
Then
2. Follows from the convexity of the rate-distortion region.
3. (0, Dmax ) is achievable ) R(Dmax ) = 0. Then R(D) = 0 for D
because R(·) is non-increasing.
R(0)
Dmax
4. Since d is assumed to be normal, (H(X),
______ 0) is achievable, and hence R(0)
________
H(X).
_______
Theorem 8.15 The following properties hold for the rate-distortion function
R(D):
1. R(D) is non-increasing in D.
2. R(D) is convex.
3. R(D) = 0 for D
Dmax .
4. R(0) H(X).
Proof
1. Let D0
D. (R(D), D) achievable ) (R(D), D0 ) achievable.
R(D) R(D0 ) by definition of R(D0 ).
Then
2. Follows from the convexity of the rate-distortion region.
3. (0, Dmax ) is achievable ) R(Dmax ) = 0. Then R(D) = 0 for D
because R(·) is non-increasing.
R(0)
Dmax
4. Since d is assumed to be normal, (H(X),
______ 0) is achievable, and hence R(0)
________
H(X).
_______
R(D)
H(X)
R(0)
The rate
distortion
region
Dmax
D
© Copyright 2026 Paperzz