Downloaded from orbit.dtu.dk on: Jul 31, 2017
A Multi-Frame Post-Processing Approach to Improved Decoding of H.264/AVC Video
Huang, Xin; Li, Huiying; Forchhammer, Søren
Published in:
IEEE International Conference on Image Processing, 2007. ICIP 2007.
Link to article, DOI:
10.1109/ICIP.2007.4380034
Publication date:
2007
Document Version
Publisher's PDF, also known as Version of record
Link back to DTU Orbit
Citation (APA):
Huang, X., Li, H., & Forchhammer, S. (2007). A Multi-Frame Post-Processing Approach to Improved Decoding
of H.264/AVC Video. In IEEE International Conference on Image Processing, 2007. ICIP 2007. (Vol. 4, pp. 381384). IEEE. DOI: 10.1109/ICIP.2007.4380034
General rights
Copyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright owners
and it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights.
• Users may download and print one copy of any publication from the public portal for the purpose of private study or research.
• You may not further distribute the material or use it for any profit-making activity or commercial gain
• You may freely distribute the URL identifying the publication in the public portal
If you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediately
and investigate your claim.
A MULTI-FRAME POST-PROCESSING APPROACH TO IMPROVED DECODING OF
H.264/AVC VIDEO
Xin Huang, Huiying Li, and Søren Forchhammer
COM Department, Technical University of Denmark, Building 343, Lyngby, DK-2800
Email:{xin,hli,sf}@com.dtu.dk
ABSTRACT
!" # $
# % & '()*+, Index Terms— "
!" #
1. INTRODUCTION
!" # - % " & !" # . /01 .(23 4 & !" # % %& .(23 /1
!" # /41 5
$ 6 5 7
6 $ ' ,
7 %&
$ %
8 ) 69
" 1-4244-1437-7/07/$20.00 ©2007 IEEE
) 4
) ) :
2. QUALITY METRIC
$
.&
% $ ';(, ' ( <, <
% q = '.)2,
6
I, P B > ;( q =
√
12 × M SE % .)2 Qstep
% 7 0 .)2 !" #
;( $ $ Qstep 'I , 'P , 'B , > % " 6
> 6
% 6
6
.)2 $ ) / 1 % ? /1 6 % 7 0 % 3. MOTION COMPENSATED UPSAMPLING
. #
'.#, ' ' @ , '@ , $, Nf # % .# % .# % 6 . 2
'.2, 6
% 6
(6
% (6
% % 6
IV - 381
Authorized licensed use limited to: Danmarks Tekniske Informationscenter. Downloaded on November 11, 2009 at 09:47 from IEEE Xplore. Restrictions apply.
ICIP 2007
% 0) & 6-7
(Δm = 0, Δn =
" % & 2
& & )
" ) &
9& xr
8
xc
2
& % 5 &4
x̂ = hr xr + hc xc
"
! 6
+
hr
)
hc
& 5*
9
4
Fig. 1 M SE Qstep QP
%
E{Xr Xr }
E{Xc Xr }
Xr
E{Xr Xc }
E{Xc Xc }
Xc
h E{XX }
r
hc
r
=
E{XXc }
&
0
X
3.1. Motion Compensated Upsampling
(
×
(
hr + hc = 1
xr
!
"
! #$
) ) × #$ % & ' : 0 9& hr
hc
hr = 1 − (1 − α)(qc /qr )
hc = 1 − hr
" ) ! %
&
"
& × ( "& & (m+Δm, n+Δn)
(m, n) (Δm, Δn) (
" ) ) %
) " & )
& "
a priori % xr
"
hr
β
'
3
+
" &
" ) % 9 ; xr
xc
α
β
9
' &
qc /qr 0 1 & 0 ≤ α ≤ 1 β ≥ 0 qr , qc ≥ 0 0 ≤ hr ≤ 1 < )
& ) ) * +$',-. /01
*
% ) xr xc
/+14
% %
9& 4
% 2
xr
min(a, b)
max(a, b)
a+b−c
$
3.2. Interpolated Upsampling
/314
x̂intra =
q̂ = hr qr + hc qc
% - /+1 c ≥ max(a, b)
c ≤ min(a, b)
- ) #
% % & % %
a b c xc & 5 %
% & & 2& 6-7
"
xr % 6-7 ( × ( % 2
(mr , nr ) xr % 8
(Δm, Δn) 6-7 % ( × ( xr % % (mr − m − Δm)V, (nr − n − Δn)H
8
% 9& #
9 9& % '
<% ) 4
xintp (m , n ) =
+
a
%
×' b
i
j
xre (i, j)K1 β 3 (|m − i|)β 3 (|n − j|)
xir (a, b)K2 β 3 (|m − a|)β 3 (|n − b|)
=
K1 K2 ! ) xre (i, j) xir (a, b)
&
β 3 (z) & /$14
β (z) =
3
3
|z|3 − 52 |z|2 + 1
2
− 12 |z|3 + 52 |z|2 −
0
≤ |z| ≤#
≤ |z| ≤+
+≤ |z|
>
4|z| + 2
IV - 382
Authorized licensed use limited to: Danmarks Tekniske Informationscenter. Downloaded on November 11, 2009 at 09:47 from IEEE Xplore. Restrictions apply.
#
(
4. DECIMATION
4.2. Edge Area Decimation
! × ! 4.1. No-edge Area Decimation
X
X
0
pl (m0 , n0 ) =
g(m, n, m0 , n0 )ph (m, n) =
Kgv (|m − m0 |)gh (|n − n0 |)w(m, n)ph (m, n)
P
g2 = (. . . , 0, a, 1, a, 0, . . .)
g4 = g2 ∗ g2 = (. . . , a2 , 2a, 1 + 2a2 , 2a, a2 , . . .)
gv = gh = g4 ∗ g4
#(0$
#(($
#(-$
w(m, n) =
#(5$
mobcalCIF
foremanCIF
cyclingCIF
alpha=0.15,beta=0.7
0.7
0.6
h
w0 #
3$ a priori #pcu $ % % #piu $ * γ #
01$ +
+ ph (m,n)−ph (m0 ,n0 )
Th
0.8
#(1$
#(2$
0
0.9
r
ph (m, n), ph (m0 , n0 ) ∈ pcu
1, ph (m, n) ∈ piu , ph (m0 , n0 ) ∈ pcu
w0 , ph (m, n) ∈ pcu , ph (m0 , n0 ) ∈ piu
0
1
0.5
0.4
0.3
+
% ph (m, n) ph (m0 , n0 )
% #pcu $ % #piu $ % ph (m, n) ,-./
w0 q(m,n)/q(m0 ,n0 )
γ
,
γ
0
7 8-329 : ;<!1 ,1. %
CIF #2/-/0$ * * =4>
(IBBP )12 * Nf 6 α β #2$ ?
#1$ # " -$ α = 0.15 β = 0.7
w(m, n)
8>
<
>:
#(6$
m0 ,n0
W(m,n)
5. EXPERIMENTAL RESULTS
"
a #
%
$ * σ ph (m0 , n0 ) ! × ! /
σ ≤ 10
m0 ,n0
W(m,n)
ph (m, n)
ph (m,n)∈F (m0 ,n0 )
wl (m, n) = e−
#!$
pl (m0 , n0 ) % ph (m, n) % &
% ph (m0 , n0 ) K ' # g = 1$ gv gh () '
w(m, n) % * () gv gh +
,-./
a=
ph (m,n)∈F (m0 ,n0 )
m,n
1,
0.5,
P
0
m ,n
W(m,n)
wd (m, n) % wl (m, n) w(m, n) wd (m, n) wl (m, n) /
1
, (m, n) = (m0 , n0 )
#(3$
wd (m, n) = 2×dis((m,n),(m ,n ))
1,
m,n
P
0
" (m0 , n0 ) pl (m0 , n0 ) =
" 4 %
% ph (m0 , n0 ) ,5.
* % F (m ,n ) % * % pl (m0 , n0 )
/
0.2
0.1
0
1
1.2
1.4
1.6
q /q
c
Fig. 2
1.8
2
2.2
r
"
hr qc /qr
@ " 1 % mobcal * >AB
mobcal foreman " 2 6 " >AB 01@ * % / IV - 383
Authorized licensed use limited to: Danmarks Tekniske Informationscenter. Downloaded on November 11, 2009 at 09:47 from IEEE Xplore. Restrictions apply.
mobcalCIF
31.5
31
30.5
PSNR(dB)
30
29.5
29
28.5
28
27.5
! "# $
Avg.PSNR of Directly Decoded Seq.
Avg.PSNR of MC Seq.
27
26.5
300
400
500
600
Fig. 4 700
800
Bitrates(Kbit/s)
900
1000
1100
1200
"# $ #&
foremanCIF
34.5
"# Fig. 3 %
#& '()* +
,.
/ 0,'+1%# 2 3 /
/
4 %
3 33.5
33
PSNR(dB)
6. CONCLUSION
34
32.5
32
31.5
31
30.5
Avg.PSNR of Directly Decoded Seq.
Avg.PSNR of MC Seq.
30
29.5
100
150
200
250
Bitrates(Kbit/s)
Fig. 5 567 8 09 1% 9 : ;
<<
/$ =
Proc. ICASSP , ((>?(( " ,@@-
5,7 :"
;1 $ /
/, = IEEE Trans. Circuits Syst. Video Technol. 6, )@>?)66 ,@@,
5>7 "AB 1%#+0,' 3 C"(> Available:
http://iphome.hhi.de/suehring/tml/download/
5'7 D3 ; $ =
Proc. SPIE >@>?>66 C 6((
5-7 CAB/D & 6''(-/6 ;D
/
= ISO/IEC International Standard
6(()
57 A"E "F
;1 /
= IEEE Trans. Image Process. 6, 'GG?'G(
1 ,@@>
5G7 0D ; $ "AB = 350
"# $ #&
PSNR improvement for individual picture
0.6
mobcalCIF@622Kbit/s
foremanCIF@222Kbit/s
0.5
PSNR Improvement(dB)
7. REFERENCES
300
0.4
0.3
0.2
0.1
0
I B B P B B P B B P B B I B B P B B P B B P B B I
Frame Type
Fig. 6 IV - 384
Authorized licensed use limited to: Danmarks Tekniske Informationscenter. Downloaded on November 11, 2009 at 09:47 from IEEE Xplore. Restrictions apply.