Fast Mode Decision And
Motion Estimation For
JVT/H.264
Pen Yin, Hye – Yeon Cheong Tourapis,
Alexis Michael Tourapis and Jill Boyce
IEEE ICIP 2003 Sep.
Outline
Introduction
 Fast Mode Decision
 Fast Motion Estimation
 Simulation Result
 Conclusion

Introduction(1/4)





High coding efficiency
Minor increase in decoder complexity
High encoder complexity
Motion estimation (ME) and mode decision
contribute most of the complexity,
In H.264 ME is separately considered from
mode decision.

ME is first checked all block types of inter modes,
then the mode decision is made by comparing the
cost of each inter mode
Introduction (2/4)

Minimal the cost defined as follow (ME)

The mode decision is made by minimizing
Introduction (3/4)
B
A
D
G
1
2
3
4
E
5
a b c
6 d 7 e 8
f gh
H
C
Assume E is best integer pel
Then 1、2 ～ 7 are searched
F
Assume 7 is best integer pel
Then a、b ～ h are searched
I
 total : 16 subpel positions
are check
Introduction (4/4)

This paper proposed a algorithms to
alleviate the encoding complexity by jointly
optimize mode decision and ME, while
maintaining coding efficiency
Fast Mode Decision(1/4)
Reduce the number of potential modes,
eliminate ME for some block types
 Decreases the number of tested intra
modes
 Modes are divided into two categories :

inter modes : Skip mode 16x16、16x8 … 4x4
 intra modes : intra 16x16、intra 4x4

Fast Mode Decision(2/4)

Two concepts :
High priority of SKIP mode
 Error surface : built by initial 3 mode 16x16 8x8 4x4


error surface monotonic:
or
 case 1 : the best two modes are 16x16 and 8x8 
only 16x8 and 8x16
 case 2 : the best two modes are 4x4 and 8x8 
only 4x8 and 8x4
 Case 3 : error surface is not monotonic all modes
Fast Mode Decision(3/4)

Step1：check SKIP mode


step2：check 16x16 and 8x8







if MinJmode= Jmode(8x8) || MaxJmode = Jmode(8x8), go to step4
if MaxJmode= Jmode(4x4), go to step5
if MaxJmode= Jmode(16x16), go to step6
step4：check 16x8, 8x16, 8x4, 4x8; go to step7
step5：check 16x8, 8x16; go to step7
step6：check sub-macroblock partition; go to step7
step7：select the best inter mode


if Jmode(SKIP) <Jmode(16x16) && Jmode(SKIP) < Jmode(8x8), go to step7;
otherwise, go to step3
step3：check 4x4


if Jmode(SKIP) < T1, select SKIP as best mode, stop; otherwise go to
step2
if the energy of the residue for best inter mode is > T2, check intra
modes; go to step8
step8：choose the best mode among all tested modes
Fast Mode Decision(4/4)

In step1,T1= Nbitsλmode ( Nbits equals the minimum number
of bits required for non SKIP inter modes )

In step2 comparing SKIP with 16x16 and 8x8



assume that the RD cost for SKIP is the minimal, so
no other modes need to be checked
In step3, check the monotonic
In step7, assume that inter modes have higher
priority than intra modes ( T2 = 0 )
Fast Motion Estimation(1/6)

ME process involves two steps：

Fast integer pel motion search：
 Based
on predictive motion estimation algorithm :
Enhanced Predictive Zonal Search (EPZS)

Fast subpel motion refinement：
 This
paper mainly discuss the subpel ME
 Propose two patterns of fast subpel ME and early
stop criterion
Fast Motion Estimation

EPZS :
ME around initial predictors
 Refinement around the best predictor using
diamond or square patterns
 refinement is performed around the second
best predictor or the median predictor

 Avoid
being trapped into local minimum
Fast Motion Estimation

First pattern :


Assume that the diamond positions are more
important than corner positions
TOTAL 12 point need to check
D
G
C
B
A
1
2
3
4
E
5
6
a b c
d 7 e
f g h
8
H
F
I
Fast Motion Estimation

Second pattern :


store the position for the best integer and its best
diamond position
3 half point need to check
D
G
C
B
A
1
2
3
4
E
5
6
a b c
d 7 e
f g h
8
H
F
I
Fast Motion Estimation

Second pattern :

For quarter pel search, the best position and second
best position in half pel search are chosen.
D
G
C
B
A
1
2
3
4
E
5
6
a b c
d 7 e
f g h
8
H
F
I
Fast Motion Estimation

Three cases :
 Case 1 :
two best pels are integer-pel  two quarter pels on
the same row/column will be tested
 Case 2 :
two best pels are at the same row/column, 3
quarter pels between these two pels will be tested.
 Case 3 :
two best pels are in a diagonal position (ex: E、8)
 3 quarter-pel (two diamond positions and one
corner position) will be tested
Fast Motion Estimation
Step1 : after integer search, examine the
current minimum SAD, if SAD <
T1, stop; otherwise, go to step2;
 Step2 : half pel positions, examine the
current minimum SAD, if SAD <T2,
stop; otherwise, go to step3;
 Step3 : examine the quarter pel positions.

Simulation Result

85% to 90% complexity reduction can be
achieved versus the reference software, while
coding efficiency is only slightly decreased.


average PSNR gain (∆PSNR)
bitrate reduction (bitrate)
Conclusion
In this paper presents a new algorithm to
jointly optimize motion estimation and
mode decision.
 Simulation results demonstrate that our
scheme can maintain coding efficiency at
considerably lower complexity.
