Ai-Mei Huang and Truong Nguyen
Image Processing (ICIP), 2009 16th IEEE International
Conference on
1
CONTENTS

INTRODUCTION

COLOR INFORMATION

MV PROCESSING FOR MCFI USING THE
COLOR INFORMATION

SIMULATIONS

CONCLUSIONS
2
Introduction(1/2)
MB (Macro Block)
16
Y1 Y2
Y3 Y4
16
Luminance(intensity)

8 Cb
8
8 Cr
8
Chrominance(color information)
Color information has been shown to be
effective in the object edges detection
 Due to its insensitivity on specular reflection
 Prevent false edge detection as compared to
luminance-based methods
3
Introduction(2/2)

Color has sharper and more consistent
variations between object boundaries
 Applications often take the color information to
assist the image segmentation process.

In our previous work [8]

In this paper, we would like to
 The color information was found very useful for the
unreliable MV detection
 Especially in the areas where the luminance
component tends to distribute uniformly.
 Examine the color information
 Analyze how the chrominance components can be
used to assist the MV processing in MCFI.
[8] A.-M. Huang and T. Nguyen, “A novel multi-stage motion vector
processing method for motion compensated frame interpolation,”
in Proc. ICIP’07, pp. 389–392, 2007.
4
Color Information(1/5)
Cb
Y
Cr

The luminance components have stronger
intensity distribution than the chrominance
components
 Conventional motion estimation often ignore the color
information due to the complexity.
5
Color Information(2/5)

Color characteristics are distinct from luminance
 Such as the insensitivity in highlight or shadow areas
 Used in preventing the false edge detection

The chrominance
improves the edge
identification for
the static text,
face features,
the cap,
and the shirts
6
Color Information(3/5)
If the moving objects have sharp edges,
the ambiguous motions seem more
unlikely to appear.
 From Fig. 1(b), we can observe that the
luminance has very smooth variations
around the face and shirts areas.

 Since the motion is mainly determined using the
luminance difference, the motion can be easily
wrong in these areas.
7
Color Information(4/5)
Many artifacts
around the nose
and the shirts in (a)
 MVs around the
shirts can only be
detected in (c)

 The luminance have
uniformly distribution
so that the encoder
always chooses the
face motion.
 The color has strong
gradients.
8
Color Information(5/5)
Generally, chrominance
residual distribution is
similar to the luminance
components.
 The pavement and
lawn have very similar
intensity.
 The color difference will
become relatively large.

9
Motion Vector Analysis

The residual energy with color
consideration be represented as follows:
 r (i, j), r (i, j), and r (i, j) are the reconstructed residual signals
of Y, Cb and Cr components of the 8×8 block, b
 α is the weight used to emphasize the degree of
color difference. Empirically set α=8 for 4:2:0 YUV
 The residues are embedded in the reconstructed
signals during the decoding process.
Y
Cb
Cr
m,n
10
MV classification process

Compare E to a predefined threshold, ε , based
on the combined residual information.
m,n
1
MB
16
16

The adjacent MBs will be merged as a
group using the residual energy
distribution.
11
Motion Vector Correction using the
Color Information

Minimizing the absolute bidirectional
prediction difference (ABPD) between
forward and backward predictions.
12
SIMULATIONS

Two video sequences, FOREMAN and FORMULA 1
 CIF frame resolution

all encoded using H.263
 with even frames skipped and the skipped frames are
interpolated
13
Visual Comparisons(1/2)


Fig. 4(c), the
artifacts around the
nose and the eye are
reduced.
These artifacts are
removed in Fig. 4(d).
 Since the chrominance
information sharpens
the residual energy.
 Unreliable MVs around
the shirts and face
areas can be identified
and be corrected
accordingly.
14
Visual Comparisons(2/2)

The intensity between
grass and pavement
is very similar.
 So the motion estimation easily
fails on the white lines areas.
15
CONCLUSIONS

We present using color information for the
MV reliability classification.

Unreliable MVs with small luminance
difference can be effectively detected.
16