INTERNATIONAL TELECOMMUNICATION UNION
FOCUS GROUP ON IPTV
TELECOMMUNICATION
STANDARDIZATION SECTOR
FG IPTV-C-0363
English only
STUDY PERIOD 2005-2008
3rd FG IPTV meeting:
Mountain View, USA, 22-26 January 2007
WG(s): 2
CONTRIBUTION
Source: ZTE Corporation
Performance test of index based video quality measurement
Title:
1. Abstract
In working document on performance monitoring for IPTV (FG.IPTV-Doc-0039, output document
of Busan October 2006 meeting), a generalized monitoring method for multi-media data based on
transmission packet loss was discussed. And a method of constructing index between packet
number and multi-media data was proposed in the contribution document of 3rd FG IPTV meeting
[1]. This contribution supplies the corresponding experiment process and results.
2. Discussion
To illustrate the validity of the index, we compute the objective video quality metric based EPSNR(Edge PSNR) model. For more details of E-PSNR, please refer to the model 2 of ITU-R Rec.
BT.1683 [2].
The test sequences in table 1 are encoded by MPEG4/AVC with baseline profile. The parameters of
video encoder are contained in table 2.
Sequence Name
Table 1. Test Sequences
Size(pixel X pixel)
YUV Format
Number of frames
Mobile_Canlender
720 X 576
4:2:0
200
Harp_Piano
720 X 576
4:2:0
200
The transmission error is simulated by dropping several numbers of video slices with different PLR
(Packet Loss Rate).
In the normal video quality measurement method, encoded video bit streams are decoded by
standard video decoder to get the damaged video sequences. The simple error concealment
algorithm, frame copy method which copies the image data from previous frame to current
damaged area, is adopted. Than the E-PSNR value is computed between the source video and the
damaged video.
Parameter
Table 2. Encoding Parameters
Option
Bitrate
2Mbps
Slice size
Fix 90 MB every slice
Contact:
Chen Rui, Guo Xiujiang
ZTE Corporation
P.R.China
Tel: +86 755 26775733
Fax: +86 755 26775774
Email: [email protected]
[email protected]
Attention: This is a document submitted to the work of ITU-T and is intended for use by the participants to the activities of ITU-T's
Focus Group on IPTV, and their respective staff and collaborators in their ITU-related work. It is made publicly available for
information purposes but shall not be redistributed without the prior written consent of ITU. Copyright on this document is owned by
the author, unless otherwise mentioned. This document is not an ITU-T Recommendation, an ITU publication, or part thereof.
-2FG IPTV-C-0363
RDOptimization
OFF
RestrictSearchRange
No restriction
Motion search algorithm
Full motion search
Number Reference Frames
1frame
Intra Period (or key frame. I frame)
IPPPPPP……
B Frames
OFF
FrameSkip
OFF
WeightedPrediction
OFF
Entropy Coding Method
CAVLC
Data Partition Mode
OFF
LoopFilterDisable
ON
Hadamard Transform
OFF
Error Transform
OFF
In the index based video quality measurement method, the damaged video is reconstructed as
follows. For more details of constructing video index, please refer to the document of 3rd FG IPTV
meeting [1].

The first frame of the video is supposed to be correct, and is copied from the source video.

The following frame is reconstructed by copying from the source video or by retrieving from
index.
 If current packet is lost, retrieve the frame number and MB’s information from the
index. The damaged image area is reconstructed by copying from corresponding area
in previous reconstructed reference frame.
 Else (current packet is not lost)
 If there is no packet loss in previous frames, retrieve the frame number and MB
address from the index. The reconstructed image area is copied from
corresponding source video.
 Else (there is packet loss in previous frames)
 Retrieve the frame number, MB information and Residual information. If the
MB is encoded by intra mode, the MB is copied from corresponding source
video. Else, the MB is reconstructed by the inter prediction from previous
frame(s), where the motion vectors and residuals are retrieved from the index.
As Figure 1 and Figure 2 show, the E-PSNR VQMs (Video Quality Metric) computed by normal
decoder and by index show the similar grade of video quality under different PLR (Packet Loss
Rate).
-3FG IPTV-C-0363
(E-PSNR VQM)
0.8
0.75
0.7
Method 1
Method 2
0.65
0.6
0.55
16%
13%
10%
8%
7%
6%
5%
4%
3%
2%
1%
0%
0.5
Packet Loss Rate
Figure 1. Comparison of E-PSNR VQM for Mobile_Calendar sequence: Method 1, damaged video is
reconstructed by standard video decoder; Method 2, damaged video is reconstructed by index.
(E-PSNR VQM)
0.8
0.75
0.7
Method 1
Method 2
0.65
0.6
0.55
16%
13%
10%
8%
7%
6%
5%
4%
3%
2%
1%
0%
0.5
Packet Loss Rate
Figure 2. Comparison of E-PSNR VQM for Harp_Piano sequence: Method 1, damaged video is reconstructed by
standard video decoder; Method 2, damaged video is reconstructed by index.
To illustrate efficiency of the index based multi-media quality monitoring method, especially under
the condition of monitoring multiple sampling points simultaneously; we simulate the process of
monitoring video quality from 12 sampling points. Each sampling point feeds back the lost packet
to the reference point, and the E-PSNR VQMs of the 12 sampling points are measured by different
methods. One (Method 1) is to reconstruct the damaged video frame using normal decoder; the
other (Method 2) is to reconstruct the damaged video frame based on index. The test environment is
an Intel P4 2.6G PC with 1G of ram, and the software source code is not optimized. Since the
computation process of VQMs of method 1 is exactly the same as that of method 2, we just
compare the time complexity of damaged video reconstruction.
-4FG IPTV-C-0363
(sec)
300
250.54
243.87
250
200
Method1
Method2
150
100
91.26
77.38
50
0
Mobile_Calendar
Harp_Piano
Figure 3. Comparison of computational complexity for Mobile_Calendar and Harp_Piano sequences: Method 1,
damaged video is constructed by standard video decoder; Method 2, damaged video is constructed by index.
Figure 3 shows the time consumption of damaged video reconstruction of 12 sampling points with
different PLR simultaneously.
The proposed index based multi-media quality monitoring method has much loser computational
complexity than the normal multi-media quality monitoring method using standard video decoder.
Specially, the time consumption of damaged video reconstruction process of method 2, which is the
main part of video quality monitoring, just need about one third computational complexity of the
method 1.
3. Conclusion
This document confirms the validity and efficiency of index based multi-media quality monitoring
method [1]. The index based multi-media quality monitoring has the same grade as the normal
multi-media quality monitoring method using standard video decoder. At the same time, the
computational complexity of index based method is much lower than the standard decoder based
method. We hope that the method in document titled “A method of indexing multi-media packet
data for multi-media quality monitoring”[1] can be included in standard document or become a part
of document as reference information.
References
[1] A method of indexing multi-media packet data for multi-media quality monitoring, contribution
document of 3rd FG IPTV meeting, Mountain View, USA, 2007
[2] ITU-R Rec. BT 1683, Objective perceptual video quality measurement techniques for standard
definition digital broadcast television in the presence of a full reference, Int. Telecommun. Union,
Geneva, Switzerland, ITU-R Rec. BT.1683, 2004.
___________