Performance Evaluation of IP Telephony
over University Network
By
M. Kousa, M Sait, A. Shafi, A. Khan
King Fahd Univ. of Petroleum & Minerals (KFUPM)
Dhahran, Saudi Arabia.
A project funded by University Fast Track
Presentation Outlines
 Definition
of VoIP.
 Performance and Quality Indicators.
 University Networks
 Assessment Designs.
 LAN Assessment Results
 Wi-FI Assessment Results.
 ADSL Assessment Results
 Conclusions.
EE Seminar, March 2009
Evaluation of VoIP over University Networks
2
What is IPT
 Special
telephone sets connected directly to
a network port (RJ45)
 Carrying voice traffic over the data network.
 Circuit-Switched versus Packet-switched
 Enabler: VoIP protocols.
EE Seminar, March 2009
Evaluation of VoIP over University Networks
3
Performance and Quality Indicators




Delay (or Latency): Time takes a packet to make its way
through a network end-to-end.
Packet Loss: Due to errors in Header of Buffer overflow.
Jitter: The measure of the variation of packet arrival time.
Mean Opinion Square (MOS): The most well-known
measure of voice quality.
MOS (Lower Limit)
User Satisfaction
4.34
4.03
3.60
3.10
2.58
Very satisfied
Satisfied
Some users dissatisfied
Many users dissatisfied
Nearly all users dissatisfied
EE Seminar, March 2009
Evaluation of VoIP over University Networks
4
Coding and Compression Algorithms

Packetization
Delay
Theoretical
Maximum MOS
Default Data Rate
(kbps)
The process of converting Analog voice signals to digital
data signals is done with a coder-decoder (CODEC).
G.711u; G.711a 64
1.0 ms
4.40
G.723.1 ACELP 5.3
67.5 ms 3.69
Codec

codec selection depends on bandwidth and quality.
EE Seminar, March 2009
Evaluation of VoIP over University Networks
5
University Networks
Bldg3 .
Bldg4 .
Bldg5 .
Bldg6 .
Bldg7 .
Bldg8 .
Bldg9 .
Bldg10 .
Bldg11 .
Bldg15 .
Bldg16 .
Bldg17 .
Bldg19 .
Bldg20 .
Bldg22 .
Bldg21 .
Bldg23 .
Bldg24 .
Bldg14 .
Bldg14 .
Bldg28 .
ITC
DCC
ITC
C
Press
Clinic
ATM
4 Mb p s
urrently
ERL
Internet
ELC
Schools
Tel .Exch
(Awalnet)
North G
ate
Bldg 34
Bldg 53
29
Main
.
d
Bl
Gate
5
g4
g
Bld
.
Bld
g
40
36
.
Bldg 54
g
Bld
Tel .Exch
Faculty Housing
Voice over IP (VoIP) assessment is performed for all four kinds of university networks.
EE Seminar, March 2009
Evaluation of VoIP over University Networks
6
University Academic Buildings considered for LAN
Assessment Test Design
S.
No.
Building
Number
Reason
1
14
Core data network
2
3
34
21
4
5
5
55
6
59
Core voice network
Largest office building have the
highest number of telephone users
and highest expected number of
incoming and outgoing calls
Tallest academic building and
situated on the edge of wireless
connection to DCC.
Building with highest number of
intermediate routers and home to
telephone operators
Largest academic building
Approx. No.
of Telephone
Users
250
35
700
100
65
300
List of University Academic Buildings, considered for VoIP Test
EE Seminar, March 2009
Evaluation of VoIP over University Networks
7
University LAN Assessment Test Design
Distribution Layer
CATALYST 3550
1000-Base LX Uplink
SYSTEM
RPS
MODE
11
STAT
UTIL
DUPLX
SPEED
1
2
3
4
5
6
7
8
9
10
1000-Base T
Uplink
12
Layer-3 Switch
B-21
Access Layer
B-5
1
3
5
7
9
11
13
15
17
19
21
23
2
4
6
8
10
12
14
16
18
20
22
24
CATALYST 3550
1
2
SYSTEM
RPS
STAT
UTIL
DUPLEX
SPEED
Layer-2 Switch
100-Base T
B-14
Backbone
Network
1000-Base LX
Uplink
B-59
Computer
Laptop
B-34
Inside a Building
Tel .Exch
B-55
LAN Assessment Physical Connectivity Diagram
EE Seminar, March 2009
Evaluation of VoIP over University Networks
8
University LAN Assessment Test Design
B-5
B-14
B-21
B-34
B-59
B-55
LAN Assessment Logical Connectivity Diagram
EE Seminar, March 2009
Evaluation of VoIP over University Networks
9
University LAN Assessment Test Design
Worst-case scenario: half of the
telephone users are busy on calls.
Calls=18
Calls originating from any
building are distributed to all
other buildings according to their
user intensity.
Calls=66
B-5
B-14
22
9
4
1
B-21
All calls are assumed to last for
3 minutes, while the inter-call
delay for the calls on the same
link was set to 5 minutes.
Calls=43
1
9
1
Figure shows concurrent
number of calls between any two
buildings.
EE Seminar, March 2009
Calls=37
3
3
Calls=7
2
B-34
6
1
26
1
Calls=13
B-59
3
Evaluation of VoIP over University Networks
B-55
10
LAN Assessment Results
Number of VoIP Probes
6
Number of links
15
Total number of concurrent calls
emulated in the test
92
Schedule Duration of Test
5 days
Call Quality Summary- G723
Unavailable
< 1%
Poor
< 1%
Good
0%
Good
Total number of calls emulated in the test 93748
Acceptable
Poor
Unavailable
Call Quality Summary- G711
Unavailable
< 1%
Poor
< 1%
Acceptable
100%
Acceptable
0%
Good
Acceptable
Poor
Unavailable
Good
100%
EE Seminar, March 2009
Evaluation of VoIP over University Networks
11
LAN Assessment Results
MOS Results
Call Quality Summary by Call Codec
4.50
4.00
MOS
3.50
3.00
2.50
2.00
1.50
1.00
M OS
EE Seminar, March 2009
G.711u (64 kbps)
G.723.1-A CELP (5.3 kbps)
4.38
3.65
Evaluation of VoIP over University Networks
12
University LAN Utilization
Building 21 LAN Utilization for a Year 2007-08
Building 14 LAN Utilization for a Year 2007-08
EE Seminar, March 2009
Evaluation of VoIP over University Networks
13
University Wireless LAN (WiFi)
•IEEE 802.11g standard which supports up to 54 Mbps.
• The wireless access points are back connected to the layer-2 switches
• Layer-2 switches are connected to the building layer-3 switch.
• Layer-3 switch forwards the data over the fiber-optic link to the
university core network.
Distribution Layer
Distribution Layer
CATALYST 3550
CATALYST 3550
SYSTEM
RPS
SYSTEM
RPS
MODE
11
STAT
UTIL
DUPLX
SPEED
1
2
3
4
5
6
7
8
9
10
12
Layer-3 Switch
Backbone
Network
MODE
11
STAT
UTIL
DUPLX
SPEED
1
2
3
4
SYST
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
8
9
10
Access Layer
CatalystSERIES
2950
10Base-T/100Base-TX
24
1
RPS
SYST
STRT
UTIL
DUPLX
SPEED
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
RPS
STRT
UTIL
DUPLX
SPEED
MODE
MODE
Layer-2 PoE Switch
Access
Point
CISCO AIRONET 1100 SERIES
WIRELESS ACCESS POINT
Laptop
Layer-2 PoE Switch
Access
Point
CISCO AIRONET 1100 SERIES
WIRELESS ACCESS POINT
Laptop
Laptop
EE Seminar, March 2009
7
Layer-3 Switch
CatalystSERIES
2950
10Base-T/100Base-TX
2
6
12
Access Layer
1
5
Access
Point
CISCO AIRONET 1100 SERIES
WIRELESS ACCESS POINT
Laptop
Laptop
Evaluation of VoIP over University Networks
Laptop
14
WiFi Assessment Tests Design,
Physical Connectivity (Bldg 59)
Room 0032 (very good signal)
Room 0072 – PC1 (good signal)
Room 0072 – PC2 (good signal,
Linked with same AP as of PC1)
Room 0081 (far from AP, poor
signal)
Room 1079 (far from AP, poor signal)
Room 2078 (Excellent signal)
Distribution Layer
Layer-3 Switch
CATALYST 3550
SYSTEM
RPS
MODE
11
STAT
UTIL
DUPLX
SPEED
1
2
3
4
5
6
7
8
9
10
12
Access Layer
Layer-2 PoE Switch
Catalyst 2950 SERIES
10Base-T/100Base-TX
1
SYST
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
RPS
STRT UTIL DUPLXSPEED
MODE
AP1
CISCO AIRONET 1100 SERIES
WIRELESS ACCESS POINT
CISCO AIRONET 1100 SERIES
WIRELESS ACCESS POINT
CISCO AIRONET 1100 SERIES
WIRELESS ACCESS POINT
AP3
R-0032
R-0081
R-0071-PC1 R-0071-PC2
CISCO AIRONET 1100 SERIES
WIRELESS ACCESS POINT
CISCO AIRONET 1100 SERIES
WIRELESS ACCESS POINT
AP4
AP2
R-1079
R-2078
EE Seminar, March 2009
Evaluation of VoIP over University Networks
15
AP5
WiFi Assessment Tests Design,
Logical Connectivity
 Three levels of traffic intensity
 Low: one call is initiated
between any pair
 Medium: 2 simultaneous calls
are initiated between any pair
 High: 4 simultaneous calls are
R-0081
initiated between any pair.
R-0032
R-1079
R-0071-PC1
EE Seminar, March 2009
R-2078
Evaluation of VoIP over University Networks
R-0071-PC2
16
WiFi Assessment Results
G711 – 1 Call
Number of Node
6
Number of Links
15
Number of concurrent calls between any two nodes
1
Number of concurrent calls per node
5x1=5
Total number of concurrent calls emulated in the test
15 x 1 = 15
Schedule Duration of Test
1 day
Total number of calls emulated in the test
2700
 The performance seems to be pretty good for more than 99% of the calls.
The average delay was always below 45 ms, which is quite acceptable.
 The lost data was negligible (less than 0.03%).
EE Seminar, March 2009
Evaluation of VoIP over University Networks
17
WiFi Assessment Results
G711 – 2 Calls: Call Quality (MOS)
Number of VoIP Probes
6
Number of Links
15
Number of concurrent calls between any two VoIP probes
2
Number of concurrent calls per VoIP probe
5 x 2 = 10
Total number of concurrent calls emulated in the test
15 x 2 = 30
Schedule Duration of Test
1 day
Total number of calls emulated in the test
5400
Call Quality Summary
Unavailable
< 1%
Poor
33%
Good
Acceptable
Poor
Acceptable
< 1%
EE Seminar, March 2009
Good
67%
Unavailable
Evaluation of VoIP over University Networks
18
WiFi Assessment Results
G711 – 2 Calls: Factors Affecting MOS
The percentage effect of each of the other three factors; delay, jitter, lost
data, is shown in figure. The source of poor quality is mainly due to
delay (51%) and lost data (41%). The effect of jitter is much smaller (8%).
Factors Affecting Call Quality
Lost Data
41%
Delay
Delay
51%
Jitter
Lost Data
Codec
Jitter
8%
EE Seminar, March 2009
Evaluation of VoIP over University Networks
19
WiFi: G711 – 2 Calls: Call Quality by Call Group
Call Quality Summary by Call Group - Top 5
4.50
4.00
MOS
3.50
3.00
2.50
2.00
1.50
1.00
59-0071-P C1-590071-P C2 : G.711u
(64 kbps)
59-0071-P C1-590032 : G.711u (64
kbps)
59-0071-P C2-590032 : G.711u (64
kbps)
59-0071-P C1-592078 : G.711u (64
kbps)
59-0071-P C2-592078 : G.711u (64
kbps)
4.38
4.38
4.38
4.38
4.38
M OS
Call Quality Summary by Call Group - Bottom 5
Room 0081 and its AP is the
source of trouble. All
communications between
0081 and other nodes are
poor, and they are the only
poor links.
4.50
4.00
MOS
3.50
3.00
2.50
2.00
1.50
1.00
M OS
59-0032-59-0081:
G.711u (64 kbps)
59-1079-59-0081:
G.711u (64 kbps)
59-0081-59-0071P C1: G.711u (64
kbps)
59-0071-P C2-590081: G.711u (64
kbps)
59-0081-59-2078 :
G.711u (64 kbps)
1.30
1.31
1.32
1.32
1.35
EE Seminar, March 2009
Evaluation of VoIP over University Networks
20
WiFi: G711 – 2 Calls: Delay
100%
314
80%
310
60%
306
40%
302
20%
298
The delay varies between 300
and 310 ms, which is on the
high side.
0%
12 1 2 3 4 5 6 7 8 9 10 11 12 1 2 3 4 5 6 7 8 9 10 11
A A A A A A A A A A A A P P P P P P P P P P P P
M M M M M M M M M M M M M M M M M M M M M M M M
Delay (ms)
Delay Evaluation by Hour
294
67 67 67 67 67 67 67 67 67 65 67 67 67 67 67 67 67 67 67 67 67 67 67 67
Go o d
A cceptable 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0%
33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33
Poor
Unavailable 0% 0% 0% 0% 0% 0% 0% 0% 0% 3% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0%
Delay (ms)
301301 301301304 302 301302 308 310 306 303303302 302303303 300 301301 301301301 301
Delay Summary by Call Group
Room 0081 and its AP is the source
of trouble. The delay on those pairs
where this link is involved exceeds
800 ms, while the delay on other
groups are very low (45 m sec).
900
800
Delay (ms)
700
600
500
400
300
200
100
0
Delay (ms)
59-0032- 59-0071- 59-1079- 59-0081- 59-0081- 59-0032- 59-0071- 59-0032- 59-0071- 59-007159-0081: P C2-59- 59-0081: 59-2078 59-0071- 59-1079 : P C2-59- 59-2078 P C1-59- P C1-59G.711u
0081:
G.711u
: G.711u
P C1:
G.711u
0032 :
: G.711u
0032 :
1079 :
826
818
814
EE Seminar, March 2009
812
812
49
47
46
46
45
Evaluation of VoIP over University Networks
21
WiFi: G711 – 2 Calls: Lost Data
Lost Data Evaluation by Hour
100%
8.29%
80%
7.79%
60%
7.29%
40%
6.79%
20%
6.29%
0%
12 1 2 3 4 5 6 7 8 9 10 11 12 1 2 3 4 5 6 7 8 9 10 11
A A A A A A A A A A A A P P P P P P P P P P P P
M M M M M M M M M M M M M M M M M M M M M M M M
Lost Data (%)
The lost data varies between
6.6% and 7.7%. In both plots,
the high values are noticed in
the hours 8:00 – 10:00 AM,
indicating a slightly more traffic
activity of the WiFi network at
this period.
5.79%
63 67 67 67 67 67 67 67 67 62 67 67 67 67 67 67 67 67 67 67 67 67 67 67
Go o d
A cceptable 4% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0%
33 33 33 33 33 33 33 33 33 36 33 33 33 33 33 33 33 33 33 33 33 33 33 33
Poor
Unavailable 0% 0% 0% 0% 0% 0% 0% 0% 0% 3% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0%
Lo st Data
6.6 6.6 6.6 6.6 7.2 6.7 6.7 6.9 7.6 7.2 7.7 7.2 7.3 7.1 7.0 7.0 6.9 6.8 6.6 6.6 6.7 6.6 6.6 6.5
Lost Data Summary by Call Group
25.00%
On those poor groups the lost
data exceeds 20%
Lost Data (%)
20.00%
15.00%
10.00%
5.00%
0.00%
Lo st Data
59-0032- 59-1079- 59-0081- 59-0071- 59-0081- 59-0032- 59-2078- 59-0071- 59-0071- 59-003259-0081: 59-0081: 59-0071- P C2-59- 59-2078 59-1079 : 59-1079 : P C2-59- P C1-59- 59-2078
G.711u
G.711u
P C1:
0081:
: G.711u
G.711u
G.711u
1079 :
1079 :
: G.711u
22.36%
21.74%
21.19%
20.48%
EE Seminar, March 2009
18.10%
0.03%
0.03%
0.03%
0.02%
0.01%
Evaluation of VoIP over University Networks
22
WiFi: G711 – 2 Calls: Sample Call Quality
Zooming in to the call level, the
figure shows three random
calls for a good group, while
second figure shows three
random calls for a poor group.
For the first group the quality is
maintained well throughout the
call duration, while the quality
is poor throughout the duration
of second call, with short
intervals when the
communication is lost.
EE Seminar, March 2009
Evaluation of VoIP over University Networks
23
WiFi Assessment Results
G711 – 4 Calls: Call Quality (MOS)
Number of nodes
6
Number of Links
15
Number of concurrent calls between any two nodes
4
Number of concurrent calls per node
5 x 4 = 20
Total number of concurrent calls emulated in the test
15 x 4 = 60
Schedule Duration of Test
1 day
Total number of calls emulated in the test
10800
Figure shows the overall
statistics of call quality. The
Figure shows that the call
quality is poor most of the
time (75%). The quality is
acceptable for 14% of the
time, and good for 10% of the
time.
EE Seminar, March 2009
Call Quality Summary
Unavailable
1%
Good
10%
Acceptable
14%
Good
Acceptable
Poor
Unavailable
Poor
75%
Evaluation of VoIP over University Networks
24
WiFi Assessment Results
G711 – 4 Calls: Factors Affecting MOS
The percentage effect of the three performance factors is shown in Figure
below. Here we can clearly see that the source of poor quality is lost data
(52 %) and delay (44%). The effect of jitter is marginal (4%) therefore no
further analysis of jitter is carried out.
Factors Affecting Call Quality
Delay
44%
Lost Data
52%
Delay
Jitter
Lost Data
Codec
Jitter
4%
EE Seminar, March 2009
Evaluation of VoIP over University Networks
25
WiFi: G711 – 4 Calls: Call Quality by Call Group
Call Quality Summary by Call Group - Top 5
4.50
4.00
3.50
MOS
3.00
2.50
2.00
1.50
1.00
M OS
59-0032-59-2078 :
G.711u (64 kbps)
59-2078-59-1079 :
G.711u (64 kbps)
59-0032-59-1079 :
G.711u (64 kbps)
59-0071-P C2-592078 : G.711u (64
kbps)
59-0071-P C2-590032 : G.711u (64
kbps)
3.98
3.89
3.55
3.25
2.96
Call Quality Summary by Call Group - Bottom 5
4.50
4.00
MOS
3.50
3.00
2.50
2.00
1.50
1.00
M OS
59-0032-59-0081:
G.711u (64 kbps)
59-0081-59-2078 :
G.711u (64 kbps)
59-1079-59-0081:
G.711u (64 kbps)
59-0081-59-0071P C1: G.711u (64
kbps)
59-0071-P C2-590081: G.711u (64
kbps)
1.20
1.20
1.16
1.04
1.03
EE Seminar, March 2009
Evaluation of VoIP over University Networks
26
WiFi: G711 – 4 Calls: Call Quality by Hour
100%
4.50
80%
3.80
60%
3.10
40%
2.40
20%
1.70
0%
Go o d
12 1 2 3 4 5 6 7 8 9 10 11 12 1 2 3 4 5 6 7 8 9 10 11
A A A A A A A A A A A A P P P P P P P P P P P P
M M M M M M M M M M M M M M M M M M M M M M M M
MOS
Call Quality Evaluation by Hour
1.00
6% 12% 8% 18%14% 11% 21%23%8% 11% 10%10% 3% 9% 7% 7% 5% 9% 5% 8% 8% 7% 11% 11%
A cceptable 23%21%20%22%27%20%22%22%13% 6% 8% 9% 11% 9% 9% 10%12% 7% 10% 11% 9% 11% 14%22%
Poor
70%66%70%59%59%69%57%55%78%82%81%79%86%83%85%78%83%84%86%81%82%79%74%66%
Unavailable
1% 1% 2% 1% 1% 0% 0% 0% 1% 0% 1% 2% 0% 0% 0% 6% 1% 1% 0% 0% 2% 3% 0% 0%
M OS
2.6 2.7 2.6 2.8 2.8 2.7 2.8 2.8 2.3 2.13 2.16 2.2 2.0 2.17 2.13 2.112.12 2.13 2.0 2.10 2.112.12 2.4 2.71
The Figure shows the effect of the data traffic
on the quality of IPT traffic (MOS ~ 2.7 in
light traffic hours 11 pm – 7 am) (MOS ~ 2.1 in
Busy Hour (BH) 8am – 9 pm)
EE Seminar, March 2009
Evaluation of VoIP over University Networks
27
WiFi: G711 – 4 Calls: Delay by Hour
100%
800
80%
640
60%
480
40%
320
20%
160
0%
Go o d
12 1 2 3 4 5 6 7 8 9 10 11 12 1 2 3 4 5 6 7 8 9 10 11
A A A A A A A A A A A A P P P P P P P P P P P P
M M M M M M M M M M M M M M M M M M M M M M M M
Delay (ms)
Delay Evaluation by Hour
0
28 33 27 38 33 25 43 43 26 29 27 30 17% 25 20 19%18% 25 16% 23 22 20 28 33
A cceptable 36 31% 35 28 32 42 22 22 31%17% 23 24 36 30 33 25 30 25 31% 25 24 27 30 31%
Poor
35 34 36 34 34 33 36 35 43 54 49 44 47 45 47 50 51% 50 53 52 53 50 43 36
Unavailable 1% 1% 2% 1% 1% 0% 0% 0% 1% 0% 1% 2% 0% 0% 0% 6% 1% 1% 0% 0% 2% 3% 0% 0%
Delay (ms)
637625 655608 617 642 615 601653720687 666697673 690 731721697 717 729 733724678 630
Figure shows that the delay has been always
excessive (average delay over 600 ms and in
busy hours it even approaches 740 ms).
EE Seminar, March 2009
Evaluation of VoIP over University Networks
28
WiFi: G711 – 4 Calls: Lost Data of Best and Worst Call by Hour
100%
35.00%
80%
28.00%
60%
21.00%
40%
14.00%
20%
7.00%
0%
Go o d
12 1 2 3 4 5 6 7 8 9 10 11 12 1 2 3 4 5 6 7 8 9 10 11
A A A A A A A A A A A A P P P P P P P P P P P P
M M M M M M M M M M M M M M M M M M M M M M M M
Lost Data (%)
Lost Data Evaluation by Hour
0.00%
0% 2% 0% 1% 1% 1% 2% 5% 2% 3% 5% 2% 0% 5% 1% 1% 3% 4% 1% 1% 2% 1% 0% 0%
A cceptable 0% 0% 0% 1% 1% 1% 1% 1% 1% 2% 0% 1% 1% 1% 3% 3% 2% 1% 1% 2% 2% 2% 1% 0%
Poor
99 97 98 97 96 97 97 95 96 95 94 95 99 94 96 90 95 94 99 97 94 94 99 100
Unavailable 1% 1% 2% 1% 1% 0% 0% 0% 1% 0% 1% 2% 0% 0% 0% 6% 1% 1% 0% 0% 2% 3% 0% 0%
Lo st Data
19. 18. 18. 17. 17. 17. 16. 16. 25. 27. 28. 26. 29. 27. 28. 28. 28. 28. 28. 26. 27. 27. 21. 18.
The figure shows the lost data evaluation, the
average percentage of lost data has been
above 16%, and exceeding 28% during Busy
hours.
EE Seminar, March 2009
Evaluation of VoIP over University Networks
29
WiFi: G711 – 4 Calls: Sample Call Quality
Finally, we show through-call
evaluation of Call Quality and
corresponding Lost Data of
some selected calls on the best
link. Two such calls are
depicted in figures.
It is interesting to show that
while a high MOS is maintained
throughout the call duration
there are intermittent periods
when call quality drops to a
poor level. These are the
periods when the link suffers
from large packet loss.
EE Seminar, March 2009
Evaluation of VoIP over University Networks
30
WiFi Assessment Results
G723 – 1 Call: Call Quality (MOS)
Number of nodes
6
Number of Links
15
Number of concurrent calls between any two nodes
1
Number of concurrent calls per node
5x1=5
Total number of concurrent calls emulated in the test
15 x 1 = 15
Schedule Duration of Test
1 day
Total number of calls emulated in the test
2700
Call Quality Summary
The performance is always
acceptable. It should be noted
that “acceptable” is the best
quality that can be obtained for
G723.
Unavailable
< 1%
Poor
< 1%
Good
Acceptable
Poor
Unavailable
Acceptable
100%
EE Seminar, March 2009
Evaluation of VoIP over University Networks
31
WiFi Assessment Results
G723 – 1 Call: Factors Affecting MOS
Therefore, that “acceptability” of the quality of the IPT call is mainly due to
codec. The average delay is in the range of 130 ms. This is higher that the average
delay for the G711 codec under same traffic load, due to the extra processing time
for compression.
Factors Affecting Call Quality
Delay
7%
Jitter
< 1%
Lost Data
< 1%
Delay
Jitter
Lost Data
Codec
Codec
93%
EE Seminar, March 2009
Evaluation of VoIP over University Networks
32
WiFi Assessment Results
G723 – 4 Calls: Call Quality (MOS)
Number of nodes
6
Number of Links
15
Number of concurrent calls between any two nodes
4
Number of concurrent calls per nodes
5 x 4 = 20
Total number of concurrent calls emulated in the test
15 x 4 = 60
Schedule Duration of Test
1 day
Total number of calls emulated in the test
10800
By increasing the number of
calls to 4 between each pair, the
quality gets severely affected.
Figure shows that only 28% of
the calls are now acceptable.
Call Quality Summary
Unavailable
< 1%
Acceptable
28%
Good
Acceptable
Poor
Unavailable
Poor
72%
EE Seminar, March 2009
Evaluation of VoIP over University Networks
33
WiFi: G723 – 4 Calls: Call Quality by Call Group
Call Quality Summary by Call Group - Top 5
4.50
The pattern of variation of
performance between
groups is very much similar
to the 2-call case of G-711.
4.00
MOS
3.50
3.00
2.50
2.00
1.50
1.00
59-0071-P C2-592078 : G.723.1A CELP (5.3 kbps)
59-0071-P C2-591079 : G.723.1A CELP (5.3 kbps)
59-0071-P C1-592078 : G.723.1A CELP (5.3 kbps)
59-0071-P C1-591079 : G.723.1A CELP (5.3 kbps)
59-2078-59-1079 :
G.723.1-A CELP
(5.3 kbps)
3.60
3.59
3.59
3.58
3.56
M OS
Call Quality Summary by Call Group - Bottom 5
4.50
4.00
MOS
3.50
3.00
Namely, the groups are
classified in two sets; always
acceptable (MOS=3.6), and
always poor (MOS=1.4).
2.50
2.00
1.50
1.00
M OS
59-0032-59-0081:
G.723.1-A CELP
(5.3 kbps)
59-1079-59-0081:
G.723.1-A CELP
(5.3 kbps)
59-0071-P C2-590081: G.723.1A CELP (5.3 kbps)
59-0071-P C1-590081: G.723.1A CELP (5.3 kbps)
59-0081-59-2078 :
G.723.1-A CELP
(5.3 kbps)
1.28
1.33
1.36
1.38
1.45
EE Seminar, March 2009
Evaluation of VoIP over University Networks
34
WiFi: G723 – 4 Calls: Call Quality of Best and Worst Group by Hour
Since data rate of G723 is low,
there does not seem to be a
Busy Hour effect on the MOS
for either set.
100%
4.50
80%
3.80
60%
3.10
40%
2.40
20%
1.70
0%
Go o d
12 1 2 3 4 5 6 7 8 9 10 11 12 1 2 3 4 5 6 7 8 9 10 11
A A A A A A A A A A A A P P P P P P P P P P P P
M M M M M M M M M M M M M M M M M M M M M M M M
MOS
Call Quality Evaluation by Hour
1.00
0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0%
A cceptable 63 71% 78 71% 56 71% 50 82 69 79 81% 75 75 71% 59 71% 69 75 84 29 69 71% 78 79
Poor
38 29 22 29 44 29 50 11% 31%21%19% 25 25 29 41% 29 31% 25 16%71%31% 29 22 21%
Unavailable 0% 0% 0% 0% 0% 0% 0% 7% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0%
M OS
3.6 3.5 3.613.5 3.6 3.6 3.5 3.6 3.613.6 3.6 3.6 3.5 3.613.5 3.5 3.5 3.6 3.6 3.5 3.6 3.5 3.613.6
100%
4.50
80%
3.80
60%
3.10
40%
2.40
20%
1.70
0%
Go o d
12 1 2 3 4 5 6 7 8 9 10 11 12 1 2 3 4 5 6 7 8 9 10 11
A A A A A A A A A A A A P P P P P P P P P P P P
M M M M M M M M M M M M M M M M M M M M M M M M
MOS
Call Quality Evaluation by Hour
1.00
0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0%
A cceptable 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0%
Poor
100 100 100 100 100 100 100 100 100 100 100 100 100 100 100 100 100 100 100 100 100 100 100 86
Unavailable 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 14%
M OS
1.16 1.15 1.24 1.271.291.481.261.24 1.231.281.291.35 1.371.261.251.241.25 1.341.281.261.251.38 1.281.23
EE Seminar, March 2009
Evaluation of VoIP over University Networks
35
WiFi: G723 – 4 Calls: Call Inside Details
The call quality
within the duration
of the call is
maintained almost
constant, for
"acceptable
groups" as well as
poor groups.
EE Seminar, March 2009
Evaluation of VoIP over University Networks
36
University ADSL Network


ADSL service provided on campus is at G-Lite rate (1.5
Mbps downlink and 0.5 Mbps uplink).
The ADSL system of university is of modular nature.
There are total of 10 DSLAMs and each DSLAM has a
capacity of approximately 100 ADSL lines. Each DSLAM
uplink supports 100 Mbps and is connected to the layer-2
switch.
LAN
Layer-2 Switch
1
3
5
7
9
11
13
15
17
19
21
CATALYST 3550
23
1
2
SYSTEM
RPS
STAT
UTIL
DUPLEX
SPEED
2
4
6
8
10
12
14
16
DSLAM 1
Computer
EE Seminar, March 2009
18
20
22
24
DSLAM 2
Computer
Computer
Computer
Computer
DSLAM 3
Computer
Evaluation of VoIP over University Networks
Computer
37
University ADSL Assessment Design
LAN
DSLAM 1-1
Layer-2 Switch
1
3
5
7
9
11
13
15
17
19
21
CATALYST 3550
23
1
2
SYSTEM
RPS
STAT
UTIL
DUPLEX
SPEED
2
4
6
8
10
12
14
16
18
20
22
24
DSLAM 1-2
DSLAM 1
DSLAM 1-1
Local
Loop
DSLAM 1-3
DSLAM 8
Local
Loop
Local
Loop
DSLAM 8-2
DSLAM 8-1
DSLAM 1-2
DSLAM 1-3
DSLAM 8-1
DSLAM 8-2
ADSL Assessment Test Logical Connectivity
ADSL Assessment Test Physical Connectivity
EE Seminar, March 2009
Evaluation of VoIP over University Networks
38
ADSL Assessment Results
ADSL Assessment Parameters used for VoIP Test


Number of nodes
5
Number of Links
10
Number of concurrent calls between any two nodes
1
Number of concurrent calls per node
4x1=4
Total number of concurrent calls emulated in the test
10 x 1 = 10
Schedule Duration of Test
1 day
Total number of calls emulated in the test
1800
For G711 codec: 100% good result, Delay < 62 ms, Jitter and
Lost Data almost negligible.
For G723 codec: Similar results, slightly lower MOS and
higher delay.
EE Seminar, March 2009
Evaluation of VoIP over University Networks
39
ADSL Assessment Results
Call Quality (MOS) for G711
Call Quality Summary
100%
4.50
80%
3.80
60%
3.10
40%
2.40
20%
1.70
0%
1.00
Go o d
12 1 2 3 4 5 6 7 8 9 10 11 12 1 2 3 4 5 6 7 8 9 10 11
A A A A A A A A A A A A P P P P P P P P P P P P
M M M M M M M M M M M M M M M M M M M M M M M M
MOS
Call Quality Evaluation by Hour
Good
Acceptable
Poor
Unavailable
100 100 100 100 100 100 100 100 100 100 100 100 100 100 100 100 100 100 100 100 100 100 100 100
A cceptable 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0%
Poor
0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0%
Unavailable 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0%
M OS
Good
100%
4.3 4.3 4.3 4.3 4.3 4.3 4.3 4.3 4.3 4.3 4.3 4.3 4.3 4.3 4.3 4.3 4.3 4.3 4.3 4.3 4.3 4.3 4.3 4.3
Call Quality Summary by Call Group
4.50
4.00
MOS
3.50
3.00
2.50
2.00
1.50
1.00 DSLA M 1- DSLA M 1- DSLA M 1- DSLA M 1- DSLA M 8 DSLA M 1- DSLA M 1- DSLA M 1- DSLA M 1- DSLA M 8
2321-21211-1DSLA M 8 DSLA M 8 DSLA M 8 DSLA M 1- DSLA M 1- DSLA M 8 DSLA M 1- DSLA M 1- DSLA M 8 DSLA M 8
-1 :
-1 :
-2 :
2:
3:
-1 :
3:
3:
-2 :
-2 :
4.38
4.37
4.37
4.37
4.37
4.37
4.37
4.37
4.37
4.37
M OS
EE Seminar, March 2009
Evaluation of VoIP over University Networks
40
ADSL Assessment
Results
Delay and Lost Data
for G711
100%
64
80%
63
60%
62
40%
61
20%
60
0%
Go o d
12 1
A A
M M
2 3
A A
M M
4 5 6
A A A
M M M
7 8 9 10 11 12 1 2
A A A A A P P P
M M M M M M M M
3 4 5
P P P
M M M
6 7
P P
M M
8 9 10
P P P
M M M
11
P
M
Delay (ms)
Delay Evaluation by Hour
59
100 100 100 100 100 100 100 100 100 100 100 100 100 100 100 100 100 100 100 100 100 100 100 100
A cceptable 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0%
Poor
0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0%
Unavailable 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0%
Delay (ms)
61 62 61 61 61 62 60 61 61 60 60 62 61 61 61 61 62 62 61 62 61 61 61 61
100%
0.05%
80%
0.04%
60%
0.03%
40%
0.02%
20%
0.01%
0%
Go o d
12 1 2 3 4 5 6 7 8 9 10 11 12 1 2 3 4 5 6 7 8 9 10 11
A A A A A A A A A A A A P P P P P P P P P P P P
M M M M M M M M M M M M M M M M M M M M M M M M
Lost Data (%)
Lost Data Evaluation by Hour
0.00%
100 100 100 100 100 100 100 100 100 100 100 100 100 100 100 100 100 100 100 100 100 100 100 100
A cceptable 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0%
Poor
0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0%
Unavailable 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0%
Lo st Data
.01%.01%.01%.01%.01%.01%.01%.01%.01%.01%.02 .03 .03 .03 .04 .04 .02 .01%.01%.01%.01%.01%.01%.01%
EE Seminar, March 2009
Evaluation of VoIP over University Networks
41
ADSL Assessment Results
Call Quality (MOS) for G723
Call Quality Summary
Unavailable
< 1%
Poor
1%
Good
Acceptable
Poor
Unavailable
Acceptable
99%
DelayQuality
Factors Affecting Call
9%
Jitter
< 1%
Lost Data
< 1%
Delay
Jitter
Lost Data
Codec
Codec
91%
EE Seminar, March 2009
Evaluation of VoIP over University Networks
42
ADSL Assessment
Results
Call Quality (MOS) by
Hour and by Group,
for G723
100%
4.50
80%
3.80
60%
3.10
40%
2.40
20%
1.70
0%
Go o d
12 1 2 3 4 5 6 7 8 9 10 11 12 1 2 3 4 5 6 7 8 9 10 11
A A A A A A A A A A A A P P P P P P P P P P P P
M M M M M M M M M M M M M M M M M M M M M M M M
MOS
Call Quality Evaluation by Hour
1.00
0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0%
A cceptable 100 100 100 100 100 100 100 100 100 100 100 100 96%97%95%100 99%100 100 87%98%100 100 100
Poor
0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 4% 3% 0% 0% 1% 0% 0% 13% 3% 0% 0% 0%
Unavailable 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 5% 0% 0% 0% 0% 0% 0% 0% 0% 0%
M OS
3.6 3.6 3.6 3.6 3.6 3.6 3.6 3.6 3.6 3.6 3.6 3.6 3.6 3.6 3.6 3.6 3.6 3.6 3.6 3.6 3.6 3.6 3.6 3.6
Call Quality Summary by Call Group
4.50
4.00
MOS
3.50
3.00
2.50
2.00
1.50
1.00 DSLA M 1- DSLA M 1- DSLA M 8 DSLA M 1- DSLA M 1- DSLA M 1- DSLA M 1- DSLA M 8 DSLA M 1- DSLA M 122-21312-111DSLA M 8 DSLA M 8 DSLA M 1- DSLA M 1- DSLA M 8 DSLA M 1- DSLA M 1- DSLA M 8 DSLA M 8 DSLA M 8
-2 :
-1 :
3:
3:
-1 :
2:
3:
-2 :
-1 :
-2 :
3.63
3.63
3.62
3.62
3.62
3.62
3.62
3.62
3.62
3.62
M OS
EE Seminar, March 2009
Evaluation of VoIP over University Networks
43
ADSL Assessment
Results
Delay and Lost Data
for G723
100%
153
80%
151
60%
149
40%
147
20%
145
0%
Go o d
12 1 2 3 4 5 6 7 8 9 10 11 12 1 2 3 4 5 6 7 8 9 10 11
A A A A A A A A A A A A P P P P P P P P P P P P
M M M M M M M M M M M M M M M M M M M M M M M M
Delay (ms)
Delay Evaluation by Hour
143
87%89%78%69%71%78%82%83%86%89%86%87%84%86%74%71%78%84%85%71%71%71%67%71%
A cceptable 13%11% 23%31%29%22%18%18%14%11% 14%13%16%14%21%29%23%16%15%29%29%29%33%29%
Poor
0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0%
Unavailable 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 5% 0% 0% 0% 0% 0% 0% 0% 0% 0%
Delay (ms)
145 145 146 146 147 146 146 146 145 145 146 146 146 146 146 147 145 145 145 145 146 147 148 146
100%
0.05%
80%
0.04%
60%
0.03%
40%
0.02%
20%
0.01%
0%
Go o d
12 1 2 3 4 5 6 7 8 9 10 11 12 1 2 3 4 5 6 7 8 9 10 11
A A A A A A A A A A A A P P P P P P P P P P P P
M M M M M M M M M M M M M M M M M M M M M M M M
Lost Data (%)
Lost Data Evaluation by Hour
0.00%
100 100 100 100 100 100 100 100 100 100 100 100 100 100 95%100 100 100 100 100 100 100 100 100
A cceptable 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0%
Poor
0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0%
Unavailable 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 5% 0% 0% 0% 0% 0% 0% 0% 0% 0%
Lo st Data
.01%.01%.01%.01%.01%.01%.01%.01%.01%.01%.01%.01%.01%.01%.01%.01%.01%.01%.01%.03 .01%.01%.01%.01%
EE Seminar, March 2009
Evaluation of VoIP over University Networks
44
Conclusions: LAN





The performance of IPT over LAN was perfect, even when
an unrealistically exaggerated traffic was assumed (50% of
users are busy on call).
The delay was at its minimum value (~ 40ms for G711 and
~130 ms for G723), and lost data and jitter loss was close to
zero.
The MOS parameter was always approaching the
maximum possible for each of the two codecs i.e. 4.38 for
G711 and 3.65 for G723.
No noticeable variation on the call quality was observed
during the span of 7 days nor during the span of the whole
day.
No difference in performance may be attributed to the
specific locations of the nodes or Busy Hour of data traffic.
EE Seminar, March 2009
Evaluation of VoIP over University Networks
45
Conclusions on ADSL






Although the ADSL link is nowhere close to the capacity of
LAN, it is a dedicated link per house.
For G711 codec call quality remained at maximum possible
throughout the duration of test.
The delay was found to be well within the limits, and jitter
and lost data were almost negligible.
All links behaved equally well without any effects of local
loop and of intra-DSALM or inter-DSLAM calls.
The results for the G 723 codec were similarly perfect,
taking into considerations the slightly lower MOS value and
slightly high delay due to the use of this compressed
codec.
The DSLAM network can safely support maximum number
of realistically expected calls from any typical house.
EE Seminar, March 2009
Evaluation of VoIP over University Networks
46
Conclusion

Lowe traffic:





The performance seems to be pretty good for more than 99% of
the calls.
The average delay was always below 45 ms
Lost data was negligible (less than 0.03%).
Moreover, all links performed equally well.
Medium Traffic:




33% of the calls became poor.
High variation in call quality between different pairs of nodes,
some being consistently “good” and others being consistently
“poor”.
By examining the poor links, we found one node common to all.
The channel between that node and the nearest Access Point is
poor, therefore affecting all the communications where this node
is involved.
EE Seminar, March 2009
Evaluation of VoIP over University Networks
47
Conclusion

High Traffic




call quality became poor most of the time (75%).
we started to notice the effect of Busy Hours of data
traffic on IPT quality.
The MOS of calls between 9 pm-8 am were found to
be 30% above that for calls between 8 am – 9 pm.
G723 results was similar in pattern to those of G711
with two differences:


The best quality for this codec is “acceptable” rather
than “good”.
It does not get much affected with data traffic
intensity.
EE Seminar, March 2009
Evaluation of VoIP over University Networks
48
Thank You
EE Seminar, March 2009
Evaluation of VoIP over University Networks
49