Embedded Passive Monitoring of
Voice over IP Service Quality
Alan Clark
Telchemy, Inc
Atlanta, GA
[email protected]
Copyright Telchemy Inc 2001
Embedded Passive Monitoring
Gateway
Gateway
IP
Network
QoS
metrics
Copyright Telchemy Inc 2001
Monitoring Agent
embedded into
VoIP Gateway
Active vs Passive Testing
• Active Measurement
– Establish looped back test call
– Send recorded speech file
– Compare sent and received files (PSQM/PESQ)
• Passive Monitoring
– Measure characteristics of live calls
– Determine Transmission Quality metric
– Estimate Speech Quality metrics
Copyright Telchemy Inc 2001
The E Model
• “Mouth to ear” transmission quality
measurement
• Produces an “R” factor typically in the
range 50 (bad) -95 (good)
• R factor can be related to MOS score,
Terminate Early (TME) etc.
• ITU G.107/ G.108 and ETSI ETR250
Copyright Telchemy Inc 2001
E Model
R = Ro - Is - Id - Ie + A
Base R value
- Noise level
Impairments that
occur simultaneously
with speech
- received speech level
- sidetone level
- quantization noise
Advantage factor
Impairments that
are delayed with
respect to speech
- talker echo
- listener echo
- round trip delay
Copyright Telchemy Inc 2001
Equipment Impairment
Factor
- CODEC
- multiplexing effects
E Model
Packet
Loss
Loss
Model
Jitter
Jitter
Model
Codec
type
Codec
Model
Ie
Delay, measured
using RTCP
Copyright Telchemy Inc 2001
R Factor
E Model
R Factor vs MOS
R Factor
MOS
4.5
100
90
4.0
80
70
3.0
60
50
0.1
1
10
Percentage of users that terminate calls early
Copyright Telchemy Inc 2001
Impairments can vary with time
• Interaction of streams in IP network causes
many impairments to vary with time
• Adaptive jitter buffer
• Call re-routing
• CODEC type changed during call
Copyright Telchemy Inc 2001
Measuring Packet Loss
Arriving
RTP
packets
Jitter
buffer
Monitor jitter and
packet loss after
jitter buffer
CODEC
RTCP statistics
report every 5 seconds
Late packets
discarded
Copyright Telchemy Inc 2001
Burst vs average loss
Zero packet loss
Burst of packet loss
Is not the same as
Non-bursty packet loss
Copyright Telchemy Inc 2001
Effects of packet loss burst
User perceived
quality profile
High
15-30
seconds
Transmission
Quality
4-5
seconds
Low
Copyright Telchemy Inc 2001
Loss Model - Markov model
3
Lost
P32
P31
P23
P13
2
Rcvd
Burst state
P22
1
Rcvd
P11
P14
P41
4
Lost
Copyright Telchemy Inc 2001
Gap state
Frame loss event driven
Lost( )
Zero packet loss
Lost( )
Lost( )
Burst of packet loss
Copyright Telchemy Inc 2001
Loss Model - Markov model
Model parameters
reconstructed at end
of call
3
Lost
P32
P31
P23
P13
2
Rcvd
Burst state
P22
1
Rcvd
P11
P14
P41
4
Lost
Copyright Telchemy Inc 2001
Gap state
Loss Model - mapping loss to Ie
Curve is CODEC
dependant
Ie (packet loss)
50
40
30
20
10
0
0
5
10
Packet Loss Rate
Copyright Telchemy Inc 2001
15
Determining QoS metrics
1. Determine “good”
and “bad” state
Ie Factor
Copyright Telchemy Inc 2001
Determining QoS metrics
t = 15
t=5
1. Determine “good”
and “bad” state
Ie Factor
2. Estimate
Instantaneous
R Factor for
each state
Copyright Telchemy Inc 2001
Determining QoS metrics
t = 15
t=5
3. Determine average Ie
1. Determine “good”
and “bad” state
Ie Factor
2. Estimate
Instantaneous
R Factor for
each state
Copyright Telchemy Inc 2001
Measuring Delay
Accumulate
frame
Encode
Transmission
CODEC
Jitter
buffer
Decode
CODEC
RTCP exchange
Copyright Telchemy Inc 2001
Delay Model
175 mS “knee”
30
R Factor
Reduction
20
10
0
0
100
200
300
End to end delay (mS)
Copyright Telchemy Inc 2001
400
“Recency” effect
60 second call
MOS 3.82
MOS 3.28
MOS 3.18
“Good” 4.3MOS
“Bad” 1.8 MOS
(3dB SNR)
Copyright Telchemy Inc 2001
Source AT&T
T1A1.7/98-031
Estimation of recency effect
Effects decay over 30-60 seconds
Average for call
Delay since last significant burst
Copyright Telchemy Inc 2001
Integration with VoIP SMS
Service
Mgt
System
Network
Mgt
System
CDR
End of call
msg (DRQ)
SNMP
Set, Get, Trap
Copyright Telchemy Inc 2001
VQmon
H.323/H.225 based reporting
H.225 Disengage Request (DRQ)
nonStandardData field
Rend
INTEGER (0..100)
Rav
INTEGER (0..100)
Rmin
INTEGER (0..100)
Rmax
INTEGER (0..100)
Copyright Telchemy Inc 2001
Embedded Passive Monitoring
C
O
D
E
C
RTP stream
IP
Network
1 Measure Packet Loss Distribution,
Jitter, Delay, CODEC type
2 Produce a transmission quality rating (R Factor)
using extension to the E-Model
3 Map R Factor to an estimated MOS Score
Copyright Telchemy Inc 2001
C
O
D
E
C