An Experimental Evaluation of
Rate Adaptation Algorithms in Adaptive
Video Streaming over HTTP
Saamer Akhshabi, Constantine Dovrolis
Georgia Institute of Technology
Ali C. Begen
Cisco Systems
February 3, 2011
Outline
 Overview of adaptive video streaming
 Experimental methodology
 Rate adaptation under available bw variations
 Smooth Streaming player
 Netflix player
 OSMF player
 Live streaming
 Competition between multiple players
 Conclusions
Outline
 Overview of adaptive video streaming
 Experimental methodology
 Rate adaptation under available bw variations
 Smooth Streaming player
 Netflix player
 OSMF player
 Live streaming
 Competition between multiple players
 Conclusions
Other Video Streaming
Approaches
 HTTP Progressive Download
Other Video Streaming
Approaches
 RTSP and RTP
Other Video Streaming
Approaches
 RTMP
Adaptive video streaming
over HTTP
 Several players support it
 Several commercial content providers use it




Microsoft Smooth Streaming Player
Netflix Player
Adobe OSMF/Zeri Player
Apple Player
HTTP Adaptive Video Streaming
HTTP Adaptive Video Streaming:
Manifest File and Fragments
HTTP Adaptive Video Streaming:
HTTP Request and Response
Outline
 Overview of adaptive video streaming
 Experimental methodology
 Rate adaptation under available bw variations
 Smooth Streaming player
 Netflix player
 OSMF player
 Live streaming
 Competition between multiple players
 Conclusions
Objectives
 Examine how commercial adaptive players react to
available bw variations
 Persistent variations
 Short-term variations (spikes)
 We control the available bw using Dummynet
 In parallel, we collect packet traces at the player
 Identify requested bitrates & measure throughput
 Also, we estimate playback buffer size (in seconds)
 Use fragment timestamps (explained later)
Experimental Methodology
Outline
 Overview of adaptive video streaming
 Experimental methodology
 Rate adaptation under available bw variations
 Smooth Streaming player
 Netflix player
 OSMF player
 Live streaming
 Competition between multiple players
 Conclusions
Smooth Streaming Player
Smooth Streaming Player
 Sample HTTP Request:
 GET
/mediadl/iisnet/smoothmedia/Experience/BigBuckBunny720
p.ism/QualityLevels(2040000)/Fragments(video=400000000)
HTTP/1.1
Smooth Streaming Player
Buffering and Steady State
• One fragment per HTTP
request
• No HTTP pipelining
•Two states:
1. Buffering state
• Request fragments as
fast as possible
2. Steady-state
• Request new fragment
every T seconds
Smooth Streaming Player
Behavior under Unrestricted Available Bandwidth
 Average throughput: running average of instantaneous
2-sec TCP throughput measurements.
 Fragment throughput: per-fragment throughput
measurement
Smooth Streaming Player
Behavior under Unrestricted Available Bandwidth
 Two successive, say video, requests sent at times t1 and t2
(t1<t2) with timestamps t‘1 and t‘2 (t‘1 < t‘2) respectively
 The playback buffer duration (in seconds) for video at
time t2 is estimated as:
B(t2) = B(t1) - (t2-t1) + (t‘2 - t‘1)
Smooth Streaming Player
Notice: Important variation in fragment sizes
 The video fragment size varies considerably (VBR encoding)
 The rate can vary widely around the nominal bitrate (2.75
Mbps)
 This can be a significant problem for adaptive video streaming!
Smooth Streaming Player
Behavior under persistent changes in
available bandwidth
 Rate adaptation occurs after long delays
 The player estimates available bw using a running average of
the per-fragment TCP throughput measurements
Smooth Streaming Player
Playback buffer size under persistent changes in
the available bandwidth
 Playback buffer size decreases when available bandwidth is less
than the requested bitrate
 Playback buffer size increases when player goes into “buffering
state” requesting fragments as fast as possible
 Together with switching to bitrate < available bw
Smooth Streaming Player
Behavior under short term available bandwidth
variations (negative spikes)
 The client reacts to some of the spikes by switching to lower
bitrate too late
 Stays at that bitrate for long after the spike has passed
Outline
 Overview of adaptive video streaming
 Experimental methodology
 Rate adaptation under available bw variations
 Smooth Streaming player
 Netflix player
 OSMF player
 Live streaming
 Competition between multiple players
 Conclusions
Netflix Player
Netflix Player
 Sample HTTP Request:
 GET /sa2/946/1876632946.wmv/range/2212059-
2252058?token=1283923056\_d6f6112068075f1fb6
0cc48eab59ea55\&random=1799513140 HTTP/1.1
NetflixPlayer
Behavior under Unrestriced Available Bandwidth
 Player accumulates 5-min playback buffer!
Netflix Player
Behavior under persistent changes in
the available bandwidth
 Initially, the player requests fragments at all possible bitrates
 Occasionally, the player requests higher bitrate than available
bw!
 Utilize large playback buffer size to optimize video quality
Smooth Streaming Player
Behavior under short term available bandwidth
variation with positive spikes
 Netflix appears to be more aggressive than Smooth
Streaming in requesting higher bitrates
Outline
 Overview of adaptive video streaming
 Experimental methodology
 Rate adaptation under available bw variations
 Smooth Streaming player
 Netflix player
 OSMF player
 Live streaming
 Competition between multiple players
 Conclusions
Adobe OSMF/Zeri Player
Adobe OSMF Player
 Sample HTTP Request:
 GET /content/inoutedit-mbr/inoutedit\_h264\_3000Seg1-
Frag5 HTTP/1.1
OSMF Player
Behavior under persistent changes in
the available bandwidth
 The client fails to select highest possible bitrate for the
given available bw
OSMF Player
Behavior under persistent changes in
the available bandwidth
 Also, player often oscillates between bitrates
 Mostly lowest and highest bitrates
Outline
 Overview of adaptive video streaming
 Experimental methodology
 Rate adaptation under available bw variations
 Smooth Streaming player
 Netflix player
 OSMF player
 Live streaming
 Competition between multiple players
 Conclusions
Smooth Live Streaming
 No major differences with on-demand streaming
 Player starts streaming with 8-seconds delay
 CableTV networks often use 7-second “profanity delay”
Smooth Live Streaming
Buffer Fill Level
 Playback delay increases over time whenever
playback buffer gets empty
 Player does not skip fragments
Outline
 Overview of adaptive video streaming
 Experimental methodology
 Rate adaptation under available bw variations
 Smooth Streaming player
 Netflix player
 OSMF player
 Live streaming
 Competition between multiple players
 Conclusions
Two Smooth Streaming players compete
 Players keep oscillating between rates even when
available bw is constant
 Synchronization can cause simultaneous bitrate drops
Two Smooth Streaming players compete
 Fairness issue: one stream may get much lower bitrate
than the other
Outline
 Overview of adaptive video streaming
 Experimental methodology
 Rate adaptation under available bw variations
 Smooth Streaming player
 Netflix player
 OSMF player
 Live streaming
 Competition between multiple players
 Conclusions and future work
Summary of the key differences
between players
 Smooth Streaming player
 Playback buffer size of 10s of seconds
 Conservative in selecting bitrate (< available bw)
 Netflix player
 Playback buffer size of few minutes
 More aggressive than Smooth player (often >
available bw)
 Zeri player
 Erratic bitrate selection
 Under Development (?)
Weaknesses in adaptation logic
 Smooth Streaming player
 Large delay in responding to persistent available bw
variations
 Erratic rate adaptations under short-term variations
 Oscillations and unfairness when multiple players
compete
 Netflix player
 Large delay in responding to persistent available bw
variations
 Erratic rate adaptations under short-term variations
 Requires large playback buffer
 Zeri player
 Adaptation logic seems broken
Future Work
 Continue the analysis of commercial players to
understand how they work
 And identify weaknesses
 Expand study of multi-player competition
 Design and implement an adaptive steaming
adaptation logic that can address all previous
issues