A Multi-Path Routing
Service for Immersive
Environments
Sherlia Shi, Lili Wang, Kenneth L. Calvert,
and James Griffioen
Laboratory for Advanced Networking
University of Kentucky
Supported by NSF, DARPA, and Intel
The Metaverse

Goal: design, implement, deploy, use and evaluate
networked, collaborative, visualization environments
that act as portals into the Metaverse*.

Basic Problem: Visualize and interact with 3D models,
simulated objects, time-dependent data, image data,
avatars, and video streams in collaboration with remote
users across the Internet

Research Areas: computer vision, graphics, networking
protocols and services
*
April 22, 2004
CCGrid: Workshop on Grids and
Advanced Networks
Neil Stephenson, Snowcrash, 1997.
An Example Metaverse App
A CFD Example
- Visualize a 3D animated
simulation
- Interact with the model
- Collaborate with colleagues
or students at remote
metaverse portals
April 22, 2004
Example
Fire Whirl Data
CCGrid: Workshop on Grids and
Advanced Networks
Metaverse Components
Model
Server
Network
Model
Server
Model
Server
April 22, 2004
CCGrid: Workshop on Grids and
Advanced Networks
Metaverse Portal
Network
April 22, 2004
CCGrid: Workshop on Grids and
Advanced Networks
Metaverse Portal
Immersive Display Environment
Based on inexpensive,
commodity, self-configuring,
networked components
April 22, 2004
CCGrid: Workshop on Grids and
Advanced Networks
Metaverse Portal
Immersive Display Environment
• rendering engines
April 22, 2004
CCGrid: Workshop on Grids and
Advanced Networks
Metaverse Portal
Immersive Display Environment
• rendering engines
• projectors
• cameras
Driven by cheap,
commodity, graphics
and video cards
April 22, 2004
CCGrid: Workshop on Grids and
Advanced Networks
Metaverse Portal
Immersive Display Environment
• rendering engines
• projectors
• cameras
• motion sensors
April 22, 2004
CCGrid: Workshop on Grids and
Advanced Networks
Metaverse Portal
Immersive Display Environment
• rendering engines
• projectors
• cameras
• motion sensors
• local area network
April 22, 2004
CCGrid: Workshop on Grids and
Advanced Networks
Metaverse Portal
Immersive Display Environment
• rendering engines
• projectors
• cameras
• motion sensors
• local area network
• control processors
April 22, 2004
CCGrid: Workshop on Grids and
Advanced Networks
Immersive, Multiprojector rendering
(18 projectors)
April 22, 2004
Walkthrough 1
View 1
Walkthrough 2
View 2
CCGrid: Workshop on Grids and
Advanced Networks
Metaverse Data

Data consists of multiple components





Same data may be represented in arbitrary ways




Texture maps
Polygonal meshes
Volumetric Data
User Control Data (position tracking, 3D mouse)
Resolutions – potentially variable resolution
Encodings
Time Series Data (=> video-like)
Generated Data
April 22, 2004
CCGrid: Workshop on Grids and
Advanced Networks
Metaverse Applications


Goal: Offer the most compelling immersive experience possible!
Assumptions:






Massive amounts of complex data to be transmitted
Cannot assume over-provisioning or QoS guarantees
User’s definition of compelling depends on several factors
Application can design for these factors
Application does not want to know details of transport
Implications:




Not all data can/needs to be sent
Data that is “the most important” to send depends on the user and the
network.
Communication channel must support cooperation between the network
and the applications.
Need to maximize bandwidth subject to (good “net citizen”) constraints
April 22, 2004
CCGrid: Workshop on Grids and
Advanced Networks
Transport API Requirements





Feedback: about transport capabilities.
Single end-to-end communication channel: need
a simple “session” abstraction -- App is not
interested in implementation details.
Data “importance”: App specifies what data to
transmit to achieve “compelling”
Sender / Receiver Cooperation: sender specifies
data, receiver defines quality
Reliability: data transmitted reliably
April 22, 2004
CCGrid: Workshop on Grids and
Advanced Networks
Feedback






Feedback could describe any “aspect” of the service
Feedback “aspects” may be network-dependent
Feedback could be at many levels of detail.
Apps don’t want to deal with the above complexity!
Apps have no desire to control the transport service
(i.e., twist transport knobs). Understanding and
mapping knobs to user satisfaction is too hard.
Simply want to know “what if ” answers! Let transport
service figure out the mapping!
April 22, 2004
CCGrid: Workshop on Grids and
Advanced Networks
New Service Abstraction
Sender
1) Specify “important” data
2) Change definition of “important” as needed
Receiver
1) Specify a prioritized delivery plan
2) Let service map the delivery plan onto the underlying network
3) Provide predictions of how well the service will perform
4) Goto 1
April 22, 2004
CCGrid: Workshop on Grids and
Advanced Networks
Prioritized Data





Specified as a hierarchy of nodes
Leaf nodes represent a semantically meaningful
data unit, called an “object”. Objects are the
smallest logical unit and are delivered as a whole
Internal nodes represent groups of objects and
define the percentage of bandwidth to give to
each subtree.
Each tree denotes a priority level
Children nodes may require ordered delivery
April 22, 2004
CCGrid: Workshop on Grids and
Advanced Networks
Example Transmission Specification
Priority 0
Priority j
1
0.2
0.5
c
a
b
Ordered
April 22, 2004
1
0.6
0.3
0.5
0.3
0.2
Unordered,
No weights
CCGrid: Workshop on Grids and
Advanced Networks
0.4
Metaverse Transport Service API




int open_session(struct sockaddr* svrAddr)
int close_session(int sessId)
int set_recv_handle(int sessId, int objId, void
*buf, int len, CallBackFunction* call_back)
MetaInfo* schedule_objects(int sessId,
MetaInfo* objects, int modified)
April 22, 2004
CCGrid: Workshop on Grids and
Advanced Networks
A Multi-path Transport Service

Goal:

Maximize throughput



Minimizing network load


Limit the hops and delay
Share bandwidth across “links” fairly


Find independent paths
Distribute load across paths
Need a sharing policy
Implication:

Need more control: Use Overlay/Grid


April 22, 2004
Control routing
It becomes a “Shared Service”
CCGrid: Workshop on Grids and
Advanced Networks
12 Mbps!
PlanetLab Example
21 Mbps!
Improving
ImprovingThroughput
Throughput(v2)!
(v1)
- use TCP unfriendly protocols
3 Mbps
April 22, 2004
714 Kbps
CCGrid: Workshop on Grids and
Advanced Networks
- use multiple TCP flows
Transport Service Tasks
1) Discover network characteristics to find highbandwidth communication channel(s) between
source and destination
2) Select a set of routes subject to constraints
limiting the load imposed on the network
3) Dynamically map object hierarchy to paths
(i.e., distributed load across the selected paths).
April 22, 2004
CCGrid: Workshop on Grids and
Advanced Networks
Residual Bandwidth Discovery



Each overlay node passively monitors current
bandwidth usage to other overlay nodes
Avoids active probing (ping, iperf, traceroute,…)
Record two variables for an active “virtual link”




Peak bandwidth
Average measured bandwidth
Difference is an estimate of residual bandwidth
Information built up over time and shared among
nodes
April 22, 2004
CCGrid: Workshop on Grids and
Advanced Networks
Path Selection






Selection algorithm must choose multiple paths
to optimize some objective function, subject to
various constraints.
How many paths should be selected?
What is the maximum length of a path?
Are some paths preferred over others?
Are the paths disjoint?
We evaluated three path selection algorithms
April 22, 2004
CCGrid: Workshop on Grids and
Advanced Networks
Random Path Selection



Randomly select paths from the source to the
destination.
Start from source and randomly select a link that
satisfies the constraints, otherwise backtrack and
try another link
Only guarantees that the virtual paths are
disjoint, not the physical paths
April 22, 2004
CCGrid: Workshop on Grids and
Advanced Networks
Closest First Path Selection





Closest First Multipath (CFM) routing uses a breadthfirst approach to select paths with the shortest hop
count.
Start from the source and select all edges
For each selected node, select all edges
When destination is reached, add path to feasible set
If more feasible paths than allowed, select from feasible
set randomly.
April 22, 2004
CCGrid: Workshop on Grids and
Advanced Networks
Widest First Path Selection




Widest First Multipath (WFM) routing selects
the paths with the highest bandwidth
Start with the source, following the widest path
If the destination is reached, the widest path
edges are removed, and the algorithm restarts
If there are more feasible paths than allowed,
paths are selected from the feasible set randomly
April 22, 2004
CCGrid: Workshop on Grids and
Advanced Networks
Mapping Data to Paths



A variety of mappings possible
Our approach sends all fragments of an object
over the same path to minimize fragmentation
effects
Our goal is to minimize the “finish time”
(adhering to the data prioritization plan), while
satisfying the buffering constraints at the
receiver.
April 22, 2004
CCGrid: Workshop on Grids and
Advanced Networks
Experimental Results
Random




Closest First
Widest First
Alpha = sharing policy (% of residual for local traffic)
When alpha is 0 (no protection) and network load is high, all algorithms perform
poorly
Medium values of alpha produce good throughput at low load and converge to the
shortest path at high load
Random tends to pick paths that are unnecessarily long and have low bandwidth
April 22, 2004
CCGrid: Workshop on Grids and
Advanced Networks
th
5
April 22, 2004
Percentile Results
CCGrid: Workshop on Grids and
Advanced Networks
Correlated Paths

Are hard to identify, because topology info is
not readily available



Traceroute is not sufficient
Not clear how important it is to know if paths
are correlated or not
We developed two passive techniques to identify
correlated links – each has its own advantages.
April 22, 2004
CCGrid: Workshop on Grids and
Advanced Networks
Summary





Metaverse-like applications require a new API
We proposed an API that allows application and
network to work together
Proposed a passive monitoring approach to
estimate available bandwidth
Designed and evaluated three algorithms for
path selection
Developed an algorithm to load balance across
paths – based on the data priority spec.
April 22, 2004
CCGrid: Workshop on Grids and
Advanced Networks