MERCURY: A Scalable
Publish-Subscribe System
for Internet Games
Ashwin R. Bharambe
To appear in NetGames’02
Game architectures

Last generation (DOOM)
•
•

Current generation (Quake)
•
•
•

Broadcast each update to every
participant
Game proceeds in locksteps
Client-server
Server maintains authoritative
game state
Sends relevant updates to clients
Generation-X
• Mercury!! 
Big Boss
What’s wrong with them ?

Very bad scaling properties
•
•

Central server
•
•
Bottleneck for computation + bandwidth
Single point of failure
Broadcast
•
•
Ridiculous on an Internet scale!
Players receive “unneeded” updates and cheat
Scaling and robustness are not plain research curiosities
• IDC expects the online-gaming market to be worth $2.3 billion by 2005
• Today's gaming servers are expensive to maintain, inflexible, and prone to
•
crash.
Most online games get stuck at about 6,000 players per server, and players on
one server can't talk to those on another, reducing the appeal of an onlinegaming community.
http://www.redherring.com/insider/2002/0117/724.html
More motivations for distributed
game architectures

Building blocks for generic distributed systems
•
•
•

Multiplayer game is a complex distributed application
Communication patterns between game components
are similar in spirit to many other applications
• Instant messengers, Chat rooms
• Service discovery, Stock tickers
• Distributed auction systems
MERCURY architecture will be applicable to these
distributed applications
You get to play games!!! 
The Problem:
Build a Distributed Game which 
Prevents “swamping” at any cost

Minimizes wastage of bandwidth

Is robust
• No Hot-Spots in the system
• Avoid flooding at all costs
• Nodes leaving or joining the system should
have minimal impact on other nodes
Outline





Publish-subscribe systems
How games can be modeled as a
publish-subscribe system
Architecture of MERCURY
• Distributed publish-subscribe system
Simulation results
Future work…
Game requirements

Information needed by a player
•
•
•

Visible and audible events happening in his “arena”
Satellite game information
•
e.g., enemies, teammates, other terrain, etc.
Downloading textures etc. relevant to the “level” in
which he is playing
Big Idea
• These interactions can be modeled as a publish•
subscribe system
With a slight difference!
What is Publish-Subscribe ?
Intelligent Network
I am available!
Publications
Subscription
Wanted: a guy!
I am too!



Publishers produce events or publications
Subscribers register their interests via
subscriptions
Network routes content
•
publications “meet” subscriptions
Example:
First-person shooting game
Publication
(50,250)
int x 100
int y 200
(100,200)
int
int
int
int
x
x
y
y
>=
<=
>=
<=
50
150
150
250
Subscription
Player
(150,150)
Virtual World
The Slight Difference

Traditional pub-sub = filter

State is very important for a game
Every publication

• No notion of an underlying persistent state
• Is matched against subscriptions and routed
• Acts as a write event on the underlying
database
Requirements of a pub-sub
system for games




Expressive subscription language
•
Different games - varied interactions
Scalability of the routing mechanism
•
Scaling with respect to number of subscriptions and
publications
Network Efficiency
•
•
Avoid flooding of subscriptions or publications
Minimize latency – important for real-time games
Previous pub-sub systems (SIENA, Elvin, Gryphon)
•
•
Centralized – OR -Use broadcast in one way or other
MERCURY:
Subscription Language




SQLish
Type, attribute name, operator, value
• Example: int x <= 200
Types: int, float, bool, string
Operators:
• Numeric: <, <=, >, >=, !=, =
• String: suffix, prefix, *
MERCURY:
Routing protocol

Each node responsible for range of attribute values

Similar to B-tree – distributed!
Division of responsibility

•
•

Simple for numeric types
•
have bounded ranges
String types
• Based on first or last few characters; e.g. ‘a-c’ or ‘aa-cz’, etc.
• Can support prefix, suffix
• Supporting substring efficiently is difficult
For each attribute, nodes logically arranged into
circle – call this overlay as Attribute Hub
Routing Illustrated



Send subscription to any one attribute hub
Send publications to all attribute hubs
Each node compares value in message to his range;
and routes along the circle
Subscription
[240, 320)
[160, 240)
Hx
[0, 80)
int
int
int
int
x
x
y
y
>=
<=
>=
<=
50
150
150
250
[0, 105)
Hy
int x 100
int y 200
[80, 160)
Publication
[210, 320)
[105, 210)
Routing Challenges

Cannot use hashing

Routing hops
• Want to support range queries
• O(n) worst case!
• Can be brought down to O(log n) using
exponentially spaced finger pointers ( similar
to Chord )

Load balance sensitive to distribution of
data values
Simulation Results
Delivery Delay
1.2

1
0.8
Centralized
0.6
Mercury
0.4
0.2
Workload = FPS-game
• Virtual world = square
• Player movements: use
mobility models from NS
0
ts20
ts50
Nodes


ts100
Without finger pointers!
Delay between pub. send and
recv by subscriber
Load: Number of publications
routed by a node
Promising Aspects



Expressive subscription language
Completely decentralized architecture
Scalability
•
•
Avoids flooding of subscriptions and publications – reduces
network traffic considerably
Distributes publications and subscriptions throughout the
network – can reduce swamping effectively
•

Problem: depends on distribution of data values
Performance
•
Simulation shows Publication delivery delay scales linearly
Currently working on…


Realistic workloads
•
Building a Quake-II+ prototype which uses
MERCURY
•
•

Introduce BOTs into Quake – collect traces!
Study packets between client-server to figure out
subscription model
Distribute the server state among different nodes
Diffusive load balancing
•
Shed off load to neighbours gradually
Questions, Comments,
Suggestions, Criticisms …