Content-Based
Publish/Subscribe:
A Re-Assessment
David S. Rosenblum
London Software Systems
University College London
OTM/DOA 2005
31 October 2005
Acknowledgments

Alexander L. Wolf
Antonio Carzaniga

Costin Raiciu
University of Lugano
University College London
OTM/DOA 2005
31 October 2005
The ‘Fire Hose’
OTM/DOA 2005
31 October 2005
Controlling the Fire Hose
OTM/DOA 2005
31 October 2005
Controlling the Fire Hose
OTM/DOA 2005
31 October 2005
Publish/Subscribe
symbol
symbol==MSFT
IBM
price = 83.47
29.34
30.17
symbol == MSFT
&&
price > 30.00
symbol = MSFT
price = 30.17
OTM/DOA 2005
31 October 2005
Publish/Subscribe Features
 Asynchronous delivery
 Multi-way delivery
 Content-driven interaction
 Anonymity
 Strong decoupling
Many applications are a natural fit
OTM/DOA 2005
31 October 2005
Some Ancient History
YEAST



Pub/sub for LANs of UNIX workstations
Centralised server implementation
Novelty: Applications




Process awareness
Office automation
Telco feature deployment
Many others
.h
OTM/DOA 2005
31 October 2005
.cpp
Some More Recent History
SIENA

Wide-area content-based publish/subscribe



Decentralised overlay network of
publish/subscribe ‘routers’
Routing and forwarding based on
subscription and notification content
Novelty:
Algorithms, Protocols, Architectures
Assumed that the applications
would naturally appear!
OTM/DOA 2005
31 October 2005
Most Recently
PreCache




Sony-funded startup to commercialise
content-based publish/subscribe
Survived 2.5 years
Successful technology development
Less successful business development



Video-on-demand (???)
Anti-virus updates
Travel alerts
OTM/DOA 2005
31 October 2005
So What Are the Killer
Applications?

Many research projects

Many novel research results

No significant deployments yet
Need to take a closer look
at some proposed approaches
OTM/DOA 2005
31 October 2005
SIENA Content-Based Routing
Subscription Forwarding
s1:1
s1
a
1
s1: “price < 700”
2
s1:a
s1:2
s1:2
s1:1
3
5
4
s1:3
6
s1:3
s1:5
8
s1:6
OTM/DOA 2005
31 October 2005
9
7
SIENA Content-Based Routing
Subscription Merging
s1 covers s2
a
1
s1:a
s1:a
s2:2
s1:2
s1:2
s2:8
s1:1
b
s2
4
s1:1
s1:1
s2:5
ss1:covers
s<2 600”
“price
2
2
s1:2
3
5
s1:3
6
s1:3
s1:5
s1:5
s2:b
8
s1:6
OTM/DOA 2005
31 October 2005
9
7
SIENA Content-Based Routing
Notification Delivery
a
1
s1:a
s2:2
s1:2
s2:8
s1:1
4
s1:1
s2:5
n1: “price = 550”
2
s1:2
3
5
s1:3
6
s1:3
b
s1:5
s2:b
8
s1:6
OTM/DOA 2005
31 October 2005
9
7
n1
Implications of SIENA’s Design





Notifications can be very frequent
But subscriptions should be relatively
infrequent
Yet there should be a lot of subscription
variation
But there should be some similar
subscriptions
And the similar subscriptions should come
from the same part of the network
Which applications are like this?
OTM/DOA 2005
31 October 2005
Other Approaches

Gryphon

Subscription flooding over tree of clusters


Hermes

Rendezvous nodes allocated to content types


Applicable if subscriptions are few and stable
Applicable if load is spread evenly by type
PreCache

Trie- and kd-tree-based subscription storage

Applicable if unsubscription occurs very infrequently
All of these limit application suitability
OTM/DOA 2005
31 October 2005
Publish/Subscribe Features
Conceptual Features





Asynchronous delivery
Multi-way delivery
Content-driven interaction
Anonymity
Strong decoupling
Infrastructure Features





Message flooding
Subscription merging
Tree-based routing
Localised forwarding
Content partitioning
Few applications can naturally exploit these features
OTM/DOA 2005
31 October 2005
Example
Stock Quotes vs Online Gaming
Stock Quotes
Online Gaming
 Message flooding
? Subscription merging
 Tree-based routing
 Localised forwarding
 Content partitioning
Message flooding
Subscription merging
Tree-based routing
Localised forwarding
 Content partitioning

?

?
One size infrastructure does not fit all
OTM/DOA 2005
31 October 2005
Matching Applications with
Infrastructures
Application
Characteristics








Notification size
Notification throughput
Notification latency
Notification variability
Subscription selectivity
Subscription stability
Locality
…
OTM/DOA 2005
Infrastructure
Characteristics
???







Number of routers
Number of routing hops
Path redundancy
Subscription replication
Matching complexity
Matching accuracy
…
31 October 2005
Example
Stock Quotes vs Online Gaming
Stock Quotes







Online Gaming
Notification size
Notification frequency
Notification variability
Notification latency
Subscription selectivity
Subscription stability
Locality
Notification size
 Notification frequency
 Notification variability
 Notification latency
 Subscription selectivity
 Subscription stability
? Locality

How do we translate these to design decisions?
OTM/DOA 2005
31 October 2005
Additional Complications

Mobility



Of publishers
Of subscribers
Of routers

Firewalls
Edge Fanout

Security

OTM/DOA 2005
31 October 2005
The Value of Information
Can we do secure content-based routing
over an untrusted infrastructure?
OTM/DOA 2005
31 October 2005
Security in Content-Based
Publish/Subscribe

Encryption used to implement many security goals




But content-based routing intrinsically requires
some transparency of content


Authentication
Confidentiality
Integrity
Infrastructure must be able to determine if a subscription
matches notification
Existing approaches have limited applicability

In large part due to need to secure multiple messages
OTM/DOA 2005
31 October 2005
A Cryptographic Protocol Based
on Yao’s Garbled Circuits




Subscriptions transformed to Boolean
circuits and then garbled based on shared
secret
Notifications encrypted with shared secret
Router evaluates circuit on encrypted
notification
Router knows result but not content!
Weak but inexpensive security
OTM/DOA 2005
31 October 2005
A Cryptographic Protocol Based
on PSM






PSM = Private Simultaneous Messages (Feige et al.)
Subscription matching transformed to graph
reachability
Notifications and subscriptions transformed to
subgraphs and encrypted based on shared secret
Router sums adjacency matrices for subgraphs
Router checks rank of resulting matrix for match
Router knows result but not content!
Better security but very expensive
OTM/DOA 2005
31 October 2005
Inherent Security Limitations
(1)

Must provide confidentiality of both
notifications and subscriptions



Range of plaintext notifications can be matched
against confidential subscription
Range of plaintext subscriptions can be matched
against confidential notification
Router must know outcome of match

This alone can sometimes be useful information

Example: Battlefield Awareness
OTM/DOA 2005
31 October 2005
Inherent Security Limitations
(2)

Router can determine subscription coverage
over time



Again, this may be useful information
Router can determine Euclidean distance
between notifications over time
Studied protocols require sharing of secret
among potentially large number of
publishers and subscribers
OTM/DOA 2005
31 October 2005
Inherent Limitations of Possible
Security Solutions

Cryptographic group membership protocols


Padding notification stream with dummy messages


Reduces throughput and increases latency of
infrastructure
 Defeats the whole purpose of the infrastructure!
Proxy publishers and subscribers


Too expensive with high subscription volatility
Increases latency of messages
Trusted infrastructure

Can be expensive to deploy for each application
OTM/DOA 2005
31 October 2005
A Generic Architecture for
Content-Based Matching
Cluster
2
Cluster
3
Cluster
1
Cluster
4
Cluster
C


Separates matching from routing
Fully-connected mesh of N nodes in C clusters


Full connectivity simulated on DHT with minimal overhead
Choose 2 of 3 configuration parameters



Subscription replication rate R (= N/C)
Notification routing hops H (1  H  C)
Load-balancing factor B
OTM/DOA 2005
31 October 2005
Conclusion
The Past




There have been many innovations in widearea content-based publish/subscribe
But researchers have ignored application
characteristics for too long
A universal infrastructure shared by all
applications is probably not feasible
Security is very difficult to achieve over an
untrusted infrastructure
OTM/DOA 2005
31 October 2005
Conclusion
The Future

We need to understand better the
relationship between application
requirements and infrastructure design

And we need to explore further the limits of
security in content-based publish/subscribe
OTM/DOA 2005
31 October 2005
Questions?
Prof. David S. Rosenblum
London Software Systems
University College London
[email protected]
http://www.cs.ucl.ac.uk/staff/D.Rosenblum/
OTM/DOA 2005
31 October 2005