Vanilla TCP?
Alasdair Allan
1
Why TCP?
• Traditional and still the best
• Because we’ve always done it that way
– not always a bad thing
– everyone knows how
• Minimum buy-in for users
• Minimum buy-in for publishers
• Fast, little if any overhead
IVOA Interop Meeting, May 2006
2
User buy-in…
• 92 lines of Perl, including;
– ACK messages
– IAMALIVE messages
– error handling
– re-connection
– message parsing
IVOA Interop Meeting, May 2006
3
Publisher buy-in…
• 130 lines of Perl, including;
– ACK messages
– IAMALIVE messages
– error handling
– re-connection
– multiple clients
IVOA Interop Meeting, May 2006
4
What we’ve done so far?
• Convenience layer and parser (Perl, C++ & Java?)
– building common cases (including GCN)
– http://voevent.sourceforge.net/
• Prototype event network live and on sky
• Limited (hard-wired logic) brokering
– using <What>
• Archiving and persistent storage
• RSS test feed(s)
– programmatic
– human readable?
IVOA Interop Meeting, May 2006
5
TCP(V) prototypes
• VOEvent over “vanilla” TCP/IP
• Acknowledgement of passed message
• Basic health checks on the endpoints
IVOA Interop Meeting, May 2006
6
“Atomic” services
•
•
•
•
•
•
Author
Publisher
Relay
Repository
Subscriber
Listener
An architecture diagram showing; publishers,
subscribers, filters (e.g. relays) and repositories,
along with the more complicated aggregators and
brokers.
IVOA Interop Meeting, May 2006
7
TCP(V) client/server
• Subscriber (client) opens a socket connection to a a
Publisher (server) and listens for events on that port.
• Subscriber (client) will send an acknowledgement to
the Publisher (server) on receipt of an event.
• Publisher (server) will periodically send an “iamalive”
packet to all Subscribers (clients).
• Subscribers (clients) will reply with an “iamalive”.
IVOA Interop Meeting, May 2006
8
Event flow
Publishers
Broker
Subscribers
# # # #
<VOEvent role=“observation”>
.
.
# # # #
.
<VOEvent role=“ack”>
</VOEvent>
.
.
.
</VOEvent>
Repository
IVOA Interop Meeting, May 2006
9
VOEvent Network
Microlensing Survey
SWIFT etc
Exeter
Data Mining
Exeter
GCN
NASA GSFC
SDSS SNe
SuperNova Survey
GCN-2
Exeter
NASA GSFC
U Wash/Stanford
Liverpool Telescope
La Palma
Exeter
Palomar-Quest
Caltech
OGLE III
Las Campanas
Caltech
Faulkes South
Australia
Faulkes North
LANL
NOAO
Hawaii
ULTRACAM
La Palma
Palomar P60
JAC
Caltech
Hawaii
Pairitel
Berkeley
UKIRT
Surveys
Tools/Services
CTIO/KPNO
Community
Key Roles
Publisher
Filter
Repository
SkyDOT
RAPTOR x 8
(database)
LANL
Author
Subscriber
Hawaii
Event Flow IVOA Interop Meeting, May 2006
VOEvent
Other
10
Alternative transport
• RSS 2.0
• eSTAR
– SOAP
– Plastic (over XMLRPC)
• Caltech
– Jabber/XMPP
• NOAO
– Java Messaging
– Mule
IVOA Interop Meeting, May 2006
11
RSS 2.0
• Parallel (on the side?) development to TCP(V)
• The vanilla of the Web 2.0 era
• What RSS is not,
– a transport layer
– a protocol for document exchange
• It’s “just” a document format
– envelope for VOEvent
IVOA Interop Meeting, May 2006
12
RSS feed
<?xml version="1.0" ?>
<rss version="2.0">
<channel>
<title>VOEvent GCN Notices</title>
<language>en</language>
<description>RSS feed for GCN notices</description>
<link>http://voeventnet.caltech.edu/GCN.html</link>
<item>
<title>MILAGRO_Source trigger 886-1</title>
<description>
<p>Possible GRB The event time is 2005-12-01T16:24:15 UT. Location RA 232.1248 Dec 62.9797 (J2000)</p>
</description>
<pubDate>Fri, 02 Dec 2005 16:09:52 PST</pubDate>
<link>http://voeventnet.caltech.edu/Notices/ivo_GCN_MILAGRO_58_886-1_2005-12-01T16:24:15.xml</link>
<author>Scott Barthelmy GCN [email protected]</author>
<comments>http://gcn.gsfc.nasa.gov/gcn3_archive.html</comments>
<enclosure url=“http://estar.org.uk/event.pl?id=886-1” length=“2440” type=“application/xml+voevent” />
</item>
.
.
.
</channel>
</rss>
IVOA Interop Meeting, May 2006
13
RSS feeds
eSTAR native feed (includes OGLE III feed)
http://www.estar.org.uk/voevent/eSTAR/eSTAR.rdf
RAPTOR/TALONS feed (includes GCN translation)
http://www.estar.org.uk/voevent/RAPTOR/RAPTOR.rdf
Caltech feed (re-published GCN translation)
http://www.estar.org.uk/voevent/Caltech/Caltech.rdf
See the eSTAR website http://www.estar.org.uk/ for more information.
IVOA Interop Meeting, May 2006
14
Where are we going?
• Event brokering (aggregators)
– time critical
– non-time critical (data mining?)
• Event archiving and persistent storage
– REST (and SOAP) interfaces
– persistence of data products
• Search service
– REST interface
• Authentication
– signatures
IVOA Interop Meeting, May 2006
15
We should not mandate transport!
• We should not tie our document or protocol standards
to a single mode of transport;
– bad design decision
– RFC 1149, see http://www.blug.linux.no/rfc1149/
– longer term problems
IVOA Interop Meeting, May 2006
16
Cathedral and the Bazaar
• Design by committee
– cathedrals
– slums
• Design by dictate
– lottery
• Design by market pressure
– betamax vs. VHS
– market pressure
IVOA Interop Meeting, May 2006
17
Conclusions
• Probably need a backbone protocol
• Probably going to be TCP(V)
– low buy-in for both publishers and subscribers
– fully supported across different platforms
– momentum is in that direction
• But other transport mechanisms are needed
IVOA Interop Meeting, May 2006
18
The 2nd HTN Workshop
July 23-26, 2006 in the Institut für Astrophysik, Göttingen, Germany
Aims
• Establish the standards for interoperability between robotic
telescope networks.
• Work towards the establishment of an e-market for the
exchange of telescope time.
• Establish the standards for interoperability with the Virtual
Observatory (VO) for event notification.
See http://www.telescope-networks.org/ for details.
IVOA Interop Meeting, May 2006