nvx-collaboration

X Platform
Collaboration
The X Platform Application Model is a model for building highly scalable and fault tolerant message
driven business applications. Over the past couple of decades, publish-subscribe has proven to be a
powerful communication model for use in distributed applications. However, commercially available
messaging platforms, although enabling application components to communicate using the publishsubscribe metaphor, fall short in offering a model and/or facilities that would elevate the platforms from
being communication toolkits to becoming true platforms for scalable and fault tolerant distributed
applications. The X Platform Application Model, described in this paper does just that. It overlays
publish-subscribe communication with more formal models for application collaboration; defines an
event-driven programming and service-oriented hosting model for application business logic
components; specifies a fault tolerance model by which application components can be redundantly
deployed in highly available configurations; defines the scalability model using which application
components can be scaled out horizontally and, finally, describes a distributed governance model that
enables applications to be configured, deployed and managed in a service grid oriented manner.
X Platform - Collaboration
Introduction
[TBD – Start off by describing in which areas messaging platform are deficient that prevent them from being considered
as full-fledged application platforms]
The X Platform Application Model is comprised of the following:

The Application Collaboration Model
This is the model specifying how application components collaborate. There are, essentially, two collaboration
models for publish-subscribe based applications. They are the Group Communication and Event Driven models.
The X Platform Application Model prescribes the use of the event driven model. This paper describes and
compares both models with a focus on highlighting the benefits and applicability of the event driven model.

The Programming and Hosting Model
This model specifies the application programming API to and describes the architecture of the container
technology used to host application business logic components.

The Fault Tolerance Model
This model specifies how application components are deployed in a highly available manner.

The Scalability Model
This model specifies how application components can be scaled out in a horizontal manner.

The Configuration and Deployment Model
This model specifies how application components are configured and deployed in a grid like manner.
The following sections describe each of the above models.
The Collaboration Model
There are, essentially, two collaboration models for publish-subscribe based systems. They are the Group
Communication and Event Driven models. The latter model has a variant that we call the Content Based Event Driven
model. This section presents each of these models with the intent of not only describing the prescribed Event Driven
model but also to differentiate it from the Group Communication model which is a less expressive model for business
applications.
Group Communication Systems
The defining characteristic of Group Communication systems are that they are, essentially, ‘Communicate by Who’
systems. In such systems, message senders send messages ‘to’ other participants i.e. senders know to whom the
messages are going. Topics represent group addresses that group participants based on some common characteristic.
For example, in a database replication system, all replicated database engines form a group that may be grouped under
Page 1
Copyright © 2008 Neeve Research & Consulting, LLC. All Rights Reserved
Private and Confidential
X Platform - Collaboration
a /Engines topic. All engines subscribe to /Engines. When a particular engine needs to replicate an update to the
other engines, it publishes the update to the /Engines topic. As another example, a set of traders may be grouped
together by the /Traders topic to which they all subscribe. An administrative command that needs to be sent to all
traders is just published to /Traders. The term ‘Group Communication’ is used to describe such systems because such
systems are typically tightly coupled, highly conversational with strong needs for members to be aware of other
members in their conversational group. Many business workflow systems are also modeled as Group Communication
systems though one could make the case that the Event Driven model is equally, if not more, applicable.
In Group Communication systems, topics serve to group messaging participants. The act of subscribing translates to
joining the group represented by the topic being subscribed to and the act of publishing serves to send messages to the
members of the group represented by the topic being published to. The objects transported in such system are
traditional ‘messages’. Message types play a significant role in such systems. The set of message types for a topic usually
defines the vocabulary of communication between the members of the group represented by the topic. Messages types
can be nouns or verbs depending on the protocol at play. For example, a system administrator would send a STOP
message to stop all /Traders i.e. the message type is a verb. On the other hand, in an instant messaging system, a
participant would send a STATEMENT message to the other participant(s) when it has something to say i.e. the message
type is a noun.
Topic hierarchies, in such systems, are used to arrange participants per a participant taxonomy organized by subgroupsupergroup relationships. For example, rather than put all traders in a single /Traders group, traders could be
organized by the country in which they reside e.g. traders are partitioned among the /<Country>/Traders groups. In
this namespace, the /<Country> group would represent everyone in the <Country> country.
Figure 1 - Group Communication Systems
Page 2
Copyright © 2008 Neeve Research & Consulting, LLC. All Rights Reserved
Private and Confidential
X Platform - Collaboration
The taxonomy tree defines the topic namespace. Each node in the tree (topic namespace) identifies a participant group
(topic) while each sub-node identifies a sub-group of the parent group. Participants subscribe to different groups to
which they belong and receive messages that are published to their groups.
Wildcards subscriptions are used to cover more than one group in a single subscription. Messages published to any of
the groups covered by a wildcard will be delivered to the participant subscribing using that wildcard. Wildcard
publication (offered in some messaging systems) publishes a message to all subscribed groups that match the published
wildcard.
A quirk of such systems is that (in almost all publish-subscribe systems] subscription to a particular group does not
automatically subscribe to parent of the group subscribed to. For example, if one extended the topic namespace from
/Traders to /<Country>/Traders then a subscription to /USA/Traders does not automatically subscribe to /USA.
This means that a message sent to everyone in USA will not be received by the traders in USA unless they also subscribe
to /USA. The assumption is that participants will explicitly subscribe to all groups they wish to belong to irrespective of
the groups’ parent-child relationship.
Topic namespaces in Group Communication systems are generally not very complex. The number of messaging
participants and participant groups are generally small thus keeping the namespaces small and shallow. Topic
namespaces in such systems are also not very dynamic. Participant taxonomies are, typically, not based on runtime
information (except for maybe the leaf node each consisting of just an individual participant whose name is dynamically
assigned) resulting in a topic namespace that is relatively static in nature. This static nature of the topic namespaces also
lend to their simplicity. Finally, the fact that although the namespaces can be hierarchically organized but parent child
relationships are not honored in subscriptions also dilutes the benefit of hierarchically organizing namespaces. This also
causes namespaces to have a simpler structure. The complexity of Group Communication systems arises mainly from the
complexity in the communication protocols that run between the messaging participants rather than from the
namespace.
Event Driven Systems
In contrast to Group Communication systems, Event Driven systems are, essentially, ‘Communicate by What’ systems. In
such systems, message senders do not send messages targeted to a specific group of participants. What they do is notify
the rest of the world of what has happened in their world i.e., they do not really know who is going to be processing
their messages. For example, a sender may notify the world of the receipt of an order by publishing a received message
to the /Orders topic. Messaging participants interested in knowing and reacting to the ‘order has been received’ event
will subscribe to the /Orders topic. The term Event Driven is used to describe such systems because the act of
publishing is analogous to the communication of an event. Such systems are typically loosely coupled, less
conversational with little or no requirement for messaging participants to be aware of whom they are communicating
with.
In Event Driven systems, transported objects (i.e. the messages) are system entities. Topic names are entity-types (note
here we distinguish between entity-types and entities – and entity-type is a category while an entity, strictly speaking, is
an instance of a given entity-type). An entity is published to notify the world that the entity set for a particular entity
type has changed i.e. a new entity has entered the system, an existing entity has changed or an entity has been removed
from the system. An entity is always published on the topic whose name reflects the published entity’s type. Participants
Page 3
Copyright © 2008 Neeve Research & Consulting, LLC. All Rights Reserved
Private and Confidential
X Platform - Collaboration
interested in monitoring changes to entities subscribe to topic names that reflect the entity types in which they are
interested. Continuing with the example above, Order is the entity type that defines the topic name on which all Order
instances (entities) are published. The act of publishing the order is to notify the world that the set of orders in the
system has changed.
Figure 2 - Event Driven Systems
Topic hierarchies, in such systems, are typically used to describe an entity taxonomy organized by subtype-supertype
relationships. For example, rather than publishing all orders to the /Orders topic, one may decide to categorize orders
by type and allow participants to monitor changes in a finer grained manner on a per-type basis. Using such a refined
namespace, member orders would be published to /Orders/Member and staged orders would be published to
/Orders/Staged.
Wildcard subscriptions are used to register interest in more than one category of entities or to register interest in finer
grained entity types. For example, one could subscribe to /Orders/... to monitor changes to the entire order set
independent of order type. Wildcard publication in event driven systems is typically never needed since entities typically
are of one and only one entity type (systems where entities could be of two different types are rare but do exist).
In Group Communication systems, it is legitimate and meaningful to publish a message to a node in the topic namespace
that is not the leaf node. What is means in that context is that a message is being sent to a group of participants that is
an aggregate of all its subgroups. However, given the subtype-supertype based organization of the topic namespace in
an Event Driven system, an entity is never published to an intermediate node in the namespace (for those who are
Page 4
Copyright © 2008 Neeve Research & Consulting, LLC. All Rights Reserved
Private and Confidential
X Platform - Collaboration
mentally jumping the gun, this behavior may no longer be valid when we extend the namespace using entity attributes –
right now we are strictly working with a topic namespace based on an entity classification).
As in Group Communication systems, in Event Driven systems where topic namespaces are based on entity taxonomies,
the namespaces are also not very complex. Entity taxonomies are available at design time keeping topic namespaces for
such systems static. Furthermore, the number of entity types in a system is also relatively small keeping the
corresponding namespaces also small and shallow.
In Event Driven systems, the messaging bus can be seen as a virtual entity cache organized by the entity taxonomy. The
act of publishing updates the cache while the act of subscribing registers watches on changes happening to the cache.
Participants collaborate by reacting to changes happening to entities in the cache.
Content Based Event Driven Systems
In most cases, the ability to be able to monitor changes to entity sets is not enough. Most event driven systems require
the ability to refine the interest using entity attributes i.e. a receiver can specify that it wants to receive only changes to
entity sets that pass a given filter criteria. For example, rather than getting all new and updated member orders, a
receiver may only be interested in new member orders that originated from a particular firm. Another receiver may be
interested in only staged orders that have been filled. The ability to do content based filtering on an entity set using a
messaging platform introduces a dynamic portion to the resulting topic namespace.
Figure 3 – Content Based Event Driven Systems
Page 5
Copyright © 2008 Neeve Research & Consulting, LLC. All Rights Reserved
Private and Confidential
X Platform - Collaboration
Messaging systems, in general, do not have any mechanism of filtering messages by message content. Those systems
that do, such as JMS using JMS Selectors, perform rather poorly. For systems that do not offer any content filtering
services, a common mechanism to implement content filtering is to factor the content on which filtering is desired into
the topic namespace and use wildcard subscriptions (and possibly publication) to express finer-grained interest in the
content. It should be noted that the use of dynamic namespaces to enable filtering is an implementation detail and is a
fall-out of the lack of other efficient mechanisms to perform content based filtering. This point is a significant one
because it is a key factor in differentiating EMP (the key component in the Programming Model) from traditional topicbased messaging systems.
Topic namespaces for such content based event driven systems can become rather complex. The hierarchical nature of
the namespace imposes structure on otherwise unstructured content (in as much as name-value pairs is considers
unstructured content). This, along with the language used to express wildcards, can be somewhat restrictive and,
therefore, requires careful analysis and thought during namespace design to identify the data elements and the order in
which they are factored into the namespace. In addition, since the content is only available at runtime, namespaces in
such systems are highly dynamic, can be quite large and grow to be quite deep.
Mixing Group Communication and Event Driven Systems
Most real world systems actually mix the Group Communication and Event Driven paradigms.
Figure 4 - Hybrid Systems
In such systems, the resulting namespace typically has group addresses at the root of the namespace. From a modeling
standpoint, essentially what this does is to scope entity caches to sub-groups of the participants on the bus. For
example, one may want have a different entity taxonomy for the trading engines as opposed to the set of IM processes
with all of them on the same bus. If such is the case, all messages targeted to the trading engines would be published to
Page 6
Copyright © 2008 Neeve Research & Consulting, LLC. All Rights Reserved
Private and Confidential
X Platform - Collaboration
topics that start with, for example, /TradingEngines. The remaining portion of the topics would conform to the entity
taxonomy defined for trading engines. Similarly for entities published to the IM processes.
Page 7
Copyright © 2008 Neeve Research & Consulting, LLC. All Rights Reserved
Private and Confidential