NINJA
The Ninja Project
Enabling Internet-scale Services from Arbitrarily Small Devices
Project of UC Berkeley Computer Science Division
Paper : The Ninja Architecture for Robust
Internet-Scale Systems and Services
http://ninja.cs.berkeley.edu/
People : Eric Brewer ,David Culler, Anthony Joseph ,
Randy Katz ,Steven Gribble, Matt Welsh
The Ninja Project
Enabling Internet-scale Services from Arbitrarily Small Devices
•Points to be covered :
Goal : The end goal of the Ninja project is to enable the
development of a menagerie of Internet-based services
which are interoperable and immediately accessible
across the spectrum of user devices ranging from PCs
and workstations to cellphones and Personal Digital
Assistants.
Examples of Applications of Ninja Vision :
* Sanctio
* Juke Box
The Ninja Project
Enabling Internet-scale Services from Arbitrarily Small Devices
The Ninja Project
Enabling Internet-scale Services from Arbitrarily Small Devices
Components of Ninja Architecture :
1) Building Robust services
* Scalability
* Availability
* Fault-Tolerance
* Consistency
* Persistence
All this in under the Growing Traffic Load
* Concept of Base : Cluster of Workstations
* Concept of vSpace :Cluster based software Platform
The Ninja Project
Enabling Internet-scale Services from Arbitrarily Small Devices
2) Device Diversity : Concept of Units
3) Adaptation by Active proxies : Concept of Distillation
* Computational power
* Network Connectivity
* Interface Capabilities of devices
e.g. Wireless Base stations, Network gateways ,
Firewalls , Gateways .
4) The Composition of Services : Composition of services
into a PATH as well as adaptation along that PATH.
* SDS (Service Discovery Service )
The Ninja Project
Enabling Internet-scale Services from Arbitrarily Small Devices
Bases : Scalable Platforms for internet Services
# Concept of Conditioning the Service: Graceful operation
during the bursts
# A programming Model :
* Splitting a Service into Stages
e.g. : web server service = reading + retrieving + return
* Design Patterns
1.Wrap
2. Combine
3. Pipeline 4. Replicate
# Java –Based I/O Substrate Implementation
Jaguar : (Java Access to generic Underlying Architectural Resources )
Extension to the Java Environment
The Ninja Project
Enabling Internet-scale Services from Arbitrarily Small Devices
#vSpace: Cluster based execution environment for
* Replication
* Load balancing
* Fault Tolerance
Concept of
* Worker : fixed size thread pool + methods
* Service definition : set of workers
* Service Publication :freezing of the code
* Clone :replication of the workers along
various other nodes
* Clone Group : set of worker clones
The Ninja Project
Enabling Internet-scale Services from Arbitrarily Small Devices
Distributed Data Structures :
# Normal D.S. Like : Binary tree, Hash Table etc.
# Properties Expected :
* High throughput
* High Concurrency
* Availability
# Strictly Consistent in Nature :
# Atomic in Nature
The Ninja Project
Enabling Internet-scale Services from Arbitrarily Small Devices
Active proxies : Execute in an environment local to devices
and hence perform context aware optimizations and
transformations on behalf of devices
1)Dynamic Service Adaptation : Distillation of service
content into a format more suitable for small devices
eg. 1) HTML
to WML for WAP phones
2)Design of the light weight protocols
2) Secure Service Access from Diverse Clients: trusted
active proxies can perform context aware transformations on data
before it arrives at a kiosk
3) Multiple device fusion :Combining capabilities of
various devices .Eg. Display of local kiosk and entering data thru
PDA
The Ninja Project
Enabling Internet-scale Services from Arbitrarily Small Devices
Paths : Composition of Services Across the Wide-Area
1)Automatic Path Creation : (APC )
* Finding path between the networks
* Creating the network connection
* Fine Tuning the performance of data flow
* Error Correction
2) Path = Operators (perform computation on data)
+ Connectors (protocol translations between
operators )
3) Operators : Strongly types and know the input they
accept and output they produce .
The Ninja Project
Enabling Internet-scale Services from Arbitrarily Small Devices
* Operators have Cost parameters associated with it which
helps in route optimization .
* Types of Operators :
* Long lived :- standard Ninja services
* Dynamically generated : light weight and short
lived transformation elements created by the
APC facility as and when required
Both these types of the operators ensure that that the path
can be reconstructed when A failure occurs .
* Example of Path : Mapquest Service
The Ninja Project
Enabling Internet-scale Services from Arbitrarily Small Devices
Logical path Creation
Physical path Creation
Path Instantiation and Execution
Path Tear Down
Path Construction Process
The Ninja Project
Enabling Internet-scale Services from Arbitrarily Small Devices
Service Location Service across the Wide Area (SDS):
* Services announce their presence
* Location of the announced services
* help clients decide the trustworthiness of the
service and vice a versa .
# Properties of the SDS :
* Fully secure
* Semantically rich
* Scalable
* Fault Tolerant
The Ninja Project
Enabling Internet-scale Services from Arbitrarily Small Devices
Design of Service Discovery Service :
1) Elements of the SLS
* Clients
* Services
* SDS servers.
2) Hierarchical arrangement of servers : Domain concept
3) Uses of XML for encoding both services and queries
4) Certificate Authority : (CA) trusted source which provide
the proof of binding between a principal and its public
and Private keys in the form of certificates .
5) Capability Manager : (CM) Manages individual access
control Lists on behalf of each authenticated service .
Hiding the services rather than disallowing the access
The Ninja Project
Enabling Internet-scale Services from Arbitrarily Small Devices
Wide Area Operation in SDS
* Information is propagated up to parent servers and
queries are partitioned among and forwarded to the
relevant servers.
* To prevent upper level servers in the hierarchy from
being loaded by the query load SDS filters the
information while it propagates the information
forward .
* SDS servers dynamically organize themselves into
potentially multiple hierarchies for data partitioning and
query routing.
The Ninja Project
Enabling Internet-scale Services from Arbitrarily Small Devices
Ninja and Related Work : Ninja is not Java
1) Jini
2) EJB
3) CORBA 4) Centaurus
5) Active networks
Future Work in Ninja
1) Challenges in automatically composing service
components to meet the needs of particular device
2) Unsolicited advertising : SPAM
3) Requirement of Services capturing bits rather than
eyeballs