Scalable Content-aware Request Distribution
in Cluster-based Network Servers
Jianbin Wei
10/4/2001
Introduction
$
$
$
$
$
$
Content-based request distribution can improve the throughput of the
whole cluster server by cache affinity and get good balancing
simultaneously.
A single front-end quickly becomes the bottleneck of whole
performance and limits the scalability of the distributed servers.
TCP splicing strategy forwards the request based on the workload of
back-end servers. Use weighted round-robin to get good load
balancing.
Decouple the role of the front-end into dispatcher and distributor.
The dispatcher decides the distribution policy.
The distributors perform the policy result by distributing the request.
Jianbin Wei
2
Performance of LARD and TCP splicing
Jianbin Wei
3
Solution
$
$
$
$
$
Decouple the role of the front-end into dispatcher and distributor.
The dispatcher decides the distribution policy.
The distributors perform the policy result by distributing the request.
The key insight is that the function of distributor can be executed
parallel while the overhead of dispatcher accounts very small.
With multiple front-end nodes, it must choose which distributor to
process the request, which is difficult to achieve good load balancing
with simple strategy.
Jianbin Wei
4
Models
Jianbin Wei
5
Prototype Implementation
$
$
$
$
A persistent TCP control connection exists between
any two nodes of the cluster and multiplexes the
messages exchanged between any two
components.
The layer 4 switch maintains the load of each cluster
node and decides the first part of client request.
After receiving update message from the dispatcher
that indicates which cluster server node is chose to
service the request, it redistributes these requests.
Upon connection termination, a close message is
sent to the switch to discard the connection state.
Jianbin Wei
6
Cluster Operation
Jianbin Wei
7
Operation (cont.)
$
$
$
$
$
The client contacts the switch, which decided which distributor to
service the request;
The distributor contacts the dispatcher to get the assignment of this
request;
The distributor distributes the request to the selected server node and
sends this information to the switch;
The server response goes to the client directly.
Any TCP acknowledgments sent by the client to the distributor are
forwarded to the server node using a forwarding module at the
distributor node. (DNS round-robin)
Jianbin Wei
8
Technologies
When using layer 4 switch
$
The choice of distributor is made by the switch, using WRR strategy
$
After a connection is handed off by the distributor, the switch is notified
and the subsequent forwarding of TCP acknowledgments to the
corresponding server node is handled by the switch.
To decrease the communication between the dispatcher and the
distributor,
$
it hashes the URL to a 32 bit integer
$
Sends several messages at the same time with dynamic degree of
batching.
Jianbin Wei
9
Experimental Results
Jianbin Wei
10
Experimental Results (cont.)
$
$
$
$
It shows the throughput results as the number of server nodes are
increased.
It indicates that the dispatcher is far from being a bottleneck.
Due to the resource share between distributor and server
component, the performance of N nodes is achieved by N-1 server
nodes in LARD strategy. However, it affords greater scalability of the
cluster server.
When it runs the real workload, it shows the similar results.
Jianbin Wei
11
Conclusion
$
$
This strategy is a scalable architecture for content-aware request
distribution in Web server clusters.
Drawback: because of the high overhead of handoff, it is necessary to
decrease this cost to improve the performance and scalability of cluster
server. (Scalable Web Server Cluster Design with Workload-Aware
Request Distribution Strategy WARD, L. Cherkasova and M. Karlsson,
HP Lab)
Jianbin Wei
12