Concurrency Issues in
Client/Server Applications
Chapters 15,16, 28
cs423-cotter
1
Server Concurrency Control
• Must choose between Iterative or
Concurrent
– How many clients may query server?
– How often will clients Query?
– What mode of query will be used?
– How long will service take?
cs423-cotter
2
Server Concurrency Control
• Must choose between Iterative or Concurrent
–
–
–
–
How many clients may query server?
How often will clients Query?
What mode of query will be used?
How long will service take?
• If concurrent, must choose the level of
concurrency
– Apparent or real concurrency
– Variable number of client threads
– Fixed Maximum number of threads
cs423-cotter
3
Server Concurrency Models
• Thread on demand
– One thread per session
– One thread per request
cs423-cotter
4
Server Concurrency Models
• Thread on demand
– One thread per session
– One thread per request
• Thread Pool
–
–
–
–
Create thread pool
Allocate threads to queries as needed.
When query has been served, return thread to pool
When all threads busy, queue or discard other
queries
cs423-cotter
5
Socket Pre-allocation
Techniques
• Associate a thread (or a thread pool) with each
different service
• For TCP, associate each thread with the service
socket, such that when a query arrives, one (and
only one) thread will have accept unblocked, will
create a new socket for this query, handle the
query, and then return to accept.
• For UDP, each request is a single datagram,
which gets assigned to a single slave thread.
cs423-cotter
6
Concurrent Connection-oriented
Master
slave
slave
slave
Port
cs423-cotter
7
Concurrent Connectionless
Master
slave
slave
slave
Port
cs423-cotter
8
Delayed Concurrency in Servers
• Concurrency helpful when the cost of a new
process (thread) is less than service time for the
request.
• May vary by request. (Some database queries
are short and others are long).
cs423-cotter
9
Delayed Concurrency in Servers
• Concurrency helpful when the cost of a new
process (thread) is less than service time for
the request.
• May vary by request. (Some database
queries are short and others are long).
• Start service iteratively and set timer. If timer
expires, create new thread to finish service.
• Start new threads based on service requested
(where multiple services are available).
cs423-cotter
10
Concurrency in Clients
• Why?
– Easier to program, because concurrency forces
modularity
– Easier to maintain, also because of modularity
– Can interact with several servers at once
– Allows the user to interact with server, even when a
call is blocked.
cs423-cotter
11
Concurrency in Clients
Example
• Database query - “residents on Elm Street”
• Response may be short (in Lone Jack)....
or very long (in KCMO)
cs423-cotter
12
Concurrency in Clients
Example
• Database query - “residents on Elm Street”
• Response may be short (in Lone Jack)....
or very long (in KCMO)
• How does the user know when to abort a “failed”
query?
• Might query server for status...
• Might terminate current query (if server is too
slow) and restart on another server...
cs423-cotter
13
Functional Concurrency in Clients
• Use multiple threads to perform different actions
–
–
–
–
–
One thread manages input (keyboard)
One thread manages input to socket
One thread manages output from socket
One thread manages control interaction with server
etc.
cs423-cotter
14
Concurrency in Clients
Multiple Servers
• Allows multiple accesses at the same time...
so that performance measures (round trip delay,
lost packets, etc.) are all taken against the same
network environment.
• Faster than iterative model
cs423-cotter
15
Concurrency in Clients
Issues
• Concurrency Model: May use real or apparent
concurrency (single threaded or multi-threaded).
• OS Support for concurrency: Some systems
allow sharing memory between threads, while
others may not.
• Functional Decomposition: What operations
might be done in parallel, and which need to be
done sequentially?
cs423-cotter
16
Concurrency in Clients
Singly threaded
• Allows memory sharing between connections
• May (or may not) overload, depending on rate of
data exchange with multiple connections
• May deadlock!!
cs423-cotter
17
Concurrent Client (TCPtecho)
#include <stdio.h>, <string.h>, <time.h>, <winsock.h>
#define BUFSIZE 4096
/* write buffer size*/
#define CCOUNT 64*1024 /* default character count*/
#define WSVERS
MAKEWORD(2, 0)
#define MIN(x, y)
((x)>(y) ? (y) : (x))
#define USAGE
"usage: TCPtecho [ -c count ] host1 host2...\n”
struct hdat {
char
*hd_name;
/* host name
*/
SOCKET
hd_sock;
/* host socket descriptor*/
unsigned
hd_rc;
/* recv character count */
unsigned
hd_wc;
/* send character count */
} hdat[FD_SETSIZE];
/* fd to host name mapping*/
char
buf[BUFSIZE]; /* read/write data buffer
*/
void
TCPtecho(fd_set *, int);
int
reader(struct hdat *, fd_set *);
void
writer(struct hdat *, fd_set *);
long
mstime(u_long *);
cs423-cotter
18
Concurrent Client (TCPtecho)
void
main(int argc, char *argv[])
{
int ccount = CCOUNT, i, hcount = 0, fd;
unsigned long
one = 1;
fd_set
afds;
WSADATA
wsdata;
if (WSAStartup(WSVERS, &wsdata))
errexit("WSAStartup failed\n");
FD_ZERO(&afds);
for (i=1; i<argc; ++i)
{
if (strcmp(argv[i], "-c") == 0)
{
if (++i < argc && (ccount = atoi(argv[i])))
continue;
errexit(USAGE);
}
/* else, a host */
cs423-cotter
19
Concurrent Client (TCPtecho)
fd = connectTCP(argv[i], "echo");
if (ioctlsocket(fd, FIONBIO, &one))
{
fprintf(stderr,"can't mark nonblocking (host %s): %d\n",
argv[i], GetLastError());
continue;
}
hdat[hcount].hd_name = argv[i];
hdat[hcount].hd_sock = fd;
hdat[hcount].hd_rc = hdat[hcount].hd_wc = ccount;
++hcount;
FD_SET(fd, &afds);
}
TCPtecho(&afds, hcount);
WSACleanup();
exit(0);}
cs423-cotter
20
Concurrent Client (TCPtecho)
void
TCPtecho(fd_set *pafds, int hcount)
{
fd_set rfds, wfds;
/* read/write fd sets */
fd_set rcfds, wcfds;
/* read/write fd sets (copy)*/
int
fd, hndx, i;
for (i=0; i<BUFSIZE; ++i) /* echo data
buf[i] = 'D';
*/
memcpy(&rcfds, pafds, sizeof(rcfds));
memcpy(&wcfds, pafds, sizeof(wcfds));
(void) mstime((u_long *)0);
cs423-cotter
/* set the epoch */
21
Concurrent Client (TCPtecho)
while (hcount) {
memcpy(&rfds, &rcfds, sizeof(rfds));
memcpy(&wfds, &wcfds, sizeof(wfds));
if (select(FD_SETSIZE, &rfds, &wfds, (fd_set *)0,
(struct timeval *)0) == SOCKET_ERROR)
errexit("select failed: error %d\n",GetLastError());
for (hndx=0; hndx<hcount; ++hndx) {
fd = hdat[hndx].hd_sock;
if (FD_ISSET(fd, &rfds))
if (reader(&hdat[hndx], &rcfds) == 0){/* host done*/
for (i=hndx+1; i<hcount; ++i)
hdat[i-1]=hdat[i];
hcount--;
continue;
}
if (FD_ISSET(fd, &wfds))
writer(&hdat[hndx], &wcfds);
}
}}
cs423-cotter
22
Concurrent Client (TCPtecho)
int
reader(struct hdat *phd, fd_set *pfdset)
{
...
}
void
writer(struct hdat *phd, fd_set *pfdset)
{
...
}
long
mstime(u_long *pms)
{
...
}
cs423-cotter
23
Deadlock in Client/Server Systems
• A situation where computation cannot proceed
because a set or 2 or more components in the
system is blocked and each component is
waiting on another component in the set.
• Cannot be released through an external input.
• Often difficult to detect or prove, because it may
depend on a particular, unusual sequence of
events.
cs423-cotter
24
Deadlock Between Client & Server
• May use request / response model to avoid
deadlock.
• Must ensure that protocol design is “tight”.
Remove ambiguities in the design.
• Assign responsibility for communication
synchronization. (One side always starts)
cs423-cotter
25
Starvation in Client/Servers
• Occurs when some clients cannot obtain service
because other clients or processes are
monopolizing the server resources.
• Results in unfair allocation of services
cs423-cotter
26
Starvation in Client/Servers
• Occurs when some clients cannot obtain
service because other clients or processes
are monopolizing the server resources.
• Results in unfair allocation of services
• Delays waiting for a client to transmit (blocking
on receive) can be managed through timeouts.
• For busy / malicious clients, servers
multithreading or non-blocking can be used.
cs423-cotter
27
Livelock in Clients / Servers
• Like deadlock, livelock results from circular
dependency among processes and resources.
• System is locked into an endless self-generating
sequence of commands that ultimately
consumes all system resources.
cs423-cotter
28
Livelock in Clients / Servers
• Like deadlock, livelock results from circular
dependency among processes and resources.
• System is locked into an endless selfgenerating sequence of commands that
ultimately consumes all system resources.
• Example:
– File Server with time stamp logging.
– Livelock occurs if a remote call to time is used
– First file request comes in.
• Time stamp requires time (which generates an incoming
message to file server).
• Incoming time response looks like a file request, which requires a
cs423-cotter
29
new time stamp.....
Planning for Concurrency
• Consider what system resources should be
consumed by the server
• Consider what the demand for the service(s) is
• Provide limits (threads, processes, memory) that
are appropriate.
• Incorporate logging into the server to track
program problems, congestion, etc.
• Consider “livelock” problems when working with
multiple services.
cs423-cotter
30