CONCURRENT PROGRAMMING
Lecture 5
Assignment 1

Having trouble?
Resolve it as early as possible!

Assignment 2:

 Handling
text-based protocol
 HTTP Proxy


Assignment 3: Concurrency
Assignment 4: Distribute file system
2
Last lecture



TCP
Socket()
Question: difference between send() and write()?
 MSG_DONTWAIT,
MSG_NOSIGNAL
3
Concurrent Programming is Hard!



The human mind tends to be sequential
The notion of time is often misleading
Thinking about all possible sequences of events in a
computer system is at least error prone and frequen
tly impossible
4
Concurrent Programming is Hard!

Classical problem classes of concurrent programs:
 Races:
outcome depends on arbitrary scheduling decision
s elsewhere in the system
 Example:
 Deadlock:
who gets the last seat on the airplane?
improper resource allocation prevents forward
progress
 Example:
traffic gridlock
 Livelock
/ Starvation / Fairness: external events and/or sy
stem scheduling decisions can prevent sub-task progress
 Example:
people always jump in front of you in line
5
Iterative Echo Server
Client
Server
socket
socket
bind
open_listenfd
open_clientfd
listen
connect
Client /
Server
Session
Connection
request
accept
rio_writen
rio_readlineb
rio_readlineb
rio_writen
close
EOF
Await connection
request from
next client
rio_readlineb
close
6
Iterative Servers

Iterative servers process one request at a time
client 1
server
client 2
connect
connect
accept
write
call read
ret read
close
read
write
call read
write
close
accept
Wait for Client 1
read
write
ret read
7
Where Does Second Client Block?

Second client attempts to conne
ct to iterative server
Client

Call to connect returns


socket

open_clientfd

connect
rio_writen
Call to rio_writen returns

Connection
request

Even though connection not yet
accepted
Server side TCP manager que
ues request
Feature known as “TCP listen
backlog”
Server side TCP manager buff
ers input data
Call to rio_readlineb blocks

Server hasn’t written anything
for it to read yet.
rio_readlineb
8
Fundamental Flaw of Iterative Servers
client 1
server
client 2
connect
accept
write
call read
ret read
User goes
out to lunch
Client 1 blocks
waiting for user
to type in data

read
write
Server blocks
waiting for
data from
Client 1
connect
write
call read
Client 2 blocks
waiting to read from server
Solution: use concurrent servers instead

Concurrent servers use multiple concurrent flows to serve multip
le clients at the same time
9
Creating Concurrent Flows
 Allow

server to handle multiple clients simultaneously
1. Processes
 Kernel
automatically interleaves multiple logical flows
 Each flow has its own private address space

2. Threads
 Kernel
automatically interleaves multiple logical flows
 Each flow shares the same address space

3. I/O multiplexing with select()
 Programmer
manually interleaves multiple logical flows
 All flows share the same address space
 Relies on lower-level system abstractions
10
Concurrent Servers: Multiple Processes

Spawn separate process for each client
client 1
server
call accept
call connect
ret connect
call fgets
User goes
out to lunch
Client 1 blocks
waiting for user
to type in data
client 2
call connect
ret accept
fork
child 1
call accept
call read
ret accept
fork
...
child 2
call
read
ret connect
call fgets
write
call read
write
close
end read
close
11
Review: Iterative Echo Server
int main(int argc, char **argv)
{
int listenfd, connfd;
int port = atoi(argv[1]);
struct sockaddr_in clientaddr;
int clientlen = sizeof(clientaddr);
listenfd = Open_listenfd(port);
while (1) {
connfd = Accept(listenfd, (SA *)&clientaddr, &clientlen);
echo(connfd);
Close(connfd);
}
exit(0);
}
 Accept
a connection request
 Handle echo requests until client terminates
12
Process-Based Concurrent Server
int main(int argc, char **argv)
{
int listenfd, connfd;
int port = atoi(argv[1]);
struct sockaddr_in clientaddr;
int clientlen=sizeof(clientaddr);
Fork separate process for
each client
Does not allow any
communication between
different client handlers
Signal(SIGCHLD, sigchld_handler);
listenfd = Open_listenfd(port);
while (1) {
connfd = Accept(listenfd, (SA *) &clientaddr, &clientlen);
if (Fork() == 0) {
Close(listenfd); /* Child closes its listening socket */
echo(connfd);
/* Child services client */
Close(connfd);
/* Child closes connection with client */
exit(0);
/* Child exits */
}
Close(connfd); /* Parent closes connected socket (important!) */
}
}
13
Process-Based Concurrent Server
(cont)
void sigchld_handler(int sig)
{
while (waitpid(-1, 0, WNOHANG) > 0)
;
return;
}
 Reap
all zombie children
14
Process Execution Model
Connection Requests
Server
Process
(Listening)
Client 1 data
Server
Process
(Client 1)
Server
Process Client 2 data
(Client 2)
Each client handled by independent process
 No shared state between them
 Both parent & child have copies of listenfd and connfd

Parent must close connfd
 Child must close listenfd

15
Concurrent Server: accept Illustrated
listenfd(3)
Client
Server
clientfd
Connection
request
Client
1. Server blocks in accept,
waiting for connection request
on listening descriptor
listenfd
listenfd(3)
Server
2. Client makes connection request by
calling and blocking in connect
clientfd
listenfd(3)
Server
Client
clientfd
Server
Child
connfd(4)
3. Server returns connfd from
accept. Forks child to handle client.
Client returns from connect.
Connection is now established between
clientfd and connfd
16
Implementation Must-dos With
Process-Based Designs

Listening server process must reap zombie children
 to

avoid fatal memory leak
Listening server process must close its copy of connfd
 Kernel
keeps reference for each socket/open file
 After fork, refcnt(connfd) = 2
 Connection will not be closed until refcnt(connfd) ==
0
17
View from Server’s TCP Manager
Client 1
Client 2
Server
srv> ./echoserverp 15213
cl1> ./echoclient greatwhite.ics.cs.cmu.edu 15213
srv> connected to (128.2.192.34), port 50437
cl2> ./echoclient greatwhite.ics.cs.cmu.edu 15213
srv> connected to (128.2.205.225), port 41656
Connection
Host
Port
Host
Port
Listening
---
---
128.2.220.10
15213
cl1
128.2.192.34
50437
128.2.220.10
15213
cl2
128.2.205.225
41656
128.2.220.10
15213
18
Pros and Cons of Process-Based Designs


+ Handle multiple connections concurrently
+ Clean sharing model
 file
tables (yes)
 global variables (no)



+ Simple and straightforward
– Additional overhead for process control
– Nontrivial to share data between processes
 Requires

IPC (interprocess communication) mechanisms
FIFO’s (named pipes), System V shared memory and semaphores
19
#2) Event-Based Concurrent Servers Using I/O Multiplexi
ng

Server maintains set of active connections
 Array

of connfd’s
Repeat:
 Determine
which connections have pending inputs
 If listenfd has input, then accept connection
 Add
new connfd to array
 Service

all connfd’s with pending inputs
Library support
 Use
library functions to construct scheduler within single
process
20
Adding Concurency: Step One

Start with allowing address re-use
int sock, opts;
sock = socket(…);
// getting the current options
setsockopt(sock, SOL_SOCKET, SO_REUSEADDR, &opts, sizeof(opts));

Then we set the socket to non-blocking
// getting current options
if (0 > (opts = fcntl(sock, F_GETFL)))
printf(“Error…\n”);
// modifying and applying
opts = (opts | O_NONBLOCK);
if (fcntl(sock, F_SETFL, opts))
printf(“Error…\n”);
bind(…);
21
Adding Concurrency: Step Two

Monitor sockets with select()


So what’s an fd_set?






int select(int maxfd, fd_set *readfds, fd_set
*writefds, fd_set *exceptfds, const
struct timespec *timeout);
Bit vector with FD_SETSIZE bits
maxfd – Max file descriptor + 1
readfs – Bit vector of read descriptors to
writefds – Bit vector of write descriptors
monitor
to monitor
exceptfds – Read the manpage, set to NULL
timeout – How long to wait with no activity before
ning, NULL for eternity
retur
22
How does the code change?
if (listen(sockfd, 5) < 0) { // listen for incoming connections
printf(“Error listening\n”);
init_pool(sockfd, &pool);
while (1) {
pool.ready_set = &pool.read_set;
pool.nready = select(pool.maxfd+1, &pool.ready_set,
&pool.write_set, NULL, NULL);
if (FD_ISSET(sockfd, &pool.ready_set)) {
if (0 > (isock = accept(sockfd,…)))
printf(“Error accepting\n”);
add_client(isock, &caddr, &pool);
}
check_clients(&pool);
}
// close it up down here
23
What was pool?


User defined structure used to store information.
A struct something like this:
typedef struct s_pool {
int maxfd;
// largest descriptor in sets
fd_set read_set;
// all active read descriptors
fd_set write_set;
// all active write descriptors
fd_set ready_set;
// descriptors ready for reading
int nready;
// return of select()
int maxi;
/* highwater index into client array */
int clientfd[FD_SETSIZE];
/* set of active descriptors */
rio_t clientrio[FD_SETSIZE]; /* set of active read buffers */
} pool;
24
void init_pool(int sockfd, pool * p);
{
int i;
p->maxfd = -1;
for (i=0;i<FD_SETSIZE;i++)
p->clientfd[i]=-1;
p->maxfd = sockfd;
FD_ZERO(&p->read_set);
FD_SET(listenfd, &p->read_set);
}
// close it up down here
25
void add_client(int connfd, pool *p)
{
int i;
p->nready--;
for (i = 0; i < FD_SETSIZE; i++) /* Find an available slot */
if (p->clientfd[i] < 0) {
/* Add connected descriptor to the pool */
p->clientfd[i] = connfd;
Rio_readinitb(&p->clientrio[i], connfd);
/* Add the descriptor to descriptor set */
FD_SET(connfd, &p->read_set);
/* Update max descriptor and pool highwater mark */
if (connfd > p->maxfd)
p->maxfd = connfd;
if (i > p->maxi)
p->maxi = i;
break;
}
if (i == FD_SETSIZE) /* Couldn't find an empty slot */
app_error("add_client error: Too many clients");
}//
close it up down here
26
void check_clients(pool *p)
{
int i, connfd, n;
char buf[MAXLINE];
rio_t rio;
for (i = 0; (i <= p->maxi) && (p->nready > 0); i++) {
connfd = p->clientfd[i];
rio = p->clientrio[i];
/* If the descriptor is ready, echo a text line from it */
if ((connfd > 0) && (FD_ISSET(connfd, &p->ready_set))) {
p->nready--;
if ((n = Rio_readlineb(&rio, buf, MAXLINE)) != 0) {
byte_cnt += n; //line:conc:echoservers:beginecho
printf("Server received %d (%d total) bytes on fd %d\n",
n, byte_cnt, connfd);
Rio_writen(connfd, buf, n); //line:conc:echoservers:endecho
}
/* EOF detected, remove descriptor from pool */
else {
Close(connfd); //line:conc:echoservers:closeconnfd
FD_CLR(connfd, &p->read_set); //line:conc:echoservers:beginremove
p->clientfd[i] = -1;
//line:conc:echoservers:endremove
}
}
}
}
27
So what about bit vectors?

void FD_ZERO(fd_set *fdset);
 Clears

void FD_SET(int fd, fd_set *fdset);
 Sets

the bit for fd
void FD_CLR(int fd, fd_set *fdset);
 Clears

all the bits
the bit for fd
int FD_ISSET(int fd, fd_set *fdset);
 Checks
whether fd’s bit is set
28
What about checking clients?

The code only tests for new incoming connections


Store all your client file descriptors


But we have many more to test!
In pool is a good idea!
Several scenarios
Clients are sending us data
 We may have pending data to write in a buffer


Keep the while(1) thin
Delegate specifics to functions that access the appropriate d
ata
 Keep it orthogonal!

29
Back to that lifecycle…
Client(s)
Server
socket()
socket()
bind()
listen()
connect()
Connection Request
select()
FD_ISSET(sfd)
accept()
write(
)
read()
Client / Server Session(s)
close()
check_clients() main loop
read()
write()
EOF
read()
close()
30
I/O Multiplexed Event Processing
Read
Active Descriptors
listenfd = 3
Pending Inputs
listenfd = 3
clientfd
clientfd
10
0
10
1
7
2
4
4
3
-1
-1
4
-1
5
12
6
5
7
-1
-1
8
-1
-1
9
-1
Active
Inactive
Active
Never Used
7
-1
12
5
-1
31
Time out?
int select(int nfds, fd_set *readfds, fd_set *writefds,
fd_set *exceptfds, struct timeval *timeout);
32
Pros and Cons of I/O Multiplexing



+ One logical control flow.
+ Can single-step with a debugger.
+ No process or thread control overhead.
 Design
of choice for high-performance Web servers and
search engines.


– Significantly more complex to code than process- o
r thread-based designs.
– Hard to provide fine-grained concurrency
 E.g.,

our example will hang up with partial lines.
– Cannot take advantage of multi-core
 Single
thread of control
33