Network Delay
Introduction
Ideally a network would allow infinite amounts of data to be transmitted over arbitrary
distances in zero time. Obviously that does not happen.
Instead we usually must accept a tradeoff between various network characteristics. Those
characteristics include throughput, error rate, delay, and jitter. And in the real world we may
also have to include the initial cost of a network, the cost to maintain the network, the schedule
for establishing the network, etc. in the tradeoff study as we define a network.
Network performance is a complicated topic because of the factors above but also because the
network is shared. Each application might have a different notion of how to trade these factors
against each other. Building a shared network that works well for a variety of applications is
therefore a challenge.
This document describes only one aspect of network performance: delay or latency. We
examine 3 different components of delay: transmit delay, propagation delay, and queuing
delay. Authors use different names for these but the reader should be able to recognize these
delays by other names. We also introduce the space-time diagram which is an important tool
for visualizing the network. Finally, we discuss the delay-bandwidth product.
Transmit delay
Communication theory tells us that we cannot send data at an arbitrarily high data rate.
Specifically, Shannon’s law indicates that for a given Signal to Noise Ratio (SNR) and a certain
bandwidth on the channel there is a maximum data rate that can be achieved.
Various complex encoding schemes are used to approach the theoretical maximum. For the
purposes of this discussion imagine that we have a very simple (and inefficient) encoding
scheme and channel.
Imagine that we use a flashlight and a color filter to send red light for a bit that is 1, blue light
for a bit that is 0, and that we send green light to indicate that we are changing to another bit.
Suppose that the receiver at the other end of the channel takes 0.5 seconds to reliably
determine what color is being sent. Then we might send green for 0.5 seconds and red for 0.5
seconds to indicate a 1 bit of data, and green for 0.5 followed by blue for 0.5 to indicate a 0 bit
of data. The transitions from red or blue to green help the receiver determine the clock rate
and the start of each bit in the channel.
Each bit takes 1 second to transmit in this channel. This inverse of the duration for a single bit
is the data rate (or throughput) of the channel. The data rate for our simple channel is
therefore 1 bit/s.
If we have N bits then it takes N bit times for all N bits to be transmitted. In other words the
transmit delay is defined as follows:
transmit_delay s 
number_of_bits
data_ratebit/s
(1.1)
Large files therefore take longer to transmit than short files for a constant network throughput.
As a more realistic example assume we want to know the transmit delay on a particular
network for a single, full-size Ethernet frame. To make the math obvious let’s assume that it is
1500 bytes long. And we have a network with an effective throughput of 80 Mbit/s. Then the
transmit delay for the frame is:
transmit _ delay s 
1500byte  8bit/byte
80  106bit/s
(1.2)
transmit _ delay s  150  10 6 s
This delay associated with transmitting bits on the medium goes by several similar names. We
call it transmit delay here but it is also called the transmit time or the transmission delay or the
transmission time by various authors.
Although it is called the transmit delay it is also the time it takes to pass any point in the
channel. If we stand anywhere between the sender and the receiver in our simplistic channel
we will see the same pattern of light colors for the same durations. The data rate at our
location is the same as at the sender or the receiver so the time for a bit to pass does not
change.
Propagation delay
If we observe the light in our simple example somewhere other than the transmitter then we
will see the same pattern of light but at a later time. The later time is associated with the
propagation of the signal carrying our bits. The further away from the transmitter or the slower
the propagation velocity the longer it will take for the signal to be observed.
Propagation delay is based on the velocity x time = distance formula from your physics class.
For copper media we often use a velocity of ¾ of the speed of light and for optical fiber we
often use 2/3 of the speed of light C where C≈3x108 m/s.
We solve the formula above for time to yield:
propagation_delay s 
distance m
propagation_velocity m/s
(1.3)
For propagation velocities roughly the speed of light it takes a long distance to accumulate
enough propagation delay to be perceived by a human, but computers are much faster. Grace
Hopper, one of the pioneers of computing, used to carry around in her pocket a “nanosecond”;
a wire whose length produced a nanosecond of delay.
Let’s update her wire to fiber. Assuming 2/3 C as the velocity on fiber:
10 9 s 
distance m
 2 / 3  3  108m/s
distance m  10 9  2  10 8
(1.4)
distance m  2  10 1
distance cm  20cm
So a delay of 1 ns occurs after roughly 20 cm of fiber. 1 meter produces about 5 ns of delay, 1
km about 5 μs of delay, and so on.
It is clear from Equation (1.3) that propagation delay is unrelated to network throughput. If we
were to improve the throughput of our simple channel to 10 bits/s that would not change how
long it takes light to propagate from the sender to the receiver.
The Physical Size of a Bit
If we want to determine the physical size of a bit on a particular channel then we can use the
transmit delay and propagation delay formulae together as follows. From Equation (1.1) we
know the transmit delay for a group of bits given a data rate. If we substitute that time in
Equation (1.3) for the propagation delay and solve for distance then we know the physical
length of a number of bits:
distance m
number_of_bits

data_rate bit/s
propagation_velocity m/s
distance m  propagation_velocity m/s 
number_of_bits
data_rate bit/s
(1.5)
Let’s look at the length of the Ethernet frame with the throughput we used in Equation (1.2)
running on fiber in Equation (1.4).
1500byte  8bit/byte
80  106bit/s

distance m
 2 / 3  3  108m/s
distance m   2 / 3   3  108m/s 
1500byte  8bit/byte
80  106bit/s
(1.6)
distance m  2  108m/s  150  10 6 s
distance m  30  103m
The Ethernet frame is 30 Km long on fiber at this data rate. This is significantly longer than the
distances in a LAN – meaning that the frame is still in the process of being sent at the same time
it is starting to be received down the hallway in another room. This is another way of saying
that in a LAN the transmit delay is more significant than the propagation delay.
In contrast, the distance between New York and Los Angeles is 1000s of Km. In that case the
start of a single frame of Ethernet might just be entering New Jersey as the end of the frame is
being transmitted in New York. A significant propagation delay must still occur before the end
of the frame finally reaches Los Angeles. And instead of physically occupying the entire channel
as it did in the LAN it occupies only a tiny fraction of this WAN channel. We will return to this
when we discuss the delay-bandwidth product below.
Queuing delay
In general we work with switched networks that use switching devices to move data across
multiple physical links. The delay added by switching devices depends to some extent on the
type of switched network, the characteristics of the protocol, and the internal design of the
switches.
In every switch we must as a minimum move bits between an input port and an output port. In
a circuit-switched network once the circuit has been set up we can visualize the switch as a
dedicated physical connection of very short length. This adds very little delay. So the
switching delay is often ignored in a physical, circuit-switched network.
But most networks are packet switched (even the virtual-circuit networks often used by the
phone company). In packet-switched networks the headers of frames passing through the
switch contain information necessary to determine the output port. This header information
must be matched with stored instructions that indicate the output port for the flow. The stored
instructions are typically a special data structure in memory that supports rapid searching (e.g.,
a routing table, or a bridge address table). The lookup will take time that grows as the size of
the data structure grows.
There are basically two types of packet-switch designs: cut-through switches and store-andforward switches. In a cut-through switch when enough of the header has been read to
determine the output port then the switch begins sending the bits of the frame on the output
port. The cut-through switch adds (as a minimum) a delay corresponding to the transmit delay
of the number of bits needed to make the decision.
In contrast, a store-and-forward switch stores the entire frame before sending any of the bits
on the output port. It therefore adds the transmit delay of an entire frame as a minimum. This
is longer than the cut-through switch – so why not use cut-through?
If the protocol supports error detection then this often “covers” the entire frame. In other
words, all of the bits of the frame must pass through the error detection computation to
declare that the frame is valid. On a store-and-forward switch an invalid frame is discarded
before any bits are sent on the output port. But on a cut-through switch most of the frame bits
will have been sent before an error is detected. For this reason many switching devices are
store-and-forward and therefore add a minimum delay of the transmit delay for a single frame.
If the frames are variable length it is often a reasonable approximation to pick an average frame
size and compute that delay for this component. Some authors call this the “switching-delay”
component of the queuing delay.
Finally, when our frame is switched to the output port it is actually queued for output. There
may be one or more frames ahead of it waiting to be clocked onto the network. This is the
basis for the name queuing delay. In the worse case, almost the entire output buffer is ahead
of our frame. This would add a delay corresponding to the transmit delay of the entire output
buffer – which is usually multiple frames. So the queuing delay has the greatest range of values
and it is this component that introduces most of the “jitter” on the network – the variability in
the delay.
In the absence of actual measurements we may simply model this as a constant for each switch.
The number is typically 1 or 2ms on a switch that is not congested.
Overall delay
We need to put these delays together to answer a question such as – how long does it take to
send a entire file consisting of multiple frames. This means the time between when we send
the first bit of frame one at the sender and when we receive the final bit of the final frame at
the receiver.
We can make this relatively easy with several assumptions. The most unrealistic of these is that
each frame experiences the same queuing delay at a given switch. If we have a multi-segment
network:
Overall_delay 
segments

i
1

switches

j
1

frames

k
1
propagation_delay i
queuing_delay j
(1.7)
transmit_delay k
The first summation is the combined propagation delay of all of the segments. Notice that the number
of frames does not factor into this part of the delay. We don’t wait for the first frame to propagate
before sending the second frame. So the propagation delay is the same regardless of how much data
travels along the path.
The second summation supposes that we have constant queuing delay for all frames so that a
single switch adds a constant value. Notice that the number of frames also does not factor into
this part of the delay. If it did we would be saying that the switch waits to receive an entire file
(the final frame) before it begins transmitting the first frame on the next segment. Remember
a switch is receiving one frame while it is transmitting another one. So under our assumption
the delay introduced to the entire file is the queuing delay associated with just one frame of the
file.
The third summation is the transmit delay for all of the frames. Here the number of frames
matters because each frame contributes to this delay component separately.
If we assume that all of the segments have the same propagation velocity then we can compute
the propagation delay for the entire length of the channel from sender to receiver and replace
the first summation with that value.
If we assume (even more unrealistically) that each switch adds the same queuing delay then we
can replace the second summation with a product of the number of switches and the per
switch queuing delay.
And the third summation can be computed for the entire file if the data rate is not changing
while the file is being transmitted.
Including those assumptions we have:
Overall_delay s 
total_distance m
propagation_velocity m s
num_switches  queuing_delay s switch

(1.8)
total_bits bits
data_rate bits s
Visualizing these delays
This can all be difficult to visualize because computer times and speeds are not part of human
experience. So imagine a convoy of trucks is carrying explosives from a factory to a mining site.
Each truck carries the same number of sticks of dynamite. The sticks of dynamite are the bits in
our analogy.
For safety reasons we place a small vehicle ahead of each trunk and another one trailing it.
These vehicles maintain a constant distance from the truck. A “frame” (a collection of bits) in
our analogy is this combination of header, payload, and trailer. Because we assume each truck
is carrying the same amount of explosives – the frames are equal size.
Unrealistically, the header from one frame can tailgate the trailer from the previous frame.
This just means that our sender (the factory) can produce dynamite as fast as the trucks can
haul it away. In network terms – the application is able to create data as fast as it can be
transmitted into the network.
Assume that all vehicles drive at the same speed (the propagation velocity) along the same
route to the same mine (the same distance). They all pass through one inspection station that
causes each truck/frame to be delayed by the same amount of time (we will assume that the
queuing delay includes the time to decelerate, wait in line, be inspected, and accelerate back to
the propagation velocity).
What we want to know is how much time between when the header of the first frame pulls out
of the factory and when the trailer of the final frame arrives at the mine. That is the total delay
for the shipment (this is an analogue to a file transfer – for streaming media we might not be
interested in waiting for the entire shipment to arrive).
First consider the propagation delay. Since we are assuming the same propagation velocity on
either side of the inspection station we don’t need to compute per-segment propagation times
and add them up.
Assume the trucks propagation velocity of 60 km/hr (or 1 km/minute) and the distance to the
mine is 60 km. If the inspection station is 15 km from the factory and 45 km from the mine
then the propagation time on the first segment is 15 minutes and on the second segment it is
45 minutes. If instead the station was halfway between the factory and the mine then it would
be 30 minutes + 30 minutes. It will always add up to 60 minutes because the total distance isn’t
changing and the propagation velocity is the same on both segments. Of course in a real
network we might have different propagation velocities and have to account for each segment
separately.
Note that we have not yet said anything about the “data rate” of this channel or about the
number of trucks. Under these assumptions whatever the interval is between the trucks it will
stay the same as they all propagate at the same velocity.
We allow time between the trucks to increase the safety of the overall process. So we are only
going to allow the trucks to leave the factory at a certain interval. This becomes the transmit
delay for a single frame in our analogy. We could divide this by the number of bits in each
frame to obtain the time for a single bit and then invert it to obtain a data rate in units of bit/s.
When the frames are equal size as in this analogy we can also work with the data rate in units
of frame/s and transmit delays in units of s/frame.
Of course this is backwards from a real network. In a real network we know the data rate and
we compute the transmit delay. But either way they are inversely related as described in
Equation (1.1).
Suppose we want the trucks to be 1 minute = 60 seconds = 1/60 hr apart. This is the transmit
delay per frame. If we have 10 trucks total (the number of frames) then the total transmit
delay for the entire convoy is 10 trucks x 1 minute/truck = 10 minutes. The data rate of our
network is 1 truck/minute.
If we stand anywhere along the route it takes the same amount of time for the entire convoy to
pass (the transmit delay for the entire convoy/file). So at the mine (the receiver) it will also
take the transmit delay for all of the convoy to arrive after the first frame’s header arrives.
How far apart will the trucks be as they travel down the road? It is their separation in time
multiplied by the propagation velocity. This is a frame’s physical size for this combination of
data rate and propagation velocity. In this example they will be 1/60 hr x 60 km/hr = 1 km
apart.
frame_extent m frame 
propagation_velocity m/s
data_rate frame/s
(1.9)
Now consider our unrealistic queuing analogy. We are assuming that each trunk experiences
the same delay, i.e., that the queue length is constant. This makes the problem much easier to
solve because we can simply add this delay to the overall time to send the convoy. Assume that
the delay each truck experiences is 2 minutes at the inspection station.
We don’t multiply this delay by the number of frames when we compute the total delay. Let’s
work that out. Imagine that the inspection is X kilometers from the factory and that the first
truck leaves the factory at time T.
First assume that the inspection station is closed so that trucks do not experience any queuing
delay. Recall that the propagation velocity is 1 km/minute. It therefore takes 1 minute to
travel a km. So the first truck passes the inspection station at time T+X. Ten minutes later at
T+X+10 the truck would be X+10 km from the factory. The second truck is 1 km behind the first
truck. It leaves the factory at T+1; at T+X it is X-1 km from the factory (i.e., one km before the
inspection); at T+X+1 it passes the inspection station; and at T+X+10 it is X+9 km from the
factory and still 1 km behind the first truck.
Now imagine instead that the inspection station is open. The first truck arrives at time T+X
again. At T+X+2 it is still X km from the factory because it is just leaving the inspection station.
At T+X+10 it will X+8 km from the factory because it has been back on the road for 8 minutes.
The second truck arrives at the inspection station at T+X+1. At T+X+3 it leaves the inspection
station. At T+X+10 it will be at X+7 from the factory because it will have been back on the road
for 7 minutes. It is still 1 km behind the lead truck.
In other words, this assumption of constant queuing delay means that the entire convoy will be
delayed by this value – but after the inspection will have exactly the same spacing, etc.
So under our assumptions we use Equation (1.8) as follows:
Overall_delay minutes 
total_distance km
propagation_velocity km minute
num_switches  queuing_delay minutes switch


total_bits bits
data_rate bits minute
60km
1km minute
1station  2minutes station

total_frames frames  bits / frame
data_rate frames minute  bits / frame
 60minutes  2minutes  10minutes
 72minutes
(1.10)
To prove that this is the same as if we break it up into small pieces – let’s ride along.
We get into the header safety vehicle for the first truck as it leaves the factory at time T. We
are at the front end of the entire convoy. We travel X km at 1 km/minute until we arrive at the
inspection station. That will be time (T+ X)/1 = T+X minutes. We are delayed 2 minutes so we
leave the inspection station at T+X+2 minutes. After traveling 60-X more km at 1 km/minute
we arrive at the mine. That takes another (60-X)/1 = 60-X minutes. So the time when we enter
the mine is T+X+2+60-X. But now we must wait for the rest of the convoy to arrive before we
can stop the timer. One minute later the trailing safety vehicle for the lead frame arrives. That
is the transmit delay for the first frame. After 10 such transmit delays have gone by (the
number of frames) the trailing safety vehicle for the final frame arrives and the convoy has
completely arrived. It is T+X+2+60-X+10 or T+72.
Suppose instead that we ride along in the trailing safety vehicle for the final truck. At time T the
first truck’s lead safety vehicle leaves the factory. At T+1 the first frame will have been sent
and the second frame will be starting to go. Every minute (the frame transmit delay) another
complete frame leaves the factory. At T+9 the header vehicle of our final frame leaves. At
T+10 we finally leave the factory in the trailer of the final frame. We travel X km at 1
km/minute – arriving at the inspection station at T+10+X. We are delayed 2 minutes and leave
the inspection station at T+10+X+2. We travel another 60-X km at 1 km/minute. We arrive at
the mine at T+10+X+2+60-X. We are the end of the convoy so that completes the arrival. Again
the arrival time is T+72.
Look carefully at the three delay factors. Notice that in this example the propagation delay is
the largest of the three. In a WAN we often see the propagation delay dominate. Because
virtually every network medium has a propagation velocity near the speed of light about all we
can do is try to shorten the path. This might entail using a different network provider that has a
more efficient route or switching from geosynchronous satellite communication to land-lines.
In general we cannot do much about the propagation delay in the network. (Keeping with our
analogy: this is why on a large construction project like a dam one builds a concrete plant very
near the project.)
In contrast, we can spend money to reduce the transmit delay. In our analogy this might mean
spending more money on the design of the trucks to make them safer. We might then be
allowed to have the trucks only 12 seconds = 1/5 minute apart on the highway. The transmit
delay for the 10 trucks would then be 2 minutes total, but the overall time would decrease only
slightly to 60+2+2= 64 minutes.
In the network we spend money to reduce the transmit delay by using a link and switching gear
with a higher data rate. But if the propagation delay is already the largest component then
spending money on a higher data rate has diminishing returns. If we had an infinite data rate in
our analogy we would still have a delay of 62 minutes.
Suppose we rebuild the factory near the mine, perhaps 5 km away. Assume we still have 1
inspection station to pass through with a delay of 2 minutes and all of the other original
assumptions are still in place (i.e., 1 minute transmit time per frame). Instead of 60 minutes,
the propagation delay is 5 minutes and the total time is 5+2+10 = 17 minutes. Note that the
front of the convoy will be arriving at the mine before the back of the convoy has left the
factory.
This case is somewhat like the situation on a typical LAN. The transmit delay is the dominant
factor. There is more payoff from increasing the data rate of the LAN switches and links. For
this nearby factory if we reduce the transmit delay to 2 minutes by improving the trucks now
we have a total time of 5+2+2 = 9 minutes.
We typically also reduce the queuing delay to some extent when we increase the data rate of
our links. Recall that for a store-and-forward packet-switched network the minimum queuing
delay is the transmit delay for a frame. Increasing the data rate reduces this delay.
And generally the switches must move data from an ingress queue to an egress queue after
making some forwarding decision by looking at the frame. The time spent doing these
operations is often reduced on more expensive network gear that has faster chips.
The reader should consider what would happen if we needed to send more trucks or if the
number of inspection stations increased.
The space-time diagram
To visualize traffic on network we can use a space-time diagram. Figure 1 is an example.
Figure 1: A Space-time diagram
The vertical direction in the diagram represents time, with time increasing from the top to the
bottom as indicated by the arrow at the lower left side.
The horizontal direction in the diagram represents space or distance. Of course we usually
cannot capture distance to scale in such a diagram but we can show the relationship between
time and distance. A vertical line at any point along the horizontal axis represents what we
would see at a specific location as time goes by.
The diagram shows two frames sent back-to-back from a device at location A, through a
switching device at location B, to a destination at location C. We see that the distance between
A and B is shorter than the distance between B and C.
For this diagram the propagation velocity of the medium is the same for both segment AB and
segment BC. We can see this by comparing the slope of the frames between AB and BC. The
slope is proportional to the inverse of the propagation velocity (i.e., time/distance). Because
the slope is the same on both segments we see that the propagation velocity is the same on
both segments.
Because the velocity is the same and the distance on the segments is different we can see that
segment BC causes a longer propagation delay than segment AB.
This diagram also shows both segments with the same data rate. Recall that the time it takes
for a frame to pass any point in the network is the transmit delay for that frame. If we draw a
vertical line somewhere between A and B, and another one between B and C we can measure
the time it takes for F1 to pass that point, i.e., the transmit delay for F1.
Notice that the vertical height of F1 is the same on either side of B, so the transmit delay is the
same on both segments. Assuming that the length of F1 has not been altered by the switching
device at B then the data rate on AB must be same as on BC. Review Equation (1.1) to verify.
Furthermore you can see in the diagram that frame F2 has fewer bits than F1. Its vertical
height is smaller which means its transmit delay is less. Assuming the data rate is not changing
between frames this means F2 has fewer bits. For modern wireless channels the data rate may
change between frames – but on a LAN running Ethernet the data rate is a constant from frame
to frame.
Finally we can see the queuing delay at B in the diagram. If B did not introduce delay then it
would be just another point between A and C. The dotted line represents the continuation for
the start of frame F1 if B did not produce any delay. The vertical distance between when F1
would have left B and when it actually left B is the queuing delay for F1.
This diagram has the same constant queuing delay we used in our analogy. F2 is shown delayed
by the same amount as F1. It could not be delayed less (since it is also using the BC data link).
If it were delayed more than F1 a gap would be visible between F1 and F2 on segment BC.
These assumptions are not necessary to use a space-time diagram – you might want to test
your understanding by exploring how the diagram would look if these assumptions were not
true.
Looking at the space-time diagram one can now see how the delays add up. To find the total
delay to send both frames we look at location C, the destination. To get the first bit of frame 1
to C we have the propagation delay for AB, then the queuing delay at B, then the propagation
delay along BC. But then we must wait for the transmit delay of both frame 1 and frame 2 for
all of the bits to arrive at C.
Notice that the propagation delay and the queuing delay are not affected by the number of
frames. And we end up using only the total transmit delay if the data rate is the same for all
frames. So we can use the total number of bits and not worry about how they are split into
frames.
Notice that this diagram depicts a typical LAN situation. Frame F1 is still being transmitted at A
while it is arriving at B. This is a visualization of a propagation delay that is less than the
transmit delay for a frame. The frame completely fills the segment.
The reader should consider what the time-space diagram would look like for a WAN.
Delay-Bandwidth Product
The delay-bandwidth product is a measure of how many bits are needed to fill a network
segment. A more accurate term would be delay-throughput or delay-date-rate product but the
term most widely used is delay-bandwidth product.
Why is this number important? The throughput of a segment is essentially an instantaneous,
maximum value. If at any moment a segment is not completely filled with bits then the real
throughput will be lower than this maximum throughput.
First let’s see where the name comes from. Recall that in Equation (1.5) we derived the
physical size of a bit on a data link given the propagation velocity and the data rate on that link.
If we set the distance to the physical extent of a link and solve for the number of bits we have
the following:
bits_in_segment 
length_of_segment m
 data_ratebit/s
propagation_velocity m/s
(1.11)
Now we substitute using Equation (1.3) to obtain:
bits_in_segment  propagation_delay s  data_ratebit/s
(1.12)
The number is most useful in a WAN context where the propagation delay is the predominant
delay factor. If we want to include queuing delay we simply add it to the propagation delay
and use that total delay instead. But generally an approximate value is adequate.
In a space-time diagram space is shown by drawing a horizontal line. This will intersect the
diagonal lines that represent the start and end of a bit or a frame. All of the locations between
these two locations are occupied by that group of bits at the time corresponding to the
horizontal line.
In Figure 1 this is illustrated using the slanted, dotted line in Segment BC to indicate the end of
the first bit (i.e., we have only 8 bits in the first frame). The bit’s transmit delay is less than the
propagation delay on Segment BC so the bit only fills a fraction of the segment. A horizontal
line in the diagram illustrates this point.
From Figure 1 we see that over the time interval shown only about half of the time was the
sender filling the pipe. Nothing follows frame F2 at the sender. So the average throughput
obtained by the sending application over that interval in the Figure would only be about half of
the theoretical throughput of the path.
“Keeping the pipe full” translates into having enough buffer space for reliable protocols that
use acknowledgements and retransmission. These protocols buffer data received from a higher
layer and must keep a copy of that data until they receive an acknowledgement (ACK) from the
receiver that the data has been received. If they run out of buffer space they must stop sending
data. A buffer cannot be re-used until the ACK has been received.
If we ignore processing delays at the receiver we will have to hold a copy of a frame in the
buffer while: 1) the frame is transmitted; 2) the frame experiences the queuing delays and
propagation delay on the way to the receiver; 3) the ACK is transmitted the other way; and 4)
the ACK experiences the queuing delays and propagation delay on the way back to the sender.
Because this is usually an issue for what are called “long, fat pipes” – networks that have both
high data-rates and long propagation delays – the usual approach is to simply ignore the
transmit delay of the ACK, the transmit delay of the frame, and even the queuing delays.
round_trip_delay_bw bits  2  one_way_propagation_delay s  data_ratebits s (1.13)
Often a “fudge factor” of 2 or 3 is used to cover the processing delays of the receiver as well as
the transmit and queuing delays that were ignored.
If we have only enough sender buffer for one frame and the round-trip delay-bandwidth
product is significantly larger, then the sender will be idle most of the time waiting for either
the propagation of the frame to the receiver or the propagation of the ACK back to the sender.
This protocol approach is called “stop and wait” and we will see it again when we study reliable
delivery. For now recognize that “stop and wait” only performs well when the transmit delay is
the dominant part of the total delay.
To avoid making the reliable protocol a bottleneck the sender side of the reliable protocol must
be willing to buffer at least the “round-trip, delay-bandwidth product” of bits. Older Windows
operating systems were configured to minimize memory use by allocating relatively small
buffer space for TCP (the reliable byte-stream protocol of the IP suite). This was often less than
the round-trip delay-bandwidth product one might see on a capable WAN with the result that
file transfer performance was substantially worse than the theoretical throughput of the WAN.
By increasing the buffer space that TCP uses for each connection to a few multiples of the
expected delay-bandwidth product the performance was significantly improved.
To illustrate the delay-bandwidth product we stretch our analogy even further. Imagine that
instead of a mine we are widening the Panama Canal and we need an extremely large amount
of dynamite for blasting. Let’s assume we need 1000 trucks of dynamite delivered. Again we
unrealistically assume that the dynamite cannot be used until it all arrives. The other
parameters are unchanged: 60 km to the blasting site; propagation velocity of 1 km/minute; a
single inspection station with a constant 2 minutes of delay, and a truck separation of 1 minute.
To make this somewhat realistic imagine that we have more trucks than we have drivers. A
driver picks up a loaded truck at the factory and takes his place in the convoy. At the blast site
he jumps out of the loaded truck and runs over to another truck that has already been emptied
(so we can ignore processing time) and joins a convoy of empty trucks and drivers on the way
back to the factory to pick up the next load. The idea is to emulate a continuous stream of
frames in one direction and ACKs in the other direction. Note that the ACKs usually have a
smaller transmit delay than the frames but in this case the ACKs will also be spaced one minute
apart. We also assume that the trucks undergo a 2 minute inspection on the return trip.
To keep costs down only the 10 truck drivers from our earlier example are available. What
happens?
We know the data rate of our link is 1 truck/minute. The propagation delay of our link is 60
km/(1 km/minute) = 60 minutes. Using Equation (1.13) adapted for our units we have:
round_trip_delay_bw trucks 
2  one_way_propagation_delay minutes  data_rate trucks minute
 2  60minutes  1truck minute
(1.14)
 120 trucks
If we want to be thoroughly accurate: a truck driver will experience a frame transmit delay on
the way out and a queuing delay, and an ACK transmit delay and queuing delay on the return.
That would add a 2 x 2 + 2 x 1 = 6 more minutes of round trip delay. At 1 truck per minute that
means 6 more trucks on the road (the total round trip delay is 126 minutes).
So it would take at least 126 truck drivers to keep the pipe full because a truck driver will not be
back to pick up his next truck for 126 minutes. What happens with just 10 truck drivers is that
the system runs at capacity for 10 minutes and then is idle for 116 minutes until the first truck
driver arrives back. Over the next 10 minutes the 10 drivers will all return and pick up another
truck and then the factory is idle again for another 116 minutes.
The effective data rate is 10 trucks / 126 minutes = 0.079 trucks/minute. Compare that to the
nominal data rate of 1 truck/minute.
When would all of the dynamite be delivered? In a round trip protocol like this we say that
delivery is complete when the ACKs have returned. If we had enough drivers it would look like
this:
Overall_delay minutes 
total_distance km
propagation _velocity km minute
num_switches  queuing_delay minutes switch

total_bits bits
data_rate bits minute

120km
1km minute
2  2minutes switch

total_frames frames  bits / frame
data_rate frames minute  bits / frame
 120minutes  4minutes  1000minutes
 1124 minutes  18.73hours
(1.15)
For the case where we have only 10 drivers the easiest way to compute the total time is to
recognize that we have all of the time above plus for every 10 trucks we also have 116 minutes
of idle time. That will occur 1000/10 = 100 times. So there will be an additional 100 x 116
minutes. That is a total of 1124 + 11600 = 12724 minutes ≈ 212 hours ≈ 8.84 days. We are
definitely not keeping the pipe full.
For a networking example consider a WAN link between two company sites. It has a data rate
of 1.5 x 106 bit/s. If the one-way delay is 40 ms then the round-trip delay-bandwidth product
is:
round_trip_delay_bw  2  1.5  106  40  10 3bits
 120  103bits
(1.16)
 15kbytes
Note that we are correctly using a decimal version (1000) for kilo here. Often when people
refer to quantities of bits (typically buffer sizes) they often use kilo to refer to 1024 = 2 10 when
strictly speaking they should be using kibi. See the Wikipedia article on binary prefixes for a
complete understanding of this issue.
What this number suggests that an early Windows box that used 8 Kbyte buffer allocations for
TCP would have performed poorly on this WAN.
Summary
Network delays are important in our understanding of protocol design and expected
performance. The relative importance of transmit delay versus propagation delay in LAN versus
WAN contexts is significant.
We have largely ignored the complexity of queuing delay and queuing theory in this discussion.
A packet-switched network is shared between multiple hosts and applications running on those
hosts. Queuing theory tells us that as the load on the network increases the delay during
switching will also increase (statistically it becomes more likely that other traffic is in front of a
frame in the output queue). Thus the simply model for queuing delay in this discussion is
provided to make you aware of the issue. We will take a bit more about queuing delay when
we discuss congestion and congestion avoidance techniques.
Finally, we mentioned some of the economics of network performance. It is important to
understand when a particular approach to a data transfer problem is affected by network
performance. In some cases spending money may not produce any significant improvement.
You can use some of the ideas in this document to explain why changing the host buffer size
might be more important than spending money on new network gear or a higher-throughput
WAN data link.