PIPELINING
Santosh Lakkaraju
CS 147
Dr. Lee
Pipelining




What is it?
How does it work?
What are the benefits?
What could go wrong?
What is pipelining?
Pipelining is a design feature that allows
individual common processor tasks to run
simultaneously, such as:
Fetch
Decode
Execute
What is pipelining?
Or, more specifically in modern computers,
Instruction read
Decode
Operand read
Execute
Operand write
What is pipelining?



No more “one instruction at a time”
processing
Processor works simultaneously on
multiple instructions
The cycle time of the processor is reduced,
thus increasing instruction issue-rate in
most cases.
How does pipelining work?

First instruction is fetched from memory
How does pipelining work?


First instruction is fetched from memory
First instruction is decoded;
second instruction is fetched
How does pipelining work?



First instruction is fetched from memory
First instruction is decoded;
second instruction is fetched
First instruction’s operands are fetched;
second instruction is decoded;
third instruction is fetched
How does pipelining work?
• First instruction is fetched from memory
• First instruction is decoded;
second instruction is fetched
• First instruction’s operands are fetched;
second instruction is decoded;
third instruction is fetched
• And on, and on, and on…
How does pipelining work?

Because each instruction demands one
stage of the processor, the maximum
number of simultaneous instructions is the
number of stages in the processor
How does pipelining work?
A short animation
of simple processor routines
How does pipelining work?
A short(er) animation
of pipelined processor routines
What are the benefits
of pipelining?


Uses most (if not all) of a processor’s
ability at all times
Efficiency goes up
Relatively minor hardware changes give
more completed cycles per second
Speed goes up
What can go wrong in
pipelines?



Data hazards
Control hazards
Structural hazards
What can go wrong…?
Data hazards

Two instructions at different stages
One instruction needs the results of
the other instruction’s operation
…
add
add
…
%r1, 5, %r2
%r2, 7, %r3
What can go wrong…?
Data hazards

Well-designed programs can anticipate this, and
spread the connected commands out with other
operations
…
add
sub
mov
add
%r1, 5, %r2
%r4, 7, %r8
%r5, %r6 (i1 read)
%r2, 7, %r3
(i1 fetch)
(i1 decode)
(i1 execute, i2 fetch)
What can go wrong…?
Data hazards


Hardware can detect some data hazards and
“forward” the information to the instructions that
need it, even as the data goes to the registers
Otherwise, programs may simply have bad data testing and good programming are the best cures!
What can go wrong…?
Control hazards



Instructions are read sequentially from
memory - even though loops may return to
previous statements
Instructions following the loop are already
in the pipeline!
Or, simple if/else blocks of code. Which
block do you get instructions from?
What can go wrong…?
Control hazards
• At worst, the entire pipeline must be
flushed: hardware can stall the influx of
new instructions until the conditional is
evaluated
What can go wrong…?
Control hazards
• Branch prediction inserts a couple
instructions of its own to guess the outcome
of the conditional
• If prediction is usually correct,
fewer cycles lost on average
• If prediction is wrong,
flush the pipeline and undo any improper
commands
What can go wrong…?
Structure hazards
• Two different instructions try to use the
same hardware resource
• One instruction will obviously get access
first, second instruction must be stalled
(maybe along with all other instructions
behind it)
• Again, careful programming will prevent
this