Packet Coalescing for Dual-Mode
Energy Efficient Ethernet: A Simulation Study
Mehrgan Mostowfi
School of Mathematical Sciences
University of Northern Colorado
Greeley, Colorado, USA
[email protected]
Packet Coalescing for Dual-Mode EEE
Slide 1 of 11
What is EEE?
An IEEE Standard (ratified in 2010) [1]
•  Introduces a Low-Power Idle (LPI) mode to Ethernet links
•  Works as follows for 10 Gb/s and less:
High transition times even for 10 Gb/s and less [2].
Would be magnified for 40 Gb/s and above.
[1] IEEE P802.3az Energy Efficient Ethernet Task Force.
[2] P. Reviriego, J. Hernandez, D. Larrabeiti, and J. Maestro, “Performance Evaluation of Energy Ecient Ethernet,” IEEE
Communications Letters, vol. 13, pp. 697-699, September 2009.
Packet Coalescing for Dual-Mode EEE
Slide 2 of 11
EEE for 40 Gb/s and above
Two modes are introduced [1]
•  Deep Sleep: Identical to LPI
•  Fast Wake:
-  Does not yield any power savings
-  In sending side to keep the sender and receiver in alignment
-  Much faster transitions (Wakeup: 0.34 us vs. 5.50 us from DS)
•  Limits power savings to receive side only in practice
Can Fast Wake effectively become a new low-power
mode for EEE?
A case can be made [2]
•  Only needs to send LPI signals
Transition times would still be too high.
[1] IEEE 802.3bj-2014 Amendment 2: Physical Layer Specications and Management Parameters for 100 Gb/s Operation Over
Backplanes and Copper Cables,“ IEEE Computer Society, June 2014.
[2] H. Barrass, “Options for EEE in 100G,” presentation at IEEE P802.3bj meeting, January 2012.
Packet Coalescing for Dual-Mode EEE
Slide 3 of 11
Overcoming High Transition Times
Packet Coalescing
• 
• 
• 
• 
Gather up a few packets, send in one burst
A sleep and wakeup transition for a number of packets
Time-based and Count-based
Has shown to be effective for EEE for 10 Gb/s and below [1]
Power
Packets arrive,
coalesced
Packet transmission
Packet transmission
Fast Wak
expires, b
Active
(100%)
Coalescing timer
expires, pkts in
buffer < Scoa l/2
Fast
Wake
(70%)
Coalescing
beg ins
Coalescing
beg ins
Deep
Sleep
(10%)
Coalescing timer
expires, pkts in
buffer ≥ Scoa l/2
Coalescing
beg ins
Packets arrive,
coalesced
Fast Wake Timer
expires, pkts in
buffer < Scoa l/2
TAtoF Tidle TFtoA
TAtoD
Fast Wake Timer
expires, pkts
in buffe r ≥ Scoa l/2
Packets arrive,
coalesced
Tcoal
TDtoA
TAtoD
Tcoal
TDtoA
TAtoF Tidle TFtoA TAtoF Tidle TFtoD
[1] K. Christensen, P. Reviriego, B. Nordman, M. Bennett, M. Mostowfi, and J. Maestro, “IEEE 802.3az: The Road to Energy
Efficient Ethernet,” IEEE Communications Magazine, vol. 48, no. 11, pp. 50-56, November 2010.
Packet Coalescing for Dual-Mode EEE
Slide 4 of 11
Notes:
• In Deep
beg ins a
first pac
• In Fast W
beg ins w
transitio
• Scoa l is a
this exa
Simulation Model
CSIM simulation library in C
•  A unidirectional EEE link with Active, FastWake, and Deep
Sleep modes
•  Ethernet link was simulated by a CSIM server facility
•  A Finite State Machine (FSM) controlled the server
•  Some variables and timers in the FSM:
Packet Coalescing for Dual-Mode EEE
Slide 5 of 11
The Finite State Machine
Packet arrived
5) -------------------Buffer packet
RESET
(Buffer empty) &&
(DFlag == FALSE)
3) ------------------------Set TTIMER to TAtoF
Start TTIMER
Packet arrived
1) ------------------Buffer packet
Active
Packet in buffer
2) --------------------Transmit packet
toFW
(Buffer empty) &&
(DFlag == TRUE)
4) ------------------------Set TTIMER to TAtoD
Start TTIMER
Packet arrived
7) -------------------Buffer packet
TTIMER expired
Fast
6) ------------------------Wake
Set FTIMER to Tidle
Start FTIMER
(FTIMER expired) &&
(FTIMER expired) &&
(Buffer is empty)
(Buffer is not empty)
9) --------------------------8) ------------------Set TTIMER to TFtoD
DFLag = (# of packets in
Start TTIMER
buffer < (Scoal/2))
Set TTIMER to TFtoA
Start TTIMER
TTIMER expired
16) ---------------------
toA
Packet arrived
15) -------------------Buffer packet
(CTimer Expired) || ((# of packets in buffer ≥ Scoal)
14) ----------------------------------------------------------------DFLag = (# of packets in
buffer < (Scoal/2))
Set TTIMER to TDtoA
Deep
Start TTIMER
Sleep
(Packet arrived) &&
(Buffer is not empty)
13) ---------------------------Buffer packet
Packet Coalescing for Dual-Mode EEE
Slide 6 of 11
toDS
Packet arrived
10) -------------------Buffer packet
TTIMER expired
11) -------------------------
(Packet arrived) &&
(Buffer is empty)
12) -------------------Buffer packet
Set CTIMER to Tcoal
Start CTIMER
Experiments
Performed on the Simulation Model
• 
• 
• 
• 
Link capacity: 40 Gb/s
Transition times: taken from the standard (closest in meaning)
Coalescing time Fast Wake: 3.00 us
Power consumptions:
-  Active and transitions: 100% (the peak power consumption)
-  Fast Wake: 70%
-  Deep Sleep: 10%
•  Two set of experiments:
-  Smooth Traffic: Poisson packet arrivals.
-  Bursty Traffic: Interrupted Poisson Process packet arrivals
•  Two coalescer sizes:
-  Small: In Deep Sleep coalesce 10 pkts or for 3.00 us
-  Large: In Deep Sleep coalesce 100 pkts or for 30.00 us
Packet Coalescing for Dual-Mode EEE
Slide 7 of 11
Results – Poisson Traffic
Coalescing brings the power consumption closer to
ideal, compared to Deep Sleep Only and no coalescing
Tradeoff: added per-packet delay
Packet Coalescing for Dual-Mode EEE
Slide 8 of 11
Results – Bursty Traffic
Similar results. Better consumption is the traffic is
bursty (some of the coalescing is done already)
Is the added delay significant?
Packet Coalescing for Dual-Mode EEE
Slide 9 of 11
Conclusions and Future Work
Conclusions:
•  Dual-Mode EEE can be an effective method of
extending idle periods between packet arrivals in order
to maximize the opportunity for sleeping.
•  The tradeoff, the increased packet delay, may be
justifiable by the energy savings gained.
Future Work:
•  The technical feasibility? (more EE than CS)
•  Other response variables?
•  Other tradeoffs?
•  Real traffic traces?
Packet Coalescing for Dual-Mode EEE
Slide 10 of 11
Questions from the audience
Thanks for listening. Any questions?
Mehrgan Mostowfi
School of Mathematical Sciences
University of Northern Colorado
Greeley, Colorado, USA
[email protected]
See the paper for more details
Packet Coalescing for Dual-Mode EEE
Slide 11 of 11