Smart Mobile Cloud
NetSys - KiVS 2013
Bernd Haberland
Alcatel-Lucent Bell Labs
COPYRIGHT © 2011 ALCATEL-LUCENT. ALL RIGHTS RESERVED.
ALCATEL-LUCENT — INTERNAL PROPRIETARY — USE PURSUANT TO COMPANY INSTRUCTION
C-RAN Requirements (1)
• OPEX and CAPEX reductions
• reduction of BBU sites
• taking benefit of pooling gains achievable in a cluster of radio sites (range 50-150)
• Pooling is done on data Radio Bearer Level (more efficient as on cell level)
• Support of LTE advanced as CoMP, HetNet (ICIC) and Carrier
Aggregation
• Low latency scheduler coordination & data transfer to joint processing points within a BBU
pool
• Reducing blocking probabilities caused by a lack of BBU processing
resources
• by offloading to remote processors
• Flexibility to allocate BBU processing resources from low traffic areas
to high traffic areas
• On Intra BBU pool level
• On Inter BBU pool level
• The vision: to support heterogeneous traffic profiles over time (enterprise, week-end events,
stadium etc.) within a radio site cluster or between radio cluster by one or different BBU pools
and guarantee a better utilization of the installed processing resources
•
C-RAN Requirements (2)
• Energy saving aspects
•Switching off BBU processors in low traffic load situations
• Dynamic Spectrum sharing in Multi-standard context
•Semi-static Re-arrangement of BBU modems by software replacement between
3G and 4G
• Recovery of failures
•A BBU processing board failure should not cause the loss of a cell or site
• A failure of the Decentralized Cloud Controller should not cause the loss of a
radio site cluster
Mobile Cloud Overall Architecture
HTN Cluster
HTN Cluster
Metro
Metro
RRH+
Metro
Metro
RRH+
RRH
RRH
…
eX2
S1
Iub
Abis
S1
Iub
Abis
DCC
BBU
RRH: Remote Radio Head
eX2: Enhanced X2 interface
(Ring only as example)
MSSeX2
DCC: Decentralized Cloud Controler
RRH
MSS-
S1
Iub
Abis
DCC
BBU
RRH
…
Metro
Metro
RRH+
eX2
BBU
BBU
MSS-BBU: Multi-site/standard BBU
eX2
DCC
DCC
HTN: Heterogeneous Network
HTN Cluster
RRH
eX2
RRH
eX2
MSS-
…
RRH
RRH
RRH
MSS-
RRH
RRH
RRH
eX2
MSSBBU
eX2
DCC
S1
Iub
Abis
In the Cloud (inter MSS-BBU)
• on User level
• on Cell level
S1
Iub
Abis
Challenge:
• Low latency transport MSS-BBU to RRH & inter MSS-BBU
Mobile Cloud
General Backhaul Architecture
RRH
eNB
RRH
eNB
RRH
eNB
RRH
eNB
RRH
eNB
RRH
eNB
RRH
eNB
RRH
eNB
RRH
eNB
RRH
eNB
RRH
eNB
MSSLast mile
DCC
BBUrouter
Last mile
router
RRH
eNB
MSSLast mile
DCC
BBUrouter
S1 + eX2
S1 + eX2
MSSRouter
DCC
BBU
Router
S1
5
COPYRIGHT © 2011 ALCATEL-LUCENT. ALL RIGHTS RESERVED.
Last mile
router
MSS-BBU Functional Architecture
cell
S1 MME
eNB
Control
Functions
RRH1
Scheduler
RB11
UP
antenna
RRH3
RB221
UP
UP
MUX/
MUX/
MUX/
MUX/
DEMUX
DEMUX
DEMUX
DEMUX
S1 - U
RB31
UP
PHYcell
RB41
UP
NB
Control
Functions
Iub
BTS
Control
Functions
LTE(FDD/
TDD)
RRH4
RRH5
antenna
Spreader/
Spreader/
Spreader/
Spreader/
Spreader/
Despr
Despr
Despr
Despread
Despr
RB51
UP
Interface II
MACMACMAC(e)hs/e
(e)hs/e
(e)hs/e
Scheduler
Scheduler
Scheduler
UP j-1
UP i
UP j
cell
Cell
Signals
eX2
To
other
MSSBBUs
DCC
UMTS
UP: User Processing stack (from S1 termination to PHYuser)
RRHm
PHYcell
UP i-1
Abis
RRH2
eX2: Enhanced X2
6
COPYRIGHT © 2011 ALCATEL-LUCENT. ALL RIGHTS RESERVED.
ALCATEL-LUCENT — INTERNAL PROPRIETARY — USE PURSUANT TO COMPANY INSTRUCTION
Traffic scaling of eNodeB functions
UP, Scheduler, Cell Control functions
Functions
Dependencies
S1 Termination
(User Plane)
# data radio
bearers
S1 data
rate
PDCP ROHC
# data radio
bearers
S1 data
rate
Configuration parameters for the
air interface (type of HC)
PDCP ciphering
# data radio
bearers
S1 data
rate
Configuration parameters for the
air interface (security algo)
RLC_MAC
# data radio
bearers
Configuration parameters for the
air interface (RLC mode)
Radio conditions 
concatenation, segmentation and
reassembly
PHYuser
# data radio
bearers
Scheduler
# data radio
bearers
Dedicated
control
# data radio
bearers
Cell control
Constant
PHYcell
Constant
MAC-PHY
rate
Radio conditions  PHYII rate
Radio conditions  resource
allocation
Possible PHY Interfaces From MSS-BBU to RRH
MAC-PHY
S1Termination/
Termination/
S1
S1
Termination/
PDCP/RLC/MAC
PDCP/RLC/MAC
PDCP/RLC/MAC
I
FEC
FEC
FEC
QAM++
QAM
QAM
+
multimultimultiantenna II
antenna
antenna
mapping
mapping
mapping
signal flow (DL)
IFFT
+
CPin
Resource III
mapping
P/S
BB
to
RF
to RRH
encode
to CPRI
User processing
IV or IV‘
Cell processing
MAC/RLC/PDCP
MAC/RLC/PDCP
MAC/RLC/PDCP
S1 Termination
S1Termination
Termination
S1
-1 +
-1
QAM
-1
QAM
+
-1
FEC
-1
FEC
-1
FEC
MAC-PHY
I
QAM +
multimultimultiII
antenna
antenna
antenna
Processing
Processing
Processing
(e.g.MRC)
MRC)
(e.g.
MRC)
(e.g.
Resource
demapping
III
CPout
+
FFT
S/P
Interface
Naming
RF
to
BB
decode
from CPRI
IV or IV‘
from RRH
signal flow (UL)
PHY
(I)
Soft-bit fronthauling (softbits + control info)
(II) Subframe data fronthauling (frequency domain I/Q + control info)
(III) Subframe symbol fronthauling (frequency domain I/Q)
(IV) CPRI fronthauling (time domain I/Q)
(IV‘) Compressed CPRI fronthauling (time domain I/Q)
8
COPYRIGHT © 2011 ALCATEL-LUCENT. ALL RIGHTS RESERVED.
ALCATEL-LUCENT — INTERNAL PROPRIETARY — USE PURSUANT TO COMPANY INSTRUCTION
UP
Data Rates From MSS-BBU to RRH Site (DL)
• Links from MSS-BBU to Macro site (Downlink)
• Each Macro site holds 3 sectors
• Transmission bandwidth of Macro site as given
below
• LTE-FDD, 4 antennas, 20 or 40 MHz
• Link level protocol not yet included on
interfaces I,II and III
• Process split at several potential interfaces
[Gb/s] per
site
IV
IV'
III
II/100%
II/30%
I/100%
I/30%
MACPHY*/
100%
Data rate (40
MHz BW,
Gb/s)
29,5
10,9
5,6
5,4
2,5
0,53
0,19
0,24
sites/
10 Gb/s link
0
0
1
1
4
18
53
41
Data rate (20
MHz BW,
Gb/s)
14,7
5,5
2,8
2,7
1,25
0,27
0,09
0,12
0
1
3
3
8
37
111
83
sites/
10Gb/s link
*average spectral efficiency: 2,0 b/s/Hz
9
COPYRIGHT © 2011 ALCATEL-LUCENT. ALL RIGHTS RESERVED.
Pros and Cons of potential PHY fronthauling interfaces
Pros
Cons
MAC-PHY
Very low bandwidth needed
Dependent on S1 data rate
Pooling only for L2/L3
All PHY functions to RRH site
Several C-RAN features not supported
PHY I
Low bandwidth needed
Load dependent
Pooling only for L2/L3 and FEC
All other PHY functions to RRH
C-RAN features only partly supported
No latency improv. for CoMP
In-band protocol needed for Modulation,
Multi-antenna proc. and PRB allocation
PHY II
Moderate bandwidth needed (Load dependent)
Pooling for all user related functions from L3-L1
Low latency support of CoMP
PHY cell functions to RRH
In-band protocol for PRB allocation
PHY III
Moderate bandwidth needed (Load independent)
Same other pros as for PHYII
No in-band protocol for PRB allocation needed
Remaining PHY cell functions as IFFT,
FFT,CPI,CPR to RRH
PHY IV’
All pros of PHYIII
High bandwidth needed
Additional processing effort for CPRI
compression at BBU pool and RRH
PHY IV
All pros of PHYIII
Deployed RRHs can be reused
Very high bandwidth needed
PHY II or PHY III could be a good choice !!!
Note: all interfaces need low latency transport interface (Scheduler within MAC)
Virtualization Principle of a UP (example LTE)
MME/SGW
S1-MME
S1-U
DCC
Router
Cell Control
Functions
BBU2
BBU1
MSS-BBU
internal
connection
UP1…UPx
Inter MSSBBU via
eX2 IF
UPx+1… UPn
DCC
Scheduler
PHY cell
MSS-BBU1
BBU3
Inter MSS-BBU
via eX2 IF
UPn+1… UPn+y
MSS-BBU2
Some definitions:
- For the users 1..x the BBU1 is the Home and the Serving BBU
- For the users x+1..n the BBU1 is the Home BBU and the BBU2 the Remote BBU
- For the users n+1…n+y the BBU1 is the Home BBU and the BBU3 the Remote BBU
11
COPYRIGHT © 2011 ALCATEL-LUCENT. ALL RIGHTS RESERVED.
ALCATEL-LUCENT — INTERNAL PROPRIETARY — USE PURSUANT TO COMPANY INSTRUCTION
Processing split for one cell for SOC realization within
one BBU Pool and comparison with GPP configurations
BBU2
BBU3
UPx+1 …UPm
BBU1
UP 1..x
Cell functions
PHYcell (MUX)
Scheduler
UPm+1…UPo
Ethernet Switch + DCC
S1
IF I or II to RRH
• It is assumed that BBU-SOCs can include various technologies such like GPPs,
DSPs or HW Accelerators
• No independent scaling depending on traffic possible for the parts.
• MAC + Upper Layer on GPP
• PHY on DSP and HW Accelerators
• complete UP allocation to the BBUx for virtualization makes sense
• GPP based configurations offer more design flexibility:
•Independent pools for MAC + Upper Layer and PHYuser with separate
virtualization
Architecture and Processing split for GPP
configurations within one MSS-BBU Pool
GPP configuration approach 1,2,3: Different Scheduler Positions
GPP++HW
HWaccelerator
acceleratorpool
pool
GPP
GPP
+ HW Accelerator
pool
PHYuser
PHYuser
PHYuser
MUX
MUX
PCIE-IB
GPPpool
pool
GPP
GPP
pool
MAC+ Upper Layers
MAC+
Upper
Layers
MAC+
Upper
Layers
Scheduler
CellScheduler
Control
Cell
Control
Cell
PCIE-IBControl
Ethernet++IB-Switch
PCIe Switch
+ DCC
Ethernet
+ DCC
S1, eX2
PCIE-IB
HWAccelerator
Acceleratorpool
pool
HW
HWPHYcell
Acc. config.
(MUX IF II)
PHYcell (MUX IF II)
PHYcell
(MUX IFControl
II)
PHYCommon
Common
Channels
PHY
Control Channels
PHY Common Control Channels
IF I or II to RRH
Fixed configuration scales with
number of cells/sites
Possible MSS-BBU HW Architecture (with SOC-BBUs)
Radio
Radio
Site k+1 Site k+2
XFP
XFP
Radio
Site m
Radio
Site 1,2
XFP
XFP
XFP
Radio
Site 3,4
Radio
Site k -1,k
Radio
Site 5,6
XFP
XFP
XFP
XFP
XFP
XFP
Fronthauling Switch
Cell
Signal
CFM
CFM
CFM
BBU1
BBU2
BBU3
(from other
MSS -BBU)
BBUk
eX2
IF
eX2
UP & Control
User Level
(from/to other
MSS -BBU)
Ethernet Switch/ Infiniband
DCC
Router/Address
Dispatcher
LTE+ GSM
WCDMA + GSM Control
Abis
Iub
S1 -U
S1 -MME
SRIO
• CFM (Cell Function Module): includes a set of cell functions: Cell control part and PHYcell
• needed for BBU HW red. (benefit from pooling gains), Recovery (BBU failure), energy saving
• Fronthaul Switch:
•needed for recovery (BBU failure), energy saving, inserting cell signals from neighbor pools
Control Flow Architecture
RBC: Radio Bearer Control
LRM: Local Resource Manager
Resource Pooling Management Algorithms (Intra RM)
• Measurements
• link bandwidth, link latency
• used processing capacity
• metric PHYuser:
• Processing effort /radio bearer setup request = f (PRB usage, MCS, no. of antennas for
TX/RX diversity or MiMO)
• Total processing effort= SUM (processing effort /per bearer setup request)
• metric for MAC and upper layers: Processing effort = f(S1 rate per radio bearer setup request,
no of radio bearer setup requests)
• Processing Resource prediction
• use Radio-Bearer_Setup_Request information
• QoS handling
• take into account dynamic Air interface behavior
• operator policies handling
• Resource Placing decision
• process the cost as a function of:
• needed processing resource, compared with available processing capacity
• link latency and bandwidth
• operator policies w.r.t. traffic distributions and energy saving aspects
• Connection management and instantiation of user processing functions
• with Open Flow Control (look-up table) or per routing header sequence
System Simulation study parameters
(simulation and simulation results contributed from Dipl.-Ing. Thomas Werthmann from the Institute of Communication Networks and Computer Engineering in
Stuttgart, Germany)
Parameter
Value
Scenario
LTE, 10 MHz uplink, 10 MHz downlink FDD, 3GPP 25.814 Macro Case 1 with some modifications
Playground
4.1 km2, 19 sites, 500 m distance, 3 sectors per site
Mobiles
Unlimited number, not moving during transmission, channel measurement and reporting, uplink transmit
power 23 dBm
Channel model
Pathloss and static shadowing
Handover
Based on SINR
Outage
Requests from mobiles dropped if SINR below -3,9 dB
Scheduling
Cell reuse one scheme, frequency-diverse and model for frequency-selective and proportional-fair
bandwidth assignment [Ellenbeck]
Link adaptation
Modulation and coding scheme with highest spectral efficiency selected
Load distribution
Randomly according to a configurable mix of uniform & Gaussian, mean in central site; non-full buffer
model
Admission Control
The 100 % reference load for each user distribution corresponds to the load at which 1% of the requests
are dropped by admission control
Request messages
Created on application layer according to a Poisson inter-arrival process; varying object sizes according to
a mix of 3 log-normal distributions, scaled by factor 0.5 [Hernandez-Campos]
Response messages
One per request; varying object sizes according to a mix of 3 log-normal distributions [HernandezCampos], maximum object size limited to 100 MB
Processing resources
Processing resources scale according to a processing resource model; for PHYuser part F( #PRBs, MCS,
#spatial antennas)
[Ellenbeck] Jan Ellenbeck, Johannes Schmidt, Ulrike Korger, Christian Hartmann: A Concept for Efficient System-Level Simulations of
OFDMA Systems with Proportional Fair Fast Scheduling, GLOBECOM Workshops 2009
[Hernandez-Campos] F. Hernandez-Campos, J.S. Marronb, G. Samorodnitsky, F.D. Smith: Variable heavy tails in Internet traffic,
Performance Evaluation 2004
Comparison NGMN and Hernandez-Campos traffic
model
• NGMN model configuration
1
• 40% HTTP users
0,8
TR 36.814
0,7
in average 15kbit/s per user
• 20% FTP users
one type of objects
reading time as in TR 36.814
in average 91kbit/s per user
• 40% Video users
0,5
0,4
0,3
0,2
0,1
object size [bytes]
1,E+10
4,E+09
1,E+09
5,E+08
2,E+08
6,E+07
2,E+07
9,E+06
3,E+06
1,E+06
4,E+05
2,E+05
6,E+04
2,E+04
8,E+03
3,E+03
1,E+03
4,E+02
1,E+02
• Hernandez-Campos model
5,E+01
0
64kbit/s fixed bit rate per user
2,E+01
•
Hernandez-Campos
0,6
7,E+00
•
NGMN FTP
3,E+00
•
NGMN HTTP
1,E+00
•
parsing time and reading time according to
weighted CDF
•
0,9
• Based on Internet data traffic measured at the
Campus of the University of North Carolina and is expected to reflect to a certain extend the today’s wireless traffic
produced from increasing use of Smartphones and tablets
• Object size distribution according to a mix of 3 log-normal distributions (implicitly includes all traffic types)
• Load configurable via negative exponential IAT
[NGMN] 3GPP TR 36.814 V9.0.0 (2010-03), Technical Report 3rd Generation Partnership Project; Technical Specification Group Radio Access
Network; Evolved Universal Terrestrial Radio Access (E-UTRA); Further advancements for E-UTRA physical layer aspects (Release 9)
[Hernandez-Campos] F. Hernandez-Campos, J.S. Marronb, G. Samorodnitsky, F.D. Smith: Variable heavy tails in Internet traffic, Performance
Evaluation 2004 (University of North Carolina)
Pooling Gains Resulting from User Load Distribution with
Traffic Model Hernandez-Campos
• Quantiles for 90%, 95%, 99% and 99.9% of the system time at its capacity limit
 Expected improvement up to a factor of
3 of the resource utilizations by the cloud
approach between the uniform to nonuniform user distribution cases
100%
90%
99,9%tile
99%tile
95%tile
90%tile
80%
used resources
• 100% hotspot users:
• less than 20% resource utilization for
90% of the time and
• less than 38% for 99.9% of the time
• 100% uniformly placed users:
• less than 65% resource utilization for
90% of the time and
• less than 91% resource utilization for
99.9% of the time
70%
60%
50%
40%
30%
20%
10%
0%
0
0,2
0,4
0,6
0,8
fraction of uniformly placed users
1
Resource Usage Trace for 50 % Uniformly Placed
Users – Sum Over All Cells
zoom
zoom
Hernandez-Campos
Comparison of Resource usage for 50 %
Uniformly Placed Users – Sum Over All Cells
NGMN
Hernandez-Campos
Comparison Resource Usage for 100 %
Uniformly Placed Users – Sum Over All Cells
zoom
zoom
zoom
Resource Usage Modeling: eNodeB Signal Flow
S1 data rate
header compression/decompression
security functions
handover support functions
discard of user plane data due to timeout
PDCP
Buffer
RLC buffer
concatenation and/or segmentation of packets
reordering of received HARQ packets
RLC
Scheduler
MAC
MAC-PHY rate
FEC/FEC-1
QAM and precoding/receive processing
3/12/2013
Cell processing
PHY2 rate
User processing
Multi-user scheduling, HARQ, ranging
PHYuser
*
PHYcell
*can be modeled with Desset
23
Processing Resource Modeling
• Reference case: single-antenna, 64-QAM, rate-1 encoding and a load of 100% [Desset]
• Scaling factors are exponents relating each contributor in the model to each parameter of the model
Scaling
exponent for
code rate
parameter C:
s2
Scaling
exponent for
antenna
parameter A:
s3
Scaling
exponent for
layers
parameter
(streams) L:
s4
Scaling
exponent for
Resource
Block
parameter R:
s5
GOPS* in
reference
scenario
for DL
GOPS* in
reference
scenario for
UL
Scaling
exponent for
modulation
parameter M:
s1
FD (linear)
30
60
0
0
1
0
1
FD (non-linear)
10
20
0
0
DL: 2 UL: 3
0
1
FEC
20
120
1
1
0
1
1
Operation
• Total Processing effort for user processing for the DL case
Ptotal, DL
PS1_ ter min ation, DL
PPDCP _ ROHC , DL
PPDCP _ ciphering, DL
PRLC _ MAC , DL
PPHYuser, DL
PScheduler, DL
with
PPHYuser, DL
PFD,lin , DL
PS 1_ ter min ation, DL
PPDCP _ ROHC , DL
PPDCP _ ciphering, DL
PFD,non
lin , DL
PFEC, DL
30 Aact / ref Ract / ref
f S1 _ data_rate
PRLC _ MAC , DL
f S1 _ data_rate,type _ of _ HC
PScheduler, DL
2
10 Aact
/ ref Ract / ref
20 M act / ref Cact / ref Lact / ref Ract / ref
f S1 _ data _ rate, RLC mod e
f resource _ allocation , # radiobearers
f S1 _ data _ rate, type _ of _ sec
*[Desset] C. Desset, B. Debaillie, V. Giannini, A. Fehske, G. Auer, H. Holtkamp, W. Wajda, D. Sabella, F. Richter, M., J. Gonzalez,
H. Klessig, I. Gódor, M. Olsson, M. Ali Imran, A. Ambrosy, O. Blume, Flexible power modeling of LTE base stations, EARTH project
24
COPYRIGHT © 2011 ALCATEL-LUCENT. ALL RIGHTS RESERVED.
ALCATEL-LUCENT — INTERNAL PROPRIETARY — USE PURSUANT TO COMPANY INSTRUCTION
Comparison Resource Block Usage and of Processing
Effort per Cell and per User
Load 40%
 Even with good SINR, higher MCSs are selected rarely
Comparison Resource Block Usage and of Processing Effort per
Cell and per User
Load 80%
• At high load, interference increases and users with high SINR are rare
• With high system load, higher MCSs become more rare
• High processing load does occur, but is uncommon even if total system load is high.
26
COPYRIGHT © 2011 ALCATEL-LUCENT. ALL RIGHTS RESERVED.
ALCATEL-LUCENT — INTERNAL PROPRIETARY — USE PURSUANT TO COMPANY INSTRUCTION
Important Inputs from the Evaluation to the Resource
Pooling Algorithms
• Processing load and air interface load information is not sufficient
• Introduce QoE metrics as additional criteria
• GBR bearer: verify that the bit rate is guaranteed
• non-GBR bearer: contribution of the RAN to the E2E latency requirement needs to be
fulfilled
• If there is some margin left for GBR bit rate or non-GBR latency, additional bearers
can be admitted by taking into account the quality requirements of the other users
• Decision is taken based on the air interface bandwidth and the processing
capacity
27
COPYRIGHT © 2011 ALCATEL-LUCENT. ALL RIGHTS RESERVED.
ALCATEL-LUCENT — INTERNAL PROPRIETARY — USE PURSUANT TO COMPANY INSTRUCTION
Summary, Conclusion
• Objectives and Requirements of C-RAN discussed
• Mobile Cloud Architecture and Baseband Pool Functional Architecture presented
• Several Processing split options incl. data rates are analyzed in pros and cons
with a conclusion:
the Interface PHY II or III for DL and for UL is a possible choice.
-- 1*10Gb/s sufficient to drive a Multistandard radio site (40MHz LTE and 3*5MHz UMTS)
• Processing split approaches and Virtualization Concepts for SOC BBUs and
GPP configurations are presented with an approach for a MSS-BBU HW Architecture
• Multi-cell System Simulation results:
• significant pooling gains (20-80%) depending on spatial distribution of users and traffic
model (implementation losses not taken into account)
• different traffic models: less resource utilization variance with NGMN Model
• Processing load and air interface load information is not sufficient
• Introduce QoE metrics as additional criteria
•Next steps:
•Include HetNets
•Extend processing resource model
•Define adjustable measurements windows
28
COPYRIGHT © 2011 ALCATEL-LUCENT. ALL RIGHTS RESERVED.
ALCATEL-LUCENT — INTERNAL PROPRIETARY — USE PURSUANT TO COMPANY INSTRUCTION