A Game Approach for Cell Selection and
Resource Allocation in Heterogeneous
Wireless Networks
Lin Gao, Xinbing Wang
Institute of Wireless Communication Technology (IWCT)
Shanghai Jiao Tong University
Outline
Introduction
 Motivations
 Related works
 Objectives
System Model and Problem Formulation
Analysis of The Two-tier Game
Convergence Algorithm and Simulation
Conclusions
A Game Approach for Cell Selection and Resource Allocation in Heterogeneous Wireless Networks
2
Motivation
 The appearance of heterogeneous OFDMA-based wireless access
networks, such as WiMAX, LTE, Wi-Fi, etc.
 Optimizing the cell selection and resource allocation (CS-RA)
processes is an important step towards maximizing the utilization of
current and future heterogeneous wireless networks.
 Distributed algorithm shows potential ability in CS-RA problem due to
the lacking of global central node in wireless networks.
An example of a
cellular system with
3 heterogeneous
base stations and 6
mobile users.
A Game Approach for Cell Selection and Resource Allocation in Heterogeneous Wireless Networks
3
Related works
 The goal of cell selection (CS) procedures is to determine
a base user to camp on.
 In [5], Hanly et al. propose a cell selection algorithm to determine
power allocation among different users so as to satisfy per-user
SINR constraints.
 In [6], Wang et al. study an HSPA based handoff/cell-site selection
technique to maximize the number of connected mobile stations,
and propose a new scheduling algorithm to achieve this objective.
 In [7], Mathar et al. provide an integrated design of optimal cell-site
selection and frequency allocation, which maximizes the number of
connected MSs and meanwhile maintains quasi-independence of
radio based technology.
 In [8], Amzallag et al. formulate cell selection as an optimization
problem called all-or-nothing demand maximization, and propose
two algorithms to achieve approximate optimal solution.
A Game Approach for Cell Selection and Resource Allocation in Heterogeneous Wireless Networks
4
Related works
 The goal of resource allocation (RA) is to determine the
radio resource (including time, frequency and power etc) in
a particular cell to a mobile user.
 In OFDMA-based system:
 Subchannel-and-power allocation algorithms for multiuser OFDM
have been investigated in [9]-[10] to maximize the overall data rate
or minimize the total transmit power.
 In [9], Wong et al. investigate margin-adaptive resource allocation
problem and propose an iterative subcarrier and power allocation
algorithm to minimize total transmit power given fixed data rates
and bit error rate (BER)..
 In [10], Jang et al. investigate rate-adaptive problem and propose a
mechanism to maximize total data rate over all users subjected to
power and BER constraints.
A Game Approach for Cell Selection and Resource Allocation in Heterogeneous Wireless Networks
5
Objectives
 In this paper, we formulate the CS-RA problem as a twotier game, namely inter-cell game and intra-cell game,
respectively.
 In inter-cell game, mobile users select the best cell according to
optimal cell selection strategy derived from expected payoff.
 In intra-cell game, mobile users choose the proper time-frequency
resource in the serving cell to achieve maximum payoff.
Inter-cell Game
Intra-cell Game
An illustration of the
inter-cell game and
intra-cell game.
A Game Approach for Cell Selection and Resource Allocation in Heterogeneous Wireless Networks
6
Outline
Introduction
System Model and Problem Formulation
 System Model
 Problem Formulation
Analysis of The Two-tier Game
Convergence Algorithm and Simulation
Conclusions
A Game Approach for Cell Selection and Resource Allocation in Heterogeneous Wireless Networks
7
System Model
 A cellular system G = (M, N) with
is the set of
base stations (BSs) and
is the set of mobile
users (MSs). Besides,
is the set of subchannels in cell j. 1
 Each MS decides which cell it should camp on and which
sub-channels (of the serving cell) it should occupy.
s4
s3
An example of the
strategy of MS s1 -selecting cell a2 and
sub-channels set
{c1 ,c2 ,c3 ,c4 ,c7 ,c8}.
a1
s2
Sub-channels c8
s2
s1
c7
s2
s1
c6
s2
s1
a2
c5
s2
c4
s1
s4
c3
s1
c2
s1
c1
s1
s4
1
Note : we assume that the cell number in each BS is 1 and thus the meaning of cell is
equivalent to BS in this paper.
A Game Approach for Cell Selection and Resource Allocation in Heterogeneous Wireless Networks
8
System Model
 We assume that each sub-channel in a cell can be used by
multiple MSs without interference by means of orthogonal
signals, e.g., separating the MSs in the time-dimension.
 We assume that the total available bandwidth on each
sub-channel is equally shared among the players selecting
this sub-channel.
s4
s3
a1
s2
Sub-channels c8
s2
s1
c7
s2
s1
c6
s2
s1
a2
c5
s2
c4
s1
s4
c3
s1
s4
c2
s1
c1
s1
The bandwidth of c8 is
equally shared by s1 and s2.
Accordingly, the power
consumption by s1 ( or s2 )
in c8 reduces to one half if
s1 and s2 use the time
orthogonal signals.
A Game Approach for Cell Selection and Resource Allocation in Heterogeneous Wireless Networks
9
System Model
 Game Model
 Players : the MSs set U = {1,2,…,N }.
 Strategies (or actions) of player u : (i) selecting a cell, i.e., xu , (ii)
selecting a set of sub-channels in the serving cell,
i.e., yu ,i = Yu ,c , Yu ,c ,..., Yu ,c  .
 Payoff function: defined in latter slide.
1
i
2
s4
s3
a1
s2
Sub-channels c8
s2
s1
c7
s2
s1
c6
s2
s1
s2 : xs2  a1 , ys2 ,a1 =  0, 0, 0, 0,1,1,1,1 ;
a2
c5
s2
c4
s1
s4
c3
s1
s4
c2
s1
s1 : xs1  a1 , ys1 ,a1 = 1,1,1,1, 0, 0,1,1 ;
c1
s1
s3 : xs3  a2 ;
s4 : xs4  a1 , ys4 ,a1 =  0, 0,1,1, 0, 0, 0, 0  ;
A Game Approach for Cell Selection and Resource Allocation in Heterogeneous Wireless Networks
10
Problem Formulation
 Key Notations
A Game Approach for Cell Selection and Resource Allocation in Heterogeneous Wireless Networks
11
Problem Formulation
 The exclusive-payoff of player u in sub-channel c of cell i is
defined as following:
 As Tc players occupying sub-channel c, the bandwidth of c
is equally shared by Tc players. Accordingly, the power
consumption of player u reduce to Pu,c/Tc . Hence, the
achieved payoff of player u in sub-channel c of cell i is :

U u ,c
Tc
A Game Approach for Cell Selection and Resource Allocation in Heterogeneous Wireless Networks
12
Problem Formulation
 The overall-payoff of player u in cell i can be written as:
where Yu,c  1 means player u occupying c and Yu ,c  0 otherwise.
A Game Approach for Cell Selection and Resource Allocation in Heterogeneous Wireless Networks
13
Problem Formulation
 The optimization problem for each player u :
where
i:
Indication of the cell player u selected.
Yu ,c : State of player u in sub-channel ck , Yu ,c  1 means player u
occupying c and Yu ,c  0 otherwise.
Pu ,c : The power of player u in sub-channel c.
A Game Approach for Cell Selection and Resource Allocation in Heterogeneous Wireless Networks
14
Outline
Introduction
System Model and Problem Formulation
Analysis of The Two-tier Game
 Optimal Power Allocation
 Analysis of Intra-Cell Game
 Analysis of Inter-Cell Game
Convergence Algorithm and Simulation
Conclusions
A Game Approach for Cell Selection and Resource Allocation in Heterogeneous Wireless Networks
15
Analysis of The Two-tier Game
 We decouple the optimization problem as three subproblems: power optimization problem, intra-cell game and
inter-cell game.
Player u
Optimal power allocation
Calculate the optimal power
allocation vector Pu,1 Pu,2 …
in all cells
The inter-cell game
……
Select cell 1
Select cell 2
The intra-cell game in 1
The intra-cell game in 2
Find the optimal sub-channel
state vector in cell 1
Find the optimal sub-channel
state vector in cell 2
……
Find the optimal sub-channel
state vector in cell M
Payoff: Uu,1
Payoff: Uu,2
……
Payoff: Uu,M
Select cell M
The intra-cell game in M
A Game Approach for Cell Selection and Resource Allocation in Heterogeneous Wireless Networks
16
Optimal Power Allocation
Optimal Power Allocation
A Game Approach for Cell Selection and Resource Allocation in Heterogeneous Wireless Networks
17
Optimal Power Allocation
 Recall the equation of achieved payoff of player u in subchannel c :
Channels Gain
Channels Gain
Sub-channels
Sub-channels
a2
a1
a1
Channels Gain
s1
Sub-channels
a3
The information
player u needed for
the calculation of
optimal power
allocation.
a3
a2
A Game Approach for Cell Selection and Resource Allocation in Heterogeneous Wireless Networks
18
Optimal Power Allocation
 Lemma 2 : The optimal power allocation for player u in
sub-channel c is:
Lagrange Equation
A Game Approach for Cell Selection and Resource Allocation in Heterogeneous Wireless Networks
19
Optimal Power Allocation
 The illustration of optimal power allocation and the optimal
absolute payoff in sub-channels. Note that the optimal
power is independent to Tc from Lemma 2.
hu ,c
Channels Gain
c1 c2 c3 c4 c5 c6 c7 c8 c9
Pu*,c
Optimal Power
*
U u ,c
channels
Optimal Absolute Payoff
Pt
0
c1 c2 c3 c4 c5 c6 c7 c8 c9
channels
c1 c2 c3 c4 c5 c6 c7 c8 c9
channels
A Game Approach for Cell Selection and Resource Allocation in Heterogeneous Wireless Networks
20
Optimal Power Allocation
 The illustration of the optimal achieved payoff in subchannels.
*
U u ,c
Optimal Absolute Payoff
Player Number
Tc
3
2
1
c1 c2 c3 c4 c5 c6 c7 c8 c9
U
U
*
u ,c
*
u ,c
channels
c1 c2 c3 c4 c5 c6 c7 c8 c9
channels
Optimal Absolute Payoff
Optimal Achieved Payoff
*
u ,c
U

Tc
c1 c2 c3 c4 c5 c6 c7 c8 c9
channels
A Game Approach for Cell Selection and Resource Allocation in Heterogeneous Wireless Networks
21
Intra-Cell Game
Intra-Cell Game
A Game Approach for Cell Selection and Resource Allocation in Heterogeneous Wireless Networks
22
Analysis of Intra-Cell Game
 The optimal sub-channels state vector for player u,
i.e., y*u ,i , can be derived as following :
U u*,c
Player u
c1 c2 c3 c4 c5 c6 c7 c8 c9
y*u ,i  (0, 0, 0, 0, 0, 0,1, 0,1)
channels
An example of the
optimal sub-channels
state vector for player u
with ku,i =2.
A Game Approach for Cell Selection and Resource Allocation in Heterogeneous Wireless Networks
23
Analysis of Intra-Cell Game
 Lemma 3 :2 The best response function for player u is:
where
k  ku ,c : the required subchannel number of player u in cell i.
{ 1 ,  2 ,...,  i }: permutation of sub-channels according to the
ascending order of Tcu .
Tcu : the number of players excluding player u in sub-channel c.
2
Note : we assume that the average channel gains of different sub-channels are
approximately the same. We use this assumption to facilitate the description of Nash
equilibrium state.
A Game Approach for Cell Selection and Resource Allocation in Heterogeneous Wireless Networks
24
Analysis of Intra-Cell Game
 The meaning of Lemma 3 is that each player will choose
the sub-channels with least other players occupied.
Tc u
Number of Players except u
Player u
3
2
1
c1 c2 c3 c4 c5 c6 c7 c8 c9
( yu ,i )*  (0, 0, 0, 0,1, 0,1, 0, 0)
channels
An example of the
optimal sub-channels
state vector for player u
with ku,i =2.
A Game Approach for Cell Selection and Resource Allocation in Heterogeneous Wireless Networks
25
Analysis of Intra-Cell Game
 Theorem 1 : A strategy profile Yi*  { y1,*i , y2,* i ,..., y*Ni ,i } is a Nash
equilibrium of the intra-cell game (in cell i) iff the following
conditions hold:
Tc
Number of Players
Nash Equilibrium
3
2
An example of the Nash
Equilibrium in a cell with
9 sub-channels.
1
c1 c2 c3 c4 c5 c6 c7 c8 c9
channels
A Game Approach for Cell Selection and Resource Allocation in Heterogeneous Wireless Networks
26
Analysis of Intra-Cell Game
 Property : all Nash equilibria sub-channel allocations
achieve load balancing over the sub-channels in a cell.
 The average of the maximal achieved payoff of player u in
cell i :
A Game Approach for Cell Selection and Resource Allocation in Heterogeneous Wireless Networks
27
Inter-Cell Game
Inter-Cell Game
A Game Approach for Cell Selection and Resource Allocation in Heterogeneous Wireless Networks
28
Analysis of Inter-Cell Game
 The optimal cell for player u, i.e., i * , can be derived as
following :
i*  arg max U u ,i  Pu*,i , yu*,i 
i
U u* ,i
A Game Approach for Cell Selection and Resource Allocation in Heterogeneous Wireless Networks
29
Analysis of Inter-Cell Game
 An example of the optimal cell for player i.
Optimal payoff in cell 2
Optimal payoff in cell 1
U u*,1
Player u
Player u
c1 c2 c3 c4 c5 c6 c7 c8 c9
channels
U u*,i
c1 c2 c3 c4 c5 c6 c7 c8 c9
channels
Player u
1
2
Cell Index (i)
A Game Approach for Cell Selection and Resource Allocation in Heterogeneous Wireless Networks
30
Analysis of Inter-Cell Game
 U u*,i can be obtained only when the intra-cell game in cell i
achieves Nash equilibria, which implies that the player u
has connected with cell i. However, the serving cell i is
indirectly determined by U u*,i . This leads to a non-causal
problem.
 Thus we introduce the mixed strategy of player u as
follows:
where
i
u
p : the probability of player u selecting cell i,
i
p
i1 u  1.
M
A Game Approach for Cell Selection and Resource Allocation in Heterogeneous Wireless Networks
31
Analysis of Inter-Cell Game
 Lemma 4 : The mixed-strategy matrix
is a
mixed-strategy Nash equilibrium if for each player u, the
following conditions hold:
where
U
*
u ,i
player u
pu1
pu2
pu3
pu5
pu4
An example of the
mixed strategy for
player u.
1
2
3
4
5
Cells Index (i)
A Game Approach for Cell Selection and Resource Allocation in Heterogeneous Wireless Networks
32
Analysis of Inter-Cell Game
 For simplicity, we consider an example with 2 BSs and 2
MSs. We assume that ku ,i  i for all players. The payoff
of two players :
strategy of 1
strategy of player 2
F2,2
BS1
MS2
F2,1
F1,2
F1,1
BS2
MS1
where
*
Fu ,i =U u ,ii : The maximum exclusive-payoff of player u in cell i
A Game Approach for Cell Selection and Resource Allocation in Heterogeneous Wireless Networks
33
Analysis of Inter-Cell Game
 We denote the mixed strategy of players by
The expected payoff of player 1 can be written as:
 The optimal p1, p2 and the mixed-strategy Nash equilibria
are shown as following :
A Game Approach for Cell Selection and Resource Allocation in Heterogeneous Wireless Networks
34
Analysis of Inter-Cell Game
 The optimal p1, p2 , p3 and the mixed-strategy Nash
equilibria for the cellular system with 2 BSs and 3 MSs are
shown as following:
A Game Approach for Cell Selection and Resource Allocation in Heterogeneous Wireless Networks
35
Outline
Introduction
System Model and Problem Formulation
Analysis of The Two-tier Game
Convergence Algorithm and Simulation
 Convergence algorithm
 Simulation Results
Conclusions
A Game Approach for Cell Selection and Resource Allocation in Heterogeneous Wireless Networks
36
Convergence algorithm
 To apply the algorithm in practical system, the following
three essential assumptions are necessary.
 First, we assume that each MS has ability to initiate inter-frequency
measurement, from which each MS can measure the average subchannels gain.
 Second, we assume that each cell periodically broadcasts the
number of MSs connecting with this cell.
 Third, we assume that each cell counts the load on each subchannel and multicast to all MSs connecting with him.
 CS-Algorithm : converge to the inter-cell game Nash
equilibrium.
 RA-Algorithm : converge to the intra-cell game Nash
equilibrium.
A Game Approach for Cell Selection and Resource Allocation in Heterogeneous Wireless Networks
37
Convergence algorithm
 CS-Algorithm
 The basic idea of the CS-algorithm :
 Problem 1: mixed strategy of one player can not be observed by
other players, which makes the calculation of expected payoff
impractical.
 Problem 2: the mixed strategy zu will degenerate to the pure
strategy due to the non-smooth characteristic of best response
functions.
A Game Approach for Cell Selection and Resource Allocation in Heterogeneous Wireless Networks
38
Convergence algorithm
 Solution of problem 1
 Lemma 5 : The expected payoff of player u is equivalent to Qu
defining as follows:
where
i ,r : The probability of r players (excluding player u itself) in cell i.
E U u*,i (r  1)  : The average of the maximal achieved payoff of player u
in cell i if there exists r other players in cell i.
A Game Approach for Cell Selection and Resource Allocation in Heterogeneous Wireless Networks
39
Convergence algorithm
 Solution of problem 2
 To overcome the degeneration problem, we introduce the
smoothed best response functions zu  ( pu1 , pu2 ,..., puM ) :
p1
Standard Best Response
Smoothed Best Response
with large 
1
Smoothed Best Response
with small 
0
p2
1
p2
The illustration of
standard best response
and smoothed best
response functions.
A Game Approach for Cell Selection and Resource Allocation in Heterogeneous Wireless Networks
40
Convergence algorithm
 The detail pseudo-code of CS-Algorithm
A Game Approach for Cell Selection and Resource Allocation in Heterogeneous Wireless Networks
41
Convergence algorithm
 RA-Algorithm
 The basic idea of the RA-algorithm : Greedy occupation
 Problem 1: the unstable sub-channel allocations caused by
simultaneously moving of different players,
 Solution of Problem 1: the technique of backoff mechanism.
A Game Approach for Cell Selection and Resource Allocation in Heterogeneous Wireless Networks
42
Convergence algorithm
 The detail pseudo-code of RA-Algorithm
A Game Approach for Cell Selection and Resource Allocation in Heterogeneous Wireless Networks
43
Simulation results
 The cell selection interval:
 The minimal scheduling interval:
 The number of cells:
 The number of MSs:
 The number of sub-channels in each cell i:
 Channel model: free-space propagation
A Game Approach for Cell Selection and Resource Allocation in Heterogeneous Wireless Networks
44
Simulation results
 Convergence of RA-algorithm (in a given cell)
Variance Ratio always
equals to 1 for any Nash
Equilibrium
A Game Approach for Cell Selection and Resource Allocation in Heterogeneous Wireless Networks
45
Simulation results
 Expectation of maximum acquired-subchannel-number of
players 1 to 15
Red: estimation results
according to Eq. (27)
Blue: simulation results
A Game Approach for Cell Selection and Resource Allocation in Heterogeneous Wireless Networks
46
Simulation results
 CLPDF learned by player 1
Dash: analytical results
Bar: learning results
A Game Approach for Cell Selection and Resource Allocation in Heterogeneous Wireless Networks
47
Simulation results
 Convergence of CS-algorithm
A Game Approach for Cell Selection and Resource Allocation in Heterogeneous Wireless Networks
48
Simulation results
 Impacts of Price and DCR on Mixed-strategy Nash
equilibrium in the inter-cell game
Fig. 2: Increasing price of BS2  Reduce the load of BS 2
Fig. 3: Increasing bandwidth of BS3  Increase the load of BS 3
A Game Approach for Cell Selection and Resource Allocation in Heterogeneous Wireless Networks
49
Outline
Introduction
System Model and Problem Formulation
Analysis of The Two-tier Game
Convergence Algorithm and Simulation
Conclusions
 Conclusions
A Game Approach for Cell Selection and Resource Allocation in Heterogeneous Wireless Networks
50
Conclusions
 In this paper, We propose a distributed cell selection and
resource allocation mechanism, in which the CS-RA
processes are performed by the mobile user independently.
 We formulate the problem as a two-tier game named as
inter-cell game and intra-cell game, respectively. We study
the two-tier game in details and analyze the existence and
property of the Nash equilibria of the proposed games.
 We analyze the structure of Nash equilibria and find some
interesting properties: load balance, load regulating,
connecting directing etc.
A Game Approach for Cell Selection and Resource Allocation in Heterogeneous Wireless Networks
51
Thank you !
Reference
 [5] S. V. Hanly, “An algorithm for combined cell-site selection and power control to
maximize cellular spread spectrum capacity,” IEEE J. Select. Areas Commun., vol. 13,
no. 7, pp. 1332-1340, 1995.
 [6] A. Sang and X. Wang, “Coordinated load balancing, handoff/cell-site selection, and
scheduling in multi-cell packet data systems,” in Proc. ACM MOBICOM ’04, pp. 302-314,
2004.
 [7] R. Mathar and M. Schmeink, “Integrated optimzal cell site selection and frequency
allocation for cellular radio networks,” Telecommunication Systems, vol. 21, pp. 339-347,
2002.
 [8] D. Amzallag, Reuven Bar-Yehuda, Danny Raz and Gabriel Scalosub, “Cell Selection
in 4G Cellular Networks,” In Proc. IEEE INFOCOM ’08, pp. 700-708, April 2008.
 [9] C. Y. Wong, R. S. Cheng, K. B. Letaief, and R. D. Murch, “Multiuser OFDM with
adaptive subcarrier, bit, and power allocation,” IEEE J. Select. Areas Commun., vol. 17,
pp. 1747-1758, Oct. 1999.
 [10] J. Jang and K. B. Lee, “Transmit power adaptation for multiuser OFDM systems,”
IEEE J. Sel. Areas Commun., vol. 21, no. 2, pp. 171-178, 2003..
A Game Approach for Cell Selection and Resource Allocation in Heterogeneous Wireless Networks
53