Power and Delay Optimal Policies for
Wireless Systems
Satya Kumar V, Anusha Lalitha, Vinod Sharma
Department of Electrical Communications,
Indian Institute of Science, Bangalore
Email: [email protected], [email protected], vinod @ece.iisc.ernet.in
Abstract—In this paper we consider a single discrete time
queue with infinite buffer. The channel may experience fading.
The transmission rate is a linear function of power used for
transmission. In this scenario we explicitly obtain power control
policies which minimize mean power and/or mean delay. There
may also be peak power constraint.
Index Terms—Fading channel, Optimal power policy, Optimal
delay policy.
I. INTRODUCTION
For last few decades Information and communication technology (ICT)/Signal Processing (SP) applications are growing
very rapidly. Consequently in near future power consumption
by these applications is going to be around 15% of the total
world consumption of electricity [1]. Thus ICT infrastructure
is contributing to global warming of the planet. This impact
is magnified by the requirement for expensive diesel fuel for
many locations in developing regions where the regular power
supply may not be available. Given the significant impact that
communications infrastructure has on the environment today,
any reduction, even incremental, in power usage will scale
to important improvements in the overall carbon footprint of
this industry. The primary objective of green communication
is to provide quality of service (QoS) at reduced energy
consumption.
In this paper we would like to address the problem of
reducing power consumption in a wireless communication
system. A cellular system consumes power mainly in the
Base Station (BS). In a BS the power is consumed mainly
in the air conditioner, data processors, power amplifier and
RF transmission. We will focus on reducing power in RF
transmission.
Goldsmith and Varaiya [5] presented an optimal power
allocation policy for a single fading link. They ignored higher
layer performances like queueing delay and emphasized on
physical layer performance. Goyal et al. [6] considered an
optimal average delay policy under the average power constraint and proved existence of an optimal policy and its
structural results and an explicit expression. Chen et al. [4]
discussed an energy efficient scheduling in presence of fading
with individual packet delay constraints when the number of
packets arrivals in a time slot and channel state information
are know before hand (optimal offline scheduling) and also
978-1-4673-0816-8/12/$31.00 ©2012 IEEE
proposed an online scheduling algorithm that performs close
to the offline case. In a multi-user scenario for time varying
channels, Neely [11] proposed an optimal power policy which
allocates power based on current channel state and current
queue backlogs and provides stability for all users under a
peak power constraint. Neely [12] proposed a new innovative
algorithm which optimizes power when there is no knowledge
of arrival rates and channel proababilities and intelligently
drops a small number of packets. Alvi et al. [10] found a
policy which minimizes energy for sending a fixed number of
packets under given delay constraints in presence of fading.
We consider the problem of minimizing the average power
under an average delay constraint when the transmit rate
is a linear function of power. In this set up we provide
explicit optimal power schemes that minimize mean delay
also when there is no fading. We also consider the case when
there is a peak power constraint. For the fading channel we
provide explicit optimal average power policies that stabilize
the queue. Finally we show existence of optimal power policies
when there are average delay constraints.
This paper is organised as follows. In Section II the system
model is described. Power and delay optimal policies under
different scenario are discussed in Sections III and IV. In
Section V, we formulate the problem as a Markov decision
problem (MDP) and show existence and optimal structure of
stationary average cost optimal policy. Section VI concludes
the paper.
II. SYSTEM MODEL
We consider a discrete time, single server queue. A slot is
of duration τ . Let Ak bits/packets arrive in slot k representing
interval [kτ, (k + 1)τ ], k ≥ 0 and can be transmitted after
time (k + 1)τ . The packets on arrival are stored in an infinite
buffer queue till transmission. In slot k, let hk be the channel
gain and Rk bits are allowed to be transmitted. Let qk denote
the queue length in bits at time kτ . Then the {qk } evolves
as (in the following we will assume that the packets can be
fragmented arbitrarily)
+
qk+1 = (qk − Rk )+ + Ak
(1)
where (x) = max(0, x). We will assume {Ak , k ≥ 0} to be
stationary ergodic.
To transmit Rk bits in slot k, the power required Pk depends
on Rk and the channel gain hk . We are concerned about
finding policies which as a function of (qk , hk ) and their
past values, find the Rk (equivalently Pk ) that will satisfy
certain objectives under different constraints. We will ignore
the power consumed in other activities.
Usually this problem is formulated as a MDP. However this
way often one computes the optimal policy numerically but
does not get a closed form expression of the policy and also
not much insight. Thus, in the following we will consider the
special case where Rk is a linear function of Pk and hk . Often
in practical situations the Rk is approximately linear ([14],
[15]). Under this assumption, we will get closed form explicit
optimal solutions for several scenarios of interest. Finally, we
will also use MDP to consider more general scenario.
First we consider the channel without fading. Then we
assume
(2)
Rk = αPk .
Fig. 1.
Sytem Model
if E[A1 ] < Rmax . Then Eπ [q] < ∞ if E[A21 ] < ∞. The
optimal mean power ≤ Pmax . If Eπ [q] = ∞ then it equals
Pmax ; otherwise = E[A]/α.
IV. O PTIMAL PO LICIES : W ITH FADING
for some α > 0 (This is a good approximation of Shannon
formula 12 log(1 + SN R) at low SNR). We will consider
scenarios where we optimize the average transmit power.
There may or may not be a peak power constraint. Also,
we will always require that the queue is stable but will
often be concerned about the delays also. By Little’s law [8]
Eπ [q] = E[A]Eπ [D] where Eπ [q] is the mean queue and
Eπ [D] is the mean delay of a bit under stationarity. Thus
a constraint on mean delay can be written in terms of mean
queue length.
Now we consider a channel with fading. We will assume
that {hk } is stationary ergodic. Let |hk | ≤ h ≤ ∞ with
probability one and Rk = αPk hk . Also assume that p =
P (hk = h) > 0 if h < ∞.
III. O PTIMAL P OLICIES : N O FADING
If there is no h < ∞ with P (h ≤ h) = 1, then there is no
policy that minimizes mean power. But the following policies
are good policies. Take h0 > 0, now P (h ≥ h0 ) > 0. Consider
the policy
�
qk
, for hk ≥ h0 ,
Pk = αhk
(4)
0,
otherwise.
We will obtain explicit policies which minimize
n−1
lim sup
n→∞
1 �
E
Pk = P .
n
(3)
k=0
The proof of the following proposition is simple and can for
example, be obtained using the exchangeable argument in [16].
Proposition 1. The policy Rk = qk minimizes (3) and also
has minimum delay. �
The policy in Proposition 1 is actually Rk+1 = Ak for
k > 1 and the delay of each packet/bit is one slot. Thus the
optimal power (3) equals E[A1 ]/α.
Often in practice there is peak power constraint Pk ≤ Pmax ,
for all k, where Pmax < ∞. Then Rmax = αPmax is the
maximum number of bits that can be transmitted in a slot.
The following policy simultaneously minimizes mean power
(3) as well as mean delay under peak power constraint.
Proposition 2. The policy Rk = min(qk , Rmax ) minimizes
mean power as well as mean delay. �
Although the policy in Proposition 2 minimizes mean power
and delay, the queue may still be unstable and then Eπ [q] =
∞. The queue is stable (has a unique stationary distribution)
Proposition 3. If h < ∞, the policy Rk = qk 1{hk =h} is
mean power optimal. �
Although the policy in Proposition 3 is mean power optimal
and stabilizes the queue, it may not be mean delay optimal.
We will consider such policies later. Also, under this policy
P = E[A]
αp .
All these policies stablize the queue. As h0 increases the
average limiting power decreases but the average delay also
increases.
If there is also a peak power constraint Pk ≤ Pmax , then
under the policy in Proposition 3, (modified to satisfy peak
power constraint : Rk = min(qk , Rmax )1{hk =h} the queue
may be unstable. Let
� ∞
x = max{x : αPmax
ydPh (y) > E[A]}
(5)
x
where Ph is distribution of h. Then if Ph is continuous at
x the optimal policy is to transmit at power Pmax whenever
hk ≥ x and
P = Pmax P [h ≥ x].
(6)
If Ph has a jump at x, then policy (6) is modified as : at x
transmit at power P < Pmax such that mean transmit rate just
exceeds E[A].
V. MEAN DELAY CONSTRAINT
Next we consider the problem when there is fading and we
want to minimize
n−1
lim sup
n→∞
1 �
E
Pk
n
n→∞
k=0
+ E[N ]E[A] < ∞
1 �
E
qk ≤ q
n
(7)
n−1
1�
E[Pk + βqk ]
n
Now from [13], an optimal discounted policy exists which
minimizes
n−1
�
lim sup E[
(Pk + βqk )γ k ]
(11)
n→∞
k=0
for an appropriately chosen q. To solve this problem, we
first use Lagrange multiplier’s ([9]) to convert this to an
unconstrained problem of finding the policy that minimizes
lim sup
n−1
1�
E[N ]E[A] 1
E[ |h ≥ h0 ]
E[Pk + βqk ] ≤
n
α
h
k=0
n−1
n→∞
lim sup
n→∞
subject to a constraint on the mean waiting time. By Little’s
law we can replace this constraint with
lim sup
Let p = P [h ≥ h0 ] and N = inf {k : hk ≥ h0 }. Then since
p > 0, E[N ] < ∞. Also, for this policy
(8)
k=0
where β > 0 is appropriately chosen. In the following
we use MDP to show the existence of an optimal policy
for this problem. For that we assume that {Ak } and {hk }
are independent and identically distributed (i.i.d) sequences,
independent of each other. Then (qk , hk ) will be taken as the
state of the system at time k, where qk ∈ {0, 1, 2, ...} and
hk ∈ R+ .
The following theorem shows the existence of an optimal,
stationary Markov policy in the class of all policies which
depend on the whole history of the system, i.e., the power
used at time k depends on (q0 , h0 ), ..., (qk , hk ).
�
�
Theorem 1: If E h1 |h ≥ h0 < ∞ for some h0 ≥ 0 with
P (h ≥ h0 ) > 0 then there is a stationary Markov policy which
minimizes (8) and at that policy (8) is finite and independent
of initial cost.
Proof: To show the existence of an optimal policy we use
Theorem 3.8 in [13]. We consider condition W in [13]. For
this observe that if the state at a time is (q, h) then the actions
possible are
q
}.
(9)
A(q, h) = {p : p ≤
αh
A(q, h) is compact and the set function (q, h) �→ A(q, h)
is upper semi-continuous. Also if action P is taken, the
cost is c(P, q, h) = P + βq. Thus the cost function is a
continuous function of (P, q, h). Furthermore, the next state
P +
then is (q − αh
) + A where A is a r.v. independent of
(P, q, h) and has the distribution of Ak . Thus the transition
function of (Pk , qk , hk ) �→ P [(qk+1 , hk+1 ) ∈ B|(Pk , qk , hk )]
is weakly continuous. Thus condition W in [13] is satisfied.
Next we show that for some initial condition (q, h), there
is a stationary policy that provides a finite average cost. We
take initial state (0, h). Also, we use the stationary policy
�
qk
if hk ≥ h0
Pk = αhk
(10)
0
otherwise
k=0
for any 0 < γ < 1. Let us denote the optimal cost for a given
γ by vγ (q, h) where (q0 , h0 ) = (q, h).
To show the existence of a policy minimizing (8), we next
need to prove that
�
supγ<1 (vγ (q, h) − inf
vγ (q , h� )) < ∞
�
�
(q ,h )
(12)
for any (q, h) in the state space. It is easy to show that vγ (q, h)
is an increasing function of q. Also, from our system equation
(1) we can see that vγ (0, h) ≡ vγ (0) is independent of h.
Thus,
�
�
�
�
�
vγ (q, h) = min{p + βq + γ vγ (q , h )dP ((q , h )|(q, h))}
p
whereP (.|.) is the transition function of the Markov chain
(qk , hk ) and hence
q
+ βq + γvγ (0).
vγ (q, h) ≤
αh
q
The inequality is obtained by taking the policy p = αh
in step
one. Since the policy (10) gives a finite discounted cost for
any γ > 0, vγ (0) < ∞ and hence
q
+ βq < ∞
(13)
vγ (q, h) − vγ (0) ≤
αh
for any (q, h) and any 0 < γ < 1. Thus we obtain the existence
of an average cost optimal policy for any β > 0. �
To obtain the Markov stationary optimal policy for the
constrained problem, from [9], we need to show that at the
optimal policy obtained in Theorem 1, the limit (8) is attained
and there is a β > 0 at which this optimal policy has the limit
in (7) equal to q. From Theorem E.10 of [7], we know that
the limit in (8) is attained.
�n−1 Also as β increases, at the optimal
policy limn→∞ E[ n1 k=0 qk ] decreases. Thus we can choose
an appropriate β to get q.
VI. CONCLUSIONS
In this paper we have considered the problem of obtaining
power control policies for a discrete time single queue that
minimizes long term average power. We require that the
queue should be stable. The transmission rate is a linear
function of power. For this case explicit optimal policies are
obtained for a channel with or without fading. There may
be peak power constraint also. These policies also minimize
mean delay for the non-fading channel. Finally, for the fading
channel the existence of optimal policies that satisfy mean
delay constraints via Markov decision problem.
In future one can consider these problems for a multiple
access channel, broadcast channel and multihop networks.
R EFERENCES
[1] A. Amanna, “Green Communications Annotated Literature Review and
Research Vision,” ICTAS, Virginia Tech, 2010.
https://filebox.vt.edu/users/aamanna/webpage/Amanna%20%20Green%20Communications%20Literature%20Review%20and
%20Research%20Proposal.pdf
[2] R. A. Berry, R. G. Gallager, “Communication over Fading Channels
with Delay Constraints,” IEEE Transaction on Information Theory, Vol.
48, no. 5, pp. 1135-1149, May 2002.
[3] Bertsekas and R. Gallager, “ Data Networks,” Second Edition, Prentice
Hall.
[4] W. Chen, U. Mitra and M. J. Neely, “Energy-efficient scheduling with
individual delay constraint Over a Fading Channel,” Proc. of the 5th
Int. Symposium on Modeling and Optimization in Mobile, Ad Hoc, and
Wireless Networks, 2007.
[5] A. J. Goldsmith, P. Varaiya, “ Capacity of Fading Channels with Channel
Side Information,” IEEE Transaction on Information Theory, vol. 43, no.
6, 1986-92, Nov 1997.
[6] M. Goyal, A. Kumar and V. Sharma, “Power Constrained and Delay
optimal policies for Scheduling Transmission over a Fading Channel,”
IEEE INFOCOM, pp. 311-320, 2003.
[7] O. Hernandez-Lerma and J. B. Lassere, “Discrete Time Markov Control
Process,” Springer-Verlang, 1996.
[8] J. D. C, Little, “A Proof of the Queuing Formula: L=λW,“ Operations
Research, vol. 9, no. 3, pp. 383-387.
[9] D. J. Ma, A. M. Makowski, A. Shwartz, “Estimation and Optimal
Control for Constrained Markov Chains,” IEEE Conf. on Decision and
Control, Dec 1986.
[10] A. Fu. E. Modiano and J. Tsitsiklis, “Optimal Energy Allocation for
Delay-Constrained Data Transmission over a Time-Varying channel,”
Proc. IEEE INFOCOM, 2003.
[11] M. J. Neely, “Optimal Energy and Delay Tradeoffs for Multi-User
Wireless Downlinks,” IEEE Transactions on Information Theory, vol.
53, no. 9, Sep 2007.
[12] M. J. Neely, “Intelligent Packet Dropping for Optimal Energy- Delay
Tradeoffs in Wireless Downlinks,” IEEE Transactions on Automatic
Control, vol. 54, no. 3, March 2009.
[13] M. Schal, “Average Optimality in Dynamics Programming with General
State Space,” Mathematics of operations Research, vol. 18, no. 1, pp.
163-172, Feb 1993.
[14] V. Sharma, U. Mukherji, V. Joseph and S. Gupta, “Optimal Energy
Management Policies for Energy Harvesting Sensor Nodes,” IEEE
Transactions on Wireless Communications, Vol. 9, no. 4, April 2010.
[15] D. Tse, V. Pramod, “ Fundamentals of Wireless Communication,”
Cambridge University, 2005.
[16] J. Walrand, “Introduction to queueing networks,” Prentice Hall, 1988.