IEEE TRANSACTIONS ON SMART GRID, VOL. 5, NO. 3, MAY 2014
1475
Game-Theoretic Demand-Side Management With
Storage Devices for the Future Smart Grid
Hazem M. Soliman and Alberto Leon-Garcia
Abstract—We study the demand side management (DSM)
problem when customers are equipped with energy storage devices. Two games are discussed: the first is a non-cooperative one
played between the residential energy consumers, while the second
is a Stackelberg game played between the utility provider and the
energy consumers. We introduce a new cost function applicable to
the case of users selling back stored energy. The non-cooperative
energy consumption game is played between users who schedule
their energy use to minimize energy cost. The game is shown to
have a unique Nash equilibrium, that is also the global system
optimal point. In the Stackelberg game, the utility provider sets
the prices to maximize its profit knowing that users will respond
by minimizing their cost. We provide existence and uniqueness
results for the Stackelberg equilibrium. The Stackelberg game is
shown to be the general case of the minimum Peak-to-Average
power ratio (PAR) problem. Two algorithms, centralized and distributed, are presented to solve the Stackelberg game. We present
results that elucidate the interplay between storage capacity,
energy requirements, number of users and system performance
measured in total cost and peak-to-average power ratio (PAR).
Index Terms—Demand-side management, distributed algorithms, game theory, PAR.
I. INTRODUCTION
A
N interesting feature of smart grids is the possibility of
a mutually beneficial relationship, a “win-win” situation,
between the users and the utility. Users would like to minimize the cost they pay to the utility, whereas the utility cares
not only about what the users pay, but also about when they
will consume, i.e., PAR [1]–[3]. In this paper, we show that
two factors help to create a win-win situation when storage devices are introduced: a smart cost function, and a suitable game.
Our methodology to reach the win-win situation is through Demand-side Management (DSM) [4]. DSM can be used for different applications, such as conservation of energy, efficiency of
power delivery, fuel substitution, and residential or commercial
load management [5], [6]. Residential load management programs try to reduce home power consumption, as well as shift it
temporarily for better utilization and cost. Since utility providers
are challenged by high peak hour usage, smart time-varying
pricing in residential load management programs helps shift
usage away from peak hours to less congested times.
Manuscript received May 06, 2013; revised August 21, 2013 and November
29, 2013; accepted January 19, 2014. Date of publication April 02, 2014; date
of current version April 17, 2014. Paper no. TSG-00356-2013.
The authors are with the School of Electrical and Computer Engineering,
University of Toronto (e-mail: [email protected]; [email protected]).
Color versions of one or more of the figures in this paper are available online
at http://ieeexplore.ieee.org.
Digital Object Identifier 10.1109/TSG.2014.2302245
Game theory is seen as a useful mathematical tool to handle
DSM problems. In particular, game theory can serve the goals
and characteristics of the smart grid as discussed in [2], [3]. The
ability of game theory to capture the competition between users,
study the possible outcomes and resulting equilibrium as well as
their stability are reasons for the wide use of game theory in the
smart grid literature. Mohsenian et al. in [4] have proposed a distributed game-theoretic algorithm for scheduling energy usage
optimized to minimize energy cost. In [7], a Stackelberg game
is introduced to optimize the energy retailers’ pricing decisions.
The case when users are equipped with storage devices was
studied in [8] using welfare theory to find optimal prices. In [9]
another formulation was proposed employing a strategy-proof
double auction. However, the papers studying energy storage
have not considered the scheduling or PAR problems, nor have
they tackled the choice of an appropriate cost function that can
meet the requirements of the system.
We extend the literature in two directions: 1) we provide a
generalized treatment of storage by introducing a novel cost
function that can model the requirements when storage is
present. Our cost function can be viewed as a generalization
of the more common quadratic and linear functions; 2) we
provide a general framework for the interaction between users
and the utility through a Stackelberg game, where the utility
decides its prices taking into consideration the reaction of the
customers, and where the users schedule their energy in order to
minimize their cost based on the prices given by the utility. We
provide distributed algorithms for both interactions, and prove
their convergence. We also establish strategy-proof properties,
as well as the relation between our framework and the minimum peak-to-average power ratio (PAR) problem. Finally, we
present an extensive set of simulation results covering a range
of possible scenarios. We find that improved collaboration between the utility and users results in lower user cost and lower
system PAR. We also explore the impact of number of users and
total amount of storage capacity on system performance. We
discover that (under the assumptions of our study) the system
performance depends solely on the total storage capacity and
not on its allocation among users. This suggests the possibility
of establishing a single user-owned storage site that acts on
behalf of all users.
The paper is organized as follows. We introduce the system
model in Section II. The energy cost minimization problem and
the energy cost minimization game are discussed in Section III.
In Section IV we present the Stackelberg game and two algorithms for solving it. Simulation results are discussed in
Section V. Conclusions are presented in Section VI.
1949-3053 © 2014 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
1476
IEEE TRANSACTIONS ON SMART GRID, VOL. 5, NO. 3, MAY 2014
II. SYSTEM MODEL
A. Power Consumption Model
In the smart grid, each user is equipped with a smart meter
that can interact with smart appliances and control them [10].
Hence, one of the roles of a smart meter is to serve as an energy consumption scheduler (ECS) that uses price information
to schedule the energy consumption of its user.
Let
denote the set of users. The total number of users is
. Let denote the total load of user
at time
. The total daily schedule for each user is
then
, and the total load of all users during
a time
is [4]
(1)
B. Energy Cost Model
Cost functions are pricing tariffs used by utility companies to
determine the price at which they sell energy to their customers.
Cost functions can be used by utility companies as incentives for
users to follow a specific consumption behavior. The utility obtains energy from multiple sources and sells it to the users. The
utility first uses energy from cheaper sources like hydro-electric
generators before moving to more expensive fuel-based generators during peak-hours [11]–[13]. The cost function determines
the prices users have to pay, and the income to the utility. Since
we use a game-theoretic framework, a smart cost function is
needed in order to minimize the impact of selfish behavior.
Let
be the cost that users have to pay to the utility
for an amount of energy
during time
. We based
our choice of the cost function on several requirements that
govern the operation of demand side management. First, the
utility provider is responsible for satisfying all the needs of all
users, hence the cost function is a function of the total consumption by all users
during some time
. Moreover, the cost
function can vary from time to time, i.e., higher cost during peak
times because of the more expensive energy used, and changes
in prices charged by the energy providers. Other assumptions
for cost functions are:
1) Cost function is an increasing function of demand, i.e.,
more energy should entail higher cost [4].
(2)
2) Cost function is convex, that is the increase in the price is
also increasing. Moreover, we assume that the cost function is strictly convex [4].
(3)
. When
3) When the user sells energy back, we have
this happens the user is paid for this energy, that is, the user
cost function is
since the user pays a negative
amount to the utility. Thus we have the requirement that,
for
,
. The quadratic cost function,
clearly does not satisfy this condition. Alternatively, we could try
which
does satisfy the negativity condition. However this choice
of cost function is not convex. We note that the linear cost
function
is increasing and convex, but
not strictly convex, so it is not guaranteed to have a unique
Nash equilibrium.
4) Another valid condition for the cost function is that, at any
given time h, the utility always makes profit, i.e., the price
at which it buys is always less than the price at which it
sells:
(4)
This condition also prevents excessive buying and selling
by the users.
1) Cost Function Rationale: The above assumptions disqualify the quadratic or linear cost functions, so we had to propose a new cost function. The function we chose is a variant of
the widely used logarithmic barrier functions, used as a penalty
function in interior point methods [14]:
(5)
where
is a pricing coefficient, determined by the utility to
give higher prices during peak-hours,
is the total load, is a
parameter that we introduce to give cost values very close to the
values given by a quadratic one, it also serves as the maximum
typical value for
.
The relation between the proposed cost function we use and
the quadratic one can be understood from its Taylor expansion.
Since
, the Taylor series expansion is
(6)
So we can see that the logarithmic cost function is essentially
a sum of linear and quadratic terms, where the parameter
is
used to govern the the weight of the linear and quadratic terms.
Hence, the proposed cost function can be viewed as a modification of the quadratic function to satisfy all the conditions we
have imposed. Fig. 1 compares the quadratic function and our
logarithmic cost function.
This cost function is monotonically increasing and strictly
convex, see Appendix A.
C. Residential Load Control
In the absence of scheduling and storage, each user will use
appliances at their preferred times. The absence of coordination among users leads to busy hours of high consumption, resulting in a high PAR [4]. When scheduling is used to optimize
energy consumption, users avoid the peak hours to avoid the
associated high prices to the extent possible [4]. However, the
users have limited flexibility in the scheduling of their appliances and hence they have limited freedom in avoiding the peak
hours. A phenomenon that is often associated with selfish optimized scheduling is the appearance of “redundant” peaks [15].
The aggregate behavior of users individually avoiding peak periods results in new “redundant” peaks during the lower price
periods [15]. The introduction of energy storage gives users
SOLIMAN AND LEON-GARCIA: GAME-THEORETIC DEMAND-SIDE MANAGEMENT WITH STORAGE DEVICES
1477
As in [4], the user is assumed to have a set of shiftable and
non-shiftable appliances. Let
be the predetermined amount
of energy required by appliance belonging to user . A user
may require this amount to be provided for the appliance during
a certain interval, beginning at
and ending at
. Then for each user
and each appliance
, the
following conditions must hold
(11)
where
. Each appliance also has a
minimum and a maximum power level that it can operate on
while working in its scheduled time. Let
denote the minimum power level and
denote the maximum power level
for appliance
, then
(12)
Each storage device has upper and lower limits on the amount
of power it can store. Let
and
be the minimum and
maximum charging levels of the storage device belonging to
user
. The following condition must hold
(13)
Fig. 1. Proposed logarithmic cost function as compared with the quadratic cost
function.
greater freedom in scheduling, so they can buy extra energy
ahead of time, and consume it later during peak times or even
sell it back to the grid.
Let
be the set of all appliances belonging to user
. Each appliance can get power either by buying power directly from the grid (external power), or by taking it from its
own stored storage device (internal power). The external power
scheduling vector for each appliance is:
where the middle term represents the net inward power for user
storage device, during all the time intervals up to time
.
III. PROBLEM FORMULATION
A. Energy Cost Minimization
Our goal is to find the optimal scheduling of all appliances
for all users that minimizes the total energy cost:
(7)
where
is the amount of power scheduled for appliance
belonging to user
during time
, to be bought
for this appliance from the grid. Similarly, the internal power
scheduling vector for each appliance is denoted by
(8)
where
is the amount of power scheduled for appliance
belonging to user
during time
, to be taken by
this appliance from the user storage device.
The power consumption schedule for storage for user
is:
(9)
where
is the amount of power scheduled by user
during time
, to be bought and stored. The total load of
user during time is then
(10)
(14)
where
,
is the optimization variable formed
by concatenating all the
variables into a single vector variable,
is a vector specifying all energy being bought from the utility provider during
time
. and are the lower and upper bounds formed
by concatenating all the lower and upper bounds in (12). is
a matrix giving the battery net inward flux in (13), and is the
battery charging limit vector.
is a matrix capturing energy
provided for each appliance in (11), and is the vector of the
required energy amount for each appliance. The requirements
we have imposed on the cost function imply that (14) is a
strictly convex optimization problem with a unique optimum
solution [14].
B. Energy Consumption Game
We use game theory to capture the competition between
users. We assume users are selfish and interested only in
1478
IEEE TRANSACTIONS ON SMART GRID, VOL. 5, NO. 3, MAY 2014
optimizing their cost, and that they might deviate from a centralized solution if they find a better schedule with less prices.
Moreover, we suppose that users wish to preserve their privacy
and are willing to share some information if it leads to lower
cost. Game theory is able to capture the competitive behavior
of the users and the stability of equilibrium solutions, which is
why we chose it as our optimization tool next.
The energy consumption game is defined as follows:
Energy Consumption Game:
• Players: All users in .
• Strategies: Each user
selects its strategy by
scheduling appliance
to minimize its own cost.
• Payoffs:
for user
, where
The parameters
are similar to the ones
in (14) but they are now specific to user . Problem (18) is still
strictly convex and admits a unique optimal solution. The algorithm basically allows each player to play his best response
strategy, announce only his total consumption, and repeat until
equilibrium is reached. The distributed algorithm is explained
in detail in Algorithm 1. The algorithm progresses by allowing
each player to solve his optimization problem, i.e., play his best
response strategy. Each step will result in the total cost decreasing or remaining the same. Since the objective function is
bounded from below, the iterations will eventually converge to
a fixed point, i.e., a NE point. Once users reach this point, they
will have no preference to change since the NE is unique, and
convergence is achieved.
(15)
Algorithm 1 Executed by each user
where
Randomly Initialize other users loads
(16)
repeat
Solve Problem (18).
represents the proportion of energy consumed by user , relative to the total energy consumed by all users. This is a proportional price sharing mechanism that divides the total cost
among users according to their consumption proportion. This
price sharing mechanism has been used in the literature, for example [4].
denotes the
consumption schedules for all users except user .
Nash Equilibrium (NE) Consider a game played between a
set
of players. For each player
, let
denote his
strategy space. A set of strategies
constitute a
NE if
(17)
Theorem 3.1: The Nash equilibrium of the Energy Consumption Game exists and is unique.
See Appendix A for proof.
Theorem 3.2: The Nash equilibrium of the Energy Consumption Game is also the global system optimal solution of the energy cost minimization problem (14).
See Appendix B for proof. The above game is a non-cooperative game. The structure of the objective function and the alignment of the NE with the system optimal point indicates that the
users’ strategies are strategic complements, not substitutes, i.e.,
they mutually reinforce each other [16].
if
changes compared to previous scheduling then
Update
to the new solution.
Broadcast the new total load
to all other users.
end if
if A new update is received then
Update
end if
until No new announcements are received.
D. Strategy-Proof Property
be the NE of the Energy Consumption Game
Let
when only a subset of users
are truthful. Also let
denote the NE when all players are truthful, which
is also the system optimal point from Theorem 3.2. Then for
any cheating user
(19)
By dividing both sides by
, we get
C. Distributed Algorithm for the Energy Consumption Game
Since
is independent of
, it can be dropped from the
optimization. This is a significant simplification and the problem
to be solved by each user becomes:
(18)
(20)
which contradicts Theorem 3.2. Hence, no user has an incentive to deviate because the total cost is divided among users in
proportion to their consumption. Since the NE point is also the
unique system optimal solution from Theorems 3.1 and 3.2, any
deviation by any user will result in increased cost not only for
the other users, but for the user himself.
SOLIMAN AND LEON-GARCIA: GAME-THEORETIC DEMAND-SIDE MANAGEMENT WITH STORAGE DEVICES
1479
players, and the objective function is continuous in both variables, the Stackelberg equilibrium is guaranteed to exist from
[17] Proposition 3.1.
IV. PRICE CONTROL
A. Pricing Problem
Thus far, we assumed that the utility provider decides pricing
apriori based on typical peak hours. Simulation results will
show that even though the PAR decreases as some users get
their storage devices, after a certain number of storage devices
the PAR will start to increase again. The reason is that users
equipped with storage devices will tend to buy extra energy
before peak hours and sell it back during the peak hours to
minimize their cost. Once too many users do this, the former
peak consumption is shifted, even though with a less value
than before, to a new hour. The new effect that also appears
when users can sell back is that they tend to sell mostly during
the high price hours, i.e., former peak hours. We call this phenomena “Reverse Peaks,” since the shift of peak consumption
is accompanied by a high tendency of users to sell their stored
energy back during high price hours. This undesirable effect
can be resolved by allowing the utility provider to become a
player in the game. When the users shift their consumption to
a newly formed peak hour, the utility will respond by adjusting
the price again. Repeated play will converge to an equilibrium
where users have no interest in shifting their loads and the
utility has no interest in changing prices. This interaction can
be modeled as a Stackelberg game. Stackelberg equilibrium is
what we called a win-win situation, whereby the users minimize their cost, while the utility maximizes its profit, and more
importantly, indirectly manages to decrease the PAR, as will be
shown later.
We assume the utility selects the price from a set of possible
prices. We also assume the users’ energy needs are will always
be satisfied. Therefore, the utility’s problem involves selecting
its best pricing option out of a reasonable set of possible prices.
A situation might happen when the set of possible prices is selected in a manner that could lead to arbitrarily high prices. We
note that this is also found in the real world when a utility operates as a monopoly. In this case, regulatory mechanisms are activated to counterbalance the tendency to increase prices. Such
mechanisms are outside the scope of this paper.
C. Equivalence Between Stackelberg Game and Minimum
PAR Optimization
We note that any cost schedule can be normalized so that the
sum of the terms in the vector is 1. Here we show that when
the set is defined as a simplex
, the solution of (21) also minimizes PAR. To show
this consider
(22)
Our objective function is convex in , and concave (linear) in
. For the class of convex-concave functions, the maximum and
minimum operations can be exchanged [14], so the problem becomes
(23)
and the solution is a saddle-point. Consider (23), when is a
simplex, the maximization over is solved by letting
for the the largest term in the summation, so (23) reduces to
(24)
Note that (23) is continuous minimax, while (24) is discrete
minimax. Since
is a monotonic increasing function in its operand
, then the function is maximized by maximizing its argument. Hence the solution of (24)
is also the solution of
(25)
is
The average energy consumption
variable-independent and can be inserted into the optimization
to obtain the minimum PAR problem [4]:
(26)
B. Stackelberg Game
A Stackelberg game is a sequential game played between a
Leader and a set of Followers. In our game, the utility plays
the role of the leader, who sets the prices first, while the users
are followers who optimize their usage to minimize the cost
they pay [2]. Using Backward Induction, the problem the utility
provider needs to solve to find the optimal prices is formulated
as follows:
(21)
is selected from
where the cost schedule
a compact set . Since the constraint sets are compact for both
D. Solving the Stackelberg Problem
1) Interior Point Solution for Minimax: We can use the interior-point algorithm developed in [18] to solve the Stackelberg
min-max optimization problem. We don’t provide the algorithm
details here due to space limitations.
2) Iterative Entropic Regularization: We propose an algorithm, based on [19], that transforms the continuous min-max
problem into a sequence of finite min-max problems using entropic regularization to smooth each finite problem.1
The algorithm involves two stage optimization problems. In
the first stage, the users receive the pricing coefficients as determined by the utility, and schedule their energy to minimize their
cost. The utility uses the players decisions to find the pricing
1Note that here we remove the negative sign in the objective function and
solve it as a min-max problem instead of a max-min problem.
1480
IEEE TRANSACTIONS ON SMART GRID, VOL. 5, NO. 3, MAY 2014
coefficients that maximize its own profit. The utility also needs
to take into account the reaction of players to these new pricing
coefficients. The utility uses an Iterative Entropic algorithm that
approximates the continuous set of players strategies by a discrete set. The main advantage of this algorithm is its distributed
nature. The inner optimization problem is the original energy
cost minimization in (14). This problem, as shown before, can
be solved in a distributed manner using Algorithm 1. The total
consumption can then be sent to the provider which is responsible for solving the outer problem, i.e., determine his optimal
pricing, thereby preserving the privacy of the users.
First, define
as
then use Algorithm 1 to find the optimum energy schedule corresponding to the current set of prices. In step (4), the set
is updated by the obtained energy schedules. Repeat the steps
until the convergence criteria as explained in [19] is satisfied.
Algorithm 2 Iterative Entropic Algorithm
Choose initial
and let
, Set
Choose
repeat
(1) Find
such that
(27)
(34)
where
is the optimization set defined by the constraints in
(14). We approximate by a finite subset
of points in .
Define
as an approximation for
given by:
(2)
if
is
then
(3) Use Algorithm 1 to find
(28)
The entropic smoothed version of
.
(4) Set
where
(5)
(6) Set
(29)
end if
if
and
then
(7)
(30)
end if
until
and is some regularization constant.
Then, in accordance with the explanation given above, the
inner optimization problem to be solved by the users is
(31)
while the outer optimization to be solved by the utility is
and
More details about the operation of the algorithm and its convergence proof can be found in [19].
V. SIMULATION RESULTS
A. Simulation Scenarios and System Parameters
(32)
where in the outer minimization problem, we minimize
instead of
and
. The algorithm uses the following
terms:
(33)
The algorithm is summarized in Algorithm 2. First, start with
some random energy schedules. At each iteration, the utility
finds the optimum set of prices by solving (34). This problem
involves minimizing the entropy-smoothed version of
up
to some accuracy . In step (3), if the difference between
and its entropy-smoothed version is below a certain threshold,
The system has 5 important parameters:
•
: number of users equipped with storage devices.
•
: Each user with a storage device is given a random
storage capacity at the beginning. This capacity is then
changed by dividing by
to study the effect of decreasing system storage capacity.
•
: This is first time in a day at which users start to make
their energy requests. This parameter is important in differentiating the no storage case from the storage one. The
time before
typically favors storage because it has low
prices.
• Storage-to-Energy Ratio: the ratio of the total storage
available to the total energy requirements. This parameter
allows us to study when storage capacity in the system will
saturate, as well as to determine whether the system has
SOLIMAN AND LEON-GARCIA: GAME-THEORETIC DEMAND-SIDE MANAGEMENT WITH STORAGE DEVICES
1481
an optimum storage capacity that achieves the minimum
PAR.
We are interested in two performance metrics, total cost and
peak-to-average power ratio (PAR).
We will study three different cases. First, when there is no
storage available in the grid and users can only schedule their
use. This is the case studied in [4] using a quadratic cost function. Second, when the users have storage capability and can sell
energy back to the grid. Third, when the users have storage capability, but can store only for their own use and can’t sell back.
Each storage-equipped user is given a random initial storage
capacity which is then varied by varying
. We divide the day
into 24 periods, 1 hour each. Analysis of cyclic behavior through
a duration of multiple days is left for future work.
B. The Energy Consumption Game
The main points we want to emphasize for the Energy Consumption Game are:
• Storage always helps; it outperforms no-storage scheduling in cost and PAR in all cases we have studied.
• The no-selling scenario results in lower PAR than the
selling scenario.
• In the no-selling scenario, there is an optimum number of
users equipped with storage that produces the lowest PAR.
• In the selling scenario, the storage is shared virtually
among all users, i.e., the performance depends on the total
storage capacity and not on how it is distributed among
users.
• Reverse peaks appear, which encourages the shift to Stackelberg games.
1)
&
: Figs. 2 and 3 shows the system performance
versus the number of users equipped with storage
for
different storage capacities
. Note that
corresponds to the first (no-storage) scenario. We plot the results
when
for the selling and no selling scenarios, and when
we decrease the storage capacity by a factor of 4.
When the users are equipped with storage devices, they
have more degrees of freedom in scheduling their appliances,
allowing them to decrease their cost. We can see in Fig. 2 that
the selling and no selling scenario start close to each other, but
soon diverge, implying that the users use selling to decrease
their cost. The gap is greater for increasing storage as more
degrees of freedom are given to the users. Hence, depriving the
users from the option of selling back generally results in higher
cost.
Fig. 3 shows that the PAR either remains the same or is
slightly decreased for the selling scenario. This is due to “reverse” peaks, where users who have storage buy extra energy
ahead of time, and sell it back during peak times. Thus, the
old peak hours become new peak hours for selling rather than
buying. A solution to this problem is to prevent users from
selling back (the third scenario we are studying). We observe a
larger decrease in PAR for the no-selling scenario.
We note that in the selling scenario, each additional user obtaining storage is akin to increasing the total storage capacity
that is shared among all users. This is apparent when we compare the two points with
&
and
Fig. 2. Total cost versus
Fig. 3. PAR versus
for different
for different
.
.
&
which have the same total storage (2) and are almost the same in total cost (66). However, this not the case in
the no-selling scenario. Here users use their storage solely for
their own favor, so the point of
&
has higher
total cost and much higher PAR than the point of
&
, reflecting that users with storage buy less during cheap
times, and the other users have less degrees of freedom. Hence
we conclude that only the total storage matters when users can
sell back. This finding can greatly simplify the problem, as the
solution will be the same as when only one user has storage, resulting in significantly fewer variables. We conclude based on
our simulations that the storage is virtually shared among users
1482
IEEE TRANSACTIONS ON SMART GRID, VOL. 5, NO. 3, MAY 2014
Fig. 4. Comparison of the total cost for the selling and no selling scenarios
when the total amount of storage is a constant.
Fig. 5. Comparison of the PAR for the selling and no selling scenarios when
the total amount of storage is a constant.
when they can sell back, and recommend the use of efficient optimization techniques to make use of this feature. This centralized optimization can be justified from the results of game theoretic competition provided in this paper. We may consider the
application of optimization techniques as part of future work.
Two parameters control the value of the cost function, the
pricing coefficient
determined by the utility, and the users’
consumption. The figures show that initially, only a few users
have storage and are able to shift their consumption to less
expensive times, making use of lower . As more users get
storage, they will buy more at these less expensive times. Eventually, the increase in cost due to users’ consumption will make
up for the decrease in cost due to , leading to the saturation
apparent in the curves. Faster saturation occurs when users have
the option to sell due to the “virtual” sharing of storage explained above in the selling case. We can also see that in the no
selling case, PAR reaches a minimum before increasing again.
This is due to the inability of users to share their storage in the
case of no selling. When almost half the users get storage, the
PAR is minimum. After that, the users shift their consumption
too much leading to an increase in PAR.
2) Energy Requirements vs. Storage Capacity: We now consider the relationship between the total storage capacity in the
system and the aggregate energy demand. Obviously, these are
the two most important parameters in the system and their relationship leads to a better understanding of system performance
as well as savings by avoiding excess storage.
In Figs. 4 and 5, we assume the storage to energy ratio is fixed
for different number of users, so the combined storage capacity
of 2 users is the same as the combined capacity of 10 users.
We consider storage to energy ratios between 10% to 100%. In
Fig. 4, we see that for the selling scenario (solid lines), the total
cost is the same for any number of users, confirming the observation that storage is shared between users. However, in the
no-selling scenario, the cost decreases monotonically as more
users get storage.
In Fig. 5, we see that for the selling scenario, the PAR is constant for any number of users. However, for the no selling scenario, there is an optimum number of users with storage, after
which the PAR will increase again. This optimum number varies
between 2 when storage capacity is low, to 5 when storage capacity is high.
3) Daily Consumption Profile: In Fig. 6, it can be seen that
users in all cases try to avoid the peak hours as much as they
can. Storage allows freedom to avoid buying at peak expensive hours. In the selling scenario, they are actually making use
of these peak prices to sell energy back to make profit. They
achieve lower cost but at the expense of higher PAR. We can
see from Fig. 6 that in the selling scenario, the extra capacity
is virtually for the whole grid. However, in the no-selling scenario, users keep their energy for themselves. This can be seen
in the greater similarity between the curves in the selling scenario (solid lines) than in the no-selling one.
C. Stackelberg Game
1)
&
: In Figs. 7 and 8, we study the effect of increasing the number of users equipped with storage. We can
see that the equilibrium in Stackelberg achieves almost uniform
pricing and consumption, resulting in very low PAR, except for
the case when storage is very low. When prices depend on the
user interaction as in a Stackelberg case, the effect of adding
extra storage to the system, whether by giving storage to more
users or increasing the storage capacity, diminishes quickly after
some threshold. At this threshold, prices become almost uniform and users don’t tend to store much energy, resulting in
PAR very close to the minimum value of 1. We can also see
the matching between the no selling and selling scenarios. This
is because whenever storage is present, users use it to decrease
SOLIMAN AND LEON-GARCIA: GAME-THEORETIC DEMAND-SIDE MANAGEMENT WITH STORAGE DEVICES
Fig. 6. Comparison of different consumption profiles with
Fig. 7. Total cost versus
.
Fig. 8. PAR versus
.
.
using Stackelberg strategy for different
using Stackelberg strategy for different
1483
, with
, with
their cost, but the utility responds by changing its prices, until
an equilibrium is reached.
2) Energy Requirements vs. Storage Capacity: In Figs. 9 and
10, we plot the total cost and PAR versus the storage over energy
ratio in the Stackelberg framework. We can see that the system
reaches the uniform pricing equilibrium after a threshold around
20%. At this point the system has converged to a uniform pricing
and the PAR has become very low.
A similar simulation was performed without employing the
Stackelberg game, and it was found that the total cost of the no
selling scenario when the Stackelberg game is employed is 70,
Fig. 9. The total cost for the selling and no selling scenarios and its relation
with the storage energy ratio using Stackelberg.
and when the Stackelberg game is not employed is 61, while the
PAR when the Stackelberg game is employed is 1.03, and when
the Stackelberg game is not employed is 1.6. For the selling
scenario, the total cost when the Stackelberg game is employed
is 70, and 50 when the Stackelberg game is not employed, while
the PAR when the Stackelberg game is employed is 1.03, and
when the Stackelberg game is not employed is 2.1. These results
show that the Stackelberg game is highly effective in reducing
PAR and they also show how the total cost that results after
the game reaches equilibrium is somewhat higher than the cost
1484
IEEE TRANSACTIONS ON SMART GRID, VOL. 5, NO. 3, MAY 2014
Fig. 10. The PAR for the selling and no selling scenarios and its relation with
the storage energy ratio using Stackelberg.
Fig. 11. Energy consumption profile with
strategy.
,
, using minimax
achieved by the Energy Consumption Game (at much higher
PAR).
3) Daily Consumption Profile: In Fig. 11 we plot the consumption profile and the optimized prices when the Stackelberg strategy is employed. When only one user has storage, the
storage capacity is too low to have uniform pricing. Optimal
prices are low at the beginning of the day, before any user has
any devices operational, and increases after that. During these
cheap times, the user buys energy as much as it can. Later when
prices are higher, the other users come into play. Once 2 or more
users get storage, the system reaches equilibrium at almost uniform pricing, resulting in almost uniform consumption and very
low PAR.
4) Convergence: In Fig. 12 we plot the convergence of the
outer iteration, when the utility receives the consumption of
each user and determines its optimal prices using Algorithm 2.
The algorithm converges in about 25 iterations.
VI. CONCLUSIONS
We have presented a game-theoretic approach to analyze the
interaction between the different users and the utility in the
smart grid in the presence of storage. A continuous non-cooperative game was used to model the interaction between the different users and a Stackelberg game was used to model their interaction with the utility. For the first game, we have shown the
difference between the selling and no-selling scenarios in terms
of cost, PAR and effect of storage. Simulation results show the
decrease in total cost and PAR when storage is present. They
show that depriving the users from the selling option is beneficial in terms of PAR. They also indicate that sometimes it is
better to give storage only to a subset of users, not all of them.
Moreover, they show how selling option makes extra storage
of benefit to all users, which is not the case with the no-selling
scenario. The reverse peak phenomenon motivates a Stackel-
Fig. 12. Convergence of outer loop iterations for the iterative entropic algorithm.
berg game. We provided two algorithms for solving the Stackelberg game and showed its equivalence with the minimum PAR
problem. Simulation results showed the very significant reduction in terms of PAR in the case of Stackelberg game.
APPENDIX
A. Proof of Theorem 3.1
in (5) as well as the local
The proposed cost function
in (18) is a composition of an affine function
one
SOLIMAN AND LEON-GARCIA: GAME-THEORETIC DEMAND-SIDE MANAGEMENT WITH STORAGE DEVICES
with a strictly convex func. From [14], this composition produces a strictly
tion
convex function. From Theorem 1 in [20], the Nash equilibrium of an game exists if it is a concave game defined over a
convex compact set. The set specified by the optimization constraints specifies a closed, bounded, and convex set [14]. Since
our game is strictly concave, cost is strictly convex, defined over
a compact and convex set, then Nash equilibrium exists.
Moreover, from Theorem 6 in [20], uniqueness is guaranteed
if
, where
is the Jacobian matrix of
the cost functions, i.e., diagonally strictly convex (DSC). In our
case, all users are optimizing the same global cost function. It
follows then
is actually the Hessian of the objective func. In other words, DSC reduces to strict contion
vexity when the objective function is the same for all users.
We have just proven that the objective cost function is strictly
convex, hence the payoff function is strictly concave, then its
Hessian is negative and the Nash equilibrium is unique.
B. Proof of Theorem 3.2
Let
be the optimal solution of (14). Then
1485
[10] P. McDaniel and S. W. Smith, “Security and privacy challenges in the
smart grid,” IEEE Security Privacy, vol. 7, no. 3, pp. 75–77, May–Jun.
2009.
[11] G. M. Masters, Renewable and Efficient Electric Power Systems.
Hoboken, NJ, USA: Wiley-IEEE Press, Jul. 2004.
[12] OECD/IEA, “The power to choose: Demand response in liberalised
electricity markets,” International Energy Agency, Organisation for
Economic Co-operation and Development, 2003.
[13] K. Herter, “Residential implementation of critical-peak pricing of electricity,” Energy Policy, vol. 35, no. 4, pp. 2121–2130, April 2007.
[14] S. Boyd and L. Vandenberghe, Convex Optimization. Cambridge,
U.K.: Cambridge Univ. Press, 2004.
[15] S. Kishore and L. Snyder, “Control mechanisms for residential electricity demand in smart-grids,” in Proc. IEEE Int. Conf. Smart Grid
Commun. (SmartGridComm), Gaithersburg, MD, USA, Oct. 2010.
[16] J. Bulow, J. Geanakoplos, and P. Klemperer, “Multimarket oligopoly:
Strategic substitutes and strategic complements,” J. Political Econ.,
vol. 93, no. 3, pp. 488–511, Jun. 1993.
[17] M. Simaan and J. B. Cruz, “On the stackelberg strategy in nonzero-sum
games,” J. Optim. Theory Appl., vol. 11, no. 5, pp. 533–555, May 1973.
[18] S. Zakovic, C. Pantelides, and B. Rustem, “An interior point algorithm
for computing saddle points of constrained continuous minimax,” Ann.
Oper. Res., vol. 99, no. 1–4, pp. 59–77, Dec. 2000.
[19] R. L. Sheu and J. Y. Lin, “Solving continuous min-max problems by
an iterative entropic regularization method,” J. Optim. Theory Appl.,
vol. 121, no. 3, pp. 597–612, Jun. 2004.
[20] J. B. Rosen, “Existence and uniqueness of equilibrium points for concave n-person games,” Econometrica, vol. 33, pp. 347–351, Jul. 1965.
(35)
Multiplying both sides by
Hazem M. Soliman (S’08) was born in Egypt in
1987. He received his B.Sc. and M.Sc. in electronics
and electrical communications engineering with
first class honors from Cairo University, Egypt, in
2009 and 2011, respectively. In 2011, he joined the
Department of Electrical and Computer Engineering
at the University of Toronto, Toronto, ON, Canada,
where he is currently pursuing his Ph.D. degree
in communication networks. His current research
interests include multi-cell MIMO, smart grids,
game theory, virtual networks, and future networks
, we get
(36)
which is the definition of Nash Equilibrium [2]. Hence the globally optimum solution is also the Nash equilibrium point.
REFERENCES
[1] S. M. Amin and B. F. Wollenberg, “Toward a smart grid: Power delivery for the 21st century,” IEEE Power Energy Mag., vol. 3, no. 5,
pp. 34–41, Oct. 2005.
[2] T. Basar and G. J. Olsder, Dynamic Noncooperative Game Theory
(SIAM Series in Classics in Applied Mathematics). Philadelphia, PA,
USA: Society for Industrial and Applied Mathematics (SIAM), Jan.
1999.
[3] W. Saad, Z. Han, H. V. Poor, and T. Basar, “Game-theoretic methods
for the smart grid: An overview of microgrid systems, demand-side
management, and smart grid communications,” IEEE Signal Process.
Mag., vol. 29, no. 5, pp. 86–105, Sep. 2012.
[4] A.-H. Mohsenian-Rad, V. W. S. Wong, J. Jatskevich, R. Schober, and
A. Leon-Garcia, “Autonomous demand-side management based on
game-theoretic energy consumption scheduling for the future smart
grid,” IEEE Trans. Smart Grid, vol. 1, no. 3, pp. 320–331, Dec. 2010.
[5] B. Ramanathan and V. Vittal, “A framework for evaluation of advanced direct load control with minimum disruption,” IEEE Trans.
Power Syst., vol. 23, no. 4, pp. 1681–1688, Nov. 2008.
[6] M. A. A. Pedrasa, T. D. Spooner, and I. F. MacGill, “Scheduling of demand side resources using binary particle swarm optimization,” IEEE
Trans. Power Syst., vol. 24, no. 3, pp. 1173–1181, Aug. 2009.
[7] S. Bu, F. R. Yu, and P. X. Liu, “A game-theoretical decision-making
scheme for electricity retailers in the smart grid with demand-side management,” in Proc. IEEE SmartGridComm, Brussels, Belgium, Oct.
2011.
[8] N. Li, L. Chen, and S. H. Low, “Optimal demand response based on
utility maximization in power networks,” in IEEE Power Energy Soc.
Gen. Meet., San Diego, CA, Jul. 2011.
[9] W. Saad, Z. Han, H. V. Poor, and T. Basar, “A noncooperative game for
double auction-based energy trading between PHEV’s and distribution
grids,” in Proc. IEEE SmartGridComm, Brussels, Belgium, Oct. 2011.
architectures.
Alberto Leon-Garcia (S’74–M’77–SM’97–F’99)
received the B.S., M.S., and Ph.D. degrees in electrical engineering from the University of Southern
California, Los Angeles, CA, USA, in 1973, 1974,
and 1976, respectively.
He was founder and CTO of AcceLight Networks,
Ottawa, ON, Canada, from 1999 to 2002, which developed an all-optical fabric multiterabit, multiservice core switch. He is currently a Professor in Electrical and Computer Engineering at the University of
Toronto, ON, Canada. He holds a Canada Research
Chair in Autonomic Service Architecture. He holds several patents and has
published extensively in the areas of switch architecture and traffic management. His research team is currently developing a network testbed that will
enable at-scale experimentation in new network protocols and distributed applications. He is recognized as an innovator in networking education. In 1986,
he led the development of the University of Toronto-Northern Telecom Network Engineering Program. He has also led in 1997 the development of the
Master of Engineering in Telecommunications program, and the communications and networking options in the undergraduate computer engineering program. He is the author of the leading textbooks Probability and Random Processes for Electrical Engineering and Communication Networks: Fundamental
Concepts and Key Architecture. His current research interests include application-oriented networking and autonomic resources management with a focus on
enabling pervasive smart infrastructure.
Prof. Leon-Garcia is a Fellow of the Engineering Institute of Canada. He
received the 2006 Thomas Eadie Medal from the Royal Society of Canada and
the 2010 IEEE Canada A. G. L. McNaughton Gold Medal for his contributions
to the area of communications.