Pricing the Duck: Cost Allocation and Net Load Variability
Jacob Mays
∗
February 15, 2017
Abstract
A basic principle of electricity market design is that cost allocation should follow cost causation as closely as possible. As renewable resources play an increasingly large role in electricity
generation, system operators have become increasingly concerned about costs related to variability in net load. This paper investigates the extent to which these costs are adequately reflected
in wholesale market prices. Using a Shapley value framework, I assess the total system cost
attributable to buyers of electricity that either exacerbate or alleviate net load variability. Numerical tests indicate that in the most popular price formation schemes currently in use, there
is potential for substantial over- or underpayment relative to cost causation. This not only represents a cross-subsidy, but also may dull the incentive to invest in equipment and implement
operating practices that reduce net load variability. In highlighting the magnitude of the effect,
this article illustrates the need for solutions at the wholesale level.
Keywords: Cost allocation, electricity markets, energy policy, energy storage, demand response, renewables integration
1
Introduction
Recent developments in advanced metering infrastructure and smart devices have created new
opportunities for electricity users to manage their consumption, allowing them to reduce their costs
(by responding to retail prices) or increase revenues (by providing ancillary services or capacity
in wholesale markets). The existing potential of end users to shift load is large in applications
such as water heating, space heating and cooling, agricultural pumping, and wastewater treatment
[23, 16]. Further growth in the ability to shift demand may come with installation of storage [6],
the increasing popularity of electric vehicles [25], and extensions to demand-side bidding formats
[21]. Taken together, these strategies can have a substantial impact on the overall shape of load in
the system [19]. Moreover, in addition to individual benefits for buyers of electricity, demand-side
participation can help the overall health of the market by ensuring reliability in times of peak usage
and preventing the exercise of market power [38].
Increased demand-side participation has become increasingly important with the spread of
renewable resources. Due to a combination of falling costs and policy goals, recent years have seen
∗
Jacob Mays is with the Department of Industrial Engineering & Management Sciences, Northwestern University, Evanston, IL and the Federal Electricity Regulatory Commission (FERC), Washington, DC (email:
[email protected]). Views expressed are not necessarily those of the Commission.
1
rapid growth in wind and solar electricity generation. In 2015, for instance, wind represented 41
percent of capacity additions in the U.S., while solar accounted for 26 percent [39]. Accordingly,
many researchers have begun to investigate the implications of asking these resources to supply
increasingly high percentages of total electricity load, with some studies even proposing that 100
percent renewable generation is possible [17, 36]. Transitioning to these resources can create a
dilemma for grid operators. While the marginal cost of energy from these sources is zero, the
variability and uncertainty that accompany them can lead to increased costs elsewhere in the
system. All other things equal, these costs may limit the share of energy these resources can
economically provide [7]. These concerns are embodied in the “duck chart” published by the
California Independent System Operator (CAISO) [3].
For system operators focused on overall efficiency, the key challenge is to ensure that any
limits to incorporation of new resources are due to their inherent characteristics and not market
design choices. Recognizing this goal, researchers and market operators have proposed an array
of strategies to limit the impact of variability and uncertainty. Broadly speaking, these strategies fall into three groups. The first of these is balancing over larger geographical areas in order to
smooth regional variation in generation and consumption, potentially requiring additional transmission capacity. The second involves improvements to scheduling and dispatch, including improved
forecasting, stochastic and robust optimization models, and better calibration of ancillary services
requirements. The third strategy, and the focus of this paper, is shaping the load curve through
use of improved pricing in combination with storage and demand resources.
Despite a great deal of both interest and technical capability, demand-side participation in
energy markets has been limited to date. In PJM, the largest wholesale market in the U.S., 96.2
percent of demand bids in the day-ahead market in 2015 were submitted without a price, with an
additional 2.3 percent submitted above the generation offer cap of $1,000/MWh [30].1 Explanations
for the disconnect between perceived potential and actual deployment must start with retail rate
design: if end-use customers see flat prices, they have no incentive to shift usage to less expensive
times. Accordingly, moving customers to time-varying retail rates is likely to provide significant
benefits [11]. Largely unexamined, however, is the role of wholesale market price formation. It is
well understood that marginal cost pricing does not fully reflect fixed costs that arise from, e.g.,
generator start-up decisions and minimum operating levels. Several authors have investigated the
implications of these non-convexities for the price formation process; a recent review of proposed
pricing schemes can be found in [20]. However, the focus in general has been on the compensation
given to generators rather than the price signal sent to loads.
This paper turns the attention to the implications of these wholesale pricing decisions for
buyers of electricity. Conceptually, the least expensive way to supply a fixed amount of energy over
1
While price responsive demands could instead participate as Demand Response, these resources represent only
0.015 percent of total energy cleared in 2015. Demand Response plays a larger role as a supplier of capacity and
ancillary services. It should be noted that this evidence does not exclude the possibility that end users respond after
seeing day-ahead or real-time prices. Such responses may do more harm than good, as they a) could occur too slowly,
b) cannot affect the price for the current dispatch interval, and c) necessitate the utilization of regulation services.
2
the course of the day occurs when the net load (i.e., total load minus generation from solar and
wind) is unchanging.2 In this setting, system operators can simply dispatch the least expensive
thermal generating units for the entire day, avoiding much of the cost associated with start-ups,
ramping, and congestion. Accordingly, a consumer that brings the overall system closer to this
ideal (e.g., by matching their consumption to the output of wind) should impart less cost on the
system. By contrast, a consumer that exacerbates variation in net load (e.g., by using more in peak
hours in the early evening) is likely to impart more cost on the system. To highlight the impact
of wholesale pricing decisions, consider loads that see real-time prices. With this assumption, each
customer’s impact should be reflected in the prices they pay: the first will benefit from the low
prices generally seen when wind is at its peak, while the second will pay the higher prices that come
from turning on more expensive units in peak hours. However, it is unclear whether current pricing
schemes fully reflect these impacts. This leads to the motivating question of this study: does the
percentage of costs caused by a consumer match the percentage of payments made by a consumer?
To answer this question, I develop a procedure that holistically measures the cost caused by
a 24-hour load profile in the day-ahead market. An important distinction for this procedure is
between the marginal costs generally used for pricing versus the incremental costs used to assess
causation. Assume that we are given a set of loads, each with a 24-hour usage profile, as well as
an order in which the loads enter the market. The incremental cost attributable to a load can
be calculated as the total dispatch cost after the load enters the market minus the total dispatch
cost before it enters. This change in cost gives a more complete estimate of the cost to serve the
load: if an additional generator is turned on in order to support the new dispatch, the start-up
cost will be directly attributed to that load. This is in contrast to allocating based on marginal
costs, which contain no information as to which load “caused” the generator to be dispatched. A
second distinction is that, whereas prices typically give the marginal cost in a single time period,
the procedure assesses the total cost for the load profile over the course of the day. Due to generator
minimum run times and ramping constraints, the total cost to serve a load profile is not necessarily
equal to the sum of its hourly parts. Throughout the paper, I will refer to this holistic cost measure
as the Total Profile Cost (TPC).
One clear feature of the above procedure is that the incremental cost will depend on the order
in which the loads enter the market. A natural response is to simply repeat the calculation for
every possible entrance order, in the end attributing to each load the average of the resultant
incremental costs. This approach can be formalized using the Shapley value solution concept from
cooperative game theory [32]. In addition to its intuitive appeal, the Shapley value has been shown
to have several features that make it attractive as a cost causation metric. First, two identical
participants will be assessed the same cost. Second, a participant who causes the same incremental
cost regardless of the other participants in the market will be assessed exactly that cost. Third,
the cost measurement for several different services combined (e.g., energy and reserves) will be
2
This is not strictly true in practice since generator offers can change throughout the day, most explicitly in the
case of unplanned outages.
3
the same as the sum of the measurements for each of them calculated individually. As the unique
partition of total cost that satisfies all three of these criteria, the Shapley value has long been seen
as a fair way of dividing total cost [33, 26].
While allocating cost based on the TPC may be appealing, two shortcomings must be addressed. The first is computational feasibility: a precise calculation requires the solution of 2n
unit commitment problems for each day under study. This paper avoids this obstacle by limiting
the analysis to a single-bus system with a small set of market participants. This strategy may
in fact be sufficient for some applications: for example, many utilities are interested in potential
cross-subsidization between a small number of customer classes (e.g., solar and non-solar residential consumers). An alternate strategy for extensions to larger numbers of market participants is
sampling [4]. The second shortcoming is that this holistic cost measure does not produce any useful
signal for short-term decision making. Rather than attempting to contort the calculated value into
such a short-term signal, I use it for post hoc analysis of the price formation process.
Underpinning this analysis is the subjective claim that under a fair cost allocation, the percentage of payments collected from each buyer should match the percentage of costs it causes [29].
If the TPC is accepted as a good measure of cost causation, then one criterion for successful cost
allocation should be conformance with this metric. Such a focus on cost causation is likely to
become increasingly important. In a setting where most market participants have roughly similar
load profiles, there is little reason to suspect significant cross-subsidization. The proliferation of distributed energy resources and increasing interest in time-varying rates, however, could drastically
alter this assumption [28].
Tests performed with data from ERCOT shows the potential for significant divergence from
this ideal. Residential customers with PV systems, representing a small proportion of total demand, alleviate load variability in the system and overpay relative to TPC by 9 percent. Business
customers with low load factors, on the other hand, exacerbate variability and underpay by 6 percent. These seemingly large deviations have important implications for demand-side participation
in the markets: to the extent wholesale prices fail to fully reflect the value of shifting demand, even
improved retail prices will not lead to fair or efficient outcomes.
A simple example as well as general models for measuring cost causation and pricing are developed in Section 2. Data and results of the numerical test are described in Section 3. Conclusions
and possible implications are discussed in Section 4.
2
2.1
Cost Causation Methodology
Motivating Example
Consider a system with two inflexible loads, l1 and l2 . Demand for l1 is 150 MW in hour t1
and 130 MW in hour t2 , while demand for l2 is 90 MW in t1 and 190 MW in t2 . Accordingly,
while both loads use a total of 280 MWh during the dispatch period, l2 contributes to variability
between time periods while l1 partially alleviates it. Load is served by two thermal generators with
characteristics described in Table 1, both of which are offline at the beginning of the period.
4
Table 1: Generator Parameters
Unit
g1
g2
Min Output (MW)
50
30
Max Output (MW)
200
300
Start-up Cost ($)
4,000
0
Energy Cost ($/MWh)
15
20
Since the demand in t2 is greater than can be supplied by either generator individually, it is
clear that both must started. The efficient dispatch for this system, with LMPs calculated by fixing
the binary start-up variables at the optimal solution, is summarized in Table 2.
Table 2: Optimal Dispatch Results
Period
t1
t2
Total
l1 usage
(MWh)
150
130
280
l2 usage
(MWh)
90
190
280
g1 output
(MWh)
200
200
400
g2 output
(MWh)
40
120
160
LMP
($/MWh)
20
20
-
By selling a total of 400 MWh at $20/MWh, g1 earns $8,000. This revenue is insufficient to cover its
operating cost of $6,000 plus start-up cost of $4,000. Wholesale markets typically have a mechanism
for make-whole payments in these situations to ensure generator incentive compatibility. In this
case, the make-whole payment amounts to $2,000. Meanwhile, g2 sets the LMP in both periods and
has no start-up cost. Hence, its revenue and cost are both equal to 160 MWh · $20/MWh = $3, 200.
The total system cost is therefore $13,200.
Because l1 and l2 use the same total quantity of energy and the LMP is the same in both
periods, they each pay 280 MWh · $20/MWh = $5, 600 for energy. It remains to allocate the
make-whole payment. Several potential strategies are available, e.g., allocating to loads based
on consumption 1) in the period during which the start-up cost was incurred, 2) over the entire
planning horizon, or 3) during the peak period. Although allocating based on strategy 1 may make
sense a priori, the result in this case is counterintuitive: despite alleviating load variability, l1 would
be responsible for more of the make-whole payment. However, lacking a standard by which to judge
these or other pricing strategies, it is impossible to say which is “correct.”
2.2
Total Profile Cost
It can be seen that marginal cost gives only a partial assessment of cost causation; at issue is
not only the cost of an additional unit of electricity in each time period, but also the interaction
between the two periods. Accordingly, a holistic measure of cost causation should consider the cost
of the entire load profile. The TPC is just such a metric. As described in Section 1, the idea behind
the TPC is to allow load profiles to enter the dispatch one at a time, calculate the incremental cost
to serve each load, and then take an average over every possible order in which they could enter.
The TPC is equivalent to the Shapley value solution concept in cooperative game theory.
While the theory behind this concept is well developed, its practical application has been limited
due to the complexity of its calculation. Nevertheless, the Shapley value has been widely applied
5
in areas such as transportation, natural resources, and electricity transmission [12, 14]. More
recent uses include evaluating deposit insurance premia in the financial sector [35], assessing input
uncertainty in simulation modeling [34], and establishing prices for cloud computing services [37].
Two studies have explored the use of the Shapley value in the context of power generation. The
analysis of [15] discusses the Shapley value alongside two other concepts from cooperative game
theory, the core and the nucleolus, for the purpose of allocating start-up costs and no-load costs.
Due to its straightforward interpretation as the average incremental cost over all possible orders in
which loads can enter the market, as well as the fairness attributes mentioned in Section 1, I limit
the discussion to the Shapley value. Turning the attention to fair compensation of large numbers of
distributed energy resources, [2] proposes a method to efficiently estimate Shapley values, testing
on coalitions of up to 1200 members. While the numerical study in this paper instead computes
exact values for a smaller number of customer classes, these results do suggest that extension to a
larger system would be possible.
There are at least two ways in which electricity markets can be considered as a cooperative
game: as a cost game in which the set of “players” is only the loads, or as a value game in which
all loads and generators participate. The first more accurately describes a traditional, verticallyintegrated utility, while the second could be more applicable to a wholesale market. We opt for the
first for both computational efficiency and conceptual clarity. The computational advantage arises
due to the nature of the Shapley value, which requires a number of calculations that is exponential in
the number of market participants. The conceptual advantage stems from the limited participation
of the demand side in current markets: any surplus calculation would rely on the assumed value of
electricity to each consumer, potentially clouding any insight from the numerical results. Instead,
we assume that the overall division of surplus between generators and loads remains the same
and investigate the distribution of that surplus among the set of loads. Further, for the sake of
discussion we assume that all loads are fixed; however, the model can be easily extended to allow
price-responsive demand by first computing the optimal dispatch, then fixing the cleared load for
all subsequent calculations.
Previous authors have assessed the “cost of variability” in the context of measuring integration
costs of wind and solar (see, e.g., [22]). It is worth clarifying that in this paper, the cost of variability
arises entirely from customers. More specifically, the required levels of ancillary services capacity
are fixed parameters, and wind and solar can be curtailed. With these assumptions, if wind or
solar were to impose a cost on the system due to its variability, it would simply be left out of the
optimal dispatch.
2.3
Unit Commitment
Given a set L of loads participating in the day-ahead market, let each l ∈ L be characterized
by a vector dl consisting of the quantity demanded for each time period t ∈ T of the day. Further,
P
let d be a vector for the total demand, such that dt = l∈L dlt . With additional notation and a
more complete formulation left to the Appendix, the total cost of the day-ahead market can be
6
expressed as a function of demand:
c(d) =
XX
minimize
u,v,p
(Kg ugt + Sg vgt +
g∈G t∈T
X
Cgs pgst )
(1)
s∈S
XX
subject to
pgst = dt
∀t ∈ T
(2)
∀a ∈ A, t ∈ T
(3)
g∈G s∈S
X
pagt ≥ rta
g∈G
(u, v, p) ∈ G C .
(4)
The objective function in Eq. 1 equals the sum of generator start-up, no-load, and energy
costs over the planning horizon. Equation 2 reflects the power balance constraint, while Eq. 3
guarantees a minimum level of ancillary services. Lastly, Eq. 4 requires that each generator follow
an operating schedule that is technically feasible, following, e.g., minimum run times and ramping
constraints. Since generator start-up decisions and on-off status in each time period are represented
by binary variables, the problem is non-convex.
2.4
Pricing Schemes
For a given d, solving the unit commitment problem gives optimal dispatch decisions for
the day-ahead market. However, in the presence of nonconvexities, market clearing prices do not
in general exist. Several pricing schemes have been proposed to support the efficient dispatch
while limiting side payments. Here we focus on marginal cost pricing with make-whole payments,
described in [24]. Tests on alternate schemes, including the dispatchable and convex hull pricing
schemes in [13], do not appreciably change the results and are included in the Online Appendix.
Under marginal cost pricing, binary variables are fixed at their optimal values and this restricted
model is re-solved. The dual variables λt associated with the power balance constraints in Eq. 2 and
λat associated with the ancillary services requirements in Eq. 3 then become the prices for energy
and ancillary services in each time period. Lastly, a make-whole payment δg for each generator can
be calculated as a function of the dispatch instructions to ensure at least zero profit within each
commitment interval.
In line with typical practice, assume that loads are allocated energy charges based on their
consumption in each period, ancillary services charges based on their load share in each period,
and make-whole payments based on their load share for the entire day. Define total demand for the
P
P DEMl
day by load l as DEMl = t∈T dlt , as well as the share of total consumption DEM%
l =
DEMl .
l∈L
Then, the total payment PMTl made by load l can be calculated as
PMTl =
X
t∈T
λt · dlt +
XX
λat · Rta ·
t∈T a∈A
dlt X
+
δg · DEM%
l .
dt
g∈G
Lastly, the percentage of overall payments charged to load l is PMT%
l =
7
P PMTl
.
l∈L PMTl
(5)
2.5
Load Profiles
The goal of this paper is to assess how the total cost of serving a 24-hour load profile differs
from the sum of its hourly parts. This task amounts to partitioning the cost c(d) across the load
profiles dl . Rather than attempting to calculate the TPC directly, however, the procedure breaks
each load profile into two parts: a reference profile matching the overall demand contour for all
the loads in the market, plus a deviation profile reflecting the hourly difference between the actual
load and reference profile.
While this split is useful conceptually to isolate the impact of variability, it is also significant
both philosophically and pragmatically. In most settings, calculation of the Shapley value proceeds
by starting from zero market participants and adding one at a time, calculating the value of the
coalition at each step. This can cause problems in the context of realistic electricity models, since
solutions to the unit commitment problem at very low levels of demand have no correspondence
to physical reality. Without redefining the required level of ancillary services, altering the status
of generators at the beginning of the planning horizon, and addressing likely violations of AC
power flow constraints, the information in these solutions is of questionable meaning. Instead, the
procedure starts with all of the load in the system, calculating the cost of the deviations. With this
re-characterization, the physical meaning of the solutions is retained: deviations tend to be small
relative to overall consumption, so the total load on the network stays closer to its original level.
Computationally, relying on relatively smaller deviations instead of the entrance of the entire load
profile allows for better warm starting of the mixed-integer program. In extensions to larger sets
of market participants, which would require Monte Carlo sampling, this method is also likely to
produce lower variance estimates of the TPC.3
Define for each load and time period a reference profile d̄l with
d¯lt = dt · DEM%
l .
(6)
This reference profile is equal to the overall demand for that time period multiplied by the fraction
of total daily demand represented by load l. Accordingly, the reference load profiles for all loads
sum to precisely the same total demand d as the original load profiles. Then, define the deviations
from this reference profile ∆dl , with
∆dlt = dlt − d¯lt .
2.6
(7)
Total Profile Cost Calculation
If all loads follow their reference profile, i.e., identical profiles scaled by share of total demand,
it is reasonable to attribute to each user the total system cost times their share of total demand.
Define this cost for each load l as φ̄l (c) = c(d) · DEM%
l . More complicated is the cost of each load’s
deviation from its reference profile. Let the set R represent all possible permutations of the load
set L, and let P r represent the set of loads that precede l in order r ∈ R. The Shapley value,
3
Note that a TPC could calculated from any reference profile. For example, an alternative reference profile is one
that assumes completely flexible demand, allowing perfect matching to the contour of wind and solar generation.
8
measuring the average incremental cost of the deviation, can be expressed as
"
#
X
X
1 X
φl (c) =
c(d +
∆di + ∆dl ) − c(d +
∆di ) .
|L|!
r
r
r∈R
i∈P
(8)
i∈P
In each permutation, the incremental cost is found by taking the difference in dispatch cost after
including the deviation of load l with all of its predecessors. Since inclusion of all deviations
by construction results in the same total load as including no deviations, it can be seen that
P
L φl (c) = 0.
The total profile cost TPCl attributable to load l can then be calculated as the cost of its
reference profile plus the cost of its deviation; i.e., TPCl = φ̄l (c) + φl (c). Write the corresponding
share of total cost causation as TPC%
l =
2.7
P TPCl
.
l∈L TPCl
Internalization of Costs and Benefits
The goal of this paper is to assess how well a given cost allocation scheme, such as that de-
scribed in Section 2.4, matches the cost causation metric developed in Section 2.6. As stated in
Section 1, this analysis assumes that an ideal cost allocation collects payments in equal proportion
to cost causation. Since total payment is greater than total system cost, with the difference representing generator profit, both are taken in percentage terms. Overpayment can then be defined
as
OVR%
l =
2.8
PMT%
l
TPC%
l
− 1.
(9)
Example Calculation
With the procedure in place, it is possible to return to the example system proposed in
Section 2.1. Since the total demand grows from 240 MW in t1 to 320 MW in t2 , and DEMl%
=
1
DEMl%
= 0.5, the reference profile for both loads is 120 MW in t1 and 160 MW in t2 . Having
2
already calculated the total system cost c(d) = $13, 200 in the optimal dispatch, we know that
φ̄l1 (c) = φ̄l2 (c) = $6, 600.
The procedure now turns to the cost of the load deviations. Since there are only two loads,
there are two possible orders in which they can enter the dispatch. First, consider if l1 enters first.
The deviation for l1 , computed from Eq. 7, is +30 MW in t1 and -30 MW in t2 . Adding this
deviation leads to a total demand of 270 MW in t1 and 290 MW in t2 . The optimal dispatch with
this demand is summarized in Table 3.
In this scenario, with lower load variability, it is no longer required to start g1 . Despite the
higher energy cost, the avoided start-up cost means that total system cost falls to 560 MWh ·
$20/MWh = $11, 200. Comparing the system cost before and after adding the deviation, the
incremental cost caused by the deviation of l1 in this ordering amounts to -$2,000. Next, the
deviation for l2 is -30 MW in t1 and +30 MW in t2 . Accordingly, when it is added to the system,
the total demand is restored to its original value in each period. It follows that the incremental
9
Table 3: Optimal Dispatch Results with l1 Deviation
Period
t1
t2
Total
l1 usage
(MWh)
150
130
280
l2 usage
(MWh)
120
160
280
g1 output
(MWh)
0
0
0
g2 output
(MWh)
270
290
560
LMP
($/MWh)
20
20
-
cost caused by the deviation of l2 in this permutation is $2,000.
Now consider the opposite order, in which l2 is added first. After including the deviation of
l2 , the total system demand is 210 MW in t1 and 350 MW in t2 . The optimal dispatch for this
scenario is summarized in Table 4.
Table 4: Optimal Dispatch Results with l2 Deviation
Period
t1
t2
Total
l1 usage
(MWh)
120
160
280
l2 usage
(MWh)
90
190
280
g1 output
(MWh)
180
200
380
g2 output
(MWh)
30
150
180
LMP
($/MWh)
15
20
-
With higher load variability, g1 must be dispatched below its maximum output to accommodate
the minimum operating level of g2 . Total system cost in this case comprises the $4,000 startup cost of g1 and energy costs of 380 MWh · $15/MWh + 180 MWh · $20/MWh = $9, 300, for
a total of $13,300. This translates to an incremental cost of $100 for l2 in this permutation,
with a corresponding incremental cost of -$100 for l1 . Having exhausted the potential orders,
compute φl1 (c) = (−$2, 000 − $100)/2 = −$1, 050 and φl2 (c) = $1, 050. Accordingly, the total
cost attributable to the two loads are TPCl1 = $6, 600 − $1, 050 = $5, 550 and TPCl2 = $6, 600 +
$1, 050 = $7, 650.
These TPC values are used to evaluate the overpayment resulting from the pricing scheme in
Section 2.4. Since the uplift payment is split based on total usage over the dispatch period, we can
calculate PMTl1 = PMTl2 = $5, 600 + $1000 = $6, 600. The resulting values for DEMl% , P M Tl% ,
T P Cl% , and OV Rl% are shown in Table 5.
Table 5: Distributive Efficiency in Example System
Load
l1
l2
Demand %
(DEMl% )
50.0
50.0
Payments %
(P M Tl% )
50.0
50.0
Causation %
(T P Cl% )
42.0
58.0
Overpayment %
(OV Rl% )
19.0
-14.0
The TPC methodology rigorously confirms the intuition that l2 , with its greater contribution to
load variability, has more responsibility for the total system cost. The price formation scheme in
use, however, does not reflect this responsibility, allocating equal cost to both loads. Relative to
the idealized split of costs reflected in the TPC, l1 overpays by 19 percent while l2 underpays by
10
14 percent. Moreover, a different rule for allocating uplift is guaranteed to be insufficient in this
example: the energy cost assigned to l1 by itself exceeds its TPC before any uplift charges are
included.
3
Numerical Study
While the example system can provide an illustration of the problem and the methodology,
it can also lead to misleading conclusions. In particular, uplift represents a much smaller portion
of total payments in typical real-world systems, averaging from $0.04 to $0.16/MWh in U.S. dayahead markets [5]. In order to better characterize the interaction between the shape of the load
curve and pricing, I constructed a test system for a set of loads participating in a single-bus system
with demand and generation characteristics modeled on ERCOT. Several system characteristics are
reported directly by ERCOT; the test uses actual wind generation, solar generation, and required
levels of each ancillary service in ERCOT for Tuesday, July 19, 2016 [9]. More complicated tasks
are the development of a set of generators with realistic cost structures and technical attributes
and a set of loads that accurately reflects the diversity of customers in the system. The resulting
data set, as well as details of its development, is included in the Online Appendix.
3.1
Generator Attributes
A complete set of offers and technical attributes for thermal generators was taken from the
FERC Unit Commitment Test System [18]. Since this test system is somewhat larger than ERCOT,
a subset of 456 generators approximating the total installed capacity of each fuel source in ERCOT
was selected [8]. Since the generator data comes from outside ERCOT and from a different year,
note that the priority is not to precisely match the actual cost seen in the ERCOT system, but
to appropriately reflect the balance between start-up, no-load, energy, and ancillary services costs
in the system. Separate bids for the provision of ancillary services were not available; the price
of each ancillary service is assumed to be entirely determined by the lost opportunity cost from
energy sales.
This set of thermal generators is supplemented by the actual hourly output of wind and solar
on the modeled day. This maximum output is treated as completely curtailable.
3.2
Load Profiles
ERCOT divides end users in the region into 28 different segments and 8 geographical zones
based on historical consumption patterns, for a total of 28 · 8 = 224 potential load profiles. In
addition to providing the number of meters assigned to each segment and geography on a regular
basis, ERCOT publishes a backcasted daily load profile for each customer segment and geography
[10]. Since precise interval data are not available for every customer, a sum of all these backcasted
profiles does not precisely match the total usage in the system; nevertheless, they provide a good
indication as to the division of total consumption between customer classes. In order to find a
subset of these 224 profiles that represents the diversity of customers in the system without adding
undue computational time, three steps were taken. First, I chose to focus on diversity between
11
customer classes rather than diversity between geographies. By taking a weighted average of each
segment profile based on their prevalence in the 8 zones, I produced a composite profile for each
customer class. Second, I manually reduced this set of 28 profiles down to 10 by removing several
with very low representation and combining similar profiles. Third, I sought a linear combination
of the remaining 10 that matched the actual hourly load seen in ERCOT as closely possible, with
each profile constrained to represent between 0 and 20% of total demand. This process resulted in
a set of 7 loads that provide a nearly precise match to actual load (R2 = 0.999). Figure 1 depicts
the contribution of the seven load profiles used in the study to total demand.
Figure 1: Load Profiles in ERCOT Test System
The shape of each individual load curve is more easily distinguished in Figure 2. Each load is
normalized to display its demand in the given hour relative to its average demand throughout the
day.
While ERCOT names are retained for the load profiles, some license is taken in the following
descriptions of the less straightforward labels. The Flat profile corresponds to certain industrial
users (e.g., Oil and Gas) that operate at consistent levels, while Business-High Load Factor profiles
correspond to other large industrial users. The Business-Low Load Factor depicts a more typical
commercial profile, while the Business-Low Consumption profile is an amalgam of many classes of
users with low demand. The inclusion of street lighting in this class likely explains the atypical
decrease in consumption from 7-8 AM and increase from 8-10 PM exhibited in its load profile.
3.3
Results
The methodology in Section 2 was then applied to this more realistic test system. The total
consumption of each load, in addition to results for the payment, TPC, and overpayment calculations, are summarized in Table 6.
12
Figure 2: Normalized Load Profiles in ERCOT Test System
Table 6: Distributional Effects in ERCOT Test System
Load
Flat
Business-High Load Factor
Business-Low Load Factor
Business-Low Consumption
Residential-High Load Factor
Residential-Low Load Factor
Residential-PV System
Demand %
(DEMl% )
20.0
16.3
15.9
4.9
20.0
20.0
2.8
Payments %
(P M Tl% )
18.9
15.8
16.8
4.8
20.5
20.7
2.6
Causation %
(T P Cl% )
17.5
15.0
18.0
4.6
21.0
21.5
2.4
Overpayment %
(OV Rl% )
7.7
4.8
-6.2
4.6
-2.7
-3.9
9.4
Notes: Demand shares represent coefficients found in regression rather than actual level in ERCOT system.
It can be seen that the load profiles with relatively higher usage in the early morning and late
evening—Flat, Business-High Load Factor, Business-Low Consumption, and Residential-PV System—are able to take advantage of the lower prices seen in those times; accordingly, their share of
total payments is lower than their share of total demand. However, as in the simple example in
Section 2, the share of cost causation as measured by the TPC is lower in each case. The opposite holds for the other three profiles, which have relatively higher usage near the system peak at
4-5 PM. LMPs in the test system ranged from $29/MWh at 4 AM to $153/MWh at 4 PM. This
variability in prices is an incomplete reflection of the cost of variability: results for the individual
customer classes range from an underpayment of 6.2 percent for the Business-Low Load Factor
profile to an overpayment of 9.4 percent for the Residential-PV System profile.
13
3.4
Alternative Price Formation Strategies
The cost of a deviation profile can be interpreted as the value of shifting consumption from
some periods to others. Under this interpretation, the overpayment calculation is an unsurprising
result of the fact that total payments are larger than total costs. As long as prices are close to
marginal cost, the value of shifting will represent a smaller percentage of total payments than it
does of total cost. However, the potential size of this effect is under-recognized in current discourse.
While marginal cost pricing remains a dominant paradigm, several system operators have
explored alternate pricing schemes that better reflect start-up and no-load costs. Tests on existing
proposals described in [13] produce only moderate changes in prices and do not affect the overall
conclusion. Potentially more successful would be Ramsey–Boiteux pricing, which sets higher prices
to customers with lower elasticity [31, 1]. Informally, customers with higher relative consumption
in peak periods can be thought of as having lower elasticity. Because it implies different prices for
different customers, most system operators have avoided Ramsey–Boiteux pricing out of fairness
considerations. However, the results of the TPC analysis suggest than an insistence on micro-level
fairness within each time period leads to macro-level unfairness over the entire day.
4
Conclusion
Starting from the premise that a fair cost allocation would match cost causation, the results
provide compelling evidence that the cost of variability is underestimated by existing price formation
schemes. This presents two issues for electricity market operators. First, to the extent customers
follow consistent usage patterns throughout the year, cross-subsidies will arise. Second, as long as
part of the cost of load variability is socialized, end-users of electricity will under-invest in reducing
it.
The analysis is presented in terms of cost allocation to a set of general load profiles. I maintain
this generality since diversity of load profiles can arise due to any number of reasons, e.g., geography,
underlying consumer preferences, retail rate design, or installation of distributed energy resources.
However, potentially the most direct implications of the results arise in the context of storage
and demand side resources. Studies of the potential of energy storage and demand response are
contingent on the value of energy arbitrage, i.e., shifting consumption from higher-cost to lower-cost
hours. The results in the test system suggest that under a cost allocation framework that matched
the TPC, revenue for demand side resources resulting from energy arbitrage could be more than
twice as large.
While this study identifies a potentially large source of inefficiency in electricity market cost
allocation, it stops short of offering a solution. Given the array of system-specific settlement
processes currently in use, as well as the competing priorities of the price formation process, a
“one-size-fits-all” approach may not be forthcoming. Nevertheless, the results indicate that system
operators concerned with net load variability would be justified in investigating a holistic cost
measure like the TPC.
14
References
[1] M. Boiteux. Sur la gestion des monopoles publics astreints a l’equilibre budgetaire. Econometrica, 24(1):22–40, 1956.
[2] Jörg Bremer and Michael Sonnenschein. Estimating Shapley Values for Fair Profit Distribution
in Power Planning Smart Grid Coalitions, pages 208–221. Springer Berlin Heidelberg, Berlin,
Heidelberg, 2013.
[3] California ISO. What the duck curve tells us about managing a green grid. Available at https:
//www.caiso.com/Documents/FlexibleResourcesHelpRenewables_FastFacts.pdf, 2013.
[4] Javier Castro, Daniel Gómez, and Juan Antonio Tejada Cazorla. Polynomial calculation of
the shapley value based on sampling. Computers and Operations Research, 36(5):1726–1730,
May 2009.
[5] Federal Energy Regulatory Commission. Staff analysis of uplift in RTO and ISO markets.
Technical report, August 2014.
[6] Fernando J. de Sisternes, Jesse D. Jenkins, and Audun Botterud. The value of energy storage
in decarbonizing the electricity sector. Applied Energy, 175:368 – 379, 2016.
[7] Paul Denholm, Kara Clark, and Matt OConnell. On the path to sunshot: Emerging issues
and challenges in integrating high levels of solar into the electrical generation and transmission
system. Available at http://www.nrel.gov/docs/fy16osti/65800.pdf, May 2016.
[8] ERCOT. Report on the capacity, demand and reserves (CDR) in the ERCOT region, 20162025. Available at http://www.ercot.com/content/gridinfo/resource/2015/adequacy/
cdr/CapacityDemandandReserveReport-December2015.pdf, December 2015.
[9] ERCOT. Grid information. Available at http://www.ercot.com/gridinfo, 2016.
[10] ERCOT. Load profiling. Available at http://www.ercot.com/mktinfo/loadprofile, 2016.
[11] Ahmad Faruqui, Ryan Hledik, and Jennifer Palmer. Time-varying and dynamic rate design.
Available at http://www.raponline.org/document/download/id/5131, July 2012.
[12] M. Gloria Fiestras-Janeiro, Ignacio Garcı́a-Jurado, and Manuel A. Mosquera. Cooperative
games and cost allocation problems. TOP, 19(1):1–22, 2011.
[13] P. Gribik, W. W. Hogan, and S. L. Pope. Market-clearing electricity prices and energy uplift.
December 2007.
[14] Mario Guajardo and Mikael Rönnqvist. A review on cost allocation methods in collaborative
transportation. International Transactions in Operational Research, 23(3):371–392, 2016.
[15] Zhaoyang Hu, Lin Chen, Deqiang Gan, and D. Chattopadhyay. Allocation of unit start-up
costs using cooperative game theory. IEEE Transactions on Power Systems, 21(2):653–662,
May 2006.
[16] Marissa Hummon, David Palchak, Paul Denholm, Jennie Jorgenson, Daniel Olsen, Sila Kiliccote, Nance Matson, Michael D. Sohn, Cody Rose, Junqiao Han Dudley, Sasank Goli, and
Ookie Ma. Grid integration of aggregated demand response, part 2: Modeling demand response
in a production cost model. Technical report, December 2013.
15
[17] Mark Z. Jacobson, Mark A. Delucchi, Mary A. Cameron, and Bethany A. Frew. Low-cost
solution to the grid reliability problem with 100% penetration of intermittent wind, water,
and solar for all purposes. Proceedings of the National Academy of Sciences, 112(49):15060–
15065, 2015.
[18] Eric Krall, Michael Higgins, and Richard O’Neill. RTO unit commitment test system. Technical
report, July 2012.
[19] Jim Lazar. Teaching the duck to fly, second edition. Available at http://www.raponline.
org/document/download/id/7956, February 2016.
[20] George Liberopoulos and Panagiotis Andrianesis. Critical review of pricing schemes in markets
with non-convex costs. Operations Research, 64(1):17–31, 2016.
[21] Yanchao Liu, Jesse T. Holzer, and Michael C. Ferris. Extending the bidding format to promote
demand response. Energy Policy, 86(C):82–92, 2015.
[22] Michael Milligan, Erik Ela, Bri-Mathias Hodge, Brendan Kirby, Debra Lew, Charlton Clark,
Jennifer DeCesaro, and Kevin Lynn. Integration of variable generation, cost-causation, and
integration costs. The Electricity Journal, 24(9):51 – 63, 2011.
[23] Daniel Olsen, Nance Matson, Michael D. Sohn, Cody Rose, Junqiao Han Dudley, Sasank Goli,
Sila Kiliccote, Marissa Hummon, David Palchak, Paul Denholm, Jennie Jorgenson, and Ookie
Ma. Grid integration of aggregated demand response, part 1: Load availability profiles and
constraints for the western interconnection. Technical report, September 2013.
[24] Richard P. O’Neill, Paul M. Sotkiewicz, Benjamin F. Hobbs, Michael H. Rothkopf, and
William R. Stewart. Efficient market-clearing prices in markets with nonconvexities. European
Journal of Operational Research, 164(1):269–285, July 2005.
[25] M. A. Ortega-Vazquez. Optimal scheduling of electric vehicle charging and vehicle-to-grid services at household level including battery degradation and price uncertainty. IET Generation,
Transmission Distribution, 8(6):1007–1016, June 2014.
[26] Martin Osborne and Ariel Rubinstein. A Course in Game Theory, volume 1. The MIT Press,
1 edition, 1994.
[27] A. Papavasiliou, S. S. Oren, and R. P. O’Neill. Reserve requirements for wind power integration:
A scenario-based stochastic programming framework. IEEE Transactions on Power Systems,
26(4):2197–2206, Nov 2011.
[28] I. J. Perez-Arriaga. The transmission of the future: The impact of distributed energy resources
on the network. IEEE Power and Energy Magazine, 14(4):41–53, July 2016.
[29] PJM.
Principles of cost causation and cost allocation.
Available at https:
//www.pjm.com/~/media/committees-groups/task-forces/emustf/20141113/
20141113-item-03c-cost-causation-slides.ashx, 2014.
[30] PJM.
Hourly day-ahead demand bids.
Available at http://www.pjm.com/
markets-and-operations/energy/real-time/historical-bid-data/demand-bid.aspx,
August 2016.
[31] F. P. Ramsey. A contribution to the theory of taxation. The Economic Journal, 37(145):47–61,
1927.
16
[32] L. S. Shapley. A Value for n-person Games, pages 307–318. Princeton University Press, 1953.
[33] Martin Shubik. Incentives, decentralized control, the assignment of joint costs and internal
pricing. Management Science, 8(3):325–343, 1962.
[34] Eunhye Song, Barry L. Nelson, and Jeremy Staum. Shapley effects for global sensitivity analysis: Theory and computation. SIAM/ASA Journal on Uncertainty Quantification, 4(1):1060–
1083, 2016.
[35] Jeremy Staum. Systemic risk components and deposit insurance premia. Quantitative Finance,
12(4):651–662, 2012.
[36] Joshua Adam Taylor, Sairaj V. Dhople, and Duncan S. Callaway. Power systems without fuel.
CoRR, abs/1506.04799, 2015.
[37] Prasang Upadhyaya, Magdalena Balazinska, and Dan Suciu. How to price shared optimizations
in the cloud. CoRR, abs/1203.0059, 2012.
[38] U.S. Department of Energy. Benefits of demand response in electricity markets and recommendations for achieving them. Technical report, February 2006.
[39] U.S. Energy Information Agency. Preliminary monthly electric generator inventory. Available
at http://www.eia.gov/electricity/data/eia860m/.
A
Notation for Unit Commitment Problem
Notation used for the problem in Equations 1-4 is adapted from [27].
Sets:
• G: set of generators
• S: set of bid steps
• L: set of loads
• T : set of time periods
• A: set of ancillary services
• G C : set of generator capacity constraints, ramping constraints, and minimum up and down
times
Elements of set A:
• regu: regulation up service
• regd: regulation down service
• spin: spinning reserve service
• nspin: non-spinning reserve service
Variables:
17
• ugt : (binary) commitment status of generator g in time period t
• vgt : (binary) startup decision for generator g in time period t
• wgt : (binary) shutdown decision for generator g in time period t
• pgst : offer cleared for generator g at bid step s in time period t
• pgt : total offer cleared for generator g at all bid steps in time period t
• pagt : amount of ancillary service a provided by generator g in time period t
Parameters:
• n: number of time periods
• Kg : no load cost for generator g
• Sg : startup cost for generator g
• Cgs : marginal cost for generator g in bid step s
of f er
: quantity offered by generator g in bid step s
• Pgs
• Pg+ , Pg− : maximum and minimum operating level for generator g
• Rga+ : maximum capacity of generator g to supply ancillary service a ∈ A
• Ug+ , Dg+ : maximum up and down per-period ramping capability of generator g
• U T g : minimum up time of generator g
• DT g : minimum down time of generator g
• dt : demand in period t
• rta : amount of ancillary service a ∈ A required in time period t
18
B
Formulation for Unit Commitment Problem
minimize
u,v,p
XX
(Kg ugt + Sg vgt +
g∈G t∈T
X
Cgs pgst )
(10)
s∈S
XX
pgst = dt
∀t ∈ T
(11)
pagt ≥ Rta
∀a ∈ A, t ∈ T
(12)
∀g ∈ G, s ∈ S, t ∈ T
(13)
∀g ∈ G, t ∈ T
(14)
pgt − pregd
≥ Pg− ugt
gt
∀g ∈ G, t ∈ T
(15)
pgt + pregu
+ pspin
≤ Pg+ ugt
gt
gt
∀g ∈ G, t ∈ T
(16)
vgt + ug,t−1 − ugt − wgt ≥ 0
∀g ∈ G, t ∈ T
(17)
vgq
∀g ∈ G, t ∈ T
(18)
wgq
∀g ∈ G, t ∈ T
(19)
pgt ≤ pg,t−1 + Ug+ + Pg+ vgt
∀g ∈ G, t ∈ T
(20)
pgt ≥ pg,t−1 − Dg+ − Pg+ wgt
∀g ∈ G, t ∈ T
(21)
∀g ∈ G, a ∈ A, t ∈ T
(22)
∀g ∈ G, s ∈ S, a ∈ A, t ∈ T
(23)
∀g ∈ G, t ∈ T
(24)
subject to
g∈G s∈S
X
g∈G
of f er
pgst ≤ Pgs
X
pgst = pgt
s∈S
ugt ≥
t
X
q=max(1,t−U T g +1)
ugt ≤ 1 −
t
X
q=max(1,t−DT g +1)
pagt ≤ Rga+
pgst , pgt , pagt ≥ 0
ugt , vgt , wgt ∈ {0, 1}
The formulation closely adheres to that found in [18], except for the exclusion of transmission
constraints. For notational simplicity, some details are omitted; the full AMPL implementation
is available in the Online Appendix. Equation 10 includes start-up, no-load, and energy costs
split into up to ten piecewise constant bid steps. Equation 11 ensures power balance, while Eq.
12 guarantees procurement of sufficient ancillary resources of each type. Demand-side resources
19
supply a significant share of spinning reserve service in ERCOT; this provision is subtracted from
the spinning reserve requirement. Excess spinning reserves count against the non-spinning reserve
requirement. Equations 13 constrains cleared power to the level offered by each generator, and Eq.
14 substitutes a dummy variable to represent the total power across all bid steps. Equations 15 and
16 ensure consistency between generator provision of energy and reserves. Equations 17-19 ensure
consistent relationships between startup, shutdown, and commitment variables given minimum up
and down times. Where appropriate, substitutions are made to account for the on-off status of
generators prior to period 1. Equations 20 and 21 incorporate ramping constraints. Equation
22 limits the capacity of reserves supplied by each generator. By default, it is assumed that all
thermal generators are able to supply up and down regulation capacity equivalent to five times
the relevant per-minute ramp rate and spinning reserves equivalent to ten times their per-minute
upward ramp rate. All offline combined cycle turbines and gas turbines are assumed to be able to
supply non-spinning reserves up to their maximum output.
20