The Valuation of Natural Gas Storage:
A Knowledge Gradient Approach with
Non-Parametric Estimation
By: Jennifer Schoppe
Advisor: Professor Warren B. Powell
Submitted in partial fulfillment
Of the requirements for the degree of
Bachelor of Science in Engineering
Department of Operations Research and Financial Engineering
Princeton University
June 2010
I hereby declare that I am the sole author of this thesis.
I authorize Princeton University to lend this thesis to other institutions or
individuals for the purpose of scholarly research.
________________________________________
Jennifer Schoppe
I further authorize Princeton University to reproduce this thesis by photocopying or
by other means, in total or in part, at the request of other institutions or individuals
for the purpose of scholarly research.
________________________________________
Jennifer Schoppe
ii
Dedication
To my unforgettable Princeton experience,
my parents who gave me the opportunity to take the journey,
and my sister who was with me through every moment.
iii
Acknowledgements
There are many people who helped bring this thesis into existence, and I
would like to extend my gratitude though the thanks they deserve extends beyond
what can be expressed on this page.
Within the ORFE Department, I would first like to thank my advisor
Professor Warren Powell for his enthusiasm, insight, and encouragement to make
my thesis better. My gratitude extends to Dr. Hugo Simao for his computational
expertise and Michael Coulon for his knowledge on energy pricing processes. A
special thanks goes to Emre Barut, for allowing this thesis the opportunity to be one
of the first to test his research in non-parametric estimators for knowledge gradient
methods. His time and energy spent in providing both computational and
theoretical support is greatly appreciated.
I would like to thank James Schoppe at Iberdrola Renewables and Richard
Adamczyk at Enstor Operating Company for providing me with invaluable resources
regarding the natural gas industry and storage.
I would like to acknowledge Doug Eshleman and Mike Hasling for their time
and patience in answering my questions as I navigated through the world of
computer programming.
I would also like to thank Stephanie Schoppe for copyediting this work.
I have been blessed to have many people in my life who have offered me love
and encouragement through all my achievements including the creation of this
thesis. They are my parents, my grandparents, my sister, and my friends. Many
thanks to all.
iv
Abstract
This thesis examines the problem of natural gas storage valuation focusing
on the valuation of high-deliverability gas storage facilities, specifically salt cavern
storage. We present capable valuation analysis and optimization methodologies that
accurately account for the various operating characteristics of real storage facilities
as well as the complex stochastic nature of natural gas prices. We use an extension
of the Knowledge Gradient Algorithm, a sequential learning method, which
incorporates non-parametric estimation in order to present a method that produces
a quality valuation with minimal computational burden. The results of the method
will present an optimal strategy for the operation of the natural gas storage facility,
as well as yield a value for the facility.
v
Table of Contents
CHAPTER 1 Introduction ..................................................................................................................... 1
1.1 Natural Gas: From Wellhead to Market ................................................................................. 4
1.1.1
Exploration and Extraction ....................................................................................... 4
1.1.2
Production and Transport ........................................................................................ 7
1.1.3
Storage .............................................................................................................................. 8
1.1.4
Distribution and Marketing ...................................................................................... 8
1.1.5
Natural Gas Markets .................................................................................................... 9
Natural Gas Spot Market ............................................................................................................10
Natural Gas Futures Market .....................................................................................................10
1.2
Traditional Use of Underground Natural Gas Storage ......................................... 11
1.2.1
Seasonal Cycling ..........................................................................................................11
1.2.2
Peaking Services .........................................................................................................13
1.2.3
Speculative Market Services...................................................................................13
1.3
Types of Underground Natural Gas Storage ............................................................ 14
1.3.1
Depleted Reservoirs ..................................................................................................15
1.3.1
Aquifers ..........................................................................................................................17
1.3.2
Salt Caverns ..................................................................................................................18
1.4
Storage Contract ................................................................................................................. 20
1.5
Overview ................................................................................................................................ 21
CHAPTER 2 Literature Review ........................................................................................................22
2.1
Spread Option Optimization Approach ..................................................................... 23
2.2
Approximate Dynamic Programming (ADP) Approach ...................................... 25
CHAPTER 3 Knowledge Gradient Methods ................................................................................30
3.1
Knowledge Gradient with Independent Measurements ..................................... 31
3.2
Knowledge Gradient for Correlated Beliefs ............................................................. 34
3.3
Knowledge Gradient with Non-Parametric Estimation ...................................... 38
3.3.1
Kernel Estimation .......................................................................................................38
3.3.2
KGNP Procedure and Derivation ..........................................................................41
CHAPTER 4 Pricing Models...............................................................................................................44
vi
4.1
Geometric Brownian Motion ......................................................................................... 44
4.2
Spot Price Process .............................................................................................................. 45
Strength of Mean Reversion (α) ...............................................................................................48
Exponential Smoothing...............................................................................................................49
Volatility...........................................................................................................................................50
Simulation Results ........................................................................................................................51
4.3
Forward Price Process ..................................................................................................... 52
CHAPTER 5 Model and Policies .......................................................................................................56
5.1
Modeling Assumptions ..................................................................................................... 57
5.2
Problem Set-Up and Terminology ............................................................................... 58
5.2.1
Parameters ....................................................................................................................59
5.2.2
Decision Variables and Constraints .....................................................................60
5.2.3
Exogenous Variables .................................................................................................62
5.2.4
System State .................................................................................................................62
5.2.5
Transition Functions .................................................................................................63
5.2.6
Contribution Function ..............................................................................................64
5.2.7
Objective Function .....................................................................................................65
5.3
Policies.................................................................................................................................... 65
5.3.1
Swing Trading ..............................................................................................................66
5.3.2
Policy 0 – Short-Term Speculation ......................................................................69
5.3.3
Policy 1 – Long Trend Speculation.......................................................................70
5.3.4
Policy 2 – Long and Short Trend Speculation ..................................................70
5.3.5
Policy 3 – Forward Hedging ...................................................................................72
CHAPTER 6 Results ..............................................................................................................................75
6.1
Model and Algorithm Set-Up.......................................................................................... 75
6.2
Policy 0: Results for Short-Trend Speculation ....................................................... 77
6.3
Policy 1: Results for Long Trend Speculation ......................................................... 81
6.4
Policy 2 – Long and Short Trend .................................................................................. 84
6.5
Policy 3 – Forward Hedge ............................................................................................... 85
6.6
Policy Analysis ..................................................................................................................... 86
vii
6.6.1
Withdrawal Rate Adjustment ................................................................................86
6.6.2
Volatility Fluctuation.................................................................................................87
6.6.3
KGNP Efficiency ...........................................................................................................89
6.6.4
Real World Testing and Comparison ..................................................................90
CHAPTER 7 Conclusion and Further Research .........................................................................93
Appendix: Code ......................................................................................................................................96
viii
Table of Figures
Figure 1- Natural Gas Powered Generation Plant ...................................................................... 2
Figure 2 - Salt Cavern Facilities ......................................................................................................... 3
Figure 3 - Preparing Hole for the Explosive Charges Used in Seismic Exploration ...... 5
Figure 4 - Shale Deposits ..................................................................................................................... 6
Figure 5 - Alaskan Pipeline Carrying Natural Gas ...................................................................... 7
Figure 6 - Natural Gas from Wellhead to End-Users ................................................................. 8
Figure 7 - Henry Hub in Erath, Louisiana ................................................................................... 10
Figure 8 - Fluctuations in Underground Storage Volume .................................................... 12
Figure 9 - Breakdown of Storage Portfolio in United States ............................................... 15
Figure 10 - Depleted Reservoir ...................................................................................................... 16
Figure 11- Aquifer Storage ............................................................................................................... 17
Figure 12 - Salt Cavern....................................................................................................................... 18
Figure 13 - Illustration of Knowledge Gradient ....................................................................... 34
Figure 14 – Knowledge Gradient with Correlated Beliefs at Measurements 1............ 37
Figure 15- Knowledge Gradient with Correlated Beliefs at Measurement 5 ............... 37
Figure 16 - Knowledge Gradient with Correlated Beliefs Estimated Surface .............. 38
Figure 17 - Plot of Kernel Functions K(u) .................................................................................. 40
Figure 18 - Henry Hub Historical Natural Gas Spot Prices 2004 - 2009 ........................ 48
Figure 19 - Natural Gas Spot Prices with Exponential Smoothing, Beta = .002 .......... 50
Figure 20 - Sample Paths for 1-Year of Simulated Natural Gas Spot Prices .................. 52
Figure 21 - Simulated Month Ahead Forward Prices for One Year .................................. 54
Figure 22 - Month Ahead Forward Prices with Spot Prices for One Month.................. 55
Figure 23 – Swing Trend with 10% Filtering............................................................................ 68
Figure 24 - Swing Trend with 5% Filtering ............................................................................... 68
Figure 25 - Policy 0: Second Measurement ............................................................................... 77
Figure 26 - Policy 0: 25 Measurements ....................................................................................... 79
Figure 27- Policy 0: 100 Measurements ..................................................................................... 79
Figure 28 - Policy 0: 500 Measurements .................................................................................... 80
Figure 29 - Policy 1: First Measurement..................................................................................... 82
ix
Figure 30 - Policy 1: 100 Measurements .................................................................................... 83
Figure 31 - Policy 1: 500 Measurements .................................................................................... 83
Figure 32 - Withdrawal Rate's Effect on Storage Value ........................................................ 87
Figure 33- Policy 0, KGNP Result for Doubled Volatility ...................................................... 88
Figure 34 - Policy 0: S-Curve ........................................................................................................... 89
Figure 35 - Policy 1: S-Curve ........................................................................................................... 90
Figure 36 - Price Paths for 2005 and 2009 Natural Gas Spot Prices ............................... 91
x
CHAPTER 1
Introduction
In late November 2009, the burgeoning glut of the U.S. natural gas supply
ventured into uncharted terrain as the Energy Information Administration (EIA)
reported the volume of working gas within U.S. storage to have hit 3.837 trillion
cubic feet (Tcf). This expansion is mirrored in the demand for natural gas, which has
called more and more supply away from its historical use in residential and industry
sectors to the power generation sector. Natural gas is playing an increasing role in
power generation, and in 2009, 23,475 MW of new generation capacity was planned
in the U.S. with over 50% being natural gas fired additions (Energy Information
Administration 2004). Figure 1 shows one of the many rapidly emerging gas
powered generation plants.
The natural gas market is rapidly changing, and natural gas storage will play
a significant role as it buffers demand and regulates supply. However, with shifting
supply and demand, other functions of storage can be utilized such as with salt
caverns, which exhibit high deliverability, high injection and withdrawal capacity,
CHAPTER 1: Introduction
that can quickly respond to changing gas prices; and thus are utilized as an arbitrage
mechanism. Storage owners and service providers will need to reexamine the value
of their facilities which is derived
Figure 1- Natural Gas Powered Generation Plant
from the expected profit from the
operations of a facility.
The problem of the valuation
of a natural gas storage facility is not
new,
and
literature
has
been
produced within the industry as well
as in academic circles. Industry professionals employ linear optimization with
Monte Carlo methods (Byers 2006) while the academic community supports the use
of stochastic optimization with an Approximate Dynamic Programming approach
(Lai, Margot and Secomandi 2008). Unfortunately, there remains advantages and
disadvantage of both techniques rendering it ambiguous to decipher the superior
procedure.
We will examine the problem of natural gas storage valuation focusing on the
valuation of high-deliverability gas storage facilities, specifically salt cavern storage
as pictured in Figure 2. We will strive to present capable valuation analysis and
optimization methodologies that accurately account for the various operating
characteristics of real storage facilities as well as the complex stochastic nature of
natural gas prices.
2
CHAPTER 1: Introduction
Figure 2 - Salt Cavern Facilities (Energy Information Administration 2004)
To solve the problem we will utilize stochastic optimization methods within
the realm of Optimal Learning. The idea encompasses a stochastic search over
policies which dictate the operational decisions of the storage facility. We assume
the policies are composed of a set of tunable parameters, and through sequential
measurements of the alternatives, estimates of each policy’s performance can be
ascertained. The goal is to choose the alternatives, and subsequently the policy,
3
CHAPTER 1: Introduction
which will produce the best performance. A strong performer within sequential
learning algorithms is the knowledge gradient which was developed by Gupta &
Miescke (1996) and later analyzed by Frazier et al. (2008). This algorithm strives to
maximize the amount of knowledge obtained with each measurement in an effort to
obtain an improved solution with the new knowledge. We will use an innovative
method newly added to the literature on the knowledge gradient algorithm that
evaluate problems with correlated beliefs using non-parametric estimation (Barut
and Powell 2010). The results of the method will account for an optimal strategy for
the operation of the natural gas storage facility, as well as yield a value for the
facility.
1.1
Natural Gas: From Wellhead to Market
Though it may be simple to flip on one’s burner in the kitchen, the process of
getting natural gas from the ground and into everyday life is actually an extensive
and complex process. This section provides an overview of the process, journeying
from exploration to marketing of the natural gas that will be sold for homes and
industry.
1.1.1 Exploration and Extraction
The process begins with geologists and geophysicists, who use technology
and knowledge of the properties of underground natural gas deposits to gather data
and make educated guesses as to where natural gas deposits may exist thousands of
feet below ground. The most important tool in exploration is the use of basic
seismology which refers to the study of how energy moves through the Earth's crust
4
CHAPTER 1: Introduction
and interacts differently with various types of underground formations. The energy
can be mapped, providing an effective way to image both sources and structures
Figure 3 - Preparing Hole for the Explosive
Charges Used in Seismic Exploration
(Energy Information Administration 2004)
deep within the Earth. In both onshore and
offshore exploration, seismic waves are
created that can travel through the Earth's
crust and generate the reflections needed
to ascertain the presence of natural gas or
the properties of a reservoir formation
(NaturalGas.org 2004). Figure 3 shows
operators
preparing
explosive
charges
a
used
hole
in
for
the
seismic
exploration.
After extensive measurements deem that the site may be enriched with a
marketable quantity of natural gas, drilling can begin. However, the exploration
team may be incorrect in its estimation of the existence of natural gas at a well site.
If this occurs, the well is labeled a 'dry well,' and production does not proceed.
Conversely, if the new well is drilled and comes in contact with natural gas deposits,
it is developed to allow the extraction of the natural gas and is labeled as a
'productive' well (NaturalGas.org 2004).
A common misconception is that all natural gas is extracted from natural gas
wells. In fact, natural gas is extracted from a variety of sources oil wells, shale, and
coalbeds. Oil wells which contain natural gas are known as “associated” wells. Even
5
CHAPTER 1: Introduction
if the well is specifically drilled for oil extraction, natural gas will almost always be a
byproduct of producing oil, since the small, light gas carbon chains come out of
solution as it undergoes pressure reduction from the reservoir to the surface. An
unconventional source of natural gas is shale, a fine-grained sedimentary rock
which can contain natural gas within its
Figure 4 - Shale Deposits
layered formation pictured in Figure 4. In
recent
years,
shale
deposits
have
accounted for an increasing share of the
natural gas production. As of November
2008, FERC estimated that there are 742
Tcf of recoverable shale gas in the United
States. A second unconventional source is
coalbed methane. Coal deposits are commonly found as seams that run
underground. Many coal seams also contain natural gas, either within the seam itself
or the surrounding rock which can be extracted and transported for production
(NaturalGas.org 2004). Table 1 lists the sources of natural gas and the amount of
natural gas extracted from each in the year 2008.
Table 1 - Sources of Natural Gas and Production Volume
(Energy Information Administration 2004)
Type of Well
Gross Production for 2008 % of Total Production
Natural Gas Well
18,011,151 MMcf
64.84%
Oil Well
5,844,798 MMcf
21.05%
Shale Deposits
2,022,000 MMcf
7.28%
Coalbed
1,898,400 MMcf
6.83%
6
CHAPTER 1: Introduction
1.1.2 Production and Transport
Although consisting of mostly methane, the gas extracted at the wellhead is
considered “raw” and can be mixed with oil or other hydrocarbons such as ethane,
propane, butanes, and pentanes. In order to maintain pipeline quality, regulations
are in place to ensure only purified natural gas is transported. Typically, some of the
needed processing is accomplished at the well site and later completed at
processing plants connected through gathering pipelines. Some of the hydrocarbons
associated with the natural gas are very valuable and are known as natural gas
liquids (NGL). NGLs have a
Figure 5 - Alaskan Pipeline Carrying Natural Gas
variety of uses and can be
sold
separately
from
the
natural gas (NaturalGas.org
2004).
Once the natural gas
has been purified, it enters
into a complex network of pipelines that make up the transportation system for
natural gas. The system is designed to quickly and efficiently transport natural gas
from the plant, to areas of high natural gas demand for use. It is composed of
interstate and intrastate pipelines which are responsible for transporting natural
gas at high speeds and pressure throughout the country and within a particular
state respectively. Natural gas will usually travel great distances before reaching the
end user. Figure 5 shows the Alaskan pipeline which carries natural gas over 800
miles.
7
CHAPTER 1: Introduction
1.1.3 Storage
Should natural gas not be required at a certain time, it can be transported to
storage facilities for when it is needed. Given our concentration on the valuation of
natural gas storage facilities, great detail of the various types of facilities and the
uses for storage will be discussed in Section 1.2 and 1.3.
1.1.4 Distribution and Marketing
Distribution is the process of delivering natural gas to end users who receive
the gas from local distribution companies (LDC). LDCs move small volumes of gas at
low pressures over shorter distances to a great number of individual users. A visual
review of the journey from wellhead to the individual users end-users can be found
in Figure 6. To perform the task, an extensive network of distribution pipeline is
present and is estimated to be over one million miles in the United States. LDCs
typically offer bundled services that combine the cost of all upstream activities,
including transportation and the purchasing price of the natural gas itself. This is
offered in one price for the customers to pay.
Figure 6 - Natural Gas from Wellhead to End-Users
(Natural Gas Depot 2009)
8
CHAPTER 1: Introduction
The increasingly important role of natural gas marketers is leading to
innovative ways of supplying natural gas to small volume users. Natural gas
marketing began in the 1980s after the deregulation of the natural gas commodity
market and the open access policy of natural gas pipelines. Natural gas marketing is
generally defined as the selling of natural gas, including selling services that a
particular purchase requires; including transportation, storage, accounting, and any
other service that is needed to facilitate the sale of natural gas.
Marketers also participate in the purchase and subsequent resale of natural
gas, thus it is not uncommon for the gas to have three to four separate owners
before it actually reaches the end-user. In addition to the buying and selling of
natural gas, marketers will use the financial markets and instruments to reduce
their exposure to risk as well as earn money through speculating market
movements.
1.1.5 Natural Gas Markets
In the United States, natural gas is traded on both the spot and futures
market. The price for natural gas is set by the New York Mercantile Exchange in the
daily transactions of the commodity and greatly impacts the activities of marketers.
Natural gas prices are also crucial to our problem of natural gas storage valuation
where prices dictate the injection and withdrawal of natural gas from storage as
speculators attempt to capitalize on market movements.
9
CHAPTER 1: Introduction
Natural gas is priced and traded at different locations throughout the country
at what is referred to as market hubs, which exist along the intersection of major
pipeline systems. Although there are over
Figure 7 - Henry Hub in Erath, Louisiana
thirty major market hubs in the U.S., this
paper will primarily concern itself with the
Henry Hub located in Louisiana. Spot and
future prices are set at Henry Hub and are
seen to be the primary price set for the
North American natural gas market.
Natural Gas Spot Market
The natural gas spot market is the daily market quoted in $/MMBtu, where
natural gas is bought and sold for immediate delivery. The spot market would be the
source to obtain the price of natural gas on a specific day. The spot market is traded
every business day, and settled, paid and delivered on the next day. For example, if
the transaction date was Wednesday, November 17, 2010 the settlement date would
be Thursday, November 19, 2010 (CMEGroup 2010).
Natural Gas Futures Market
Natural gas is traded using forward contracts which are widely traded on the
New York Mercantile Exchange (NYMEX). One natural gas contract has an energy
value of 10,000 MMBtus. Natural gas forward contracts are Henry Hub contracts,
meaning they reflect the price of natural gas for physical delivery at this hub. The
prices of 72 forward contracts are available every business day for trade. A feature
10
CHAPTER 1: Introduction
of the forward contracts is that they will expire on the third to last business day of
every month. For example, the February 2010 contract will expire on January 27,
2010. When a contract expires and is settled, the buyer receives the gas every day
for the specified month, receiving 1/30 of the contract every day (CMEGroup 2010).
Both markets will be employed in testing our procedure to value natural gas
storage. Chapter 4 will introduce modeling methods which will be utilized to
simulate the price processes of the spot and future markets of natural gas.
1.2
Traditional Use of Underground Natural Gas Storage
Storage facilities were developed to allow the production capacity of natural
gas to be moved from one point in time to another. Natural gas that reaches its
destination is not always needed right away, so it is injected into underground
storage facilities where it can be stored for an indefinite period of time. This
capability is utilized in Seasonal Cycling, where a facility stores gas to meet
variations in seasonal load. Other uses for natural gas storage include Peaking
Services, where facilities use gas capacity to respond to quickly changing conditions
effecting demand of natural gas, and Speculative Market Services where high
deliverability storage quickly responds to changing gas prices capitalizing on price
movements at market centers.
1.2.1 Seasonal Cycling
Production of natural gas produces a constant supply from the various
sources mentioned in Section 1.1.2, contrary to the varying demand. Traditionally,
the demand of natural gas is cyclical, with demand higher during the winter months
11
CHAPTER 1: Introduction
due to the need to provide heat in residential and commercial locations.
Corresponding to this seasonal pattern, natural gas will be injected into storage
facilities throughout the non heating season, which typically runs from April to
October. This is when production of natural gas exceeds demand. Subsequently, the
excess natural gas stored during the summer is withdrawn during the heating
season, which includes the months from November to March, when production falls
short of the demand. Storage facilities involved in seasonal cycling are large and
capable of holding enough natural gas to satisfy the long term seasonal demand
requirement. Without natural gas storage, supply would be limited in satisfying
increased demand during winter months. It is vital in its role of ensuring that excess
supply delivered during summer months is available during the winter. Figure 8
depicts the seasonal pattern of natural gas storage withdrawal and injection.
(NaturalGas.org 2004)
Figure 8 - Fluctuations in Underground Storage Volume
(Energy Information Administration 2004)
12
CHAPTER 1: Introduction
1.2.2 Peaking Services
Demand for gas during the winter can be highly volatile due to the
uncertainty of weather patterns. An unexpected drop in temperature calls for the
immediate delivery of natural gas from storage. This peak demand seen in winter is
now being replicated during the summer as the new trend of natural gas fired
electric generation, increases the demand for natural gas during the summer
months due to the demand for electricity. Electricity demand is also dependent on
the variability of weather as residential and commercial buildings power their air
conditioners to combat the summer heat. Following these fluctuations of electricity
load, the demand for natural gas oscillates from day to night and from weekday to
weekends. Thus, not only is storage needed to store excess natural gas during lowpeak hours, but storage facilities must be able to provide high-deliverability of the
natural gas to respond to the sudden short-term periods of high demand
(NaturalGas.org 2004).
1.2.3 Speculative Market Services
After 1992, with the introduction of the Federal Energy Regulatory
Commission’s Order 636, interstate pipeline companies who owned all the natural
gas flowing through their system including the gas held in storage were required to
operate their storage facilities on an open access basis as part of the deregulation of
the natural gas market. This has expanded the use of storage from its role as a backup supply source in times of excess demand to an agent of the financial markets.
Now, marketers can move gas in and out of storage in order to capitalize on the
changes in price levels as well as in combination with financial instruments such as
13
CHAPTER 1: Introduction
futures and other option contracts. In order to adapt to its new role of profiting from
market conditions, natural gas storage has focused on more flexible operations with
high deliverability and rapid cycling in their inventories (Energy Information
Administration 2004).
1.3
Types of Underground Natural Gas Storage
There are three main types of underground storage: depleted gas and oil
reservoirs, aquifers, and salt caverns. The breakdown in percent of each storage
type that makes up the portfolio of U.S. storage is shown in Figure 9, and the specific
characteristics of each are described in Sections 1.3.1 - 1.3.3. However, all have
similar operational characteristics. All underground storage has a capacity
measured in Bcf (billion cubic feet), which can be divided amongst the amount of
working gas and base gas within a facility. Often when a capacity of a storage facility
is quoted, it is referring to the working gas capacity seeing as it is the amount which
can be withdrawn and injected into a facility. Every storage facility comes attached
with a maximum and minimum withdrawal and injection rates which are typically
expressed in Bcf/day. The injection and withdrawal rates for natural gas fluctuate
based on the amount of pressure (PSI) within the storage facility. Withdrawal rates
share a direct relationship with pressure while injection rates maintain an indirect
relationship. Pressure for a facility is also bounded by a maximum and minimum
quantity which is determined by the volume, depth, and structure of a facility. These
operational characteristics determine the operational flexibility of a facility.
14
CHAPTER 1: Introduction
Figure 9 - Breakdown of Storage Portfolio in United States
(Energy Information Administration 2004)
1.3.1 Depleted Reservoirs
The most prominent type of underground storage due to their wide scale
availability is depleted gas or oil reservoirs. These storage facilities are gas or oil
reservoir formations that have already been tapped of all their recoverable resource
through earlier production, leaving an underground formation geologically capable
of holding natural gas. As a result, storage facilities of this nature are abundant in
producing regions (Dietert and Pursell 2000).
Of the three types of underground storage, depleted reservoirs are the
cheapest and easiest to develop, operate, and maintain. Using an already developed
reservoir for storage presents the opportunity to reuse the extraction and
distribution equipment left over from when the field was productive, reducing the
cost of conversion to gas storage. However, the maturities of these reservoirs
require a substantial amount of maintenance.
15
CHAPTER 1: Introduction
In order to sustain pressure in depleted reservoirs, the facility maintains
equal parts base and working gas. However, depleted reservoirs, having already
been filled with natural gas, do not require the injection of what will become
physically unrecoverable gas seeing as it already exists in the formation. Depleted
reservoirs with high permeability and porosity are ideal for natural gas storage,
porosity lending itself to the amount of natural gas it can hold and permeability
determining the rate of flow of
Figure 10 - Depleted Reservoir
natural gas through the formation.
This
in
turn
determines
the
injection and withdrawal rate of
working gas.
Disadvantages
depleted
uncertainty
reservoirs
of
of
using
are
the
capacity.
The
configuration of the geological
formation is never fully used since it runs the risk that injected gas may diffuse into
the outer veins of the formation and becomes inaccessible. Other disadvantages
include a reservoir’s limited cycling capabilities, where working gas volumes are
usually cycled only once per season. In addition, reservoirs are characterized with
having low deliverability and thus would not be well suited for peaking services. It is
most typically employed for seasonal cycling (NaturalGas.org 2004).
16
CHAPTER 1: Introduction
1.3.1 Aquifers
Aquifers are underground permeable rock formations that act as natural
water reservoirs. When reconditioned, these formations may be used as natural gas
storage facilities where gas is injected on top of the water formation displacing the
water further down within the structure. Most of these facilities are located in the
Figure 11- Aquifer Storage
upper Mid-West where there is a
lack of depleted oil and gas
reservoirs (Dietert and Pursell
2000).
Advantages
storage
include
of
their
aquifer
close
proximity to markets where other
geological reservoirs are not readily available. Deliverability rates may also be
enhanced due to the presence of an active water drive which increases the storage
facilities overall pressure. The high deliverability allows the working gas volumes to
cycle through the facility more than once per season.
Aquifer storage is the least desirable form of storage due to its physical and
economic disadvantages. A significant amount of time and money is spent testing
the suitability of an aquifer for natural gas storage and subsequently developing the
infrastructure needed for an effective natural gas storage facility. In addition, in
aquifer formations, base gas requirements are as high as 80 percent of the total gas
volume. Unlike base gas from depleted reservoirs, this base gas is unrecoverable in
17
CHAPTER 1: Introduction
aquifer storage due to the risk of facility damage. This high base gas requirement
increases the initial cost of capital for aquifer storage projects, thus limiting their
number. Most aquifer storage facilities were developed when the price of natural
gas was low, meaning this base gas was not very expensive to give up. However,
with higher prices, aquifer formations are increasingly expensive to develop and
most often other storage type projects will be contracted (NaturalGas.org 2004).
1.3.2 Salt Caverns
Salt cavern storage sites are high pressured, solution-mined cavities in
existing salt dome caverns located at depths several hundred to several thousand
feet below the earth’s surface. They are accessible by one or more wells per cavern.
Figure 12 depicts a typical salt cavern facility. Most of the salt cavern facilities are
located in the high producing region of the Gulf Coast, with some bedded salt
deposits located in the high consumption states of western Pennsylvania and New
York (Dietert and Pursell 2000).
Figure 12 - Salt Cavern
Underground salt formations
are well suited to natural gas storage
allowing for little injected natural
gas to escape from the formation
unless specifically extracted. The
walls of a salt cavern have the
structural strength of steel making it
resilient against degradation over
18
CHAPTER 1: Introduction
the life of the facility. Bas gas requirements are the lowest of all three storage types,
requiring on average only 33 percent of total gas capacity to the natural gas storage
vessel which maintains very high deliverability rates, exceedingly higher than that
of depleted reservoirs and aquifers. This allows for natural gas to be more readily
withdrawn, sometimes on as little as an hour’s notice, which is well suited for
satisfying unexpected surges in demand. The caverns also offer operational
flexibility having the ability to cycle working gas four to five times a year, reducing
the per-unit cost of each thousand cubic feet of gas injected and withdrawn. This
multiple cycling capability coupled with its high deliverability is why salt caverns
are well suited for peaking services as well as responding to volatility in natural gas
market prices for commodities traders.
Drawbacks of this form of storage are volume limitations where each cavern
size typically ranges from 5-10 Bcf of working gas, considerably smaller than
capacity capabilities of depleted reservoirs and aquifers. In addition, start-up costs
generated during cavern development are substantial, and the disposal of saturated
salt water produced during the solution mining can be detrimental to the
environment (NaturalGas.org 2004).
A summary of the three different storage facilities and their defining
characteristics and uses can be found in Table 2.
19
CHAPTER 1: Introduction
Table 2- Storage Facility Characteristics (Eydeland and Wolyniec 2003)
Withdrawal
Operating
Costs
Major
Use
120-200
Days
60-120 Days
High with
some fuel
losses
Seasonal
Cycling
High
deliverability, high
cycling, low
capacity
20 Days
5-20 Days
Low with
minimal fuel
losses
Peaking
Services
Low deliverability,
low cycling, high
capacity
120-200
Days
60-120 Days
High with
some fuel
losses
Seasonal
Cycling
Facility
Description
Injection
Depleted
Fields
Low deliverability,
low cycling, high
capacity
Salt
Caverns
Aquifers
1.4
Storage Contract
In order to use any type of storage facility above, a storage contract must be
entered into. A natural gas storage contract will specify the term date for the party’s
use of the storage, the type of storage facility, as well as the physical constraints and
operational costs of the facility. A specific catalog of these physical and operational
components can be found in the example contract below.
Table 3 - Example Storage Contract for Depleted Reservoir Storage (Enstor 2009)
Term
2/1/2010-1/31/2012
Type
Gas Reservoir
Depth
7000 ft
Maximum Working Gas Capacity
30 Bcf
Initial Working Gas Capacity
0 Bcf
Maximum/Minimum Injection Rate
400 MMcf/day – 50 MMcf/day
Maximum/Minimum Withdrawal Rate
1 Bcf/day – 200 MMcf/day
Fuel Injection Loss Spread
1.0%- 2.5%
Maximum/Minimum Facility Pressure
1000 – 3500 psi
20
CHAPTER 1: Introduction
1.5
Overview
Now possessing a foundation in the natural gas industry, including the
application of storage, we are able to advance towards our goal of valuing a storage
facility in the form of a salt cavern. The analysis will begin in Chapter 2 with a study
of linear optimization with Monte Carlo methods discussed by Byers (2006) as well
as the Approximate Dynamic Programming approach introduced by Lai et al.
(2008). In Chapter 3, we will describe Knowledge Gradient Algorithms in particular
a knowledge gradient method which employs non-parametric estimation that will
be utilized in solving the valuation problem. Chapter 4 presents the pricing process
used to model the complex stochastic nature of natural gas spot and future prices.
The simulations garnered in Chapter 4 will be utilized within the model outlined in
Chapter 5, which describes the operational functions of a storage facility and will
derive the costs and rewards achieved from its operation. The policies exercised in
making the operational decisions for the storage facility will also be exhibited in
Chapter 5. Chapter 6 will present the results of the knowledge gradient method,
bestowing the policies of Chapter 5 with optimal parameters that maximize the
operation of the storage facility, allowing for the deduction of the proper value for
the salt cavern storage. Chapter 7 will impart any conclusions deduced from the
results as well as extensions in the development of the methods used to solve the
storage valuation problem.
21
CHAPTER 2
Literature Review
Valuing a contract is difficult because it entails the dynamic optimization of
inventory decisions with capacity and operational constraints while combating the
uncertainty of natural gas pricing dynamics. Several approaches have been
developed to tackle this problem: Forward Optimization optimizes the withdrawal
and injection schedules given the current forward prices and storage constraints,
Forward Dynamic Optimization (Spread Option Optimization) values the storage as a
portfolio of spread options which is valued based on the forward curve, and
Stochastic Dynamic Programming which depends on directly modeling the price
processes and storage flows given the dynamics (Eydeland and Wolyniec 2003).
This review will encompass the two latter approaches given that they are favored by
natural gas industry professionals and academic scholars respectively.
CHAPTER 2: Literature Review
2.1
Spread Option Optimization Approach
Within the natural gas industry, there is a preference of modeling the full
dynamics of the futures term structure using high dimensional forward models.
Though this high dimensionality may hinder a stochastic dynamic programming
approach, practitioners have found satisfaction in the solution rendered by spread
option valuation methods combined with a linear program (LP) which is embedded
within a Monte Carlo simulation.
Byers (2006) depicts this process as a two stage valuation problem. In the
first stage, linear optimization is used to determine spot market transactions, which
is the actual purchase and sale of the physical gas. The linear optimization model is a
profit maximization problem, where profit is defined by
𝑃𝑟𝑜𝑓𝑖𝑡 =
𝑁
𝑏𝑖𝑑
𝑡=1(𝑃𝑡 𝑤𝑡
+ 𝑃𝑡𝑎𝑠𝑘 𝑖𝑡 − 𝑐𝑡𝑖 − 𝑐𝑡𝑤 ) 𝛿.
Parameters include the withdrawal and injection volume, 𝑤𝑡 and 𝑖𝑡
respectively, the number of contract periods within the tenure of the facility, N, the
cost of withdrawal and injection, 𝑐𝑡𝑤 and 𝑐𝑡𝑖 , and the price variables calculated from
the forward price curve and bid-ask spread curve, 𝑃𝑡𝑏𝑖𝑑 and 𝑃𝑡𝑏𝑖𝑑 .
As with most optimization problems, the maximum profit obtained through
the trafficking of the commodity is subject to constraints. Physical and operational
constraints are put in place to ensure that the purchase and sale of natural gas does
not violate the withdrawal and injection capacities of the storage facility during a
specified period. These capacities depend on the maximum capacity of the facility,
23
CHAPTER 2: Literature Review
the current inventory, and the future commitment to deliver or accept the delivery
of expiring natural gas futures contracts.
This profit maximization problem given as an LP is actually solved many
times during the second stage of the valuation which iterates over the first stage
calculating the marginal contributions of changes in the storage portfolio from day
to day using Monte Carlo methods.
The result of this technique is an efficient and easily obtained value for
natural gas storage assets. This technique is extremely attractive to industry
professionals due to the simplicity in the linear structure of the model which can be
solved utilizing simple optimization methods such as the simplex method. Software
for solving linear optimization problems is widely available and can handle
substantial sized models.
However, the linear optimization structure of the model which allows for
efficient computation is also the root of debate regarding the valuation results.
Computing the natural gas storage valuation as a linear optimization problem
assumes that the storage facility operations are deterministic. In fact, natural gas
storage decisions follow a non-deterministic process meaning that the system's
state is determined both by the process's predictable actions and by random
elements. Natural gas storage decisions heavily depend on the stochastic processes
exhibited in the pricing of natural gas which in turn affects the operational
constraints of the facility. Due to the uncertainty, natural gas storage decisions can
be viewed as stochastic in nature. Thus, the LP structure would not be equipped to
24
CHAPTER 2: Literature Review
fully capture all the characteristics of the process and a valuation produced by such
a model would be deficient. In order to truly portray the operational decisions of a
natural gas storage system, a stochastic model would need to be incorporated,
requiring stochastic optimization methods in order to solve for the value of the
facility. Section 2.2 will discuss the use of Approximate Dynamic Programming
(ADP) as a stochastic optimization method and evaluate its valuation capabilities in
comparison to the method and results produce by linear optimization.
2.2
Approximate Dynamic Programming (ADP) Approach
Due to the uncertainty and stochastic nature of natural gas storage decisions,
scholars such as Lai et al. (2008), gravitated toward stochastic dynamic
programming (SDP) as the natural approach to solving the storage valuation
problem. Solving dynamic optimization problems is rooted in the application of
Bellman’s equation which solves simpler sub-problems, computing the value of a
decision at a certain point in time in terms of the payoff of the current decision and
the value of the remaining decisions within the problem that result from the prior
decisions. Bellman’s equation for stochastic problems can be described as
𝑉𝑡 𝑠𝑡 , 𝑎𝑡 = max 𝐶 𝑥, 𝑎 + 𝛾𝔼 𝑉𝑡+1 𝑠𝑡+1 , 𝑎𝑡+1 |𝑠𝑡 , 𝑎𝑡
∀ 𝑡 ∈ 𝑇,
where C(x, a) is the contribution made at time t from taking action a while in
state x, and γ discounts the expected value of the next state (Puterman 1994).
Storage value is determined by the expected cash flow generated by the
operational decisions. These decisions are dependent on the current operational
characteristics of the storage facility in addition to the market conditions at the
25
CHAPTER 2: Literature Review
time. Given the decision to inject or withdraw x and the operational and market
characteristics that make up the state at time t, the value of storage can be
expressed by the following Bellman’s equation
𝑉𝑡 𝑋𝑡 , 𝑭𝑡 = arg max 𝐶𝐹𝑡 𝑥, 𝑆𝑡 + 𝑒 −𝑟𝑑𝑡 𝔼 𝑉𝑡+1 𝑋𝑡+1 , 𝑭𝑡+1 |𝑭𝑡
∀ 𝑡 ∈ 𝑇,
subject to
−𝑊𝑡 ≤ 𝑥 ≤ 𝐼𝑡 ,
𝑋𝑡+1 = 𝑥 + 𝑋𝑡 ,
0 ≤ 𝑥 ≤ 𝑋𝑚𝑎𝑥 ,
𝐶𝐹𝑡 𝑓, 𝑆𝑡 = 𝑓 ∗ 𝑆𝑡 ,
where
𝑋𝑡 = the inventory level at time t with 𝑋𝑚𝑎𝑥 being total capacity
𝑥 = the amount of flow in any given period t
𝑊𝑡 = the maximum withdrawal capacity at time t
𝐼𝑡 = the maximum injection capacity at time t
𝑠𝑡 = the gas price at time t which is determined by a specific stochastic process
𝑭𝑡 = the forward curve at time t
𝐶𝐹𝑡 = the cash flow at time t
26
CHAPTER 2: Literature Review
The solution approach for solving a discrete SDP is defined by backward
induction, which can be implemented by first considering the last time a decision is
made and choosing what to do in any situation at that time. Using this information,
one can then determine what to do at the second-to-last time of decision. This
process continues backwards until one has determined the best action for every
possible situation.
Though it appears an easily obtained solution can be reached, the presence of
the forward curve vector, 𝑭𝑡 , within system state that complicates the ability to
reach a solution. 𝑭𝑡 is a vector of forward prices at time t for all the futures contracts
that mature at various times in the future. Over 72 contracts are traded in the
futures market at one time, yet utilizing even a fraction of them within the state
variable subjects the problem to the “curse of dimensionality.” This means that the
high dimensionality of the state space and the manipulation of exponentially large
volumes of information render the solution unfeasible by backwards induction.
To combat this problem Lai et al. (2008) develops a technique based on ADP
methods to value the storage of natural gas and renders the high-dimensional model
of the forward curve more pliable.
This approach transforms the intractable
stochastic dynamic program model of the storage problem into a manageable lower
dimensional Markov decision process.
The approach to reducing the computationally intractable SDP model is to
develop an approximate model using information reduction. Lai et al. (2008),
removes price related state variables from the state definition of the exact model.
27
CHAPTER 2: Literature Review
Though the amount of variables removed may fluctuate based on the solver’s
discretion, it is important to note that the more price-related variables are removed,
the easier it will be to solve efficiently. The contraction within the ADP technique
occurs by computing the optimal value function by conditioning on the possible
values of the next month’s futures price and the next months futures price at 𝑡 = 0,
instead of conditioning on the whole forward curve. So now the vector 𝑭𝑡 , is
composed of two scalar variables, 𝑭𝑡,𝑡+1 and 𝑭0,𝑡+1 . The state variable now includes
the current inventory, spot price, and only two variables which are associated with
the forward curve within each period t. Thus, the ADP model of the problem is
vastly reduced from the previous model and is given by
𝑉𝑡𝐴𝐷𝑃 𝑋𝑡 , 𝑭𝑡 = 𝔼 𝑣𝑡𝐴𝐷𝑃 𝑋𝑡 , 𝑭𝑡,𝑡+1 |𝑭0,𝑡+1 ∀ 𝑡 ∈ 𝑇,
where,
𝐴𝐷𝑃
𝑣𝑡𝐴𝐷𝑃 𝑋𝑡 , 𝑭𝑡 = arg max 𝐶𝐹𝑡 𝑥, 𝑆𝑡 + 𝑒 −𝑟𝑑𝑡 𝔼 𝑉𝑡+1
𝑋𝑡+1 , 𝑆 𝑡+1 |𝑭𝑡,𝑡+1
.
The conditioning on the next month’s futures price at time 0, 𝑭0,𝑡+1 , can be
seen in the first equation and yields the expected value of the value function
displayed by the second equation, which is conditioned on the next month’s futures
price at time t, 𝑭𝑡,𝑡+1 .
It is reported that the implementation of this ADP model can generate values
that on average are 97% of optimal storage value, while the LP model of Section 2.1
reports a larger underestimate of the value of storage with an average of 75% (Lai,
Margot and Secomandi 2008). It appears that ADP dominates the LP model in terms
28
CHAPTER 2: Literature Review
of the actual valuation of the storage facility. However, looking at cpu run time as an
alternative indicator, ADP can be considered as a suboptimal policy with extensively
higher cpu requirements, averaging 250 cpu seconds compared to the .0038 cpu
seconds required by the LP model. It is evident that despite the improvements in the
actual valuation of natural gas storage, greater computational burdens are
undertaken to produce such results. The fact that ADP solution methods are
considered slower and less efficient, little impression is made on industry
practitioners. Thus, it is important to find a method that will best balance valuation
quality and computational efficiency.
Chapter 3 will discuss the stochastic optimization method known as optimal
learning, focusing on the stochastic search method known as the Knowledge
Gradient Policy. We will utilize this process in order to find a valuation method for
natural gas storage that deploys stochastic optimization methods lacking in the LP
model, yet will yield less computational burdens found in the ADP model.
29
CHAPTER 3
Knowledge Gradient Methods
Within the natural gas industry, decisions for the operation of a storage
facility are based on policies which may be simple or complex in nature but all of
which are based on heuristics. Letting the problem of valuing a natural gas storage
facility be defined as computing the expectation of these operational decisions
based on such a policy, we can define the problem within the framework of the
stochastic optimization method known as optimal learning. The idea encompasses a
stochastic search over policies which dictate the operational decisions of the storage
facility. We assume the policies are composed of a set of tunable parameters where
a range of suitable alternatives for the parameters will be established. The method is
defined by the sequential measurements of the alternatives that estimate each
policy’s performance. Typically, a budget of measurements is spread over a set of
alternatives. The goal is to choose the alternatives, and subsequently the policy,
which will produce the best performance. Within the scope of our problem, the
CHAPTER 3: Knowledge Gradient Methods
performance we wish to optimize is profit produced by the utilization of the storage
facility and subsequently the value of the storage facility.
The problem described above is considered a ranking and selection problem.
A policy which has had much success within the problem class is known as the
Knowledge Gradient policy (KG). First developed by Gupta & Miescke (1996) and
later analyzed by Frazier et al. (2008), the knowledge gradient policy is a myopic
policy that strives to solve ranking and selection problems by choosing alternatives
that maximize the amount of knowledge obtained by a single measurement in an
effort to obtain an improved solution with the new knowledge. This is done by
looking one time step into the future when making a measurement decision.
This chapter will not only discuss the Knowledge Gradient policy but also
extensions made to the concept. This includes the correlation of measurements
which applies to our problem of optimizing the natural gas policies where the
measurement of one policy provides information regarding another. A new method
within the realm of correlated measurement problems is also discussed within the
chapter and will be the means of obtaining the valuation solution to the natural gas
storage problem.
3.1
Knowledge Gradient with Independent Measurements
To visualize the idea of the knowledge gradient policy we first consider
problems with independent measurements. This means when measuring alternative
x, no information is gained for the value of x’≠ x.
In this setting, the true values of the alternatives are distributed by
31
CHAPTER 3: Knowledge Gradient Methods
𝑌 ~ Ν(μx , σ2x ),
where the parameters are unknown (Powell and Frazier 2008). However, assuming
that at iteration n our state of knowledge is given by
𝑆 𝑛 = (μn , βn ),
where 𝛽 𝑛 is the precision of alternative x defined by
𝛽𝑛 =
1
,
𝜍 2,𝑛
a prior predictive distribution of Y for each alternative x can be obtained which is
normally distributed with mean 𝜇𝑥𝑛 and variance 𝜍𝑥2,𝑛 (Frazier and Powell 2008). If
measurements were to cease after n iterations, the best option would yield a
solution represented by
𝑛
𝑉 𝑛 𝑆 𝑛 = max𝑥′ ∈𝜒 𝜇𝑥′
.
If measurements continued, the next state of knowledge would be 𝑆 𝑛 +1 (𝑥),
where x has been chosen to be measured, producing 𝑦𝑥𝑛+1 , the observation of the
n+1 measurement. Defining ε as the noise of a measurement, a generalized
description for an observation can be written as
𝑦𝑥 = 𝑌𝑥 + 𝜀.
With a new observation, a posterior predictive distribution for Y can be determined
with mean 𝜇𝑥𝑛+1 and variance
1
𝛽 𝑥𝑛 +1
(Frazier and Powell 2008). Using Bayes’ rule, the
updated values for 𝜇𝑥𝑛+1 and 𝛽𝑥𝑛+1 are given by
32
CHAPTER 3: Knowledge Gradient Methods
𝜇𝑥𝑛+1 =
𝛽𝑥𝑛 𝜇𝑥𝑛 + 𝛽 𝜀 𝜇𝑥𝑛+1 , 𝑖𝑓 𝑥 𝑛 = 𝑥
,
𝜇𝑥𝑛 ,
𝑜𝑡𝑕𝑒𝑟𝑤𝑖𝑠𝑒
𝛽𝑥𝑛+1 =
𝛽𝑥𝑛 + 𝛽 𝜀 , 𝑖𝑓 𝑥 𝑛 = 𝑥
.
𝛽𝑥𝑛 ,
𝑜𝑡𝑕𝑒𝑟𝑤𝑖𝑠𝑒
This allows for the computation of the best solution after n+1 measurements
represented as
𝑛+1
𝑉 𝑛+1 𝑆 𝑛+1 (𝑥) = max𝑥′ ∈𝜒 𝜇𝑥′
.
𝑛+1
Going back to the perspective at iteration n, it is important to note that 𝜇𝑥′
𝑛
is a random variable, while 𝜇𝑥′
is not. The objective is to choose an alternative x that
maximizes the incremental increase between the best value at n and what is
believed to be the best value at n+1. This is given by
𝑣 𝐾𝐺,𝑛 = max𝔼 𝑉 𝑛+1 𝑆 𝑛+1 𝑥
𝑥∈𝜒
− 𝑉𝑛 𝑆𝑛 𝑆𝑛 ,
(3.1)
which is known as the value of the knowledge gradient (Frazier, Powell and Dayanik
2008). Equation 3.1 can be interpreted mathematically as a gradient of 𝑉 𝑛 𝑆 𝑛 with
respect to the measurement chosen which is maximized over the knowledge gained
from that measurement. The knowledge gradient policy can then be expressed by
𝑋𝐾𝐺,𝑛 = argmax𝔼[𝑉 𝑛+1 𝑆 𝑛+1 (𝑥) − 𝑉 𝑛 𝑆 𝑛 |𝑆 𝑛 ] .
𝑥∈𝜒
A simple example of the knowledge gradient policy can be seen in Figure 13. Here,
five choices are considered , where the estimate mean of choice 4 appears to be the
best solution. Looking at the distribution of alternative 5, there is a distinct
33
CHAPTER 3: Knowledge Gradient Methods
probability that choosing alternative 5 will produce a value greater than the current,
best witnessed in part of the distribution curve that exceeds choice 4. If this occurs,
𝑉 𝑛+1 will increase; or be 0 if it does not. The expected increase of 𝑉 𝑛+1 is the
knowledge gradient 𝑣 𝐾𝐺 .
Figure 13 - Illustration of Knowledge Gradient (CASTLE Laboratory 1997-2009)
3.2
Knowledge Gradient for Correlated Beliefs
Many problems are characterized as having one measurement providing
information about what might be observed from other measurements. This is
known as correlated beliefs, and the knowledge gradient concept has been extended
by Fraizer (2009) for greater accessibility to this problem class, naming the policy
the Knowledge Gradient for Correlated Beliefs (KGCB).
The move from independent to correlated measurements involves a
transition to working with vectors of means and covariance matrices. In this setting,
the values of the alternatives are distributed by
34
CHAPTER 3: Knowledge Gradient Methods
𝑌 ~ Ν(μ, Σ),
where μ is a vector of means with element μx , and the means of the alternatives are
represented by x. Let Σ be the covariance matrix with element Cov(x , x’) (Frazier,
Powell and Dayanik 2009).
Assuming that at iteration n our state of knowledge is 𝑆 𝑛 = (μn , Σ n ), the
knowledge gradient with correlated beliefs will act similarly to its independent
counterpart by selecting an alternative that is expected to increase the current best
estimate given by
𝑣 𝐾𝐺,𝑛 = argmax𝔼[max𝑥 ′ 𝜇𝑥𝑛+1
− max𝑥 ′ 𝜇𝑥𝑛′ |𝑆 𝑛 , 𝑥 𝑛 = 𝑥].
′
𝑥∈𝜒
After taking a measurement of alternative x, 𝑦𝑥𝑛+1 is observed as the n+1st
measurement, where the observation is normally distributed with mean 𝑌𝑥 and
variance 𝜆𝑥 which is defined as
1
𝜆𝑥 = 𝛽 𝜀 .
Transitioning to the new state 𝑆 𝑛+1 = (μn+1 , Σ n+1 ) obtains a posterior distribution
for our beliefs with parameters μn+1 and Σ n+1 which are calculated from the
Bayesian updating formulas given by
𝜇 𝑛+1 = 𝜇 𝑛 +
𝑦 𝑛 +1 −𝜇 𝑥𝑛
𝜆 𝑥 +Σ 𝑛𝑥 ,𝑥
Σ 𝑛+1 = Σ 𝑛 +
35
Σ 𝑛 𝑒𝑥 ,
Σ 𝑛 𝑒𝑥 𝑒𝑥𝑇 Σ 𝑛
𝜆 𝑥 +Σ 𝑛𝑥 ,𝑥
,
CHAPTER 3: Knowledge Gradient Methods
where 𝑒𝑥 is a column vector of zeros except for a single 1 where x is indexed
(Frazier, Powell and Dayanik 2009). For simplicity, 𝜇 𝑛+1 can be written as
𝜇 𝑛+1 = 𝜇 𝑛 + 𝜍(Σ 𝑛 , 𝑥 𝑛 )𝑍,
where Z is a standard normal random variable and
𝜍 Σ𝑛 , 𝑥 𝑛 =
Σ 𝑛 𝑒𝑥
𝜆 𝑥 +Σ 𝑛𝑥 ,𝑥
.
Integrating this into the 𝑣 𝐾𝐺 formula for the KGCB, it can be restated as
𝑛
𝑣 𝐾𝐺𝐶𝐵 ,𝑛 = max𝔼 max
[(𝜇𝑥𝑛′ + 𝜍𝑥 ′ Σ 𝑛 , 𝑥 𝑛 )𝑍 𝑆 𝑛 , 𝑥 𝑛 = 𝑥 − max 𝜇𝑥′
′
𝑥∈𝜒
𝑥′ ∈𝜒
𝑥
where the policy will choose the alternative which maximizes this value (Frazier,
Powell and Dayanik 2008).
In Figure 14, the implementation of the KGCB policy can be seen in a series of
graphs that juxtapose a graph of the measurements and the value of the knowledge
gradient, 𝑣 𝐾𝐺𝐶𝐵,𝑛 , at three distinct points in time. The first shows the first
measurement and its effect on the value of the knowledge gradient. Whenever a
point is measured, knowledge is gained not only about the measurement but also
about surrounding points which are seen by the drop in the knowledge gradient at
the point of measurement and its surrounding area. This is due to the correlation of
the measurements. The policy will continue to select areas of high uncertainty to
measure.
36
CHAPTER 3: Knowledge Gradient Methods
Figure 14 – Knowledge Gradient with Correlated Beliefs at Measurements 1
(CASTLE Laboratory 1997-2009)
Figure 15 portrays the fifth measurement where the policy has chosen to
measure a point within the middle region of the surface due to the uncertainty of the
region. Knowledge is once again gained not only about the specific point we
measured, but also the region surrounding the point.
Figure 15- Knowledge Gradient with Correlated Beliefs at Measurement 5
(CASTLE Laboratory 1997-2009)
The graphs in Figure 16 displays the final estimate of the surface based on all the
measurement decisions of the policy.
37
CHAPTER 3: Knowledge Gradient Methods
Figure 16 - Knowledge Gradient with Correlated Beliefs Estimated Surface
(CASTLE Laboratory 1997-2009)
3.3
Knowledge Gradient with Non-Parametric Estimation
The KGCB policy assumes that we have prior knowledge regarding the
covariance structure. For problems outside of this class, Barut (2010) introduces the
Knowledge Gradient with Non-Parametric Estimation (KGNP), a sequential learning
method that aggregates a set of estimators derived from multiple kernel regression
with different bandwidths. Unlike KGCB, this policy does not assume prior
knowledge of the covariance structure.
3.3.1 Kernel Estimation
Kernel estimation methods are at the core of KGNP, and thus this section
strives to provide a brief overview in order to illuminate the KGNP procedure
discussed in Section 3.3.2.
Consider an unknown function where the parametric form is unclear. We
have observed the following points related by the function, 𝑥1 , 𝑦1 … (𝑥𝑛 , 𝑦𝑛 ) and
define the conditional expectation of Y given X as
38
CHAPTER 3: Knowledge Gradient Methods
𝑚 𝑥 = 𝔼 𝑌 𝑋 = 𝑥].
This allows us to use the common procedure of estimating the value of the
function by weighing nearby points,
𝑛
𝑖=1 𝑤 𝑖 𝑦 𝑖
𝑛
𝑖=1 𝑤 𝑖
𝑚 𝑥 =
.
𝑚 𝑥 can be estimated as a locally weighted average, using a kernel as a weighting
function (Nadaraya 1964). This can be seen in the Nadaraya-Watson estimator
𝑚𝑕 𝑥 =
𝑛
𝑖=1 𝐾𝑕 (𝑥−𝑥 𝑖 )𝑦 𝑖
𝑛 𝐾 (𝑥−𝑥 )
𝑖
𝑖=1 𝑕
,
where h is a positive number known as the bandwidth, which is dictated at the
discretion of the modeler, and K is the kernel function where
𝐾𝑕 𝑥 − 𝑥𝑖 =
1
𝑕
𝐾(
𝑥−𝑥 𝑖
𝑕
).
Kernel functions are always positive and chosen based on the bounded region they
support. A compilation of kernel functions frequently applied as estimators are
exhibited in Table 4 and are graphically displayed in Figure 17.
39
CHAPTER 3: Knowledge Gradient Methods
Table 4- Commonly Used Kernel Functions
Kernel Function
K(u)
Uniform
Triangular
Epanechnikov
𝐾 𝑢 =
1
𝟙
2 { 𝑢 ≤1}
𝐾 𝑢 = 1 − 𝑢 ∗ 𝟙{ 𝑢 ≤1}
𝐾 𝑢 =
3
1 − 𝑢2 ∗ 𝟙{ 𝑢 ≤1}
4
𝐾 𝑢 =
15
1 − 𝑢2
16
2
∗ 𝟙{ 𝑢 ≤1}
Triweight(tricube) 𝐾 𝑢 =
35
1 − 𝑢2
32
3
∗ 𝟙{ 𝑢 ≤1}
Quartic(biweight)
Guassian
Cosine
𝐾 𝑢 =
𝐾 𝑢 =
1
2𝜋
1 2
𝑒 −2𝑢
𝜋
𝜋
𝑐𝑜𝑠 𝑢 ∗ 𝟙{ 𝑢 ≤1}
4
2
Figure 17 - Plot of Kernel Functions K(u)
40
CHAPTER 3: Knowledge Gradient Methods
Picking a suitable kernel function for estimation is simplified in Hardle et al.
(2004) who shows that different kernel functions share similar properties in
estimation rendering the choice of function negligible. The challenge now reverts to
the selection of the bandwidth size. The bandwidth can be thought of as the width of
a window centered at the data point and giving weight to any points located in the
window. In KGNP, explained in Section 3.3.2, the bandwidth selection will be
established by fixing a set of bandwidths and reweighing the estimates proposed by
each according to the accuracy of their measurements (Barut and Powell 2010).
3.3.2 KGNP Procedure and Derivation
We begin by establishing the value for the knowledge gradient for alternative
x to be
𝑣 𝐾𝐺,𝑛 = max𝔼[max𝑥 ′ 𝜇𝑥𝑛 ′+1 − max𝑥 ′ 𝜇𝑥𝑛 ′ |𝑆 𝑛 , 𝑥 𝑛 = 𝑥].
(3.2)
𝑥∈𝜒
In the KGNP approach, the estimate of 𝜇𝑥𝑛+1
is formed by the weighting of
′
estimators, 𝜇𝑥𝑘,𝑛+1
, where k denotes a different kernel estimation method and/or
′
bandwidth within a set of kernel estimation methods denoted by K . The estimator,
𝜇𝑥𝑛+1,𝑘
, can be written as
′
𝜇𝑥𝑛+1,𝑘 =
𝑥 ′ ∈𝜒
𝛽 𝑥 ′ Κ 𝑕 .𝑘 𝑥,𝑥 ′
𝑥 ′ 0,𝑛 + Κ 𝑕 .𝑘 𝑥,𝑥 𝑛 (𝛽 𝑥𝑛𝑛 𝑥 𝑛0,𝑛 +(𝛽 𝑥𝜖 𝑛 𝑦 𝑥𝑛𝑛+1 )
𝑛 +1
′
𝑥 ′ ∈𝜒 𝛽 𝑥 ′ Κ 𝑕 .𝑘 (𝑥,𝑥 )
,
where h represents the bandwidth used within kernel regression method k. The
function K(. , .) is the kernel weight used to obtain the local estimate which will be
different for every bandwidth (Barut and Powell 2010).
41
CHAPTER 3: Knowledge Gradient Methods
The total precision obtained by making measurement x is given as
𝐴𝑘𝑛+1 𝑥, 𝑥𝑛 =
𝑥′∈𝜒
𝛽𝑥𝑛+1
Κ 𝑕.𝑘 (𝑥, 𝑥 ′ ) + 𝛽𝑥𝜖𝑛 Κ 𝑕.𝑘 𝑥, 𝑥𝑛 .
′
Thus, 𝜇𝑥𝑛+1,𝑘 can be expressed as
𝜇𝑥𝑛+1,𝑘
=
𝜇𝑥𝑛,𝑘
+
𝛽𝑥𝜖𝑛 Κ 𝑕.𝑘 𝑥, 𝑥𝑛
𝐴𝑘𝑛+1
𝜇𝑥 𝑛 − 𝜇𝑥𝑛,𝑘 + 𝜍 𝑥, 𝑥𝑛 , 𝑘 𝑍,
𝑥, 𝑥𝑛
where Z is a standard normal and
𝜍 𝑥, 𝑥𝑛 , 𝑘 =
( 𝜍𝑥𝑛𝑛
2
+ 𝜍𝑥2𝑛 )
𝛽𝑥𝜖𝑛 Κ 𝑕.𝑘 𝑥, 𝑥𝑛
𝐴𝑘𝑛+1 𝑥, 𝑥𝑛
.
Now 𝜇𝑥𝑛+1
′ , given 𝑥𝑛 at time n, becomes
𝜇𝑥𝑛+1
′
𝑤𝑥𝑛+1,𝑘
=
1−
𝑘
𝛽𝑥𝜖𝑛 Κ 𝑕.𝑘 𝑥, 𝑥𝑛
𝐴𝑘𝑛 +1 𝑥, 𝑥𝑛
𝜇𝑥𝑛,𝑘
𝑤𝑥𝑛+1,𝑘
+ 𝜇𝑥 𝑛
𝑘
𝛽𝑥𝜖𝑛 Κ 𝑕.𝑘 𝑥, 𝑥𝑛
𝐴𝑘𝑛+1 𝑥, 𝑥𝑛
𝑤𝑥𝑛+1,𝑘 𝜍 𝑥, 𝑥𝑛 , 𝑘 ,
+ 𝑍
𝑘
where the estimates produced by k and h are weighed according to w (Barut and
Powell 2010). However, because future weights 𝑤 𝑛+1 are not known, predictive
weights are used given by
𝑤𝑥𝑛,𝑘
𝑥 =
′
(
2
𝜍𝑥𝑛,𝑘
𝑘𝜖𝐾
where
42
−1
+ 𝛿𝑥𝑘 )−1
,
CHAPTER 3: Knowledge Gradient Methods
𝜍𝑥𝑛,𝑘
2
= 𝑉𝑎𝑟 𝜇𝑥𝑛+1,𝑘
𝑛 +1
′ 2
′ 0,𝑛 )
𝑥′ ∈𝜒 (𝛽 𝑥′ Κ 𝑕 .𝑘 𝑥,𝑥 ) 𝑉𝑎𝑟 (𝑥
( 𝑥′ ∈𝜒 𝛽 𝑥𝑛′+1 Κ 𝑕 .𝑘 (𝑥,𝑥 ′ ))2
=
=
𝑥′ ∈𝜒
𝑛 +1 Κ
′ 2
𝛽 𝑥′
𝑕 .𝑘 𝑥,𝑥
( 𝑥′ ∈𝜒 𝛽 𝑥𝑛′+1 Κ 𝑕 .𝑘 (𝑥,𝑥 ′ ))2
.
The variable 𝛿𝑥𝑘 is the estimated bias of the kth kernel of alternative x. Kernels with
higher bias get less weight among the kernels exhibiting high variance (Barut and
Powell 2010).
Integrating our result for 𝜇𝑥𝑛+1
′ with Equation 3.2, we are able to determine
the knowledge gradient for KGNP to be
𝑣 𝐾𝐺,𝑛 = 𝔼 max𝑥′ ∈𝜒 𝑎𝑥𝑛′ (𝑥) + 𝑏𝑥𝑛′ (𝑥)𝑍 𝑆 𝑛 − max 𝜇𝑥𝑛 ,
𝑥′∈𝜒
where
𝑤𝑥𝑛+1,𝑘 1 −
𝑎𝑥𝑛 𝑥𝑛 =
𝑘
𝛽𝑥𝜖𝑛 Κ 𝑕.𝑘 𝑥, 𝑥𝑛
𝐴𝑘𝑛+1 𝑥, 𝑥𝑛
𝜇𝑥𝑛,𝑘 + 𝜇𝑥 𝑛
𝑤𝑥𝑛+1,𝑘
𝑘
𝛽𝑥𝜖𝑛 Κ 𝑕.𝑘 𝑥, 𝑥𝑛
𝐴𝑘𝑛+1 𝑥, 𝑥𝑛
,
and
𝑤𝑥𝑛+1,𝑘 𝜍 𝑥, 𝑥𝑛 , 𝑘 .
𝑏𝑥𝑛 𝑥𝑛 =
𝑘
This method will be utilized in determining optimal parameters within the
policies developed in Chapter 5 for the optimal utilization of a storage facility.
Results concerning the policies’ performance will be presented in Chapter 6
43
CHAPTER 4
Pricing Models
Decisions to inject and withdraw from a storage facility rely heavily on the
spot and forward prices of natural gas. It is important to accurately simulate these
pricing processes in order to accurately induce appropriate operational responses
of the salt cavern storage facility. The modeling process consists of considering the
market’s qualitative properties, selecting a model that matches the properties and
estimating the parameters based on historical data. From these an accurate model of
the market can be produced. This chapter will demonstrate the modeling processes
used to simulate the spot and futures pricing process for the natural gas market,
utilizing historical prices reported by NYMEX at Henry Hub in Louisiana.
4.1
Geometric Brownian Motion
An industry standard for modeling price evolution is the use of geometric
Brownian motion (GBM). GBM has been extensively studied, and its properties are
well known, including its ability to generate only positive random numbers which is
CHAPTER 4: Pricing Models
important for financial applications when the numbers represent prices (Eydeland
and Wolyniec 2003).
The GBM process in its standard form is expressed as
𝑑𝑆 𝑡
𝑆𝑡
= 𝜇 𝑑𝑡 + 𝜍𝑑𝑊𝑡 .
Here, 𝑑𝑆𝑡 signifies a random movement of the spot price 𝑆𝑡 over a small interval of
time 𝑑𝑡. The drift or rate of return of an asset as well as the volatility of the asset are
constants represented respectively by μ and σ. Lastly, 𝑑𝑊𝑡 denotes the increments
of standard Brownian motion over the same period of time. Increments of 𝑑𝑊𝑡 are
normally distributed with mean zero and standard deviation 𝑑𝑡 .
𝑊𝑡 ~ Ν 0, 𝑑𝑡
4.2
Spot Price Process
Though excellent for a variety of financial applications, geometric Brownian
motion in its standard form is ill suited to describe the evolution of energy prices.
Particularly, the natural gas market requires a model which describes the presence
of mean reversion within the spot prices. This attribute is present due to the
pressures of supply and demand on the price caused by the seasonal use of natural
gas. High demand during the winter months will put upward pressure on the price
of natural gas, and similarly the limited demand and thus increased supply of
natural gas during the spring will place downward pressure on the price. Amongst
these high and low swings, the price is centered around an average that the market
wants to regress to as it moves further away. The mean reversion within natural gas
prices can be captured in the stochastic process known as the Ornstein-Uhlunbeck
45
CHAPTER 4: Pricing Models
process. Using variables related to the topic of natural gas, the process can be
described by the following stochastic differential equation
𝑑𝑆 𝑡
𝑆𝑡
= 𝛼(𝜇𝑡 − log 𝑆𝑡 ) 𝑑𝑡 + 𝜍𝑡 𝑑𝑊𝑡 .
(4.1)
Here, μ(t) is the mean reverting level or price, and α represents the strength
of the mean reversion or the rate at which the price will try to revert back to the
mean reverting price. The volatility, 𝜍𝑡 , is now a function of time due to its
dependence on the changing mean reverting level. The process of obtaining each
parameter will be discussed in later sections of this chapter. The other variables
retain their identity from the standard GBM process (Eydeland and Wolyniec 2003).
From Equation 4.1, a closed form solution for the distributions of the
logarithms of prices can be obtained. Utilizing a simple change of variables
𝑆𝑡 = 𝑒 𝑋𝑡
(4.2)
from which we obtain
𝑋𝑡 = log 𝑆𝑡 .
Applying Itô’s Lemma, the process can be written as
𝑑𝑋𝑡 = 𝛼 𝜇𝑡 − 𝑋𝑡 𝑑𝑡 + 𝜍𝑡 𝑑𝑊𝑡 ,
where 𝑑𝑋𝑡 is the change in the log price of natural gas. Introducing a new variable
𝑌𝑡 = 𝑒 𝛼𝑡 𝑋𝑡
and again utilizing Itô’s Lemma, the process of 𝑑𝑌𝑡 can be expressed as
𝑑𝑌𝑡 = 𝛼𝜇𝑡 𝑒 𝛼𝑡 𝑑𝑡 + 𝜍𝑡 𝑒 𝛼𝑡 𝑑𝑊𝑡 .
46
(4.3)
CHAPTER 4: Pricing Models
Since 𝑑𝑌𝑡 is a normal random variable for every t, the summation of these
increments from the current time to the future time will produce the random
variable
𝑌𝑡+Δ𝑡 ~ Ν 𝑌𝑡+Δ𝑡 + 𝜇𝑡 𝑒 𝛼Δ𝑡 , 𝜍𝑡
𝑒 2𝛼 Δ 𝑡
2𝛼
.
Hence, by the definition of 𝑌𝑡 given in Equation 4.3,
𝑋𝑡+Δ𝑡 ~ Ν 𝑒 −𝛼Δ𝑡 𝑋𝑡 + 𝜇𝑡 (1 − 𝑒 −𝛼Δ𝑡 ), 𝜍𝑡
(1 − 𝑒 −2𝛼Δ𝑡 )
2𝛼
(4.4)
which is a closed form solution for the distribution for the logarithms of natural gas
spot prices (Eydeland and Wolyniec 2003). However, before simulations can be
implemented, parameters α, μ(t), and σ(t) must be determined. The parameters will
be estimated using the historical data of Henry Hub Natural Gas spot prices from the
years 2004-2009, which can be seen in Figure 18.
47
CHAPTER 4: Pricing Models
Figure 18 - Henry Hub Historical Natural Gas Spot Prices 2004 - 2009 (Enstor 2009)
16.00
14.00
Price ($/MMBtu)
12.00
Spot
Price
10.00
8.00
6.00
4.00
2.00
0.00
12/30/03
02/27/05
04/28/06
06/27/07
08/25/08
10/24/09
Date (mm/dd/yy)
Strength of Mean Reversion (α)
The parameter α is the strength or rate of mean reversion, which is the rate
at which the price will move toward a certain level. To estimate α, a linear
regression is implemented on the historical spot prices. Here, the log return of the
𝑆
spot prices, log⁡
( 𝑡+1
), are regressed against the log of the ratio between the
𝑆
𝑡
𝜇
historical average, 𝜇 , of the data and the spot price, log⁡
(𝑆 ). The slope of this
𝑡
regression is the rate of mean reversion. For this data set, α was computed to be
.0063.
48
CHAPTER 4: Pricing Models
Exponential Smoothing
The mean reverting level for this process, 𝜇𝑡 , is time dependent due to the
assumption that the long-term mean will alter over the course of many years. From
Figure 18, the average price of natural gas in 2009 appears to be approximately
$4.00 per MMBtu, while years prior show an average of $7.00 or higher. Thus, the
technique of exponential smoothing is utilized to capture the movement of this longterm mean. Similar to a moving average, exponential smoothing assigns
exponentially decreasing weights over time instead of assigning equal weights to
past observations allowing current measurements to hold more influence. A simple
expression for the exponential smoothing algorithm is
𝜇𝑡 = 𝛽𝑆𝑡 + (1 − 𝛽)𝜇𝑡−1
(4.5)
where β is the smoothing factor between [0,1] (Brown and Meyer 1960). Thus, the
smoothed statistic, in this case the mean reverting level, 𝜇𝑡 , is a function of the
weighted average of the new information receive, the spot price
𝑆𝑡 , and the
previous smoothed statistic, 𝜇𝑡−1 . Tuning β will change the smoothness of the
observations produced form the process. The smaller the β the smoother the
observations, and conversely the larger the β the more the observations will track
the noise of the original data. For the Henry Hub Natural Gas Spot Prices a β of .002
was chosen in order to capture a gradual shift in the long-term of the spot price. The
implementation of exponential smoothing can be seen in Figure19.
49
CHAPTER 4: Pricing Models
Figure 19 - Natural Gas Spot Prices with Exponential Smoothing, Beta = .002
16.00
Spot Price
14.00
Price ($/MMBtu)
12.00
Exponential
Smoothing
Price
10.00
8.00
6.00
4.00
2.00
0.00
12/30/03
02/27/05
04/28/06
06/27/07
08/25/08
10/24/09
Date (mm/dd/yy)
Volatility
Earlier it was shown that the distribution of 𝑋𝑡 , the logarithms of prices , had
a standard deviation of
𝜍𝑡 = 𝜍𝑡
(1 − 𝑒 −2𝛼Δ𝑡 )
2𝛼
(4.6)
where 𝜍𝑡 is the volatility seen in the pricing process described by Equation 4.1.
Rearranging Equation 4.6, the volatility can be written as a function of the standard
deviation of the logarithm of 𝑆𝑡 .
50
CHAPTER 4: Pricing Models
𝜍𝑡 = 𝜍𝑡
2𝛼
(1 − 𝑒 −2𝛼Δ𝑡 )
(4.7)
The value of 𝜍𝑡 can be estimated using either exponential smoothing similar
to the form in Equation 4.5 or from a rolling window of historical spot price data by
applying the formula
𝜍𝑡 =
1
𝑚−1
𝑚
𝑖=1
log 𝑆𝑖 − log 𝑆𝑖−1
Δ𝑡
1
−
𝑚
𝑚
𝑖=1
log 𝑆𝑖 − log 𝑆𝑖−1
2
Δ𝑡
where 𝑆𝑖 represents the price of natural gas at the closing of each day, which is
measured from historical data during a window of m time periods prior to the
current time period (Eydeland and Wolyniec 2003). Using this estimate in Equation
4.7, a time dependent volatility for the pricing process can be determined.
Simulation Results
Having selected a mean reversion model and estimated the appropriate
parameters based on the historical data, the model expressed in Equation 4.1 can be
used to simulate the evolution of the spot prices for natural gas. Figure 20 illustrates
several sample price paths constructed from the simulations of a year of natural gas
spot prices.
51
CHAPTER 4: Pricing Models
Figure 20 - Sample Paths for 1-Year of Simulated Natural Gas Spot Prices
7.50
7.25
7.00
6.75
Price ($/MMBtu)
6.50
6.25
Path 1
6.00
Path 2
5.75
Path 3
5.50
Path 4
5.25
Path 5
5.00
4.75
4.50
4.25
4.00
01/01/10 02/20/10 04/11/10 05/31/10 07/20/10 09/08/10 10/28/10 12/17/10
Date (dd/mm/yy)
4.3
Forward Price Process
In order to implement a forward pricing process, it is important to utilize the
simple relationship between the spot price, denoted by 𝑆𝑡 , and the price of the
forward contract maturing at T, denoted by 𝐹𝑡,𝑇 .
𝐹𝑡,𝑇 = 𝔼 𝑆𝑇 | 𝑆𝑡
This equality states that the forward price is the expectation of the spot price at time
T given the information available about the spot price at time t. The expectation is
taken over all the paths of the mean reverting process defined in Equation 4.1.
Computing the forward price depends heavily on the spot price process established
52
CHAPTER 4: Pricing Models
in Section 4.2. To begin, the change of variables established in Equation 4.2 is
utilized and allows the forward price to be expressed as
𝐹𝑡,𝑇 = 𝔼 𝑒 𝑋𝑇 | 𝑒 𝑋𝑡 .
From Equation 4.4, it is known that 𝑋𝑇 is normally distributed with mean
𝜇 = 𝑋𝑡 𝑒 −𝛼(T−t) + 𝜇𝑇 1 − 𝑒 −𝛼
𝑇−𝑡
(4.8)
and variance
𝜍2 =
𝜍𝑡 2
(1 −
2𝛼
𝑒 −2𝛼
𝑇−𝑡
(4.9)
).
The parameters include 𝜇 𝑇 which is defined as the mean reverting level of the spot
price at time T. This will be approximated by 𝜇𝑡 due to the assumption that the mean
reverting level at time T will on average be the same as at time t. The computation
for 𝜇𝑡 , as well as for the mean reversion rate α and the volatility 𝜍𝑡 will have been
conducted during the spot pricing process by the techniques outlined in Section 4.2
(Eydeland and Wolyniec 2003).
Having a closed form solution for the distribution of 𝑋𝑇 and the estimates for
the parameters, the expectation can be solved using the moment generating
function of a normal random variable yielding the price of a forward contract to be
1
2
𝐹𝑡,𝑇 = 𝔼 𝑒 𝑋 𝑇 | 𝑒 𝑋𝑡 = 𝑒 𝜇 +2 𝜍 .
Thus, the calculation of the forward price at time t will immediately follow the result
of the computation of the spot price by Equation 4.1 for the same period (Eydeland
and Wolyniec 2003).
53
CHAPTER 4: Pricing Models
In Figure 21, a single sample path of the simulation that utilizes the forward
pricing process for a year’s worth of month-ahead contracts is seen with the
respective spot price simulation.
Figure 21 - Simulated Month Ahead Forward Prices for One Year
6.80
6.60
Month-Ahead
Price
Spot Price
6.40
Price ($/MMBtu)
6.20
6.00
5.80
5.60
5.40
5.20
5.00
01/01/10 02/20/10 04/11/10 05/31/10 07/20/10 09/08/10 10/28/10 12/17/10
Date (mm/dd/yy)
From Equation 4.9 it can be seen that the volatility from the forward price
simulation will converge to zero as time approaches T. This allows for the
convergence of the forward price to the spot price at time T which is characteristic
behavior for the forward price. This relationship is given in Equation 4.10 and can
be seen clearly in Figure 22 which displays simulated forward and spot prices for
one month.
lim𝑡→𝑇 𝐹𝑡,𝑇 = 𝑆𝑇 .
54
(4.10)
CHAPTER 4: Pricing Models
Figure 22 - Month Ahead Forward Prices with Spot Prices for One Month
One Month Forward Simulation
6.40
Price ($/MMBtu)
6.20
Month Ahead
Price
Spot Price
6.00
5.80
5.60
5.40
5.20
5.00
02/01/10
02/06/10
02/11/10
02/16/10
Date (mm/dd/yy)
55
02/21/10
02/26/10
CHAPTER 5
Model and Policies
Consider a natural gas commodities trader holding a one year salt cavern
storage contract. The contract details are presented in Table 5. The trader is
participating in the physical trading of natural gas and will use the contracted
storage facility to hedge his activity within the natural gas market. This allows him
to make the decision of buying and injecting gas into the facility, withdrawing and
selling the gas, or storing the gas for later use. The trader wishes to maximize the
terminal profit from trading natural gas at the end of his storage contract. This
requires the trader to make his decisions to inject, withdraw, or store based on not
only the price of natural gas but also the operational capabilities of the storage
facility. The decisions of the trader will generate a series of cash flows that when
discounted will ascertain the terminal profit as well as the value of the storage
facility at a given point in time. This chapter will model these decisions and
operations of the storage facility as well as describe the policies used to direct the
decisions.
CHAPTER 5: Model and Policies
Table 5 - Salt Cavern Storage Contract (Enstor 2009)
Term
01/01/2010 - 12/31/2010
Type
Salt Cavern
Depth
3500 ft
Maximum Working Gas Capacity
7.5 Bcf
Initial Working Gas Capacity
0 Bcf
Maximum Injection Rate
.3 Bcf/day
Maximum Withdrawal Rate
.5 Bcf/day
Fuel Injection Loss
1.5%
Maximum/Minimum Facility Pressure
.85 PSI/foot – .2 PSI/foot
Compressor
25,000 horse power
5.1
Modeling Assumptions
A series of assumptions were made in creating this model in order to achieve
simplicity and ease in its comprehension. They are as follows:
Assumption 1: We consider the scope of the trading activity negligible
compared to overall market activity, meaning a decision to inject or withdraw
natural gas in order to participate in market transactions will hold no influence over
its price on the spot and futures market. For example, on April 2, 2010 over 147,000
forward contracts were traded equating to 1,470,000,000 MMBtu of natural gas
(CMEGroup 2010). Considering that the maximum that could be injected or
withdrawn in one day by the storage facility in Table 5 is 300,000 and 500,000
57
CHAPTER 5: Model and Policies
MMBtu’s, only .021% and .034% of the traded volume, it can be established that a
single gas storage facility is a small player within the market.
Assumption 2: Decisions will be restricted to a maximum of once a day. This
is due to the fact that simulations of the spot and forward price of natural gas
described in Chapter 4 are limited to producing daily evolutions in price. This cuts
the complexity of producing finer discretizations of the change in price. This
restriction in the frequency of gas prices results in the ability to only make decisions
as the price changes, that being once a day. It would be illogical to make multiple
decisions on stagnant information.
Assumption 3: The physical delivery of natural gas purchased or sold on the
spot market is subject to the rules and regulations of NYMEX and must take place
the day after the market transaction occurred (CMEGroup 2010). A spot purchase
of 10,000 MMBtu purchased on Monday, April 12, 2010 must be physically delivered
to the buyer on Tuesday, April 13, 2010.
Assumption 4: The physical delivery of natural gas purchased or sold on the
futures market is subject to the rules and regulations of NYMEX and takes place on
the month the contract matures and is delivered evenly throughout each day of the
month (CMEGroup 2010). Therefore, it is assumed that 1/30 of the contract is
delivered each day of the maturity month, given that a month contains 30 days.
5.2
Problem Set-Up and Terminology
Due to the availability of daily pricing data for natural gas, the problem will
be modeled in terms of days and months. If D is the set of days and M is the set of
58
CHAPTER 5: Model and Policies
months, then D = {1,2,3,..,360} and M = {1,2,3,…,12}. The model is constructed
according to a six element structure that highlights the decision variables,
exogenous information, state variables, transition functions, contribution function,
and objective function of the problem. This section outlines this framework detailing
each of the elements in accordance with the problem at hand.
5.2.1 Parameters
𝑋𝑚𝑎𝑥 - upper volume limit of working gas in the storage facility (Bcf)
𝑑- depth of cavern in feet
𝐼
𝑟𝑚𝑎𝑥
- the maximum injection rate at the storage facility (Bcf/day)
𝑊
𝑟𝑚𝑎𝑥
- the maximum withdrawal rate or deliverability at the storage facility
(Bcf/day)
𝐼
𝑟𝑚𝑖𝑛
- the minimum injection rate at the storage facility (Bcf/day)
𝑊
𝑟𝑚𝑖𝑛
- the minimum withdrawal rate or deliverability at the storage facility (Bcf/day)
𝑝𝑚𝑎𝑥 - the maximum pressure within the storage facility (PSI/foot)
𝑝𝑚𝑖𝑛 - the minimum pressure within the storage facility (PSI/foot)
𝐶𝑚𝑎𝑥 - the maximum horsepower of wellhead compressor
𝛼- the percent fuel rate loss for injection at a storage facility
𝛾- a risk-free discount factor
59
CHAPTER 5: Model and Policies
5.2.2 Decision Variables and Constraints
The decision whether to inject or withdraw is considered with respect to the
prices of both the spot market and the forward market. Thus, this problem is
composed of two decision variables x and y, which represents the volume of natural
gas injected or withdrawn based on a transaction in the spot or forward market
respectively.
𝑥𝑑,𝑑+1,𝑚 - the volume of natural gas injected or withdrawn on day d+1 from facility
based on the transaction on spot market on day d and month m, (Bcf)
𝑥𝑑,𝑑+1,𝑚 > 0 𝑖𝑓 𝑖𝑛𝑗𝑒𝑐𝑡𝑒𝑑
𝑥𝑑,𝑑+1,𝑚 = 0 𝑖𝑓 𝑛𝑜 𝑖𝑛𝑗𝑒𝑐𝑡𝑖𝑜𝑛 𝑜𝑟 𝑤𝑖𝑡𝑕𝑑𝑟𝑎𝑤𝑎𝑙 𝑜𝑐𝑐𝑢𝑟𝑠
𝑥𝑑,𝑑+1,𝑚 < 0 𝑖𝑓 𝑤𝑖𝑡𝑕𝑑𝑟𝑎𝑤𝑛
𝑦𝑚 ,𝑀 - the volume of natural gas to be injected or withdrawn from facility in month
M based on the transaction in futures market that took place on day d during
month m, (Bcf)
𝑦𝑑,𝑚 ,𝑀 > 0 𝑖𝑓 𝑖𝑛𝑗𝑒𝑐𝑡𝑒𝑑
𝑦𝑑,𝑚 ,𝑀 = 0 𝑖𝑓 𝑛𝑜 𝑖𝑛𝑗𝑒𝑐𝑡𝑖𝑜𝑛 𝑜𝑟 𝑤𝑖𝑡𝑕𝑑𝑟𝑎𝑤𝑎𝑙 𝑜𝑐𝑐𝑢𝑟𝑠
𝑦𝑑,𝑚 ,𝑀 < 0 𝑖𝑓 𝑤𝑖𝑡𝑕𝑑𝑟𝑎𝑤𝑛
The sign of a decision corresponds to the increase or decrease in capacity.
The positive decision variables, 𝑥𝑑,𝑑+1,𝑚 , 𝑦𝑚 ,𝑀 , will represent injections of natural
60
CHAPTER 5: Model and Policies
gas while negative decision variables will reflect a withdrawal of natural gas from
the facility.
The volume of gas injected or withdrawn per day is constrained by the
operational capabilities of the storage facility. Thus, the decision of how much to
inject and withdraw is bounded by the withdrawal capacity and injection capacity of
the facility at time t, where 𝐼𝑑+1 and 𝑊𝑑+1 are considered as system-state variables
and are defined in Section 5.2.4.
𝑥𝑑,𝑑+1,𝑚 +
𝑦𝑑,𝑚 −1,𝑚
≤ 𝐼𝑑+1
30
𝑥𝑑,𝑑+1,𝑚 +
𝑦𝑑,𝑚 −1,𝑚
≤ 𝑊𝑑+1
30
It is also important to note the division of the forward decision variable to
reflect the daily distribution of a forward contract throughout the month as stated in
Assumption 4.
An additional constraint arises due to Assumption 3 and 4 where all natural
gas transactions must be settled by the delivery date. For the spot market this is the
day following the transaction, the operation time of injection and withdrawal must
take place within the 24 hour time block of the delivery date. Thus, the decision
variables are bounded by the daily injection and withdrawal rate, 𝑟 𝐼 and 𝑟 𝑊 , for the
specified day of delivery.
1
𝑟𝑑𝐼 +1
𝑥𝑑,𝑑+1,𝑚 +
1 𝑦𝑑,𝑚 −1,𝑚
≤1
𝐼
30
𝑟𝑑+1
𝑤𝑕𝑒𝑛 𝑥𝑑,𝑑+1,𝑚 , 𝑦𝑑,𝑚 −1,𝑚 > 0
61
CHAPTER 5: Model and Policies
1
𝑟𝑑𝑊+1
1
𝑟𝑑𝐼 +1
1
𝑟𝑑𝑊+1
𝑥𝑑,𝑑+1,𝑚 +
1 𝑦𝑑,𝑚 −1,𝑚
≤1
𝐼
30
𝑟𝑑+1
𝑤𝑕𝑒𝑛 𝑥𝑑,𝑑+1,𝑚 < 0, 𝑦𝑑,𝑚 −1,𝑚 > 0
𝑥𝑑,𝑑+1,𝑚 +
1 𝑦𝑑,𝑚 −1,𝑚
≤1
𝑊
30
𝑟𝑑+1
𝑤𝑕𝑒𝑛 𝑥𝑑,𝑑+1,𝑚 > 0, 𝑦𝑑,𝑚 −1,𝑚 < 0
𝑥𝑑,𝑑+1,𝑚 +
1 𝑦𝑑,𝑚 −1,𝑚
≤1
𝑊
30
𝑟𝑑+1
𝑤𝑕𝑒𝑛 𝑥𝑑,𝑑+1,𝑚 < 0, 𝑦𝑑,𝑚 −1,𝑚 < 0
The rates within these constraints are considered System State variables and will be
discussed in Section 5.2.4.
5.2.3 Exogenous Variables
𝑠𝑑+1,𝑚 - the change in the natural gas spot price at day d, month m ($)
𝐹𝑑+1,𝑚,𝑚 +1 - the change in the natural gas futures price at day d and month m for
physical delivery during month m+1 ($)
5.2.4 System State
𝑋𝑑 - the volume of working gas at the storage facility at day d. 0 ≤ 𝑋𝑑 ≤ 𝑋𝑚𝑎𝑥 , (Bcf)
𝐼𝑑 - Injection Capacity at day d, (Bcf)
𝑊𝑑 - Withdrawal Capacity at day d, (Bcf)
𝑟𝑑𝐼 - the injection rate at the storage facility at day d, (Bcf/day)
𝑟𝑑𝑊 - the withdrawal rate or deliverability at the storage facility at day d, (Bcf/day)
𝑝𝑑 - the pressure within the storage facility at day d, (PSI)
62
CHAPTER 5: Model and Policies
Together, these variables compose the state of the system which provides the
information needed to make decisions whether to inject or withdraw natural gas.
𝑆𝑑 𝑋𝑑 , 𝐼𝑑 , 𝑊𝑑 , 𝑆𝑑 , 𝐹𝑑,𝑚 ,𝑚+1 , 𝑟𝑑𝐼 , 𝑟𝑑𝑊 , 𝑝𝑑 - the state of the system/storage at day d,
5.2.5 Transition Functions
Transition of Market Prices:
𝑠𝑑+1 = 𝑠𝑑 + 𝑠𝑑+1
𝐹𝑑+1 ,𝑚 ,𝑚+1 = 𝐹𝑑,𝑚 ,𝑚 +1 + 𝐹𝑑+1 ,𝑚 ,𝑚 +1
Transition of Storage Capacity and Operational Elements:
𝑊𝑑+1 = 𝑋𝑑+1 + 𝑥𝑑,𝑚 +
𝑦𝑑,𝑚 −1,𝑚
30
𝐼𝑑+1 = 𝑋𝑚𝑎𝑥 − 𝑊𝑑+1
𝑋𝑑+1
𝑦𝑑,𝑚 −1,𝑚
𝑋𝑑 + 𝛼 𝐼 𝑥𝑑,𝑚 + 𝛼 𝐼
,
30
𝑦𝑑,𝑚 −1,𝑚
𝑋𝑑 + 𝛼 𝐼 𝑥𝑑,𝑚 +
,
30
𝑋𝑑 ,
=
𝑦𝑑,𝑚 −1,𝑚
𝑋𝑑 + 𝑥𝑑,𝑚 + 𝛼 𝐼
,
30
𝑦𝑑,𝑚 −1,𝑚
𝑋𝑑 + 𝑥𝑑,𝑚 +
,
30
𝐼
𝑟𝑑+1
𝑖𝑓 𝑥𝑑,𝑚 , 𝑦𝑑,𝑚 −1,𝑚 > 0
𝑖𝑓 𝑥𝑑,𝑚 > 0, 𝑦𝑑,𝑚 −1,𝑚 < 0
𝑖𝑓 𝑥𝑑,𝑚 = 0, 𝑦𝑑,𝑚 −1,𝑚 = 0
𝑖𝑓 𝑥𝑑,𝑚 < 0, 𝑦𝑑,𝑚 −1,𝑚 > 0
𝑖𝑓 𝑥𝑑,𝑚 , 𝑦𝑑,𝑚 −1,𝑚 < 0
𝐼
𝐼
𝑟𝑚𝑎𝑥
− 𝑟𝑚𝑖𝑛
𝐼
=−
∗ 𝑋𝑑+1 + 𝑟𝑚𝑎𝑥
𝑋𝑚𝑎𝑥
𝑊
𝑟𝑑+1
=
𝑊
𝑊
𝑟𝑚𝑎𝑥
− 𝑟𝑚𝑖𝑛
𝑊
∗ 𝑋𝑑+1 + 𝑟𝑚𝑖𝑛
𝑋𝑚𝑎𝑥
63
CHAPTER 5: Model and Policies
𝑝𝑑+1 =
𝑝𝑚𝑎𝑥 − 𝑝𝑚𝑖 𝑛
∗ 𝑋𝑑+1 + 𝑝𝑚𝑖𝑛
𝑋𝑚𝑎𝑥
𝐶𝑑+1 =
𝐶𝑚𝑎𝑥
∗ 𝑝𝑑+1
𝑝𝑚𝑎𝑥
Transition of Operational Costs:
𝑠𝑝𝑜𝑡 ,𝐼
𝑐𝑑+1
= 𝐶𝑑+1 ∗ 42.44 ∗ 60 ∗ (
1
𝐼
𝑟𝑑+1
∗ 24 ∗ 𝑥𝑑 ) ∗ 𝑠𝑑 /1000000
𝑠𝑝𝑜𝑡 ,𝑊
𝑐𝑑+1
= .01 ∗ 𝑥𝑑 ∗ 1000000
𝑓𝑜𝑟𝑤𝑎𝑟𝑑 ,𝐼
𝑐𝑑+1
= 𝐶𝑑+1 ∗ 42.44 ∗ 60 ∗ (
𝑓𝑜𝑟𝑤𝑎𝑟𝑑 ,𝑊
𝑐𝑑+1
1
𝐼
𝑟𝑑+1
= .01 ∗
∗ 24 ∗
𝑦𝑑,𝑚 −1,𝑚
30
𝑦𝑑,𝑚 −1,𝑚
) ∗ 𝐹𝑑,𝑚 −1,𝑚 /1000000
30
∗ 1000000
5.2.6 Contribution Function
The reward for an action is denoted by 𝑟 𝑥𝑑,𝑚 , 𝑦𝑑,𝑚 −1,𝑚 , 𝑠𝑑 where 𝑥𝑑,𝑚 and
𝑦𝑑,𝑚 −1,𝑚 are associated with the operational decision to injection, withdraw or do
nothing during day d. Rewards for injections will be negative quantities considering
injections are equated with commodity purchases and cash outflows. Withdrawals
will have positive rewards corresponding with commodity sales and cash inflows.
64
CHAPTER 5: Model and Policies
𝑟 𝑥𝑑,𝑚 , 𝑦𝑑,𝑚 −1,𝑚 , 𝑠𝑑
− 𝛼 𝐼 𝑠𝑡 𝑥𝑑,𝑚 ∗ 106 + 𝑐𝑡𝑠𝑝𝑜𝑡 ,𝐼 + 𝛼 𝐼 𝐹𝑡,𝑇
𝑦𝑑,𝑚 −1,𝑚
𝑓𝑜𝑟𝑤𝑎𝑟𝑑
∗ 106 + 𝑐𝑡
30
,𝐼
𝑖𝑓 𝑥𝑑,𝑚 , 𝑦𝑑,𝑚 −1,𝑚 > 0
− 𝛼 𝐼 𝑠𝑡 𝑥𝑑,𝑚 ∗ 106 + 𝑐𝑡𝑠𝑝𝑜𝑡 ,𝐼 + 𝐹𝑡,𝑇
𝑦𝑑,𝑚 −1,𝑚
30
𝑓𝑜𝑟𝑤𝑎𝑟𝑑 ,𝑊
∗ 106 − 𝑐𝑡
𝑖𝑓𝑥𝑑,𝑚 > 0, 𝑦𝑑,𝑚 −1,𝑚 < 0
0
=
𝑠𝑡 𝑥𝑑,𝑚
𝑠𝑡 𝑥𝑑,𝑚
𝑖𝑓 𝑥𝑑,𝑚 , 𝑦𝑑,𝑚 −1,𝑚 = 0
𝑦𝑑,𝑚 −1,𝑚
𝑓𝑜𝑟𝑤𝑎𝑟𝑑 ,𝐼
𝑠𝑝𝑜𝑡 ,𝑊
∗ 106 − 𝑐𝑡
+ 𝛼 𝐼 𝐹𝑡,𝑇
∗ 106 − 𝑐𝑡
30
𝑖𝑓 𝑥𝑑,𝑚 < 0, 𝑦𝑚 −1,𝑚 > 0
𝑦𝑑,𝑚 −1,𝑚
𝑓𝑜𝑟𝑤𝑎𝑟𝑑 ,𝑊
∗ 106 − 𝑐𝑡𝑠𝑝𝑜 𝑡,𝑊 + 𝐹𝑡,𝑇
∗ 106 − 𝑐𝑡
30
𝑖𝑓 𝑥𝑑,𝑚 , 𝑦𝑑,𝑚 −1,𝑚 < 0
5.2.7 Objective Function
The storage value can be computed by solving the following optimization function
𝑉 𝑋𝑖 , 𝑌𝑖 = max𝜋𝜖 Π 𝔼
360
𝑑=0 𝛾
∗ 𝑟(𝑆𝑑 , 𝑋𝑑𝜋 ( 𝑆𝑑 ), 𝑌𝑑𝜋 (𝐹𝑑,𝑚 ,𝑚 +1 )) ,
where 𝛾 is a discount factor associated with the specific time. The objective function
attempts to maximize the expectation of the sum of the discounted rewards over the
year according to a policy 𝜋, which is determined from a set of possible values П.
5.3
Policies
Speculators in the natural gas market aim to earn the maximum profit from
the purchase and sale of natural gas. Thus, policies are derived from trading
algorithms which aim at discerning a proper time and hopefully low price to buy
and conversely a high price for which to sell. A common trading algorithm is “swing
65
CHAPTER 5: Model and Policies
trading” where the asset is bought or sold at or near the end of an up or down price
swing, or price trend. The determination of when these swings occur varies and the
policies will explore a few of the swing trading techniques in order to best try and
capture the trends of the natural gas market. Policies will also be tested which make
use of the seasonality of prices and make decisions based at what point in the year is
the decision being made. This section will describe all the policies tested in the
model.
5.3.1 Swing Trading
As stated earlier, swing trading endeavors to buy and sell an asset when the
price approaches the end of a trend. For example, during a downward trend in price,
a trader will aim to buy the asset at the bottom of the trend, riding the swing
upwards and anticipating the sell of the asset at the top of price oscillations. Various
swing trading strategies have been developed in order to anticipate when these
swings occur. One simple system popular in the 1970s was Donchian’s Four-Week
Rule, which instructed the trader to buy when the price of an asset rose above the
high of prior four weeks and sell when the price fell below the low of the same four
weeks. Another technique stipulates a purchase of the asset after three consecutive
highs and a sale following three consecutive lows.
In order to anticipate the price oscillations of the natural gas market, a
technique based on the percentage of price movement has been chosen. This
technique removes actual price from the decision, moving away from strategies
which base decisions on tangible price levels. A good image for this technique is a
66
CHAPTER 5: Model and Policies
filter, where small price variations are removed from the decision process, leaving
the algorithm less susceptible to false trends (Merrill 1977). A simple example of the
strategy would be to base buy and sell decisions on price movements of 5% or
better over a certain period of time. Figures 23 and 24 display the strategies which
only identifies price changes of greater than 5% and 10% respectively. The red line
displays the trend, and the max and min of these lines represent a sell and buy
decision.
The policies utilized in the model in Section 5.2, will incorporate both long
term and short term windows of time independently for evaluating the price change
as well as combine the two perspectives in order to gain greater accuracy in
predicting market trends. The policies will also be tuned in order to discern the
optimal percentage of price movement, as well as other parameters for which to
base the buy and sell decisions on. This will be achieved through the various
Knowledge Gradient methods described in Chapter 3.
67
CHAPTER 5: Model and Policies
Figure 23 – Swing Trend with 10% Filtering
6.25
6.00
5.75
Price ($/MMBtu)
5.50
5.25
5.00
4.75
4.50
4.25
4.00
0
50
100
150
200
250
300
350
200
250
300
350
Day
Figure 24 - Swing Trend with 5% Filtering
6.25
6.00
5.75
Price ($/MMBtu)
5.50
5.25
5.00
4.75
4.50
4.25
4.00
0
50
100
150
Day
68
CHAPTER 5: Model and Policies
5.3.2 Policy 0 – Short-Term Speculation
This policy concerns itself with earning the maximum profit by capitalizing
on short-term price movements of natural gas. The policy will be utilizing swing
trade methods where the change in price will depend on the pricing data from the
prior three days. Thus, a decision will be made if the percent of change in price after
three days breaches a particular level. The decision level will also be a percent, and
may be set differently for the percent gain which stipulates a sell decision and the
percent loss which indicates a sell decision. The variables of the policy can be
explicitly expressed as
ρ = the percentage level of price increase needed to instigate withdrawal
φ = the percentage level of price decrease needed to instigate injection
𝜃𝑑 = the percentage level of price increase or decrease at time d based on pricing
data of last 3 days
The last question remaining before being able to express the policy is how
much natural gas should be injected or withdrawn when the criteria for the
percentage change is reached. An assumption is made that the maximum amount of
natural gas would be bought or sold in order to capitalize on the price which has
matched the policy’s criteria. The amount that can be purchased or sold is bounded
respectively by the maximum capacity the facility could inject or withdraw on the
𝐼
day of delivery d+1. It is expressed by the injection and withdrawal rates 𝑟𝑑+1
and
𝑊
𝑟𝑑+1
defined in Section 5.2.4. Thus Policy 0 can be expressed in mathematical
notation by
69
CHAPTER 5: Model and Policies
𝐼
𝑥𝑑,𝑑+1 = 𝑟𝑑+1
∗ 𝟙{ 𝜃 𝑑
<𝜑}
𝑊
− 𝑟𝑑+1
∗ 𝟙{𝜃 𝑑 >𝜌}
where 𝑥𝑑,𝑑+1 is the decision made on day d for the amount of natural gas to be
injected or withdrawn at time d+1. Variables φ and ρ are tunable and obtaining their
optimal values will be achieved through Knowledge Gradient Algorithms discussed
in Section 3.4.
5.3.3 Policy 1 – Long Trend Speculation
This policy is identical to the policy described in Section 5.3.2, except the
percentage change in price will be determined by taking into account the price four
weeks prior to the current day instead of three days prior. The variables are the
same and similarly φ and ρ will be tuned to represent the optimal operation of the
policy.
5.3.4 Policy 2 – Long and Short Trend Speculation
The policy described here extends the policies represented in Sections 5.3.2
and 5.3.3 by encompassing a look at both long and short term trends. This is
expected to enable the algorithm to better identify the transition between trends up
and down having information on both the long and short market activity.
𝜃𝑑𝑠𝑕𝑜𝑟𝑡 = the percentage level of price increase or decrease at time d based on
pricing data of last three days
𝑙𝑜𝑛𝑔
𝜃𝑑
=
the percentage level of price increase or decrease at time d based on
pricing data of last four weeks
70
CHAPTER 5: Model and Policies
For this policy there will be four tunable parameters expressing certain
criteria from which a decision to inject, withdraw, or do nothing will be instated.
ρs = the percentage level of price increase for short term trend
φs = the percentage level of price decrease for short term trend
ρl = the percentage level of price increase for long term trend
φl = the percentage level of price decrease for long term trend
Based on the price at day d and the percentage change from four weeks and three
days, a decision will be made. For example, if a certain price has a four week percent
gain in price that surpasses ρl as well as a three day gain that surpasses ρs , then the
assumption can be made that the price of natural gas is in a steady upward trend.
Thus, the decision will be to do nothing in order to follow the trend upwards and
withdraw to sell at an even higher price. This could be indicated when the four week
percentage gain in price does not exceed ρl , and the three day percentage is a gain
that does not exceed ρs or is a loss. These scenarios suggest the beginning of a
downward trend, and thus a peak in natural gas price. A decision should be made to
withdraw and try to capture the price peak before the downward trend gets too far
into its run. Table 3 displays all possible scenarios surrounding the criteria and the
resulting decisions. Once again, a decision to withdraw or inject would call for the
sale or purchase of the maximum amount the facility could withdraw or inject on
the day of delivery, d+1.
71
CHAPTER 5: Model and Policies
Table 6 - Mixed Trend Decisions
#
4
Week
Trend
𝜃𝑑
1
Up
Yes
2
Up
3
3 Day 𝜃 𝑠𝑕𝑜𝑟𝑡 >𝛒
𝐬
𝑑
Trend
𝜃𝑑𝑠𝑕𝑜𝑟𝑡 < 𝛗𝐬
Trend
N/A
Up
Yes
N/A
Up
Nothing
Yes
N/A
Up
No
N/A
Flat
Sell
Up
Yes
N/A
Down
N/A
Yes
Down
Sell
4
Up
Yes
N/A
Down
N/A
No
Flat
Nothing
5
Up
No
N/A
Up
Yes
N/A
Flat
Nothing
6
Up
No
N/A
Up
No
N/A
Flat
Nothing
7
Up
No
N/A
Down
N/A
Yes
Down
Sell
8
Up
No
N/A
Down
N/A
No
Flat
Sell
9
Down
N/A
Yes
Up
Yes
N/A
Flat
Buy
10
Down
N/A
Yes
Up
No
N/A
Flat
Buy
11
Down
N/A
Yes
Down
N/A
Yes
Down
Nothing
12
Down
N/A
Yes
Down
N/A
No
Flat
Nothing
13
Down
N/A
No
Up
Yes
N/A
Up
Buy
14
Down
N/A
No
Up
No
N/A
Flat
Nothing
15
Down
N/A
No
Down
N/A
Yes
Down
Nothing
16
Down
N/A
No
Down
N/A
No
Flat
Buy
𝑙𝑜𝑛𝑔
>𝛒𝐥
𝑙𝑜𝑛𝑔
𝜃𝑑
<𝛗𝐥
Decision
5.3.5 Policy 3 – Forward Hedging
This policy will utilize the forward market and its ability to hedge the
positions taken on in the spot market. Participating in transactions in the forward
market will protect from unexpected price fluctuations in the future. For example, if
a decision to inject natural gas is made, the trader is anticipating a rise in prices in
72
CHAPTER 5: Model and Policies
the future. However, this leaves the trader exposed to severe losses if prices drop.
To avoid this exposure, the trader could decide to lock in the future price of natural
gas by selling on the forward market. If the trader sells an amount of natural gas on
the forward market equivalent to what was purchased on the spot, then there is
guaranteed profit for the trader even if the price of natural gas on the spot market
falls. The downside to this strategy is that covering the spot position 1:1 with a
forward transaction eliminates the chance for large gains by an upwards movement
of the spot price. Thus, a strategy would be to balance the risk and reward of the
spot position by hedging only a certain percentage of the spot position with a
forward position.
This policy takes the decision of the amount to inject or withdraw from the
spot market and hedge a percentage of the position with a decision to inject or
withdraw in the future using the forward market. In order to avoid the complexities
of a high dimensional problem, the spot decision will be derived from the policy in
5.3.4, using the optimal parameters found and fixing them for the current policy.
The variables for this policy are expressed by
𝜂 = the percent of spot position to hedge using forward transaction
ρs = the percentage level of price increase for short term trend
φs = the percentage level of price decrease for short term trend
ρl = the percentage level of price increase for long term trend
φl = the percentage level of price decrease for long term trend
73
CHAPTER 5: Model and Policies
where, the last four being the optimal parameters found for Policy 2 which will be
fixed. As in prior policies, 𝑥𝑑,𝑑+1 is the decision made on day d for the amount of
natural gas to be injected or withdrawn from the spot market at time d+1. This will
be derived from the fixed parameters above in order to produce a policy for the
forward decision expressed by
𝑦𝑑,𝑚 ,𝑚 +1 = 𝜂 ∗ 𝑥𝑑,𝑑+1 ,
where, 𝑦𝑑,𝑚,𝑚 +1 is the forward decision made on day d and month m for delivery in
month m+1, and η is a percent that will be tuned in order to obtain the optimal
hedging percentage.
74
CHAPTER 6
Results
With the aggregation of the natural gas storage model and the policies
containing finely tuned parameters courtesy of the KGNP algorithm, estimates of the
value of natural gas storage can be obtained. This chapter will present the results of
the KGNP algorithm and subsequently the value of storage associated with each of
the policies in Chapter 5. We endeavor to validate our results by garnering
responses from our policies to the exposure of varying sample paths. We also
compare our results to the current industry value for storage provided by Enstor
Operating Company, a natural gas storage services company owned by Iberdrola
Renewables.
6.1
Model and Algorithm Set-Up
The characteristics of the natural gas storage facility which was utilized as
parameters within the storage model can be found within the storage contract in
Table 7.
CHAPTER 6: Results
Table 7 - Salt Cavern Storage Contract with Model Parameters
Term
01/01/2010 - 12/31/2010
Type
Salt Cavern
Depth
3500 ft
Maximum Working Gas Capacity
7.5 Bcf
Initial Working Gas Capacity
0 Bcf
Maximum Injection Rate
.3 Bcf/day
Maximum Withdrawal Rate
.5 Bcf/day
Fuel Injection Loss
1.5%
Maximum/Minimum Facility Pressure
.85 PSI/foot – .2 PSI/foot
Compressor
25,000 horse power
For all policies, no assumptions were made as to the performance of specific
alternatives, and thus equivalent priors were utilized within the KGNP algorithm.
The bandwidths needed for the KGNP algorithm reflect the range of alternatives
used within the policies. Due to comparable ranges amongst the policies, we were
able to apply the four bandwidths throughout without variation. Both the priors and
bandwidths for the KGNP algorithm used to solve our storage valuation problem can
be found in Table 8.
Table 8 - KGNP Priors and Bandwidths (h)
𝜇𝑥0
𝛽𝑥0
𝛽𝑥𝜖
𝑕1
𝑕2
𝑕3
𝑕4
0
1
1014
1
106
.05
.6
2
5
76
CHAPTER 6: Results
6.2
Policy 0: Results for Short-Trend Speculation
Denoted as Policy 0 in Chapter 5, this policy capitalizes on the short-term
price movements of natural gas to maximize operating profit of the storage facility.
The function of the KGNP algorithm is to perform sequential measurements over the
alternatives of the policy’s parameters, and after a budgeted amount of
measurements will produce parameters that will show the best performance from
the policy. For this policy, the KGNP algorithm produces parameters for ρ, the
percentage level of price increase needed to instigate withdrawal, and φ, the
percentage level of price decrease needed to instigate injection, where ρ and φ ∈
(0%,10%).
Before revealing the parameters chosen by the KGNP algorithm, we will take
a graphical look at the KGNP algorithm measurement choices by juxtaposing the
measurement estimates with a plot of the knowledge gradient.
Figure 25 - Policy 0: Second Measurement
77
CHAPTER 6: Results
Figure 25 shows the first two measurements chosen by the KGNP algorithm.
The algorithm will strive to pick measurements that will result in the maximum
knowledge obtained. Typical of knowledge gradient policies, KGNP has chosen to
first measure at the extremities of the alternatives. The first measurement is at (ρ,
φ) = (0%, 0%) where it results in a negative estimate for the value. Notice in the
graph of the estimation that the point equating to alternative (0%, 0%) is not the
only alternative to shift in value. In fact, a great deal of alternatives in proximity to
the measured alternatives are estimated to have a negative value as well, despite
having not been measured. This is due to the correlation between alternatives which
are assessed within the KGNP algorithm, allowing us to make assumptions
regarding the estimates other alternatives. The second measurement is taken at
another extremity, (0%, 10%), yet with greater success, finding that this
alternatives yields a large value for the measurement. Despite two different values
received from the two measurements, the graph depicting the value of the
knowledge gradient displays a similar account of the knowledge obtained regarding
the alternatives.
The estimates and knowledge gained with prior measurements is important
when determining the next measurement. Figure 26 shows several subsequent
measurements branching out from (0%, 10%) and finding equally successful
estimates at nearby alternatives. Yet as shown by the knowledge gradient, the value
of knowledge is still high for many of the alternatives, leading us to assume that the
KGNP algorithm will procure measurements with regards to those alternatives in
subsequent iterations.
78
CHAPTER 6: Results
Figure 26 - Policy 0: 25 Measurements
This is continued in Figure 27 as the surface begins to form from the
measurements surveyed by the KGNP algorithm.
Figure 27- Policy 0: 100 Measurements
79
CHAPTER 6: Results
After 500 measurements, the KGNP estimated surface for the annual profit
produced from the operations of the storage facility with parameters ρ and φ can be
observed in Figure 28.
Figure 28 - Policy 0: 500 Measurements
The maximum value of the surface is what the KGNP has deemed the best
estimate for the annual profit from storage operations, using the corresponding
parameters are considered optimal for this policy. These values are summarized in
Table 9.
As expressed in the objective function in
ρ (%)
1.0%
φ (%)
5.5%
Maximum Value ($) $15,578,000
Table 9 - Policy 0: Results
Section 5.2.7, the value of storage is equal to the
expected cash flows rendered by its operations.
Thus, storage takes on the value of $15,578,000
80
CHAPTER 6: Results
for Policy 0, where a 1% price increase in the spot price dictates the withdrawal and
sale of natural gas, and a 5.5% decrease in price calls for the purchase and injection
of natural gas.
An interesting observation can be found in the chasm between ρ, the price
increase required to withdraw and φ, the price decrease required for injection.
These values indicate that a larger percentage of decrease in price must occur
before injecting gas, compared to the percent increase required for withdrawal. This
can be contributed to the significantly higher costs necessary for injection versus
withdrawal, due to the cost of using the compressor as well as the 1.5% of fuel loss
associated with the injection process. The result of this higher cost for injection is
that greater incentive is required to purchase the natural gas from the market in
order to account for the added costs. Thus, our policy waits for a larger drop in price
before making the decision to inject.
6.3
Policy 1: Results for Long Trend Speculation
As stated in Chapter 5, Policy 1 capitalizes on the long-term price movements
of natural gas to maximize operating profit of the storage facility, looking at trends
within four weeks of the current date. The parameters ρ and φ maintain their
identity from the previous policy. From our results for Policy 0, we expect that the
parameters in Policy 1 will show the same disparity between the price increase
required to withdraw, and the price decrease required for injection. We also expect
the values for ρ and φ to increase due to the larger range in price movement that
can result over a four week period. Therefore, the KGNP algorithm will measure
alternatives where ρ and φ ∈ (0%,15%).
81
CHAPTER 6: Results
We show again the evolution of the estimated surface and knowledge
gradient surface where Figure 29 shows the first measurement. Once again, the
algorithm tested first at an extremity. This figure also clearly displays the effects of
correlation in the surrounding alternatives many of which have adjusted estimates
based merely on the first measurement.
Figure 29 - Policy 1: First Measurement
In Figure 30, the evolution of the surface is extended through more
measurements, and clearly shows the varying degrees of correlation between the
measurement seen in the middle of the surface and surrounding alternatives as
their estimated values cascade down from the pinnacle of the measured alternative.
82
CHAPTER 6: Results
Figure 30 - Policy 1: 100 Measurements
After 500 measurements, the KGNP algorithm procures the estimated surface
for the storage value which can be observed in Figure 31.
Figure 31 - Policy 1: 500 Measurements
83
CHAPTER 6: Results
Table 10 - Policy 1: Results
ρ (%)
2.0%
φ (%)
9.5%
Maximum Value ($)
$18,579,000
The maximum value of the surface and the
corresponding parameters are expressed in
Table 10. In every respect, our expectations
have been met. There is still the discrepancy between ρ and φ, due to the larger cost
for injection versus withdrawal; and both parameters received a boost from their
values in Policy 0 in order to adjust to the larger fluctuations in price that occur
during a four week interval.
6.4
Policy 2 – Long and Short Trend
Recalling from Section 5.3.4, in order to use both the long term and short
term price information, this policy contains four tunable parameters expressing
certain criteria from which a decision to inject, withdraw, or do nothing will be
instated.
ρs = the percentage level of price increase for short term trend
φs = the percentage level of price decrease for short term trend
ρl = the percentage level of price increase for long term trend
φl = the percentage level of price decrease for long term trend
However, unlike the technique used in Policy 0 and Policy 1, these four
parameters will not be tuned by the KGNP algorithm simultaneously due to
dimensionality constraints. Thus, two of the parameters will be held constant as the
KGNP algorithm estimates the additional parameters. The KGNP algorithm will then
be run again with the fixed parameters now being the estimates from the previous
84
CHAPTER 6: Results
KGNP process. This exchange between the parameters acting as fixed variables and
being tuned by the KGNP algorithm will continue until the value for storage appears
to converge.
Table 11 describes the exchange in tuning the parameters, where the
highlighted parameters are the ones tuned by the KGNP algorithm.
Table 11 - Policy 2: Results
KGNP Run
ρs
φs
ρl
φl
Max Value
1
1%
5%
7.5%
7.5%
$13,953,000
2
2%
9%
7.5%
7.5%
$14,722,300
3
2%
9%
6.5%
13%
$18,502,000
…
…
…
…
…
…
Optimal
Estimate
2.5%
9.5%
1%
14.5%
$18,742,000
6.5
Policy 3 – Forward Hedge
Policy 3 determines the appropriate percentage of the spot decision to hedge
using a purchase or sale within the forward market. We attempt to hedge decisions
from Policy 2 by holding the optimal parameters found in Policy 2 constant as the
KGNP algorithm searches over the percentages. Unlike the technique in Policy 2, an
exchange in the fixed variables does not occur since we are attempting to acquire
the specific hedging strategy for a particular policy. We expect the value obtained
for this policy to be less than that of Policy 2 since reward is being given up in
exchange for protection from risk.
85
CHAPTER 6: Results
Table 12 – Policy 3 Parameters and results
ρs
2.5%
φs
9.5%
ρl
1%
φl
14.5%
Percent Hedged
60%
Max Value
6.6
$6,120,500
Policy Analysis
This section we will analyze our policies using fluctuating variables
embedded within our model, such as the withdrawal rate and market volatility. We
will also compare the results of our policies with actual data from recently traded
storage volumes.
6.6.1 Withdrawal Rate Adjustment
With our optimal parameters determined in 6.2 and 6.3, Policy 0 and Policy 1
were simulated one hundred times each, increasing the maximum withdrawal rate
.01 Bcf/day every iteration. Therefore, we were able to achieve storage values for
each policy with withdrawal rates ranging from 0 to 1 Bcf/day. The results are
graphically represented in Figure 32.
From the figure, we can see a deficiency in the value of storage when the
withdrawal rate is between 0 and .05 Bcfs/day for the two policies. This is due to
the fact that at these low deliverability levels, only an insignificant amount of gas
has the ability to be withdrawn in order to capitalize on market movements. To gain
some perspective on these rates, we establish that .05 Bcf/day is equivalent 50
MMcf/day, (1000 MMcf = 1 Bcf), a deliverability rate that is common amongst
86
CHAPTER 6: Results
depleted reservoir storage facilities. As mentioned in Chapter 1, because of their low
deliverability, depleted reservoir systems are used for seasonal cycling, which
slowly injects natural gas during the spring when demand is low, and steadily
withdraws it in the winter to meet heavy demand. Thus, this graph highlights the
actual limitations of low deliverability storage in roles that require rapid
operational maneuvering such as peaking services or market speculation.
Figure 32 - Withdrawal Rate's Effect on Storage Value
2.50E+07
2.00E+07
1.50E+07
Storage Value ($)
1.00E+07
5.00E+06
0.00E+00
-5.00E+06
0
0.2
0.4
0.6
0.8
1
-1.00E+07
Policy 0
-1.50E+07
Policy 1
-2.00E+07
-2.50E+07
Withdrawal Rate Bcf/Day
6.6.2 Volatility Fluctuation
The natural gas market is one of the most volatile markets, and thus to
ensure that our policies would be stable when they encounter the volatility actually
within the market, we have decided to create a heightened reality by augmenting
the volatility within the pricing process two-fold. In Table 13, the numerical results
87
CHAPTER 6: Results
can be found for Policies 0, 1, and 2, relating the value of storage with the standard
volatility and then with the doubled volatility.
Table 13 - Volatility Fluctuation Results
Policy
Policy 0
Policy 1
Policy 2
𝜍𝑡
$15,578,000
$18,579,000
$18,742,000
2*𝜍𝑡
$3,164,145
-12,400,000
$9,210,000
The results show a significant decrease in value upon the augmentation of
the volatility. However, it should be noted that two of the three policies retain
positive values despite the surge in volatility. Figure 33 displays the value surface of
Policy 0 computed by the measurements of the KGNP. Recalling the surface in Figure
28 for the standard volatility, it can be seen that a shift in volatility will also shift the
values of the policies, though not significantly enough to remove all value. However,
this is just the case for Policy 0, seeing as the shift in volatility has greatly impacted
the value of storage in Policy 1, implying that longer trend windows are more
susceptible to swings in volatility.
Figure 33- Policy 0, KGNP Result for Doubled Volatility
88
CHAPTER 6: Results
6.6.3 KGNP Efficiency
When discussing the use of Approximate Dynamic Programming methods on
storage valuation in Chapter 2, the central criticism of the technique rested in the
computational burden even after reductions in dimension. Thus, in order to put
forth the KGNP algorithm as a viable method for solving the storage valuation
problem, we must determine its expedience in ascertaining the maximum value. We
do this as we are conducting the measurements, comparing at each iteration, how
many measurements have been taken to the maximum value achieved at that time.
Each run of the KGNP for each policy was budgeted 500 measurements to be
allocated amongst the numerous alternatives. However, it is obvious that 500
measurements were not needed in the case of Policy 0, where five runs of the KGNP
algorithm converged to a maximum value by 100 measurements. In fact, three of the
five samples converged to the maximum before ten measurements.
Figure 34 - Policy 0: S-Curve
18,000,000
16,000,000
Storage Value ($)
14,000,000
12,000,000
Sample 1
10,000,000
Sample 2
8,000,000
Sample 3
6,000,000
Sample 4
4,000,000
Sample 5
2,000,000
0
0
10
20
30
40
50
60
70
# of Measurements
89
80
90
100
CHAPTER 6: Results
Similar results can be seen in the sample plots of Policy 1, where all sample
paths converge to their maximum values well before 100 measurements have been
reached.
Based on these two figures, it appears that the KGNP algorithm provides us
with an approach that allows us to obtain the best estimate for storage value with
the minimal amount of measurements and computational time as possible.
Figure 35 - Policy 1: S-Curve
22,500,000
20,000,000
17,500,000
Storage Value ($)
15,000,000
Sample 1
12,500,000
Sample 2
10,000,000
Sample 3
7,500,000
Sample 4
5,000,000
Sample 5
2,500,000
0
0
10
20
30
40
50
60
70
80
90
100
# of Measurements
6.6.4 Real World Testing and Comparison
This section will utilize real natural gas prices and storage value data to
provide a glimpse at the validity of our policies and valuations within the natural gas
industry. As described in Chapter 4, the prices used to influence and compute all the
policies are simulated data based on a form of the mean-reverting process, and thus
90
CHAPTER 6: Results
our data never experiences the real price movements and trends of the market. To
correct this, we will test our policies and model on actual historical data. We will be
testing from the past ten years, two unusual years in regard to price paths, 2005
which exhibited a rapid growth to more than double its initial price, and 2009
where prices fell below $2 per MMBtu.
The price paths for 2005 and 2009 can be seen in Figure 36, with the results
of the simulations directly below in Table 14.
Figure 36 - Price Paths for 2005 and 2009 Natural Gas Spot Prices
$16.00
$14.00
Price ($)
$12.00
$10.00
$8.00
2005
$6.00
2009
$4.00
$2.00
$0.00
1/1
3/12
5/21
7/30
Month/Day
10/8
12/17
Table 14- Results from 2005 and 2009 Pricing Data
Year
2005
2009
2010 (simulated)
Policy 0 (1% , 5.5%)
$19,800,000
$6,339,567
$15,578,000
Policy 1 (2% , 9.5%)
$-8,940,877
$-4,510,426
$18,579,000
Amongst the volatile cycles of these years, Policy 0 not only retained the
value of storage, but attained a new pinnacle in the year 2005 with soaring prices.
91
CHAPTER 6: Results
The poor performance of Policy 1 was anticipated after its poor reaction to volatility
in Section 6.6.2. Based on the analysis conducted here and in Section 6.6.2, it would
appear that Policy 0 is the superior policy in terms of reaction to volatility
fluctuations.
Lastly, we compare each of the storage values estimated by our policies to
the value of equivalent storage volume traded at Enstor Operating Company, a
natural gas storage service firm, which participates in the sales of traditional firm
storage contracts. From an Income Statement for 2009 for Enstor, we were able to
obtain the annual value for a storage facility of comparable to the facility used
within our model (Enstor 2009). From Table 15, we find that the valuations
determined by our model utilizing KGNP algorithms computes numerical results
comparable to the values placed on facilities by the natural gas industry.
Table 15 - Table of Storage Value from Industry and Policies
Policy
Enstor
Policy 0
Policy 1
Policy 2
Annual Value
$13,500,000
$15,578,000
$18,579,000
$18,742,000
92
Value per unit
.15 MMBtu/month
.173 MMBtu/month
.206 MMBtu/month
.208 MMBtu/month
CHAPTER 7
Conclusions and Further Research
In our attempt to solve the vexing valuation problem for natural gas storage,
we deploy new methods within stochastic optimization within the structure of an
operationally acute model coupled with intuitive price centric policies.
The model we describe is purposefully structured to accurately account for
the various operating characteristics of real storage facilities. This includes the
explicit modeling of operational properties taking into account the variations in
pressure within the storage facility and the energy costs associated with the use of
the compressor during injection. Both of these operational components are usually
absent within other models. However, like the Approximate Dynamic Programming
approach discussed in Chapter 2, our model lacks the full evolution of the forward
curve, where we confine our decisions to the current spot price and month ahead
forward price. Further research could be pursued through navigating the
computational challenges of natural gas storage as a high dimensional stochastic
optimization problem.
CHAPTER 7: Conclusion and Further Research
Opportunity for further work within our framing of the natural gas storage
valuation problem would involve the further development of the operational
policies. We acknowledge that industry professionals who are responsible for
making the operational decisions for natural gas storage employ indicators outside
of the price evolution of the natural gas market. These may include gross production
reports regarding natural gas, total volume within U.S. storage, and weather, all of
which would offer a richer and more complex experience with the storage valuation
problem.
An exciting aspect of this work is the opportunity to utilize a new approach
when solving the storage valuation problem. The Knowledge Gradient is still a new
field of research within stochastic optimization with new methods and extensions
rapidly appearing. One of these emerging ideas is the Knowledge Gradient with NonParametric estimation which embarked on its maiden voyage upon our goal to
determine an accurate and computationally efficient method for valuating natural
gas storage.
Through the use of KGNP, policies were obtained that displayed
comparable results to the value of gas storage reported by the natural gas industry.
However, the creativity within the policy development aspects of this work was
stunted by the dimensional limitation of the KGNP algorithm which could only cope
with the manipulation of two parameters. Should the KGNP increase its dimension
threshold, it will be an even greater asset in solving stochastic optimization
problems such as this one.
94
CHAPTER 7: Conclusion and Further Research
Based on observing appropriate responses to various testing, as well as
receiving comparable results to current industry valuations of natural gas storage,
we have added to the literature an instrument that successfully deploys the strict
use of natural gas prices in policy development for the operational decisions
surrounding a natural gas storage facility.
95
Appendix: Code
Appendix: Code
//Natural Gas Storage Model and Pricing Simulations
//By Jennifer Schoppe with Professor Hugo Simao
import
import
import
import
import
import
import
import
import
java.io.BufferedReader;
java.io.File;
java.io.FileOutputStream;
java.io.FileReader;
java.io.IOException;
java.io.PrintStream;
java.util.ArrayList;
java.util.Random;
java.util.StringTokenizer;
public class ModelSimTesting {
//function will read in simulated price data that was computed in MatLab
public static double[] readExternalFile(String fname) {
int numRecs = 0;
// the return variable
double[]prices_array = new double[2175];
int i = 1;
try {
// setup buffered reader to read in the file one record at a time
File filePtr = new File(fname);
FileReader fileReader = new FileReader(filePtr);
BufferedReader bufReader = new BufferedReader(fileReader);
// read the file, record by record
i++;
String nextRec = bufReader.readLine();
while (nextRec != null) {
// parse record: three fields, separated by 1 or more tabs
StringTokenizer st = new StringTokenizer(nextRec, "\t");
// make sure that the parsed record did contain two fields
if (st.countTokens() == 1) {
double spot =
Double.parseDouble(st.nextToken().trim());
prices_array[numRecs] = spot;
numRecs++;
}
else {
// issue an error message
System.out.println("Cannot parse rec # "+ i +": "+
nextRec);
System.out.flush();
}
// get the next record
i++;
nextRec = bufReader.readLine();
}
// close the input files
bufReader.close();
fileReader.close();
}
catch (IOException e) {
System.out.println("Caught exception " + e + " while reading rec #"
+ i + " of file " + fname);
System.out.println("Exiting!");
System.out.flush();
System.exit(-1);
}
96
Appendix: Code
return prices_array;
}
public static double[][] pairs(double step){
//produces array of all possible choices for the percents
double [][] pairs = new double [2][361];
int count = 0;
double upper = 1;
for(int i =0; i<19;i++){
double lower = 1;
for(int j = 0;j<19;j++){
pairs [0][count] = upper;
pairs [1][count] = lower;
count++;
lower = lower + step;
}
upper = upper + step;
}
return pairs;
}
public static double[][] simulate(double[]prices_array) {
double [][] prices = new double [2][360];
ArrayList<Double> spot = new ArrayList<Double>();
ArrayList<Double> ExpS = new ArrayList<Double>();
for (int i = 0; i<prices_array.length; i++){
spot.add(prices_array[i]);
}
//mean reversion speed calculated using MATLAB regression
double alpha = .0063;
//Performing Exponential Smoothing
int N = 2175;
double Beta = .002;
//initialize starting smoothing price and price vector
double E = 0;
for (int i = 0; i< 5; i++ ){
double e = spot.get(i);
E = E + e;
}
ExpS.add(E);
for (int i = 1; i < spot.size(); i++) {
ExpS.add(spot.get(i-1)*Beta + (1-Beta)*ExpS.get(i-1));
}
//Spot Price Simulation
double dt = 1;
for (int x = 0;x<12;x++){
for (int y =0; y<30;y++){
Random generator = new Random();
double dz = generator.nextGaussian();
//Volatility calculation
//volatility window
int m = 360; //year
int n = spot.size();
double Sum = 0;
for(int i = n-m; i<n; i++){
double sum = 0;
for(int j = n-m; j<n; j++){
sum = sum + (Math.log(spot.get(j))Math.log(spot.get(j-1)));
97
Appendix: Code
}
Sum = Sum + Math.pow((Math.log(spot.get(i))Math.log(spot.get(i-1))-(1/m)*sum),2);
}
double sig = Math.sqrt((1/(double)(m-1))*Sum);
double sigHat = sig*Math.sqrt(2*alpha)/Math.sqrt(1Math.exp(-2*alpha));
int g = spot.size();
double S_old = (spot.get(g-1));
double E_old = (ExpS.get(g-1));
double dxt = alpha*(Math.log(E_old)-Math.log(S_old))*dt +
sigHat*dz;
spot.add(S_old + dxt);
prices[0][x*30+y] = S_old + dxt;
ExpS.add((S_old+dxt)*Beta + (1-Beta)*E_old);
double delta = 30-y;
double mu = Math.log(S_old)*Math.exp(-alpha*delta) +
Math.log(E_old)*(1-Math.exp(-alpha*delta));
double sd = (Math.pow(sigHat,2)/(2*alpha))*(1-Math.exp(2*alpha*delta));
double z = generator.nextGaussian();
double y_t = mu + sd*z;
prices[1][x*30+y] = Math.exp(y_t);
}
}
return prices;
}
public static double[]CapChange(double Spot, double Forward, double rateI, double rateW,
double Imax, double Wmax, double commitment, double contracts, double
history,double upper, double lower) {
//Output: spot injection/withdrawal and forward months injection/withdrawal
decision
double[]amount = new double[2];
//decision for spot
double x = 0;
double y = commitment;
double change = 100*((Spot-history)/history);
//for gain in price value
if (change>0) {
if (change>= upper){
x = -rateW;
}
else {
x = 0;
}
}
else {
if(Math.abs(change)>=lower) {
x = rateI;
}
else {
x = 0;
}
}
double z = 0;
//constraints for spot decision
double check1 = 0;
double check2 = 0;
while(check1==0 || check2==0){
if (x>0){
if (x + y/30 > Imax) {
x = Imax - y/30;
check1 = 1;
}
else{
98
Appendix: Code
check1 = 1;
}
if (y>0){
if((1/rateI)*(x+y/30)>1) {
x = rateI - y/30;
check2 = 1;
}
else{
check2 = 1;
}
}
else {
if ((1/rateI)*x + (1/rateW)*Math.abs(y/30) >1) {
x = (1-(1/rateW)*Math.abs(y/30))*rateI;
check2 = 1;
}
else {
check2 = 1;
}
}
}
else {
if (Math.abs(x + y/30) > Wmax) {
x = -Wmax - y/30;
check1 = 1;
}
else{
check1 = 1;
}
if (y>0){
if(((1/rateW)*Math.abs(x)+(1/rateW)*(y/30))>1) {
x = -(1-(1/rateI)*y/30)*rateW;
check2 = 1;
}
else{
check2 = 1;
}
}
else {
if (Math.abs((1/rateW)*x+(1/rateW)*(y/30)) >1) {
x = -rateW*(1+(1/rateW)*(y/30));
check2 = 1;
}
else {
check2 = 1;
}
}
}
}
// constraints for forward decision
if (contracts >0) {
if (z > 0) {
if(contracts + z > Imax) {
z = Imax -contracts;
}
}
else {
if (Math.abs(z)>Wmax) {
z = -Wmax;
}
}
}
else {
if (z>0) {
if (z >Imax) {
z = Imax;
}
}
else {
if (Math.abs(z+contracts)>Wmax) {
z = -(Wmax-Math.abs(contracts));
99
Appendix: Code
}
}
}
amount[0]= x;
amount[1]= z;
//amount[0]= x;
//amount[1]= z;
return amount;
}
public static double Capacity(double Current, double loss,double s, double f) {
//Output: new capacity of storage facility
double X = 0;
if (s>0 && f>0){
X = Current + loss*s + loss*f/30;
return X;
}
else if (s>0 && f<0){
X = Current + loss*s + f/30;
return X;
}
else if (s==0 && f==0){
X = Current;
return X;
}
else if (s<0 && f>0){
X = Current + s + loss*f/30;
return X;
}
else {
X = Current + s + f/30;
return X;
}
}
public static double Pressure(double max, double min, double CapMax, double Cap) {
//Output: pressure within storage facility
//Inputs: max pressure, min pressure, max capacity, current capacity
double p = ((max-min)/CapMax)*Cap + min;
return p;
}
public static double InjRate(double max, double min, double CapMax, double Cap) {
//Output: fuel loss rate for injection into storage facility
//Inputs: max rate, min rate, max pressure, current pressure
double rate = -((max-min)/CapMax)*Cap + max;
return rate;
}
public static double WithRate(double max, double min, double CapMax, double Cap) {
//Output: fuel loss rate for withdrawal from storage facility
//Inputs: max rate, min rate, max pressure, current pressure
double rate = ((max-min)/CapMax)*Cap + min;
return rate;
}
public static double Compression(double max, double pmax, double p) {
//Output: horsepower of compressor for injection
//Inputs: max horsepower of compressor, max pressure, current pressure
double rate = (max/pmax)*p;
return rate;
}
public static double InjSpotCost(double compress, double rate, double amount,
double spot) {
//Output: spot injection cost
//Inputs: compression, Injection rate, amount to be injected, spot price
double cost = Math.abs((compress*42.44*60)*((1/rate)*24*amount)
*spot/1000000);
return cost; //in Bcf
100
Appendix: Code
}
public static double WithSpotCost(double amount) {
//Output: withdrawal cost
//Inputs: withdrawal rate, storage cost, current pressure
double cost = Math.abs(.01*amount*1000000);
return cost; //in Bcf
}
public static double InjForCost(double compress, double rate, double amount,
double forward) {
//Output: spot injection cost
//Inputs: compression, Injection rate, amount to be injected, spot price
double cost = Math.abs((compress*42.44*60)*((1/rate)*24*(amount/30))
*forward/1000000);
return cost; //in Bcf
}
public static double WithForCost(double amount) {
//Output: withdrawal cost
//Inputs: withdrawal rate, storage cost, current pressure
double cost = Math.abs(.01*(amount/30)*1000000);
return cost; //in Bcf
}
public static double Reward(double spot, double forward, double Cost_I_Spot,double
Cost_W_Spot,double Cost_I_Forward,double Cost_W_Forward,double s,double
f,double loss){
// Output: daily reward
double R = 0;
if (s==0 && f==0){
R = 0;
return R;
}
else if (s>=0 && f<=0){
R = -((loss*spot)*s*1000000 + Cost_I_Spot) +(forward)*Math.abs(f)
*1000000 - Cost_W_Forward ;
return R;
}
else if(s>=0 && f>=0) {
R = -((loss*spot)*s*1000000 + Cost_I_Spot +(loss*forward)*f*1000000
+ Cost_I_Forward) ;
return R;
}
else if (s<=0 && f>=0){
R = (spot)*Math.abs(s)*1000000 - Cost_W_Spot -((loss*forward)
*f*1000000 + Cost_I_Forward) ;
return R;
}
else {
R = (spot)*Math.abs(s)*1000000 - Cost_W_Spot +(forward)*Math.abs(f)
*1000000 - Cost_W_Forward ;
return R;
}
}
public static double model(double Capacity, double upper, double lower){
// Output: produces Annual Value for Natural Gas
String fname =
"\\C:\\Users\\Jennifer\\Documents\\Thesis\\PriceData.txt\\";
double[] prices_array = readExternalFile(fname);
double [][] prices = new double [2][360];
prices = simulate(prices_array);
FileOutputStream fout;
try
{
// Open an output stream
101
Appendix: Code
fout = new FileOutputStream
("\\C:\\Users\\Jennifer\\Documents\\Thesis\\Simulations.txt\\");
// Print a line of text
for(int i = 0; i < 360; i++){
new PrintStream(fout).print(prices[0][i] + "\t");
new PrintStream(fout).print(prices[1][i] + "\n");
}
// Close our output stream
fout.close();
}
// Catches any error conditions
catch (IOException e)
{
System.err.println ("Unable to write to file");
System.exit(-1);
}
//establish parameters for model
double X_max = Capacity; //maximum level of working gas that can be put in
a storage facility
double rate_I_max = .3; //maximum injection rate for storage facility
double rate_I_min = 0; //minimum injection rate for storage facility
double rate_W_max = .5; //maximum withdrawal rate for storage facility
double rate_W_min = 0; //minimum withdrawal rate for storage facility
double p_max = .85; //maximum pressure within the storage facility
double p_min = .2; //minimum pressure within the storage facility
double compress_max = 25000; //max horsepower for compressor
double loss_I = .985; //fuel rate loss for injection
double d = 3500; //depth of storage facility
double gamma = 0; //risk free discount factor
double history = 0; //for swing trading
//establish initial variables
double[] X = new double[361]; //initial capacity of storage facility
double[][] x_t = new double[2][361];//initialization of spot decision
double[] max_With = new double[361];//initialization of max Withdrawal
Capacity
double[] max_Inj = new double[361];//initialization of maximum Injection
Capacity
double[] rate_I = new double[361]; //injection rate for storage facility
double[] rate_W = new double[361]; //withdrawal rate for storage facility
double[] p = new double[361]; //initialization of pressure
double[] cost_I_spot = new double[361];
double[] cost_W_spot = new double[361];
double[] cost_I_forward = new double[361];
double[] cost_W_forward = new double[361];
double[] compress = new double[361];//horsepower of compressor
double[] reward = new double[361]; //initializing reward, daily revenue
//initialize variables
double Spot_t = 0;//initialization of spot price
double Forward_t = 0;//initialization of forward price
double contracts = 0; //number of futures contracts to be fulfilled
double price = 0; //price of contracts to be fulfilled
double forward_commitment = 0; //amount injected or withdrawn due to
forward contract commitments from prior month
double price_commitment = 0;
double Value = 0; //initializing yearly revenue
int Count = 0; //initializing counter
102
Appendix: Code
//day 0 values
X[0] = 0;
x_t[0][0] = 0;
x_t[1][0] = 0;
max_With[0] = 0;
max_Inj[0]= X_max - max_With[0];
rate_I[0] = rate_I_max;
rate_W[0] = rate_W_min;
p[0] = p_min;
cost_I_spot[0] = 0;
cost_W_spot[0] = 0;
cost_I_forward[0] = 0;
cost_W_forward[0] = 0;
compress[0] = 0;
reward[0] = 0;
double[] spot = new double[360];
double[] forward = new double[360];
for (int i = 0; i<360; i++){
spot[i] = prices[0][i];
forward[i] = prices[1][i];
//System.out.print("spot = " + spot[i]+"\t");
//System.out.print("forward = "+forward[i]+ "\n");
}
//loop of year long operation of storage facility
double AnnualProfit = 0;
for (int i = 1; i <= 360; i++) {
//keeps track of where we are in months
if (Count == 31){
contracts = 0;
Count = 1;
}
else {
Count++;
}
max_With[i] = X[i-1] + x_t[0][i-1];
max_Inj[i] = X_max - max_With[i];
X[i] = Capacity(X[i-1], loss_I, x_t[0][i-1],forward_commitment);
p[i] = Pressure(p_max, p_min, X_max, X[i]);
rate_I[i] = InjRate(rate_I_max, rate_I_min, X_max, X[i]);
rate_W[i] = WithRate(rate_W_max, rate_W_min, X_max, X[i]);
compress[i]= Compression(compress_max, p_max, p[i]);
cost_I_spot[i] = InjSpotCost(compress[i], rate_I[i], x_t[0][i1],spot[i-1]);
cost_W_spot[i] = WithSpotCost(x_t[0][i-1]);
cost_I_forward[i] = InjForCost(compress[i], rate_I[i],
forward_commitment,price_commitment);
cost_W_forward[i] = WithForCost(forward_commitment};
reward[i] = Reward(spot[i-1], price_commitment,
cost_I_spot[i],cost_W_spot[i], cost_I_forward[i],
cost_W_forward[i], x_t[0][i-1],forward_commitment,loss_I);
Value = Value + reward[i];
if (i<4) {
history = 0;
for(int j =0;j<i;j++){
history = history + spot[j];
}
history = history/(double)4;
}
else {
history = spot[i-4];
}
double[] decisions = new double[2];
decisions = CapChange(spot[i-1], forward[i-1], rate_I[i],
103
Appendix: Code
rate_W[i],max_Inj[i],max_With[i], forward_commitment,
contracts, history,upper,lower);
x_t[0][i] = decisions[0];
x_t[1][i] = decisions[1];
if (Count <30) {
if(x_t[1][i] != 0) {
if (contracts*x_t[1][i]<=0){
price_commitment = forward[i-1];
contracts = contracts + x_t[1][i];
}
else {
price_commitment = (price_commitment*
Math.abs(contracts) + forward[i-1]*
Math.abs(x_t[1][i]))/Math.abs(contrac
ts+x_t[1][i]);
contracts = contracts + x_t[1][i];
}
}
}
else {
forward_commitment = contracts/30;
}
AnnualProfit = AnnualProfit + reward[i];
}
FileOutputStream fout2;
try
{
// Open an output stream
fout2 = new FileOutputStream
("\\C:\\Users\\Jennifer\\Documents\\Thesis\\ModelOutput.txt\\");
for (int i = 0; i < 360; i++){
// Print a line of text
new PrintStream(fout2).print(x_t[0][i]+"\t");
new PrintStream(fout2).print(x_t[1][i]+"\t");
//System.out.print(price_commitment+ "\t");
new PrintStream(fout2).print(X[i]+"\t");
new PrintStream(fout2).print(forward_commitment+ "\t");
new PrintStream(fout2).print(max_Inj[i]+"\t");
new PrintStream(fout2).print(max_With[i]+"\t");
new PrintStream(fout2).print(rate_I[i]+"\t");
new PrintStream(fout2).print(rate_W[i]+"\t");
new PrintStream(fout2).print(cost_I_spot[i]+"\t");
new PrintStream(fout2).print(cost_W_spot[i]+"\t");
new PrintStream(fout2).print(cost_I_forward[i]+"\t");
new PrintStream(fout2).print(cost_W_forward[i]+"\t");
new PrintStream(fout2).print(reward[i]+"\n");
}
fout2.close();
}
// Catches any error conditions
catch (IOException e)
{
System.err.println ("Unable to write to file");
System.exit(-1);
104
Appendix: Code
}
//System.out.println("Annual Value = " + Value);
return AnnualProfit;
}
/*
* *
* @param args
*/
public static void main(String[] args) {
// TODO Auto-generated method stub
//only needed for testing
double AnnualProfit = model(.01,.015, Capacity);
System.out.println(AnnualProfit);
}
}
105
Bibliography
Bibliography
Barut, A. Emre, and W.B. Powell. Optimal Learning for Sequential Sampling with NonParametric Estimation (Working Paper). 2010.
Brown, Robert G., and Richard F. Meyer. The Fundamental Theorem of Exponential
Smoothing. 1960.
Byers, Joe Wayne. "Commodity Storage Valuation: A Linear Optimization based on
traded instruments." 2006.
CASTLE Laboratory. 1997-2009. http://www.castlelab.princeton.edu/ (accessed 1
2010, April).
CMEGroup. 2010. http://www.cmegroup.com/trading/energy/natural-gas/naturalgas-last-day.html (accessed 04 02, 2010).
Dietert, Jeff A., and David A. Pursell. Underground Natural Gas Storage. Simmons &
Company International, 2000.
Energy Information Administration. August 2004.
http://www.eia.doe.gov/oil_gas/info_glance/natural_gas.html (accessed
November 3, 2009).
Enstor. "Gas Dailys: Historical Daily Spot Prices 1999-2009." 2010.
Enstor. "Statement of Income for Caledonia Storage." 2009.
106
Bibliography
Eydeland, Alexander, and Krzysztof Wolyniec. Energy and Power Risk Management:
New Developements in Modeling, Pricing, and Hedging. Hoboken: John Wiley &
Sons, 2003.
Frazier, P., W.B. Powell, and S. Dayanik. A Knowledge Gradient Policy for Correlated
Normal Rewards. 2009.
—. A Knowledge Gradient Policy for Sequential Information Collection. 2008.
Gupta, S.S., and K.J. Miescke. Bayesian Look Ahead One-stage Sampling Allocations for
Selection of the Best Population. 1996.
Hardle, Wolfgang, Marlene Muller, Stefan Sperlich, and Axel Werwatz.
Nonparametric and Semiparametric Models. Berlin: Springer, 2004.
Lai, Guoming, Francois Margot, and Nicola Secomandi. An Approximate Dynamic
Programming Approach to Benchmark Practiced-based Heuristics for Natural
Gas Storage Valuation. 2008.
Merrill, Arthur. Filtered Waves. New York: Analysis Press, 1977.
Mes, Martijn R.K., W.B. Powell, and P. Frazier. Hierarchical Knowledge Gradient for
Sequential Sampling. 2009.
Nadaraya, E.A. "On Estimating Regression." Theory of Probability and its
Applications, 1964.
Natural Gas Depot. 2009. www.naturalgasdepot.com (accessed April 3, 2010).
107
Bibliography
NaturalGas.org. 2004. http://www.naturalgas.org/naturalgas/storage.asp#depleted
(accessed November 12, 2009).
Powell, W.B., and P. Frazier. ""Optimal Learning"." TutORials in Operations Research,
2008: 213-246.
Powell, Warren B., and Ilya Ryzhov. Optimal Learning. Hoboken: Wiley, 2010.
Puterman, M.L. Markov Decision Processes. Wiley, 1994.
U.S. Department of Energy. 2009.
http://www.energy.gov/energysources/naturalgas.htm (accessed November
22, 2009).
108