P2P energy exchange agent
platform featuring a game theory
related learning negotiation
algorithm
Nicola Capodieci
Facolta di Ingegneria
Universita degli Studi di Modena e Reggio Emilia
Guest student at Rijksuniversiteit Groningen
Master Degree Thesis
2010/2011
RuG Supervisors: Prof Marco Aiello, Andrea Pagani
Unimo Supervisor: Prof. Giacomo Cabri
18th April 2011
ii
To Davide Capodieci
Acknowledgements
Contents
List of Figures
v
List of Tables
vii
1 Introduction
1.1
1
Political and legislative context . . . . . . . . . . . . . . . . . . . . . . .
2 Related works
1
5
2.1
Global effects on energy markets . . . . . . . . . . . . . . . . . . . . . .
5
2.2
Software platforms and other technologies . . . . . . . . . . . . . . . . .
7
2.3
Balancing, pricing and learning approaches . . . . . . . . . . . . . . . .
8
3 Technologies and Architecture
3.1
3.2
11
JADE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
11
3.1.1
Agent description . . . . . . . . . . . . . . . . . . . . . . . . . . .
14
3.1.2
Agent messaging . . . . . . . . . . . . . . . . . . . . . . . . . . .
18
3.1.3
Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . .
21
XML . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
36
3.2.1
38
Web Services usage . . . . . . . . . . . . . . . . . . . . . . . . . .
4 Interconnection with the Smart Meter
41
4.1
Smart Meter: definitions and usage . . . . . . . . . . . . . . . . . . . . .
41
4.2
Mobility issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
44
4.3
Communication features and typical requests . . . . . . . . . . . . . . .
45
4.4
Simplified demand forecasting . . . . . . . . . . . . . . . . . . . . . . . .
48
iii
CONTENTS
5 Security issues
53
5.1
Problem description and JADE-S . . . . . . . . . . . . . . . . . . . . . .
53
5.2
Tools used, testing and data encryption . . . . . . . . . . . . . . . . . .
55
5.3
Other security aspects . . . . . . . . . . . . . . . . . . . . . . . . . . . .
61
5.3.1
Authentication . . . . . . . . . . . . . . . . . . . . . . . . . . . .
61
5.3.2
Authorization . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
62
5.3.3
Message integrity . . . . . . . . . . . . . . . . . . . . . . . . . . .
62
Agent based malicious attacks . . . . . . . . . . . . . . . . . . . . . . . .
63
5.4.1
Attack scenarios against the agent trading platform . . . . . . .
64
Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
66
5.4
5.5
6 Supply/demand balancing, forecasting and pricing
67
6.1
Supply forecasting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
67
6.2
Demand forecasting . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
68
6.3
Balancing equation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
70
6.4
Pricing methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
73
7 Trading strategies
77
7.1
Problem description . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
77
7.2
Agent learning using game theory (Buyer’s side) . . . . . . . . . . . . .
80
7.3
Bidding learning algorithm . . . . . . . . . . . . . . . . . . . . . . . . .
87
7.4
Test results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
89
8 Conclusions
93
8.1
Results’ discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
8.2
Improvements and future implementations . . . . . . . . . . . . . . . . . 102
References
93
105
iv
List of Figures
3.1
Main agents general view . . . . . . . . . . . . . . . . . . . . . . . . . .
16
3.2
Agents’ description in two different approaches . . . . . . . . . . . . . .
17
3.3
The JADE asynchronous message passing . . . . . . . . . . . . . . . . .
19
3.4
Simulation GUI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
33
3.5
Peer market statistic . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
35
3.6
Buyer agent’s log . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
37
3.7
Typical XML reply for a weather forecasting request . . . . . . . . . . .
40
4.1
Smart Meter Interface overview . . . . . . . . . . . . . . . . . . . . . . .
43
4.2
Smart Meter Interface GUI . . . . . . . . . . . . . . . . . . . . . . . . .
51
5.1
Different levels of security . . . . . . . . . . . . . . . . . . . . . . . . . .
56
5.2
JADE sniffer agent GUI . . . . . . . . . . . . . . . . . . . . . . . . . . .
57
5.3
Commview GUI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
58
5.4
Commview log file . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
59
5.5
ACL Message format . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
63
6.1
Buyer’s Demand Forecasting algorithm scheme . . . . . . . . . . . . . .
69
6.2
Energy consumption example scenario . . . . . . . . . . . . . . . . . . .
72
7.1
Stochastic Game approach modeled with a Mealy state machine. . . . .
86
7.2
Fuzzy logic styled diagram for controlling bids. Buyer’s side 1) Do nothing 2) Rise stakes/bids 3) Increase expected money to spend . . . . . . .
89
7.3
Simulated Learning Strategy results . . . . . . . . . . . . . . . . . . . .
92
8.1
2010 electricity cost comparison . . . . . . . . . . . . . . . . . . . . . . .
94
v
LIST OF FIGURES
8.2
Explanation of demand response effects on a quantity (Q) - price (P)
graph . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
97
8.3
Amortizing time and predicted earnings: solar device . . . . . . . . . . . 100
8.4
Amortizing time and predicted earnings: wind device . . . . . . . . . . . 101
8.5
Graph showing the dependency from the weather to the supply capacity
of a solar panel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102
vi
List of Tables
7.0
El Farol Bar payoff matrix . . . . . . . . . . . . . . . . . . . . . . . . . .
82
7.1
initial state 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
84
7.2
initial state 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
84
7.3
final state 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
84
7.4
final state 4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
85
8.1
Wind and Solar power in the Italian biggest islands (Sardegna, Sicilia)
domestic environment . . . . . . . . . . . . . . . . . . . . . . . . . . . .
vii
99
LIST OF TABLES
viii
1
Introduction
1.1
Political and legislative context
The energy producing and distributing sectors have always been coupled with lots of
issues and problems who bothered governments as well as big energy producers: the
traditional model of distributing electric energy (which is still widely used by most
countries) is the same as the one theorized during the last century because it is still
featuring a monopolistic view of the energy in which every consumer (domestic or
business related) is bounded by its own country’s rules to be provided by a single (or
extremely restricted set of) big energy producer company (henceforth called Genco).
In this situation, there is no space for innovation towards a more decentralized and
liberal view of the matter because there is no competition due to the limited number of
present legally allowed actors. In addition, traditional Gencos have no direct interest in
optimizing their production model since every wrong planned decision who could lead
in costs’ rising will directly propagate to end users, who cannot easily contract with
other suppliers. If that was not enough, the traditional model was pondered when the
world’s consciousness regarding environmental issues and upcoming depletion of old
energy sources was not as present as nowadays, when international commissions that
have the duty of measuring the impact on the planet caused by this approach, were
introduced. By the time of the first conferences kept by those authorities, governments
pushed Gencos to be more keen towards domestic users needs as well as start investigating about moving into a more ”greener” and responsible consumption oriented
model. That resulted in a slightly more open market, also with the de-bounding from
1
1. INTRODUCTION
the mere production sector with the distribution one involving now a separate apparatus of transmission service operators (TSOs).
However, all of the previous considerations, are not the only improvements made
in order to radically transform the used model from the traditional one to a new one.
Entities involved needed further motivations that were supplied by a relatively recent
awareness about environmental issues as well with scientific findings related the study
of renewable energy sources. In addition to that, last 50 years shown how every country
is directly affected to the periodical economical crisis as well as political instabilities in
those Nations that are primary responsible for the production and export of raw energy production material around the world. In other words, the traditional model was
expensive, had no stimulus for innovation, pollution and production waste disposal’s
issues where left untreated for long time and left our economy strictly dependant from
causes apparently very far from the end user point of view.
The market then opened towards the introduction of other Gencos in the same countries, leaving end users to stipulate contracts according their needs and introducing a
soft market competition, still heavily regulated by governments. The real heap towards
a completely new paradigm has been accomplished with the new trend of installing
of more and more solar panels and/or wind turbines into both domestic and business
environment: at first, the user could produce perfectly clean energy using renewable
sources and decide to give back to the electricity net the surplus energy in order to
obtain bill discounts from the service provider.
Even if all of this could appear as the definitive solution, still remains a wide improvement margin especially regarding energy distribution and responsible consumption: in this model we did not solve all the issues related in overloading a single energy
production line, we do not have a clear overview on how to produce as much energy
as necessary without creating environmental harms or other inconvenient economical
side effects and the number of users who manage to fulfil their energetic needs with
renewable sources still remains a sparse minority, but what if we extend the possibility
of TSOs to lend their services to domestic users featuring renewable energy devices in
order to let them participate as consumers and at the same time producers, forming
2
1.1 Political and legislative context
a completely new form of actor, that we decide to call ”prosumers”? Doing so, will
give the possibility to ordinary consumers to retrieve their needed energy from the prosumer neighbour that can supply both those domestic environments, creating a new
kind of distribution net that is (most of the time) extremely decentralized with sellers
introduced in a free market in which the only winner is the consumer that every time
could choose between different offers for the same service. In addition, new sources
of energy are easily obtainable in this open market in which every participant could
retrieve information from the outside world in order to adjust prices and production’s
plan according to the real market trend of raw material and/or weather forecasting for
an optimized management of solar panels and wind turbines.
We can summarize all of these considerations in figure ??: at first we had the
monopolistic approach in which every energy buyer was bounded by a single Genco
that was also responsible for the transmission and distribution services. Secondo step
was the introduction of a limited competitiveness among the markets with more then
one Gencos,
3
1. INTRODUCTION
4
2
Related works
2.1
Global effects on energy markets
Due to the size of the problem and to the fact that this whole new concept of producing
and distributing energy among a p2p net touches several aspects of our everyday lives,
the related literature about this matter is huge. Switching from a centralized paradigm
to the new system implies legislative aspects that are linked to the different legislation
present in different countries: the effort each country has to make in order to prepare
itself (legally and economically speaking) to this radical change are extremely diverse
and therefore almost every country should be studied as a separate scenario. Even
if all of this is not properly treated in this thesis, we can therefore, study and get to
understand more about this by reading some common legislation for EU zone countries.
The European parliament, taking into account that the most recent economic crisis
made all our governments perfectly conscious of our strong dependency from old and
not renewable energy sources coming from eastern countries, emanated different regulations that are just suggesting to the EU members how to act in order to change the
monopolistic current structure of energy distribution in something more environmental care related. For this purpose the Regulation (EC) No 663/2009 of the European
Parliament and of the Council of 13 July 2009 for establishing a programme to aid
economic recovery by granting Community financial assistance to projects in the field
of energy distribution has been issued and published in the official journal of EU parliament emanations under the issue Official Journal L 200 , 31/07/2009 P. 0031 - 0045
5
2. RELATED WORKS
1.
In that regulation, also distribution of Gas and carbon capture are treated. Other
discussions and comments about these new regulations can be obtained here (1).
Concepts like smart grids/meters are also deeply treated by the same European
commission: for sake of being exhaustive enough we can therefore list some of the most
recent speeches kept by Günther Oettinger (member of the European Commission,
Energy sector):
• SAP event on Smart Grids, (25/01/2011)
• Towards competitive, sustainable and secure energy? - EPC Breakfast Policy
Briefing, Brussels , (25/01/2011)
• EPPSAs 5th Annual Technology Evening (24/01/2011)
• Europäische Energiepolitik und die Rolle der Schweiz, Schweizerischer Stromkongress,
Bern (10/01/2011)
Getting into this matter with a more engineering oriented point of view, several
researchers proposed their solution to the distribution, demand/supply balancing and
learning issues. Basically, starting from the European laws and taking into account the
current situation of their country they tried to simulate how this deregulation could
be helped by using modern software agent based approach. In Croatia some tests have
been performed (2).
Obviously, Europe is not the only area in which this transformation is beginning
to being discussed and investigated: an equivalent of the previous legislation paper
applied to the USA and Australian situation has already been studied (3), (4).
In addition to all of that, the IEEE scientific magazine Power & Energy for electric
power professionals 2 , is a huge source of related works. According to the editorial
publishing of the Volume 8, issue number 5 (Sept/Oct 2010), over the course of the past
five years, that magazine published an important number of papers relating the studies
1
2
http://eur-lex.europa.eu
http://www.ieee-pes.org/
6
2.2 Software platforms and other technologies
for preparing software and hardware architectures ready to be compliant with the new
energy deregulation laws. In that particular issue, publicized papers are focussing of
the impact of these new solutions on the energy markets especially in Spain, Germany,
Sweden, U.K and Latin America.
2.2
Software platforms and other technologies
Related works on platforms, technologies and mathematical abstractions for modelling
pricing, demand/supply balancing and strategic learning are widely treated in literature, conferences and published papers issued by magazines and university researches:
the subject is vast and there are alternative approaches that differ from the ones hereby
presented.
As far as the technologies are concerned, we already introduced the fact that this
whole project will be handled by an agent platform and later (in chapter three), we
introduce JADE and start an exhaustive discussion in order to motivate this choice.
Although, agents are not the only solution proposed by researches in this field: an alternative architecture featuring web based approach in a Java powered framework with
extensive usage of JSP/Servlet pages as resources have been previously investigated (5),
while first feasibility studies about the agents approach are dated back to year 2000
and presented during the Advances in Power System Control, Operation and Management, 2000. APSCOM-00. 2000 International Conference. Even if those studies often
name no more existing/used/updated technologies like ZEUS or CORBA (see (6)), it
still remains interesting how they predicted the deregulation trend and tried to use
this supposed new approach with autonomous software agents. Few years from that,
other researchers proposed several centralized or not peer based models like SEPIA
and MASCEM (7), (8), that started to introduce also the concept of learning for every
agent involved in the market simulation.
7
2. RELATED WORKS
2.3
Balancing, pricing and learning approaches
Balancing the total consumers’ demand with suppliers’ forecasting energy values can
be done in several ways. Each of these solutions are based on assumptions on how the
current energy market will evolve once the deregulation process reaches its final stages:
our solution (shown in chapter 6), tries to be as much as linked to reality, however it
lacks of a in depth study on electricity leaking, losses during the transportation process
and other not fully accurate data analysis. Other researches show a more complicate
and mathematical based approach in order to figure out how to solve this problem, even
using long-term view of the market abstraction. As far as our project is concerned, due
to the current difficulty to improve the wind strength forecasting used in predicting the
supply capacity and the efficacy of pricing energy according the continuously swinging
of raw materials’ prices, a short-term approach is used instead. For sake of having a
complete overview, however, also other ways to study the issues were investigated and
therefore hereby reported (5), (9), (4).
For others strictly short-term related models, Dutch researchers from the Eindhoven
University of Technology and from the Energy research centre of the Netherlands proposed a model in which the balancing has been obtained by seeing everything like
frequencies variations of load lines and the consumers’ demand, showing which precautions have to be taken in order to have a functioning model (10).
Other researchers simulated the benefits of creating a sort of energy production/consumption
profile to be inserted in the agent platform (11) and by doing so, the system gets a more
accurate idea for the balancing problem while single users receive important suggestion
on how to optimize their domestic consumption.
Combining the previous considerations with some concepts related the strategy
of learning that every agent has to implement in order to achieve a complete and
functioning model of the reality, an interesting study has been made (12) to highlight the
visible improvement for controlling LMPs (local marginal prices) with or without the
usage of a learning algorithm. Other researches (13) show comparisons and description
of previously adopted agent based models for the short term approach in the energy
8
2.3 Balancing, pricing and learning approaches
markets. Even if in that paper, the authors did not considerate the actual possibility
that web services could communicate to a requester agent, letting it becoming more
flexible to the changes in the external world, it is still interesting to read how the same
problem could be approached first with a long term model and then with a mixed
strategy set that combines the best from both the approaches.
9
2. RELATED WORKS
10
3
Technologies and Architecture
3.1
JADE
During the introduction chapter, the importance and the quality improvements for the
electric energy distribution service due to the introduced higher granularity of stipulating contracts, have already been pointed out. Although, stipulating and negotiating
contracts several times during a single day forces our architecture to be independent:
it is not thinkable to have a constant human user taking care of all the operations and
not just because of the added granularity, but also because finding best sellers in a
future net with thousands of possible supplier should be done in a autonomous way.
A simple web application or service could not be enough in order to handle such
a complicate problem and therefore a more independent, adaptive and autonomous
system should be used as main technology leaving human intervention to the sole adjusting of initials values and other preferences: software agents (already described in
the previous sections) are the perfect candidate for this kind negotiating tasks.
The challenge now is how to choose the right agent platform in order to link all
the good characteristics of agents with an easily updatable environment, a known and
not bounded to the operating system programming language with distributing, security and fault tolerance oriented set of libraries and/or extensions. This was not as
easy as it sounds because on the net it is possible to find traces of a multitude of agent
programming platforms and even discarding the ones that are no more supported or up-
11
3. TECHNOLOGIES AND ARCHITECTURE
dated, the set of possibilities remains wide: therefore, some fixed parameters should be
chosen and evaluated in order to find out which platforms could be used in this project.
Other institutes tried to give an answer to this platform related problem and the
results are basically electing JADE (Java Agent Development Environment) as best
or one of the best agent platform for general purpose uses. In the reference (14) for
instance, the fixed comparing parameters were:
• Standard compatibilities for agent messaging
• Inter platform messaging capabilities
• Mobility strength: how much data could an agent maintain while migrating between another host/peer. Two under classes have to be defined:
– Strong mobility: Code and execution state information are moving within
the agent.
– Weak mobility: Just code is sent during an agent migration.
• Availability and documentation
• Security policies
From the above parameters I think we need to add:
• Quantity and quality of possible updates (extensions, additional libraries with
additional functionalities)
• Customization possibilities
• Quality of support service
Always according to (14), the platforms that better satisfy the first batch of requirements are Grasshopper(that is developed by IKV++ Technologies AG) placing
itself at the first place, immediately followed by JADE (actual copyrights are owned
by Telecom Italia S.p.A.). The Grasshoper solution scored higher than JADE due to
the slightly better implementation of the mobility issues, however it is not an open
source solution and it appears that the current official website is outdated, while the
12
3.1 JADE
one indicated in (14) is now referring to the official site of some alcoholic beverage.
Other resources indicates a no more existing website.
On the other hand, JADE is satisfying all the above parameters with good results,
therefore becoming the best agent platform for this particular project.
Listing all of its most important features:
• It complies with FIPA specifications. See next section for further information.
• Logging, permission grants and added security possibilities for inter platform
communications
• Wide range of extensions for additional features like security, possibility to embed
JADE instances in smaller devices, web service integration etc... and others still
in development
• Easy to distribute.
• Open source, allowing an expert programmer to handle extensions and libraries
in order to obtain a full customization of the system.
• Availability and documentation are quite good. Manuals and tutorials are not
always up to date, but the developer’s forum is an excellent resource also because
of the replies by people who designed this platform.
• Already used and tested in many other important projects.
• The presence of a very intuitive graphic user interface
• The presence of several already implemented agents, that could be used both for
testing and debugging.
The most important feature is that it relies on Java and treats every agent as a
separate special thread, but it is faster than a conventional Java based thread. In
addition to that, it supports all the operating systems that can handle a JVM.
13
3. TECHNOLOGIES AND ARCHITECTURE
3.1.1
Agent description
In previous sections, the concept of agent has already been explained in detail and all
other theory related issues that was not treated in this thesis could be easily verified in
official JADE guides, tutorials and reference books. Now it is the time to give a complete overview of the agent set that this project features, describing the kind of agents
involved in the energy market with a in depth explanation of the most important steps
of their behaviours. In addition to that, a clear distinction of how an agent is suppose
to act in a hypothetical real environment and what actually does inside the simulation
software, has to be presented to the reader in order to obtain a better understanding
of the problem.
The platform used for simulations has different kinds of agents according to their
roles. Main and auxiliary agents are present:
• Main Agents
– Consumers: Energy buyers
– Prosumers: Producers and Consumers of electricity; they can produce a
limited quantity of energy (compared to a Genco) thanks to their wind
turbines and/or solar panels.
– Gencos: Big Energy Producer Companies
• Other Agents
– Time Agent: Synchronization duties
– Weather Agent: Weather Forecasting Duties
– Balancer Agent: Total energy demand balancing duties
An addition to the last set should be specified: in the simulation environment, the
first agent created is the AgentCreator. It creates all the other agents.
It is extremely important to notice that every host can run a JADE instance with
more than one agent: according to the method chosen to distribute the entire architecture a buyer could be (for instance) also the weather agent or the balancer, by having
14
3.1 JADE
an host running these two types of agent. Using the same principle, that is how a
Genco could also buy energy (typically prosumers’ unsold supplies): also in this case,
we have to think about a computer running more than one agent. However, in these
situations, more efforts are required to balance the demand/supply.
As we can see in figure 3.1, every agent is defined by multiple parameters. Some of
them are common to everyone, others are specific for the role chosen by the agent. Every variable’s name is pretty much self explanatory, except for the ”position” one that
refers to the fact that in our simulation the city observed is divided in six sub-areas:
every area has its own identifying number (from 0 to 5) that represents the distance
between different peers. Distances simulate the different costs of transporting a single
energy unit from one city area to another since it reasonable to think that in the future every country will have the electricity supply services and the transmission service
operations (done by transmission service operator: TSO) being completely separated.
So distances are related to TSO contracts and every seller may have a different TSO
operator that charges the seller a fixed amount of money for transporting energy from
one area to another and the bigger is the distance, the higher gets the price. Obviously
those prices paid by sellers (prosumers and Gencos) will influence the final price that
they are going to propose to their possible buyers. In the simulation environment, after being created by the AgentCreator, every sellers already have a randomized priced
contract with a TSO: that is because we want to focus on the negotiating part of the
project: however in a future real implementation, we can think about a TSO agent
being registered to the DF/AMS service offering fixed priced contracts to energy producers. This latter explanation is just one of the many differences between the supposed
real implementation and the simulated environment.
Every type of agent has different duties inside the system. Some of those duties
are real world related, that means we are referring to behaviours directly connected
to the functioning of the real market energy exchange system. E.g.: finding the best
seller/buyer.
Other part of behaviours are created in order to set the simulation environment, so
they will not be present in an hypothetical finished software, but still, they have been
implemented in order to analyse and make possible the whole simulation system. E.g.:
Graphic Interface Implementation. In order to have a better understanding of which
15
3. TECHNOLOGIES AND ARCHITECTURE
Figure 3.1: Main agents general view - Showing a generic class diagram of every main
agent
16
3.1 JADE
behavioural step belongs to the real of the simulated implementation, we can look at
figure 3.2.
Figure 3.2: Agents’ description in two different approaches - Shows the difference in behaviour of every main agent between a simulated environment and a realistic
implementation
TheWeather Agent now randomizes weather while in the state of art architecture
should retrieve forecasting from a web service (see the following sections for more related information). Forecasting information are also used by consumers, but just for
simulation purposes: in a real world it is not about using those information, but is more
being influenced by weather when we are thinking about consumptions: this point will
be better explained in the forecasting related chapter of this thesis.
There are also some part of behaviours that are shared between the two worlds:
that means in the real world and in the simulation environment the system has to be
17
3. TECHNOLOGIES AND ARCHITECTURE
aware of those variables, but in this scenario, they are treated in order to fulfil simulation purposes.
All the buyers (except for one), are randomizing data about previous consumption.
In the real environment, this should be done by interconnecting with a Smart Meter:
in our simulation, due to the fact that we are provided by just one of those devices,
only one agent (called enhancedBuyer), could actually send a request to a real Smart
Meter, installed in a real domestic context. See chapter 4 for more information about
this interconnection.
3.1.2
Agent messaging
Before seeing every behavioural step that every agent go through during a negotiation
round, should be specified that the core of every link between agents are represented
by messages. In our simulation those are of two kinds: synchronization messages and
market related messages.
In JADE, the messaging system is fully compliant to the FIPA standard, winning
the interoperability tests between different platforms in Seul (Jan. 1999) and London
(Apr. 2001). Those tests were executed according to the FIPA official site and therefore
summarized in (15).
Some information that we need to know and explain before describing the content
and the classification of every single message sent and/or received by an energy market agent, are about what kind of information can an agent specify when creating an
ACLMessage.
First of all, as seen in figure 3.3, the paradigm chosen is for an asynchronous passing;
in fact the clear presence of queues between a single sender and a set of receivers make
us think about that every agent has its own mailbox in which can extract and therefore
process more messages. If and when a single agent should pick up or not a message is
up to the programmer, but usually, the receiver who has blocked its execution due to
the waiting for a message, is being waken up by its container.
Other important parameters for a single message:
18
3.1 JADE
Figure 3.3: The JADE asynchronous message passing - The JADE asynchronous
message passing paradigm as seen in JADE OFFICIAL TUTORIAL
• Content: the content of the message itself. Not much to say about it.
• Performative: the purpose of the message, or better the communicative intention.
Independently from the content, an agent could decide to try to better understand
what a sender is saying by just fetching this part of the message. Typical usage
in our project:
– INFORM and INFORM REF: generic informative messages used for synchronization purposes. The latter one is used as synch signal for final operations (e.g: a buyer informs a balancer that he has stipulated a contract
successfully)
– PROPOSE, REFUSE and ACCEPT PROPOSAL: used in market related
messages, the first one is an agent proposing a bid (sent by buyers) or a
starting price (in case it is a seller), the second for refuse a single offer and
the last one for accepting it.
– CANCEL: sent by a prosumer in order to abort the current auction with the
specified buyer.
• Language: syntax used to express the content. Not used in our project.
• Ontology: Not used in our simulation, but for future and more extended versions,
it may become useful to process messages according to their pre determined ontology. Other researches have specified agent ontology here (16).
19
3. TECHNOLOGIES AND ARCHITECTURE
We have several methods in order to set the previous parameters to the desired
values, while we have (according to JADE documentation) one strongly suggested way
to block an agent behaviour until it gets a message. This code HAS to be implemented
inside the action() method of the agent behaviour extended class:
...
public void action()
{
ACLMessage msg = myAgent.receive();
if(msg != null) {
...
//message received and ready to be processed here }
else
block();
}
...
In this way we can indefinitely block an agent scheduled behaviour until a message
wakes it up. We can also specify a maximum wait for a message to arrive by setting
an int value as the parameter of the block() method above. That number should be
expressed in milliseconds.
In order to have a complete control on the messaging system, JADE also provides methods for managing the queue like createMessageQueue(), getCurQueueSize(),
getQueueSize(), postMessage(ACLMessage msg), putBack(ACLMessage msg) and setQueueSize(int newSize). Altrough, a specific method in order to clear the queue does not
have an official implementation: in our simulation the following alternative has been
used:
while (myAgent.receive() != null) { ; }
FIPA standards are not just related to the message format, but it also gives information about how to name an agent in a distributed and heterogeneous platform.
20
3.1 JADE
Agent naming will be one the topics treated in the next section.
3.1.3
Implementation
This section will explain each behaviour for each agent involved in the simulation environment. Before that, we have to specify that every java class that extends a JADE
agent concept, has a setup() method which runs just once on its starting, a takeDown()
method that is called every time an agent calls the doDelete() function and describes
the last action that an agent should take before closing itself and a series of behaviours
that depending on their implementation could run multiple times. In almost all of the
following agent classes, most of the behaviours are multi step behaviours: the agent
go through all the implemented steps until the last one, then it could start from the
step number 0 if it is ready to start another negotiating round or call the doDelete()
function in order to kill himself if we reached the end of the simulation.
In order to start the simulation, the following command should be execute from a
command line environment:
java -cp lib/jade.jar;classes jade.Boot -gui -agents ac:simulation.AgentCreator.AgentCreator
As we can see, we are telling JADE to start its GUI with a specific first agent: the
AgentCreator, which has to be called by specifying its related class package and by
assigning it a name (ac, in this case). Since this last agent is the first to be created, it
will be the first to be explained.
AgentCreator
Agent method: setup()
In this init method, the agent just creates the prototype for the behaviour method.
It will have a single oneShotBehaviour behaviour.
private class CreateEverything extends OneShotBehaviour: Action()
21
3. TECHNOLOGIES AND ARCHITECTURE
Creates and starts the weather agent and the time agent. After that, it will run a
30 times executed iteration in which will create a single enhancedBuyer, 29 ordinary
Buyers, 7 prosumers e 3 Gencos. Eventually it will also create the balancer and kills
itself with the method doDelete(). Every created and started agent, has a name that
starts with ’c’ if it is a consumer, ’s’ if it is a prosumer and ’g’ if it is a genco; to this
letter, the iteration number is added in order to have a unique name space as well.
Agent method: takeDown()
Terminates its execution and state its intention by printing a statement in the command line.
Weather Agent
Agent method: setup()
It randomizes weather, using a function that outputs three numbers:
• Temp: related to the current temperature degree.
• Solar pw: related to the average solar rays incidence power.
• Wind pw: wind strength information.
Also it registers itself at the DF, specifying its willing to offer a Weather Forecasting
Service. Eventually it define its main behaviour.
private class Behaviour: Action()
If first run asks to the Time Agent the current hour and then completes the randomizing of the forecasting according the time of the day. If not the first run it just listens
for INFORMATIVE messages containing the ”WF” string: those are other agents asking for weather forecasts; this message triggers a reply directly to the sender.
22
3.1 JADE
By receiving an INFORM REF performative message that contains the word ”DIE”,
causes the agent to call the takeDown() method.
Agent method: takeDown()
Terminates its execution and states its intention by printing a statement in the
command line. Also by doing this, it will de-registers from the DF service
TimeAgent
Agent method: setup()
In this init method, the agent registers at the DF. It will have a single CyclicBehaviour behaviour.
new CyclicBehaviour: Action()
The current implementation is not ready: it should be rewritten in order to have
two simultaneous behaviours (parallelBehaviour) :
behaviour A: wait for other agents demanding the global time
behaviour B: wait for an EOT (end of time interval) message to send to the balancer
agent; see the BalancerAgent’s explanation for details.
In the actual implementation, the single behaviour forces the agent to listen and
reply to whoever asks for the current time.
Agent method: takeDown()
Terminates its execution and states its intention by printing a statement in the
command line. Also by doing this, it will de-registers from the DF service.
Balancer Agent
23
3. TECHNOLOGIES AND ARCHITECTURE
The balancer agent dumps a log file in the /logs/balancer/balancerName.txt file
keeping tracks of all of its operations as well as received messages.
Agent method: setup()
In this init method, the agent registers at the DF, clears all vectors and initialize
the graphic user interface. It will feature a single multi step behaviour.
new Behaviour: Action()
Step 0:
Searches in the DF for the weather and time agents.
Step 1:
Searches in the DF for sellers: both Gencos and prosumers. Every time it discovers
a new one, the balancer sends to it a INFORM type of message, with a ”IN” sting
content that is used to tell to sellers that they could now inform the balancer about
their name and position. If it is talking with a prosumer, also receives information
about how much energy the prosumer is going to produce for the next time interval.
Once the balancer gets a reply, it draws an agent representation on the GUI.
Step 2:
Same as the previous step, but with consumers (buyers). Together with name and
position, the balancer also fetches information about the single balancer demand forecasting so at the end of this step the balancer could easily calculated the total demand
of the observed area.
Step 3:
The balancer agent sends an INFORM message to the weather agent (content:
”WF”): by doing this it will be able to display some information about the future
24
3.1 JADE
weather in the GUI. Also in this step, the balancer is performing calculations in order
to balance correctly the demand/supply equation. See the next chapters for more information about this issue: basically the balancer sends the production threshold to every
Genco and (this last one is not yet implemented) probability data about the learning
strategy for buyers. After that it sends a GO message to all participants in order to
start the negotiating process.
Step 4:
Listens for CONFIRM messages sent from buyers who have stipulated a contract
with a seller: every time it gets this kind of messages it will update the GUI showing a
direct link between the interested buyer and the seller. Once every buyer has reached
an agreement it sends INFORM REF signals that could be RST (reset signal that
forces everyone to start over from step 0) in case we have other spare time intervals,
or else could it be a DIE killing signal for shutting down every alive agents (except the
balancer itself and the enhancedBuyer) in order to end the simulation and have a look
at the GUIs for statistics etc... .
It also listens for EOT signals from the TimeAgent. Those signals indicate that the
time dedicated for the contracting operations is almost expired: users without a contract will receive SUBSCRIBE kind of message that is able to force the buyers without
contract, to stipulate an agreement with the preferred (so nearest) Genco. All of this
is just partially implemented.
Agent method: takeDown()
Terminates its execution and states its intention by printing a statement in the
command line. Also by doing this, it will de-registers from the DF service. Also closes
the output stream for the log file.
Buyer Agent
25
3. TECHNOLOGIES AND ARCHITECTURE
The buyer agent (sometimes referred as consumer), could be enhanced or regular.
In our simulation we have 29 normal buyer agents and one enhancedBuyer that has
the possibility to interconnect itself to a Smart Meter or another middleware device
connected to a Smart Meter.
All the buyer agents dump a log file in the /logs/consumer/consumerName.txt file
keeping tracks of all of their operations as well as received messages.
Agent method: setup()
In this init method, the agent registers at the DF, clears all vectors and initialize
the graphic user interface in case it is the enhancedBuyer designated. It will feature a
single multi step behaviour.
new Behaviour: Action()
Step 0:
Searches in the DF for the weather and time agents.
If enhanced, it will contact via HTTP the web server between the Smart Meter and
apply its forecasting algorithm: in order to do that the buyer will parse the XML server
reply. For further information, see the related chapter of this thesis.
Step 1:
Searches in the DF for services: time, weather and balancing. Then casts a blocking receive procedure to wait for the IN signal from the balancer: it replies with an
INFORM message containing its name, position and forecasted demand.
Step 2:
In this step the buyer just waits for the GO signal that will start the negotiation:
after that it will contact every seller; since the strategy learning part of this project has
26
3.1 JADE
been tested in a different code every buyer in this simulation will first ask every seller
to propose their starting prices by sending an INFORM message stating their name
and position. Those prices are going to be sorted putting the cheapest sellers in lower
indexes of the designed vector of sellers.
Step 3:
The buyer will contact every Genco and will receive an INFORM message stating
their position. According to this variable, the buyer designs its preferred genco according to the nearest distance.
Step 4:
It starts by removing not suitable prosumers from the respective vector: a not suitable seller is a supplier that does not have enough energy to satisfy the consumer’s
request. If at the end of this step, that vector remains empty, then the buyer should
immediately jump at the seventh step in order to contact Gencos. In the other scenario,
if we have at least one suitable prosumer, the buyer uses the cheapest starting price
to calculate (adding a random percentage) its expected budget to spend in the current
negotiating interval; keep in mind this is not a complete implementation of the learning
strategy described in chapter seven.
Step 5:
Here the buyer will send the first bid to the best seller (cheapest suitable prosumer)
according to its pre determined stake to add to the starting price proposed by the seller
and then starts listening for other messages:
• ACCEPT PROPOSAL: jump to step 6.
• REFUSE: the seller refuses the bid. If the buyer’s actual strategy allows the buyer
to be refused again by the same seller, it will rise the stake and sends the bid
again by staying in thi step. Otherwise it will remove the seller from the suitable
27
3. TECHNOLOGIES AND ARCHITECTURE
seller vector and start again with another prosumer: the refusing prosumer will
be informed by this decision with a CANCEL message from the consumer.
• CANCEL: An abort contracting message featuring a CANCEL performative, will
be received by the buyer if the seller reached the sold out of its supplies. Also in
this case, the buyer will remove that prosumer from the vector of possible sellers.
• SUBSCRIBE: sent by the balancer to inform the buyer that too much time has
passed and it has not reached an agreement: the buyer is redirected to the seventh
step with a forcing command to obtain the contract from the nearest (earlier was
called preferred) Genco.
Step 6:
Sends data about the obtained contract to the balancer agent, and waits until it
receives an RST or DIE signal.
Step 7:
Contract every Genco in order to obtain all their current prices and then accept
the cheapest of all the proposed. After that, the consumer just move back to the sixth
step.
Agent method: takeDown()
Terminates its execution and states its intention by printing a statement in the
command line. Also by doing this, it will de-registers from the DF service. Also closes
the output stream for the log file.
Prosumer Agent
The prosumer agent dumps a log file in the /logs/sellers/prosumerName.txt file
keeping tracks of all of its operations as well as received messages.
Agent method: setup()
28
3.1 JADE
In this init method, the agent registers at the DF, clears all vectors and initializes/randomizes all the necessary variables. It will feature a single multi step behaviour.
new Behaviour: Action()
Step 0:
Searches in the DF for the weather and time agents.
Step 1:
Sends an INFORM message containing the string ”WF” to the weather agent. Once
received the answer, the seller/prosumer agent extract the value relating the temperature and the solar or wind value depending what kind of renewable sources it can use.
Step 2:
With the just obtained data, the agent calculates its future production capacity
and waits for an IN signal from the balancer. Once received it, it answers by stating
its name, position and calculated supply capacity.
Step 3:
Wait for consumers’ proposals. The length of this wait depends on the number
of buyers: this variable could be discovered by searching the DF and once collected
enough offers, the agent move to the following step.
Step 4:
The seller here, refuses every offer except for the highest one; if this last offer is
bigger then the expected earning for this agent, the prosumer will reply with an ACCEPT PROPOSAL, otherwise it will send a refusal signal (or accept anyway if its
strategy allows it to do). In case of an abort contracting signal sent by a buyer, it just
29
3. TECHNOLOGIES AND ARCHITECTURE
erase that buyer from the respective vector and go back to the third step in order to
listen for other proposals.
If the prosumer have no more supplies to sell, it moves to the next and final step.
Step 5:
Listening for RST or DIE signals from the balancer agent. Obviously, the agent
could receive these kind of messages also in the previous steps: just think about a seller
who don’t manage to sell all of its supply, but the balancer in order to coordinate the
entire architecture has to restart every contracting operation from scratch once the
negotiating time interval its over.
Agent method: takeDown()
Terminates its execution and states its intention by printing a statement in the
command line. Also by doing this, it will de-registers from the DF service. Also closes
the output stream for the log file.
Genco Agent
The Genco agent dumps a log file in the /logs/gencos/gencoName.txt file keeping
tracks of all of its operations as well as received messages.
Agent method: setup()
In this init method, the agent registers at the DF, clears all vectors and initializes/randomizes all the necessary variables. It will feature a single multi step behaviour.
new Behaviour: Action()
Step 0:
30
3.1 JADE
Searches in the DF for the weather and time agents.
Step 1:
Waits for an IN signal from the balancer. Once received it, it answers by stating
its name, position and a maximum production capacity threshold.
Step 2:
It waits for the suggested production threshold, a value that is calculated by the
balancer. For more information about this number, see the balancing/pricing chapter
of this thesis.
Step 3:
The Genco at this point, just listens for every kind of message: both contracting
from buyers and RST/DIE from the balancer. In the first case it answer by proposing
a price that is depending from distance from the buyer and if the seller has reached
the threshold or not (once again for more information, see the balancing/pricing chapter of this thesis), in the latter scenario it will die or restart over from step number zero.
Agent method: takeDown()
Terminates its execution and states its intention by printing a statement in the
command line. Also by doing this, it will de-registers from the DF service. Also closes
the output stream for the log file.
Other important aspects
First of all, simulating more then 40 agents in a single computer is not an easy
tasks. Receiving unexpected messages during all the steps of every kind of agent is
very common: an unexpected message could be for instance, receiving a weather forecasting string while we are expecting a contracting message from a buyer or vice versa;
31
3. TECHNOLOGIES AND ARCHITECTURE
in these situations, every agent should treat these messages in order not to parse a
numeric content like it was an alphanumeric string or vice versa, since it will cause the
agent to die prematurely throwing a java exception.
For the same reasons, it is also common not receiving messages or experiencing
delays during the sending/receiving operations. In order to contrast these scenarios,
every agent in every step has different preventing measures like time-out waiting times
for contacting messages and the possibility to send stimulation messages to agents that
were supposed to send synchronization signals, but the agent that was supposed to
received it did not obtain it.
In addition to all of that, even if it was not tested, the current implementation
supports more than one balancer, weather and timing agents. They are all retrieved
from the DF and saved in different vectors: that allows the requesting agent to contact
different auxiliary agents if the one that they had previously contacted is not functioning any more. The simulation code also supports the possibility to add agents after
the beginning of a time negotiating interval: if an agent wants to join the platform it
could do it any time and it will start participating from the following time interval. As
previously stated, a host can run multiple agents becoming for example, both a seller
and a buyer, just by creating two agents in the same platform. However, since the
simulation code was not tried in a distributed environment (other studies for how to
distribute the architecture according the current state of art are still in progress), this
solution (theoretically possible with the actual implementation) could not be tested;
it is also to be specified that in case of a multiple agent hosts with sellers and buyers
running in the same device, additional code may be required in order to balance the
equation energy asked/energy offered to that specific peer of the net.
Graphic interface
For a better understanding of all of this scenario, the simulation outputs both log
files for each ”real” agent and a graphic interface showing the stipulated contracts,
weather for the current time interval and a graph that compares expected prices with
obtained prices after the auction session.
32
3.1 JADE
Figure 3.4: Simulation GUI - Simulation showing forecasts, contracts and market
strategy results for a single consumer peer
33
3. TECHNOLOGIES AND ARCHITECTURE
Looking at figure 3.4, on the upper left corner: this is a view of the city in which we
can see consumers (purple coloured, starting with letter ’c’) prosumers (blue, ’s’) and
gencos (red, ’g’). All the black lines are connection that symbolizes agreed contract
between peers: in this particular screen shot, no prosumers succeeded in selling energy,
probably due to bad weather conditions (displayed in the right upper corner). Clicking
on a consumer peer allows the graph to be filled by some kind of a plot. The red line
symbolizes the expected price (calculated as the lowest starting price of all prosumers,
plus a randomized percentage of added cost). The black line is what the agent actually
managed to pay. Prices have to be intended as for a single energy unit. We have chosen
to divide the day in six periods of time, varying for each interval, the weather and the
energy demand for every single consumer.
Here is another graphic example (3.5):
As we can see, the red line never changes over time, that is due to the fact that with
no learning algorithm there is no attempt to adjust the auction’s results. Defining as
win the situation in which the black line is below or overlaps the red one and as fail, the
opposite situation, it is clear that in the last one the agent, not only, is always failing,
but in the first three intervals it keeps spending more and more money on a single energy
unit, before stabilizing on a constant value. The previous graph, shows a win situation
in just one interval, while in the other he loses in an unpredictable trend. The important
issue here, is to determine what kind of variables are responsible of those unpredictable
and different behaviours. Looking at the log files is it clear that the time factor is critical. Every consumer has now just one strategy: they contact every prosumers before
trying to have a Genco energy contract, so if they contact a good prosumer after it
has already dispatched all of its supply energy to other buyers, the observed consumer
will just waste other time waiting for an ACLMessage.CANCEL/REFUSE or (worst
case) waiting for a limited but still calculable amount of time an answer that may be
never been created. In the meanwhile other cheap sellers can stipulate other contracts,
even with gencos, causing in the latter case, to move forward the supply threshold of a
specified Genco and therefore rising prices.
With a learning algorithm is it possible to try to limit this kind of issues: the algorithm
can ask the consumer to react by rising the stakes of every single bid that a consumer
34
3.1 JADE
Figure 3.5: Peer market statistic - The difference experienced by a buyer between
what it was expecting to spend and what actually paid for an electric energy contract
35
3. TECHNOLOGIES AND ARCHITECTURE
can propose to a seller and that is a very effective way to reach the seller expectation
and therefore get an acceptable contract at a reasonable price. Although this is not a
definitive way to solve this problem since if every prosumer has already dispatched all
of his energy offer, the buyer is forced to contact gencos.
Time constraints are very difficult to simulate in a single host since every agent is a
thread and is always difficult to know and analyse which of the more the 40 agent has
the priority inside a simulation session. Another variables to study is the possibility
to lose messages during the auction. Log files of different simulation sessions show
that this kind of unexpected behaviour can actually happen and even if the simulation
can arrive at its end with every consumer having a signed contract, an unpredictable
(usually little) number of ”ruined” and therefore ”expensive” contracts are issued. The
reliability of sending/receiving messages, especially in the real world, outside any other
simulation, is still an open issue. The other variable is obviously the demand. If a
consumer has a very high demand and if the weather does not allow prosumers to produce the right amount of energy to fulfil those needs, the observed consumers is forced
to begin contacting gencos, resulting in higher price and increasing the time critical
impact on the system.
With no learning algorithm, variables like bid stake, expected price and number of
acceptable refusals before changing seller are set in the first run of the simulation and
never changed. Even if they are randomized according a possible interval of realistic
values, randomizing this kind of important parameters can lead to unlikely scenarios.
Here is how a consumer’s log file looks like (3.6)
3.2
XML
XML stands for ”extensible mark-up language” and it represents a modern approach
for modelling complex data systems using a similar set of rules and syntax like the
more known HTML, with added features regarding tags, elements and attributes that
may have arbitrary structure with the possibility to extend and easily upgrade the interchange format according to the user’s need.
36
3.2 XML
Figure 3.6: Buyer agent’s log - Buyer agent’s log file that shows how different bids are
sent to a prosumer, before contacting the Gencos and stipulating the cheapest contract
37
3. TECHNOLOGIES AND ARCHITECTURE
Due to the mentioned features and description, using the XML format is certainly
the preferred language that our architecture could use when interfacing itself with
the external world: in order to achieve a complete awareness of the surrounding environment of the agent platform, every agent will feel the need to ask for a weather
forecasting service, retrieve information about the real market trend and raw material
prices or communicating with lower level hardware and/or middleware devices. For
some of these needs, we can rely on contacting web services, while for the second we
may think about an HTTP request towards a middleware platform capable to host a
functioning web server and so we are talking about existing solutions; however different
devices/services most of the times are presented with different parameters or syntaxes
and we want to avoid the trap of constructing a fixed architecture that is capable to
communicate with a restricted type of services and devices.
For example, our architecture now only deals with a weather forecasting web service
and a Smart Meter; in a future and more complex implementation we should also think
about interfacing with web services that monitors the real energy market fluctuating
prices or interconnect an agent with other electric devices for example for monitoring
the conditions for transmitting power lines. In this scenario in which we know that our
architecture will expand its possibilities but we are not actually sure how and when
will it happen, the usage of an extensible language like XML become a really appealing
solution.
In the next subsection we will understand more about how to use web services in
our architecture, while in chapter 4, a complete overview of the intercommunication
with a Smart Meter will be given to the reader.
3.2.1
Web Services usage
Defining what exactly is a web service is apparently harder than describing how do they
work. That is because stating that a web service is just a modern implementation of
the old client-server paradigm over the internet in which the first asks for a service to a
server, is extremely reductive and misleading; in order to have a better understanding
we can rely on the official definitions:
38
3.2 XML
”Definition: A Web service is a software system designed to support interoperable machine-to-machine interaction over a network. It has an interface described in a
machine-processable format (specifically WSDL). Other systems interact with the Web
service in a manner prescribed by its description using SOAP messages, typically conveyed using HTTP with an XML serialization in conjunction with other Web-related
standards.”
The previous definition is taken from the World Wide Web Consortium (W3C) and
gives us other precious information: the definition focusses on the concept of interoperability and machine-processability (usage in an autonomous way) to describe how
to interact with those web services and it features an extendible and flexible language
that we already introduced. In other words, web services are software entities spread
on the internet and they provide plenty of services that can be discovered and used
by different systems dealing with a completely arbitrary architecture that specifically
designed auxiliary languages can describe to an autonomous system in order to be comprehended and wrapped as a part of bigger architecture.
Linking this technology with Java (and therefore a JADE agent) is fairly easy due
to the ever increasing presence of official and/or third part’s libraries that can interact
and parse all the protocols and language involved in using a web service. The most
famous library package is JAXR, that can also deal with the XML parsing of the SOAP
message and the discovery and finding of the different services via UDDI. For more information regarding the theory behind of all it, see the second chapter for furthermore
references.
A good usage of a web service applied to our project would be for instance a
weather forecasting request: once discovered a functioning service, we can ask him for
its WSDL scheme that will be useful for the agent to parse it in order to understand
what parameters are going to be inside the XML formatted reply. Example: from this
site (Global Weather Service from WebserviceX.net) we can extract the WSDL scheme,
in which for every field of the upcoming XML, the agent can learn name and variables’
type. Also information about the needed inputs for that services: in that particular
example, the agent knows that in order to receive the forecasting, it has to send its
39
3. TECHNOLOGIES AND ARCHITECTURE
country and city in which it is situated. That agent will get an answer like this one in
figure 3.7
Figure 3.7: Typical XML reply for a weather forecasting request - Every field
has been specified to the agent thanks to the previous WDSL format exchange. Source:
http://www.webservicex.net/globalweather
In figure 3.7 we can see how a prosumer agent can extract information about wind,
temperature and sky visibility that will use in order to forecast its future production
capacity, while the other values (shown with green arrows) can be used by agents who
would like to forecast their next energy consumption instead.
From the same web service provider we could also find the GeoIpService that allows
the requester agent to know where it is currently located looking up the IP geographic
context: we can therefore combine that data as input for the weather forecasting service
described just above in order to give the same demand/supply forecasting capabilities
to agents that are constantly moving since they are created in a portable device.
40
4
Interconnection with the Smart
Meter
4.1
Smart Meter: definitions and usage
Smart meters are one of the most important parts when talking about smart grids.
Smart Grids are particular types of energy distribution nets that they can adapt themselves creating constant communications with users in order to avoid consumption peaks
over a single or limited set of suppliers. In this scenario a smart meter is just a new
generation energy consumption reader that differs from the old ones by having some
additional features:
• A bidirectional communication between users and supplies of the net
• Could be easily interfaced with other external devices
• Could handle information about tariffs and costs
• Higher time granularity for electrical readings
Focussing again on the trading agent platform, these particular devices are the
the perfect choice having all the above listed features in order to have a complete autonomous system. They have been produced when the deregulation process started and
they are already spread in some countries, however right now different smart meters
may comply to different standard and different levels of out-of-factory customization
41
4. INTERCONNECTION WITH THE SMART METER
possibilities, forcing researches in the smart grid field to deal with middleware platforms and not directly with the smart meter.
In our project, the issue of communicating with a smart meter or with another
middleware related device, has been investigated, tested and discussed since it is one
of the most important part of the designed architecture.
This part of the project is linked with another one, which was proposed and solved by
Hanzehogeschool students J.J. Huzeling and J.C.H. Bos, supervised by Mr. Kamphuis.
The purpose of their work is to design and develop an interface module for the smart
meter that has to have an easy access to the outside world using the internet standard
communication protocols.
In their project plan, they had to choose, investigate, assembly and test all the
necessary processors and other electronic devices needed in order to solve this problem.
We studied and agreed together on the best way to interface their middleware device
to a buyer agent described in our architecture. Also the possibility to implement our
agent(s) in their device has been widely discussed and researched.
They succeeded in building a small device and installing it in one of their house in
order to keep track of the amount of electrical usage in a real domestic environment,
with additional features allowed by implementing a small embedded web server that
allows communication over the TCP/IP stack, sends and receives HTTP get requests
and stores important quantities of past consumption readings.
By using those characteristics, their device has also the possibility to be interfaced
with a Google Power Meter account in order to have some graphic evidences of the
examined domestic consumption added to all other features that this new Google web
based application can offer to the users. More information about their project can be
found in: SMI Project Home page 1 .
In figure 4.1, we can see the basic scheme of their solution.
1
http://meter.info.tm
42
4.1 Smart Meter: definitions and usage
Figure 4.1: Smart Meter Interface overview - All the most important macro component of the Smart Meter interface project. The agent can communicate to the device
that features an installed and fully functioning web server
The main purpose for a generic buyer agent while communicating with a smart meter is retrieve information about previous and actual consumption of electric energy in
order to forecast the energy demand that this agent is going to buy during the following
energy negotiating interval.
Another task is represented by updating all the data about costs once the agent has
finished to stipulate a new energy contract.
While the first task was successfully implemented and tested, the second one was not,
and that is because as already stated, different smart meters has different levels of
customization and the one used for this platform could modify those information just
by sending the device back to the original factory.
It is important to highlight that for the agent, there is no difference in communicating directly with a smart meter or another related external and/or remote device, since
the only parameters to change when dealing with a web server are address, port(s) used
and name of the resources.
More information about smart grids, deregulation and forecasting demand, can be
found in the introduction chapter and in chapter 6.
43
4. INTERCONNECTION WITH THE SMART METER
4.2
Mobility issues
Together with the Hanze students, we studied the possibility to embed all the necessary
part of the agent architecture in the same electronic device they were (at that time)
starting to design. They proposed a couple of microprocessor families and our task was
to find out which was the best in order to support a small distributed instance of JADE.
They proposed the ARM7 and the Microsolutions Microchip families, with their
respective commercialized models called LCP2378 and MIPS32 M4K. The first one
features a 32bit architecture, 80 MHz CPU frequency and 512 Kb of internal memory,
while the second one features a different 32 bit architecture coupled with a bit slower
CPU (72 MHz) but it can rely to additional 58 Kb of SGRAM to the standard 512 Kb.
Both of them are basically micro controllers in which the user can install an RTOS
in order to have a complete control over all the features these two chips can offer.
Choosing a RTOS compatible microcontroller chip is indeed a wise choice: a more traditional styled operating system could not use the advantages of querying the smart
meter with the permitted reading granularity. However, dealing with this kind of system can create difficulties when dealing with the software implementation/installation
issues.
Even using an RTOS and embedding everything in their device could barely satisfy
the basic requirements of having a distributed instance of JADE even with the LEAP
add-on for added mobility and/or the use of a very light version of a Java Virtual Machine (like Jamaica Virtual Machine, specifically designed for real time systems).
However, according to the impossibility to simulate the implementation without
actually try with real components and to other researches that basically failed to reach
this goal, I came to the conclusion that (as other papers state like (17)) embedding a
JADE instance even using all the extensions in order to reduce to the minimum the
resources’ usage, it is not possible with both of the CPU proposed. Even with more
performant CPUs from the same families would be difficult and could lead to unacceptable slow response times even in the luckiest hypothesis.
44
4.3 Communication features and typical requests
However it is still possible, to embedded everything in a mobile phone or other
mobile devices with a different kind of operating system installed. Unfortunately, in
this case, other security related issues arise: see chapter 5 for more information on this
matter.
Hanze’s students opted for the ARM7 solution and we agreed to start implementing all that is needed for the communication between an agent and their device.The
following sections describe how this message passing is designed.
4.3
Communication features and typical requests
An agent behaviour, as already pointed out, is written in Java that has a wide development kit allowing the programmer to perform HTTP requests quite easily. Since we
can rely on a functioning web server that is installed in the previously described middleware device, using HTTP requests is probably the best choice when talking about
communication. The only issue here is to create a secured connection in order to avoid
certain attacks: by using uClinux in combination with openSSL it is theoretically possible to set a secure communication session between the two parts. A simpler but less
secure solution is to use the unique identifying number featured in the smart meter
as a parameter request during the HTTP message exchange: every request made by
the agent to the device should start with the agent communicating that number to the
embedded web server, that answers just if that number is correct. This last solution
is certainly not the best one, but at least we can rely on a basic security level for
communication between agents and their own smart meter interface. Communications
between middleware interface and the Google Power Meter system already features a
secured connection (SSL) thanks to Google API.
HTTP get request with protocol version 1.1 was used during the tests.
The first point we discussed was if the communication for obtaining consumption
readings for the agents should follows a push or pull paradigm. We chose for the pull
solution: the agent asks the smart meter for the data and repeats the request when
45
4. INTERCONNECTION WITH THE SMART METER
necessary. The disadvantage of a push solution is that the examined agent could receive
a lot of not necessary readings while dealing with negotiating part of its algorithm and
this could lead to unexpected behaviours or problems while receiving messages with
complete different purposes.
The second most important point was to agree on a common standard to pass information. This language should be easily updatable, extensible and should not be
an obstacle for future implementation. In addition to that we had to take notice that
different smart meters may have different parameters’ name. As already presented in
chapter 3, the XML format is probably the best choice. In addition to that, it is one
of the language the agent needs to know how to parse in order to communicate with
all the web services involved.
JAVA standard environment does not have a pre existing import for parsing XML
files, so an external library source was used. JDOM XML
2
parsing libraries have been
used in order to have a simple but at the same time efficient set of classes to facilitate
the parsing.
The following is a list of possible ways to query the middleware device connected
to the smart meter.
• GET /echo.cgx?echo int=5 HTTP/1.1
This is used for testing the possibility to send values to the device by the agent.
Due to the fact that the only data that a buyer would send to its smart meter
is the one related to the new stipulated tariff, this request will be implemented
when smart meters without the limitation of changing factory settings will be
available. Basically we are sending a char value to the web server and it is just
echoing back in order to let us understand if it got that number or not and if the
agent was able to receive it.
If we use 5 as parameter we can expect this response:
<form>
2
www.jdom.org
46
4.3 Communication features and typical requests
<echo>You said: 5</echo>
</form>
• GET /smart meter slim.cgx HTTP/1.1
This is the first request made by the enhanced buyer. The server sends to the
agent an XML response in which all the most important smart meter parameters
are transmitted. In our simulation, from this request, we can extract the field
related to the actual applied tariff and use it to calculate the parameter related
to the expected money to spend of the buyer agent. The following is an example
of a typical response:
<form>
<m name>SmartMeterSim</m name>
<m iden>0505514284290634 </m iden>
<eu low>489.777</eu low>
<eu normal>489.777</eu normal>
<ep low>.114</ep low>
<ep normal>.114</ep normal>
<t indic>normal</t indic>
<e watt> 60</e watt>
<e ampere>85</e ampere>
<switch>on</switch>
<cost>0.01</cost>
<save></save>
<button> </button>
<time>11-01-24.10:21:44 </time>
</form>
The first parameter is about the unique Smart Meter identifying number, In
addition to that, an agent could also know if certain switch are on or off, have an
average consumption reading expressed in Watt, Ampere or Voltage and a final
timestamp.
For more information about the meaning of each of those parameters could be
found inside the SMI communication reference v0.2, that is a part of the Hanze’s
project.
47
4. INTERCONNECTION WITH THE SMART METER
• GET /forecast req.cgx HTTP/1.1
This is the second request made since the beginning of the negotiation round. It
gives us information about today’s and yesterday’s consumption. The information
is divided in four values related to the actual consumption calculated from the
beginning of the current hour and the average consumption about yesterday at
the same hour of the new request, but also yesterday for the previous hour and
the next hour. In order to have a better understanding, we use an example: the
agent is performing a forecast request at 11:44:
<form>
<previous hour yesterday>
Usage between 11-01-10.10:00 and 11-01-10.11:00 is 000000.100
</previous hour yesterday>
<current hour yesterday>
Usage between 11-01-10.11:00 and 11-01-10.12:00 is 000000.200
</current hour yesterday>
<next hour yesterday>
Usage between 11-01-10.12:00 and 11-01-10.13:00 is 000000.300
</next hour yesterday>
<current hour today>
Usage between 11-01-11.11:00 and 11-01-11.11:44 is 000001.953
</current hour today>
</form>
All values are expressed in KWh.
This is used for a basic demand forecasting algorithm that will be explained in
the next section.
There are also other ways to query the web server, but it is out of the scope of the
present work to provide all the details.
4.4
Simplified demand forecasting
For simulation purposes only, a simplified forecasting algorithm has been added to the
behaviour of a single buyer (called enhanced buyer), while all other buyer agents are
just randomizing demand values. That is because we can rely on just one Smart Meter
48
4.4 Simplified demand forecasting
and having all the agents doing forecasts using the same /forecast req.cgx directed to
same device will result in a unrealistic scenario in which every consumer is searching
for the same amount of energy.
In addition, we specify that this kind of algorithm just shows how it is possible to retrieve multiple data from a Smart Meter and use those values to complete a forecasting:
the state of art related to demand forecasting uses a more complicated system based
on a neural net that is instructed using a large data set featuring a lot of information
that are not easy to retrieve: see chapter 6 for more information.
This simplified algorithm basically adjust what an agent is planning to consume in the
next time interval by detecting the gap between what it is actually consuming with the
average consumption of the previous day during the same hours of the today’s request.
As already reported in the previous section, we have four values:
• Phy : Average consumption about yesterday at the previous hour of the current
time of today’s request
• Chy : Average consumption about yesterday at the same hour of the current time
of today’s request
• Nhy : Average consumption about yesterday at the next hour of the current time
of today’s request
• Cht : Average consumption about today, calculated as an average value between
all the readings made from the beginning of the current hour and exact time of
the new forecasting request.
Algorithm:
cht/timeElapsedF romBeginningOf T heHour ∗ 60 = avgCht
From the current consumption related to a certain number of minutes past the beginning of the hour of the current request, the average in the whole hour is then calculated.
avgCht − Chy = ytGap
Calculating the gap between yesterday and today consumption.
(phy + chy)/2 − nhy = yGap
Calculating the gap between different hour related to yesterday.
F ORECAST = avgCht + sign(ytGap) ∗ yGap
49
4. INTERCONNECTION WITH THE SMART METER
Adding the gap between the two days to the average of the whole current hour gives
us an idea on how much energy should a buyer ask during the next negotiating round.
The FORECAST value then is normalized between the interval [0 - 5] in order to
compete with other buyers’ values, that have randomized their previous consumption
data following that (simulation adapted only) interval.
It has to be specified that, every time an agent is performing this kind of request,
the target interface just query its memory and its log file(s) in order to retrieve data
related to the interested hours. It can happen that some of those data could be missing and one or more fields of the XML response could show a NO DATA FOR THAT
TIMESTAMP ERROR exception message: if one or more fields feature that error, then
even the enhanced buyer is forced to randomize the values explained in this and in the
previous section.
The enhanced buyer also displays a graphic user interface (GUI) that is composed
as shown in figure (4.2):
The table on the top part of the GUI provided, the buyer retrieves the cost value
(that is the tariff applied for the current hour), and use it to calculated the expected
money to spend for the next negotiating round. It is done by multiplying by two that
value (that is related to a one hour time interval, while our simulation has a different
time interval division) and then the buyer adds or subtracts a random percentage up to
the 20%. This is done just for the first time interval, the expected money to spend value
is supposed to be adjusted in the future by applying the market strategy algorithm.
50
4.4 Simplified demand forecasting
Figure 4.2: Smart Meter Interface GUI - An enhanced buyer could display in a
window the results of the smart meter slim.cgx and forecast req.cgx requests
51
4. INTERCONNECTION WITH THE SMART METER
52
5
Security issues
5.1
Problem description and JADE-S
A generic data exchange between hosts that are using internet as the mean of communication, has to be secured when those data represent sensible information. This is
really important when studying an agent based auction system, because data issues like
tariffs, costs and consumptions are clearly very sensible and could be easily intercepted
my malicious users. So the entire architecture has to satisfy the following requirements:
• Data Encryption
• Secure Authentication
• Communication Integrity
The first one simply means that all communications must be encrypted so that they
are not clearly visible by other users that are spying communications using sniffing
software or other instruments. The second refers to the importance of being recognized as a trusted user. Even if a malicious agent could not understand an encrypted
communication, it could think about stressing the system by faking its identity and
starting stipulating contracts on the behalf of other real users. The latter one is about
ensuring that the encrypted message has not been changed during the transmission: a
compromised message during a negotiation can drastically alter auction performances.
Since Jade is the architecture chosen for this experimentation, a good start in order to
solve all these problems is to test the performance of JADE-S security add-on, downloadable in the JADE official site as an extension. After its installation in the Jade
53
5. SECURITY ISSUES
directory, we still have to specify that we want to use the security services: this is
due to the fact that a secure communication consumes an important amount of system
resources, so JADE turns down all those services by default if it is not clearly specified.
Since the test environment resides in a single host, the tests is based on a simple two
agent architecture in which, each of them lives in a different main agent container. The
architecture is still distributed since each container is linked to a different system port.
• Platform 1: JADE platform uses port 1099
• MTP Service in Main Container 1: uses port 7778
• Platform 2: JADE platform uses port 2099
• MTP Service in Main Container 2: uses a random port, obviously different from
all the previous ones.
• MTP service is the service that is responsible for sending and receiving HTTP
messages in a distributed architecture.
Specifying that security services are needed, can be done using those additional
commands while booting JADE graphic user interface:
javacp < jade−classes >; < jade−s−classes >; < common−codecbase64Classes >
; < example − classes > jade.Boot -gui
-services
jade.core.security.SecurityService;jade.core.security.signature.SignatureService;
jade.core.security.encryption.EncryptionService
jade security authentication loginmodule SingleUser
< senderagentname >:package.senderAgentClass
< receiveragentname >:package.receiverAgentClass
Also system privileges can be set by using command lines and that is really important since if not otherwise specified, an agent can directly give commands to the AMS,
causing for instance the shutting down of a container and/or an entire platform. This
issue will be treated later in this section.
54
5.2 Tools used, testing and data encryption
5.2
Tools used, testing and data encryption
Lets take two agent that are about to communicate:
A sender agent named Alice;
A receiver agent named Bob;
Alice sends different messages to Bob: each message has different degrees of encryption
and the purpose of this test is to determine the best way to communicate in order
to hide the content of the message. Different sniffing techniques and instruments are
used to intercept their communication and before digging into the details of the test
it is essential having some information about those software and approaches used.
First of all: simulating the whole architecture in a single host, running a Windows 7
operating system, and trying to sniff a communication between different ports in the
same machine. Sniffing loopback traffic is usually a very easy task on a UNIX based
system, but in a Windows Environment this particular type of traffic is seen in a very
different way and intercept it is a tricky problem to solve.
Solutions tried:
• Installing and configuring a Microsoft Loopback adapter Driver.
• Adding a route to my local machine going through the network gateway, doing
so, every network traffic from my machine to itself will use the physical network
interface, it will then go to the gateway, back to the starting point.
• Trying a commercial software.
The latter solution is actually the only working: several sniffers have been used to
intercept agent communications;
• Wireshark. A very famous sniffer that can support plenty of communication
protocols; unfortunately this software was unable to see my Microsoft Loopback
adapter, causing the interception of just the packets going in or out the internet,
without showing a port to port communication in the same host. Free software.
• Archisoft EtherSnoop. It has the same purpose as the previous one, but it is
lighter and easier to configure. Nonetheless also this solution failed for the same
issues as the previous sniffer. Free software.
55
5. SECURITY ISSUES
• JADE standard Sniffer Agent. That is an useful debugging instrument that can
be found in the jade compiled classes path. It can intercept every communication
between different agents living in a distributed architecture, stating what kind
of ACL message has been sent and showing also its content. The sniffer agent
is a working solution, but using just a debugging instrument (that can be easily became ineffective by setting correctly agent privileges) does not give a full
understanding of the interception problem.
• Commview Evaluation version. That is the final answer to the loopback issue;
Commview is a commercial and expensive software that can be used in our tests.
The evaluation version has several limitations like a 30 days limited usage time
and also it hides the content of most of the intercepted messages. This latter issue
can be solved with a bit of luck: sending a lot of messages increases the probability
that the message intercepted in which its content has not been hidden, could be
the one that we need to investigate.
Figure 5.1: Different levels of security - How different levels of security can contrast
the action of different sniffing tools
Figure 5.1 represents a summary of security aspects. Starting from the top line there
is the lowest security level that JADE can provide; that is just the default options and
by doing so, every communication can be retrieved with a really little effort in this case.
The figure shows that also the debugging instrument provided by JADE has enough
privileges to sniff sensible information.
The security helper is the method that you have to invoke in order to set the agent
to code/decode the message it is going to send/receive. If the security service has not
56
5.2 Tools used, testing and data encryption
Figure 5.2: JADE sniffer agent GUI - With no encryption service at all, sniffing
agents could intercept the conversation between agents Bob and Alice
57
5. SECURITY ISSUES
been started, then that method throws a warning (not an exception) causing the agent
to die before start sending ACL messages. If the service is running but we are not
invoking the helper, the agent can just avoid to be sniffed by the JADE sniffing agent
(as seen in figure 5.2). In order to get the full protection of the sent data we have to
use both. Unfortunately this latter solution is not 100% secure as expected.
In order to explain the issue it is best to just take a look at some screenshots that show
how CommView works and how it was able to intercept information in figure (5.3).
Figure 5.3: Commview GUI - CommView screenshot showing sniffed data in a loopback
communication
58
5.2 Tools used, testing and data encryption
On the top part of figure 5.3 are shown which hosts and ports are interested by
the communication, including a time stamp and some information about the protocols
used. The part below is the most important because it shows the raw data captured.
At the bottom are shown information about protocols, port, MAC addresses and other
useful information. All of this can be easily read after creating a log file shown in figure
5.4:
Figure 5.4: Commview log file - CommView logs of captured communications
59
5. SECURITY ISSUES
Figure 5.4 is just an extract of a log entry that refers only to a single communication. Data like addresses, agent name, standards used can be easily seen (most of these
are located in the top red box), while the encryption only works on the content part of
the message (red box situated at the bottom).
One of the parameter used to chose the architecture was its capacity to handle the
security issues in a very trustful way and JADE seemed to be a winner due to the fact
that it contains extensions designed to provide a secure architecture. It does, but it
has also some limits as we can see above. The limits shown by performing the test are
indeed quite surprising and actually there are other limits as well.
The most serious one it is represented by the fact that the security service cannot
encrypt platform managing controls. Also, JADE-S documentation is scarce and outdated: it lacks of examples and it gets confused when talking about limitations in
different versions. It is also not completely clear (due to the incomplete documentation) if JADE-S can actually support a security service for agents mobility and/or
LEAP extension.
Although using JADE-S is easy and fast. Implementing the highest security level is
done in a very transparent way, but still signature and encryption are enabled by the
fact that each agent holds an asymmetric private and public key pair. Now, if a malicious attacker fake a message by spoofing its sender field using a name shown in a
intercepted message, it probably will not be able to succeed since it does not have the
right key combination in order to set a trusted communication. We can solve some of
the JADE-S problems by using another method that is included among JADE services:
IMTPoverSSL.
The main features of IMTPover SSL are:
• Fully relies on Internal Mail transfer protocol over a TSL/SSL secure connection.
• Encrypts every horizontal commands (messages between platforms and containers).
• It uses two files for each container: a keystore file for storing authentication
passwords and a truststore file for keeping track of every trusted user.
60
5.3 Other security aspects
• It needs certificate in order to determine which user is trusted and which it is
not.
• Completely supports LEAP extension, so it can be used in device with limited
resources.
• Like with JADE-S we can specify what combination of security service is needed.
This last solution is independent from the previous one, so it could be possible to
use both in order to encrypt not only ACL messages between agents but also all the
data across different containers. Nonetheless it has to be specified that in this way,
every container needs a certificate and that can be a really big issues when taking this
whole model in a real environment with thousands (if not more) containers filled with
users.
JADE-S is still in a early development stage; maybe in the future all of those limitations
will be solved, allowing the developer to use a simpler but effective secure environment.
5.3
Other security aspects
Data encryption is just one of the many aspects that have to be examined in order to
have a complete overview about the security problem in an agent based architecture.
The following sections will present a description about other important aspects of this
matter.
5.3.1
Authentication
JADE-S relies on JAAS API that includes some sort of a dialogue box in which a user
had to specify username and password in order to be booted in a container. Passwords
are (by default) stored in the same host as the main container in a plain text file:
this is obviously a serious and unacceptable way to deal with the security issues of
the entire architecture. Fortunately JAAS can interface itself with other password
storing methods, just by changing the login module: some studies found out that
LDAP directory service has not only the possibility to interact with JADE, but it also
delivers a really high level of security in storing password files. That is possible using
JNDI interface to call LDAP methods over a TSL secure connection authenticated via
61
5. SECURITY ISSUES
DIGEST MD5 (18).
Authentication in higher levels (during information exchange between peers) can be
obtained with:
• Trusted certifications (especially with IMTPoverSSL).
• Pair of public/private keys.
5.3.2
Authorization
In order to obtain a full and secure control over the entire architecture, every agent
must have permissions set in advance: failing to do so can cause the sudden shut down
of the entire architecture. Setting permission policies is something that has undergone
several changes during the development of different JADE versions. Nowadays JADE
can count on a quite flexible way to deal with this issue: permissions can be granted
or denied by modifying an access control list that is stored in a certain policy text
file, keeping in mind the fact that everything that is not clearly and explicitly allowed
is seen as denied by the system. That policy file can be treated like the password file
described in the earlier section and it shows if the following grants are allowed or denied
(referring to a single agent):
• Right to create and/or kill a Main Container
• Right to create and/or kill an agent Container
• Right to create and/or kill a one or more agents
• Right to interface with the AMS
• Right to send/receive messages from other agents
5.3.3
Message integrity
Integrity is obtained by using signatures. Those are handled by the signature service
that is always coupled with the encryption service (see section 5.1). Same transparency
level applies to this issue as the one seen while investigating the encryption system.
The figure 5.5 is the structure of a standard FIPA/ACL message as it is presented
in the official JADE guide. Security services like encryption and signatures are applied
62
5.4 Agent based malicious attacks
Figure 5.5: ACL Message format - Basic components of an agent message in a JADE
instance
in the whole payload section of the message. It explains why, in section 5.1, we discovered that important information like addresses can be easily intercepted using common
sniffers.
If the message is not encrypted or not correctly signed, the message is discarded and
the sender gets an ACL FAILURE message. On the other hand, if a first contact between two agents is seen as a trusted communication, JADE does not force the system
to check an already proved signature.
5.4
Agent based malicious attacks
There are plenty of studies relating what kind of attacks can an agent based architecture
experience (19), so it will not be that useful to repeat what other people have already
said about the matter. Instead, it will be more interesting trying to figure out how this
particular simulated solution of the auction system will react during those attacks.
63
5. SECURITY ISSUES
5.4.1
Attack scenarios against the agent trading platform
A DoS (denial of service) attack is probably the easiest way to attack either an agent
or an entire platform. Studies demonstrated how flooding an agent/AMS with a large
number of useless messages can drastically decreases performances in an exponential
trend. Talking about the auction based energy market, decreasing performances means,
usually, a growth of needed time to complete a negotiation: it has to be reminded that
useless messages have to be received in order to be discarded and so if there are countless of them trying to overflow the local buffer, not only they could provoke a crash
in the system, but before that they could alter the result of an auction. A solution
could be represented by using the loginmodule service that is provided by the JADE-S
extension: each agent has to login to its platform by providing a valid and recognized
username and password couple. Doing so, just authenticated (and so traceable) users
can attempt a DoS attack to a platform.
Having a username and password implies some sort of registration to a special authority. This matter has to be explained in a more detailed way.
See previous sections in order to understand how to encrypt and distribute password/certification files.
Attack scenario 1
A seller agent is communicating with a buyer. They are both legit users correctly authenticated on their respective platforms, using all the services provided by the JADE-S
extension.
A malicious agent has managed to sniff those communications. As discussed earlier, this latter agent cannot understand what those agents are saying (i.e content of
the message), but it can retrieve name and address of both sender and receiver. The
attacker could spoof its IP and fake an FIPA/ACL message making the receiver believe
that it is a real user: the malicious agent does not have the correct pair of key in order
to create an understandable message, but the attacker can send a large number of messages in order to corrupt the communication. The solution has to be implemented in an
another level of the architecture: internet providers have to check if packets outgoing
64
5.4 Agent based malicious attacks
their subnets are (probably) IP spoofed or not.
Attack scenario 2
The attacker can compromise a legit host, used by a trusted user that owns valid
username and password couple and all JADE-S services running.
Possible consequences:
• jadeSecurity.jar file can be altered, creating fake security services
• Agent code could be modified: a normal agent can become a weed agent, a
flying Dutchman or just a generic malicious agent (additional behaviours could
be added)
• Cause the agent termination
A weed is an agent with no specific purposes; in this case it is not a seller, or a
buyer or any other type of recognized agent and can be used to perform DoS attacks.
A flying Dutchman is an agent that cannot terminate, allowing the doDelete() method
to be ineffective or not called at all.
All of these attacks can be avoided by setting correctly system privileges and rights:
everything JADE related must not run with an administrator account.
Attack scenario 3
In an original, not compromised JADE architecture that is using security services,
incoming and outgoing messages have to pass through some special filters in order
to be coded/decoded. If a host becomes compromised, it is reasonable to think about
implementing additional filters before the actual real filters, causing to disturb all outgoing/incoming communications. Researchers at the Karlsruhe University (19) managed
to implement that attack successfully, by connecting a malicious host with a corrupted
JADE-S version to the target platform. That is possible because the jadeSecurity.jar
file is not signed.
65
5. SECURITY ISSUES
Attack scenario 4
The same researchers from Karlsruhe (19) managed to perform another attack. The
target was the AMS caching system. Each AMS has a cache in which it stores up to
100 agents in order to speed up communications. So a malicious agent could send more
than 100 useless messages to agents situated in at least more than one container in
order to strongly decrease system performances.
5.5
Conclusions
JADE-S can actually provide just a limited security level. If all the precautions are
taken, message payload interception, unwanted horizontal commands and spoofing can
be avoided while cache overloading, message buffer overflow and other generic DoS
based attacks are unavoidable. Also security implementations in a mobile environment
is yet not completely supported: if those issues will not be fixed in the new JADE(S) versions, new solutions involving additional security layers have to be tested and
implemented.
The lack of agents mobility security support is probably the most serious limitation
among the others listed above: allowing an agent to move in an another platform can
be an useful strategy due to the predetermined fault tolerance strategy.
66
6
Supply/demand balancing,
forecasting and pricing
6.1
Supply forecasting
As already pointed out in previous chapters (1, 2, 3), there is the possibility to forecast
the amount of supplies that a single prosumer can offer during a certain negotiation
round using specifically designed web services. Knowing in advance meteorological
parameters (e.g: wind strength, sky visibility etc...) and using these values together
with all the constants related to the quality of the prosumers’ solar panels and/or wind
turbines, we can give to the system an idea of the future supply capacity. An external
balancer agent calculate the sum between every capacity of every prosumer and use
this in order to have a better overview of how much electricity should be produced by
the Gencos, avoiding them to produce more energy that is necessary. It is reasonable
for a buyer trying to have a cheaper contract by contacting a nearer house that features
renewable energy sources.
Unsold energy units represent a challenge of our project, since it is a scenario that
the system should avoid in an autonomous way due to the following reasons:
• Buffering and/or storing electric energy is expensive and difficult.
• Assuming that most Gencos are transforming non renewable energy sources, the
act of producing more energy than the needed quantity will be harmful for the
environment and it can contribute to the depletion of used resources.
67
6. SUPPLY/DEMAND BALANCING, FORECASTING AND PRICING
• Other demand balancing reasons will be better explained in the next sections.
The elaborating phase of all of this data, done by the Balancer Agent, gives a
suggested threshold for each Genco. Afterwards a Genco can choose to follow that
suggestion or not by its own risk, without affecting the consumer in case of unsold
energy: it is strictly a Gencos’ interest avoiding this kind of situations according to this
proposed model.
In the next sections of this chapter, this whole discussion will be linked with other
concepts like demand forecasting and pricing methods in order to have a complete
overview about the delicate issues of balancing the demand with supply to avoid production surpluses.
6.2
Demand forecasting
Demand forecasting is one of the most important point in order to balance the equation
between how much electricity the consumers are going to ask and how much energy
sellers could or should produce to satisfy everyone without having the unsold energy
being wasted. It represents an important term of the balancing equation.
The challenge is to find the best algorithm or model in order to have a forecast demand that should not present an error (calculated between the real encountered value
and the predicted one) that is considered as acceptable.
In the previous chapter we have already presented a simplified way to forecast this
demand, but of course, it is very far from the state of art proposed by other researches:
that algorithm should be seen as a trying test for the communication with the smart
meter.
The state of art of energy forecasting lies in a complete implementation of different
prediction schemes that differ from each other depending if we are trying to forecast the
demand related to a week day or to a so called abnormal usage day ( e.g.: weekend days
or national holidays). In this context weather information too plays a really important
role: retrieving those data from a web service (like prosumers do, as already explained
68
6.2 Demand forecasting
in the previous section) could inform the buyer agents about future temperatures and
humidity or atmospheric pressure values that influence the usage (and therefore the
energetic consumption) of electric devices (e.g: air conditioning systems etc...) .
So, basically, both prosumers and consumers are interfacing themselves through a
web service architecture, but they are extracting different values.
Weather is just one of the many variables used inside a state of the art prediction
algorithm: in order to be completely efficient, the neural net implementation should
be instructed also with a certain number of previous consumption readings. In our architecture all of that is possible thanks to the on board memory chip embedded inside
the middleware device between the smart meter and the outside world.
A simplified scheme of this complex algorithm is pictured in figure 6.1. It is important to notice the presence of a feedback path with another learning block. This last
one is used to force the system to learn and update the parameters of the neural net
when the difference between the forecasted value and the real obtained consumption
value cannot be considered as acceptable.
Figure 6.1: Buyer’s Demand Forecasting algorithm scheme - Shows how actual
and previous consumption data, added to weather forecasting information on a multiple
prediction scheme, can be the input of the designed neural net
The implementation and following test of a very similar solution like the one just
69
6. SUPPLY/DEMAND BALANCING, FORECASTING AND PRICING
described has been already tested by other researches (20), obtaining an error range
[1 − 3]%, that is extremely low. All those data were referring to the New York City
situation between the July 1, 2003 and June 30, 2004.
6.3
Balancing equation
Different publications (9), (10) agree that the balancing equation that shows the relation between the total energy demand of an area and the sellers’ supply capacity
available in the same examined zone should be linked with a pricing system with offers and deals starting from the LMPs (local marginal prices, calculated depending on
the method to fix this balance and the prices for the real market trend and other raw
material availability). Several mathematical models are presented, but mostly all of
them are different way to set to zero the algebraical sum between demand on a side
and supply to the other (9).
Here follows a simplified mathematical approach that considers all the discussions
already explained in this section. For every assumption, we try to give a reasonable
explanation of all our choices.
Given:
Gcx as Genco num.x;
x ∈ Ng number of Gencos
P ry as Prosumer num.y;
y ∈ Mp number of Prosumers
D total demand of the observed Area and Time Interval;
Ti time interval(s)
Ct number of consumers in the observed Area and Time Interval
The ability of producing an amount of energy is influenced mostly by the market
of raw materials for the Gc production line, while the prosumers have to deal with the
local weather.
Producing more than the quantity that they are supposed to supply is risky for the
seller since we assume we are trying to avoid buffering or storing of surplus energy.
70
6.3 Balancing equation
Demand D is calculated with a specific algorithm of demand forecasting, but no matter
which kind of statistics we are going to use in order to solve that, we have to specify
that the demand refers to a pre-determined interval of time. That is because we are
trying to deal with a real time market experience in order to avoid overloading electric
load lines and Generating Company production lines: if a specific Genco or a single
load line has to deal with random peaks of consumption, the seller then has to buy
more energy in foreign markets, causing unexpected price increase. Our model should
then try to distribute the energy avoiding these kind of situations.
Each day is divided by a certain number of time intervals, depending on what kind of
granularity is going to be observed during the simulation (hours, minutes, seconds...).
Obviously D is just the sum of all the demands (at a certain time) needed for all
the consumers in the area. It has to be balanced with the simple equation (with S
indicating supplies):
PN g
i=1 SG ci
+
PM p
j=1 SP rj
=
PCt
k=1 DC tk
The previous equation does not count unavoidable leaks and calculating errors. On
the other side, if the supply and demand forecasting are efficient and precise enough,
we can rely to an easy to implement model for simulations. The meaning is fairly clear:
the sum between the two big production sources (Gencos and prosumers) should be
equal to the total consumer demand; from previous sections, we know how to estimate
the second term and part of the first term of the equation, while we have to suggest a
production threshold for the Gencos’ related summatory terms.
Example:
It is presented with the sole purpose of explaining the previous equations, dimensionless and generic values are used to indicate electric consumption.
Given:
Ng = 2
Mp = 3
71
6. SUPPLY/DEMAND BALANCING, FORECASTING AND PRICING
Ct = 6
The market day is divided in three time intervals
Randomizing consumption values for all the six participants and creating the graph
shown in figure 6.2, is it clearly visible how energy consumption is distributed and the
trend during the three intervals of the observed day.
Figure 6.2: Energy consumption example scenario - This graph shows how the energy peaks are distributed over time after fixing all the needed parameters for this example
scenario
As we can see, all buyers experience peak of consumption during the central interval, with the exception of number 4 and 2. We can assume that during T1, buyers will
stress the energy market and our goal is to distribute the energy involving every seller:
again, we are trying to avoid overloading lines and transmissions devices.
Translating the graph in numbers:
Given:
D(consumer1)(t0 ) = 1 D(consumer1)(t1 ) = 3 D(consumer1)(t2 ) = 2
D(consumer2)(t0 ) = 1 D(consumer2)(t1 ) = 2 D(consumer2)(t2 ) = 3
D(consumer3)(t0 ) = 4 D(consumer3)(t1 ) = 5 D(consumer3)(t2 ) = 3
D(consumer4)(t0 ) = 2 D(consumer4)(t1 ) = 2 D(consumer4)(t2 ) = 2
D(consumer5)(t0 ) = 2 D(consumer5)(t1 ) = 4 D(consumer5)(t2 ) = 2
72
6.4 Pricing methods
D(consumer6)(t0 ) = 1 D(consumer6)(t1 ) = 6 D(consumer6)(t2 ) = 1
Calculating total amounts for each interval:
D(tot)(t0 ) = 11 D(tot)(t1 ) = 22 D(tot)(t2 ) = 13
The next step consists of deciding how it is possible to divide production expectations among Gencos and prosumers.
Assuming that the weather will allow prosumers to produce a 25% of the total foreP
PM p
cast demand: (( Ct
k=1 DC tk /
j=1 SP rj ) = K )
Supposing that K will be constant for the next time intervals:
SG ci = ((1 − K) ∗ D(tot)(t0 ) )/Ng
SG ci = (0.75 ∗ 11)/2 = 4.125
As for the remaining time intervals:
SG ci = ((1 − K) ∗ D(tot)(t1 ) )/Ng
SG ci = (0.75 ∗ 22)/2 = 8.25
SG ci = ((1 − K) ∗ D(tot)(t2 ) )/Ng
SG ci = (0.75 ∗ 13)/2 = 5
Those numbers represent the suggested threshold that the balancer agents calculates
and then suggest to every Genco participant. We will have a better understanding of
the meaning of this value in the next section, while explaining the priced model chosen
for the simulations.
6.4
Pricing methods
The pricing model issue is a fundamental part of the project because it is related not
just to the whole balancing/forecasting calculation of demands and supplies, but it is
73
6. SUPPLY/DEMAND BALANCING, FORECASTING AND PRICING
also the key concept in order to evaluate the market strategy learning algorithm that
every agent has to implement after every negotiation round.
Every seller has its own transmission contracts with a specific TSO (Transmission
service operator), while every contract has different pricing and that effects the total
energy production cost. This kind of mixed approach of pricing single energy units,
depending on how the buyers/sellers are going to act during the negotiation and according to their distance between seller and consumer was already being investigated
by other researches (5)
The pricing strategy should also take into account extreme scenarios that may differ
from the one presented in the previous sections. For example, if there is no sun or wind
whatsoever, the prosumers may not be able to produce energy and so the Gencos are
supposed to handle this kind of situations. On the other side if the weather is experiencing particularly hot summers or cold winters the demand would probably rise and
also the prosumers production capacity would change.
Another assumption is that the cost of transporting energy depends on how far the
energy is transmitted; we can therefore assign to each agent (buyer/seller) a number
that refers to its position in the observed area, so the transmission costs will rise as the
difference between the buyer position and the seller position became wider. Translating
all of the above in equations, the total cost of a unit of energy in a given time interval
can be written as:
Costenergy = (|Buyerposition − Sellerposition | + 1) ∗ ContractT SO + Costproduction
The latter term of the equations is influenced by the weather, market prediction
and balancing forecasting.
This is the basic equation chosen in order to model such a complex reality. Price calculating in this method just represents what in this thesis are called ”starting prices”,
meaning that those prices are the ones that are going to be communicated to consumers
when requested to sellers, but they may not be the definitive final contract price. As
74
6.4 Pricing methods
already stated, if a buyer is willing to retrieve a contract from a prosumer, then an
auction bidding system should follow starting to a price calculated using the same variables and formulations.
This is not true when talking about contracting with a Genco. Keeping in mind
that our objective is to distribute consumption usage among different production lines,
another point to discuss is how much energy a generating company can produce without
having to buy some of it in a foreign (and more expensive) market. Lets assume that
every Genco has their supply threshold already calculated as shown in the previous
section, and once reached it, the Genco has to buy energy abroad (clear signal that we
are stressing the load line of that seller).
A energy unit costs:
Costenergy if below supply threshold
Costenergy ∗ Ec ∗ Df t if above supply threshold
With: Ec : external cost constant (> 1) and Dft: distance from threshold
Surpassing the threshold is also harmful for the environment.
The prosumer equation is very similar to the last one, with the exception of the EC
constant: a solar or wind powered house can sell just the amount of energy it was able
to produce. That quantity is usually lower than a Genco supply.
When the market opens, every buyer contacts every seller (with an order of communication that depends on the market strategy chosen i.e: contacting nearest seller first,
starting with prosumers and then Gencos . ) in order to stipulate an agreement for the
next interval of time. So if the day starts at T0, during that time we begin to negotiate
for the electric contract for the T1 interval; of course we have to specify that the time
of the negotiation has to be considerably shorter than the length of the current interval.
Possible scenarios:
75
6. SUPPLY/DEMAND BALANCING, FORECASTING AND PRICING
• A buyer contacts a Genco asking for a pre-determined (predicted) amount of
energy. The Genco answers with a fixed price that depends on the distance from
the production threshold, just following the equations above; higher prices happen
when we stress the Genco power supply units so we may prefer to contact other
Gencos or begin to send offers (bids) to the prosumers in order to obtain a better
price for energy unit.
• A buyer first contacts all prosumers sending the same bid to all of them; if no
one accepts he could rise the bid and repeat the sending to the same receivers
as before. So the consumer engages a loop that could end if one or more of the
following conditions are satisfied:
– Running out of Time
– Reached an agreement
– Bids are beginning to appear too much expensive
Time is very important considering that especially while negotiating with the Gencos, first costumer get better deal since we are widely below the supply threshold.
If the bids are beginning to rise more than the buyer’s expected budget, he may consider asking other Gencos.
Choosing which strategy is better, is a decision that has to be made by the agent
using specifically designed learning algorithms that can adjust their behaviour according
if the expectations are met (cheapest contracts). Learning algorithm will be treated in
chapter 7.
76
7
Trading strategies
7.1
Problem description
Three different kind of agents are running in the system;
• Buyer,
• Prosumer,
• Genco.
We have a large number of buyers, a limited set of prosumers and very few Gencos.
Buyers are energy consumers; they do not produce energy so they are searching for
electricity by stipulating contracts related to a specific time interval. Each market day
is divided several time intervals and for each, every buyer has to decide in advance who
is going to be its energy supplier for the next time interval.
In the simulation software developed, every day is composed of six time intervals;
Buyers can predict how much energy they need for the following time interval. This
can be obtained by reading previous electric measurement and by applying an energy
consumption forecasting algorithm. It is important to perform this forecasting before
any negotiation, so that the buyer should check which seller could be a suitable choice
according to the energy availability of the possible supplier. Every buyer is in competition with other buyers: each consumer has the goal to stipulate the cheapest contracts
following two different macro strategies;
1) Participating in an auction system handled by prosumers.
77
7. TRADING STRATEGIES
2) Chose the right time to stipulate a contract with the cheapest Genco.
We can see that the strategy 1) is clearly a competition: suppliers have limited resources and so just a restricted set of buyer can win an auction while all the others will
lose.
Strategy number 2) has a competitive side as well, but this concept will be better explained later.
Prosumers produce and consume energy; they produce a smaller quantity of electricity
(compared to Gencos) by using renewable sources of energy: we can think of an house
with solar panels or wind turbines. If the amount of energy is more than what they
consume in a given time interval, they may decide to sell surplus electricity to other
neighbours (buyers). Prosumers have also information about weather conditions in order to have a forecast on the amount of energy to be produced.
Hypothesis
• A buyer can stipulate a contract with a prosumer after winning an auction round,
based with sealed bids; every buyer sends different offers to a suitable prosumer.
• Prosumers starting price is considerably lower than Gencos initial contract prices.
• Weather conditions during an observed interval can prevent a prosumer to generate enough electricity to be sold.
• Prosumers communicate to buyers an initial starting price that is influenced by
contracts with energy transmission operator (henceforth called TSO) and a random cost due to the devices used to produce electricity. (e.g. Maintenance costs)
• The energy produced by prosumer has to be sold quickly and cannot be stored
or buffered.
Every prosumer is in direct competition with other sellers: they have to propose an
appealing starting price and make an intelligent use of refusing bids in order to rise the
price without letting buyers contacts other sellers.
Gencos are big energy generator companies. They have an theoretical infinite amount
of energy supply, but we have to point out other features as well. When dealing with
78
7.1 Problem description
buyers, there is no auction system: every contract can be stipulated much faster compared to the prosumers auction system.
Hypothesis
• Gencos prices are higher than prosumers starting price.
• Gencos prices depends on TSO contracts, raw material price, random costs and
(most important) threshold exceeding costs. (It will be better explained in the
following section)
• A Genco receives a notification from a buyer; then it just calculates the price
according to the variables written in the above points and communicates the
final price back to the buyer.
Gencos’ threshold system
A key point is how much energy a generating company can produce without having
to buy some of it in a foreign (and more expensive) market. So let’s assume that every
Genco has a supply threshold, and once reached, the Genco has to buy energy abroad
(clear signal that we are stressing the load line of that seller).
Cu =
(
Costenergy if below supply threshold
Costenergy × (EC × A) if above supply threshold
given Cu : Single Energy unit cost
EC: external costs constant (> 1)
and A: number of energy units above the threshold (> 0)
Surpassing the threshold is also harmful for the environment.
Asking the Genco for contracts when this threshold is already surpassed will lead to
more expensive contract prices. Those prices rises as we get further from the specified
threshold.
Hypothesis
79
7. TRADING STRATEGIES
• A balancer agent keeps track on how much energy prosumers can supply; subtracting this amount to the total energy demand in an observed area should give
an idea on the quantity of energy that Gencos have to produce. Dividing this
quantity by the number of functioning Gencos gives an advice on how many energy units every Genco is supposed to produce in order not to generate more
energy than it is needed.
• Even in this case, storing or buffering energy is not the best solution.
• The threshold is calculated according to the previous observations and to the
amount of raw resources that the given Genco can handle.
7.2
Agent learning using game theory (Buyer’s side)
An approach in order to find a good learning algorithm for the auction problem between
agents, could be represented by an accurate use of the game theory. This theory is
strongly related to concepts like lose/win situations, Nash equilibrium and repeated
game matches in order to reach a certain state of equilibrium between competitive
(and not cooperative) agents. We can think about a lose situation when an agent is not
satisfied by the amount of money he earned or spent, while the fact that we have to
think about a game that is going to run multiple times (different rounds) can be linked
to our day division in different time intervals in which (every time) the agent tries to
get a better deal after having adjusted his behaviour (change in strategy).
During these kind of games, reaching the Nash equilibrium means that we have found
the right strategy (or mixed strategies combination) that each agent does not want to
change, independently of which strategies other agents had chosen. This concept may
seems a little strange, but keeping in mind the famous prisoners dilemma it all becomes
clear: the Nash equilibrium is not the right strategy in order to let a single agent (or
limited number of agents) win the match letting the others (loser agents) paying the
consequences, but it is a strategy that could, in the worse case, minimize the loss or
even represent victory in the lucky case. It is obvious how those kind of behaviours are
a very appetizing goal for every market participant in our auction system.
As we can see in this short introduction, game theory concepts can be extremely useful
in our project, but the main challenge that arise is to adapt a pre-existing, studied
80
7.2 Agent learning using game theory (Buyer’s side)
and solved game that really looks like our project. Some challenging aspects arise: can
we start from the same equations and then adapt them in order to fit our needs? Are
those games as complicate as our auction based system?
In this chapter we try to give an answer to these challenges and, in case some of those
answers are positive, to describe how it is possible.
Step 1: Find the most similar known game compared to the agent auction
system
We can think about El Farol Bar minority game as the most similar game to our
auction problem.
Just to summarize the game, El Farol Bar is a really existing bar situated somewhere in
New Mexico (US). Every Thursday night it delivers Irish parties with discounted beer
prices, becoming really appetizing for the local potential costumers. So every person
living near that bar, wants to go there in that particular night, but there is something
that we have to specify:
• Given a N population of the nearby area, and a threshold T of people attending
the Irish night event.
• The night at the bar is considered as enjoyable if the number n (≤ N ) of people
attending it during a particular Thursday is below the threshold T (win situation), otherwise it would be better for the single person to remain at home (lose
situation, the pub is too crowded). That is why it is called minority game: we
have two different behaviours and a single agent can win if he had chosen the
path taken by the minority of the population (as seen in table 7.0).
The similarity with our problem are shown in this two-paths way of thinking: if
every agent contacts Gencos, that means that we are overloading the production lines
of these big energy producers causing high prices and environmental issues. Likewise,
if every agent contacts (or tries to do so) the same restricted set of prosumers, only a
few number of participant could get a nice deal, due to the fact that a prosumer can
deliver a little amount of energy, especially compared to a Genco.
In addiction to the bar game, participants are competitive and they are not able to
81
7. TRADING STRATEGIES
Table 7.0: El Farol Bar payoff matrix
Action
Crowded Bar
not Crowded Bar
Attend
L
H
Do not Attend
M
With payoff score
M
H>M >L
M is unconditioned
communicate with each other, so they cannot organize themselves in order to create
some sort of shift, in which they can split and chose (for a fraction of them) not to
go on this Thursday, but go to the next one instead. There are similarities also here:
energy bidding agents cannot communicate in this way, and they have to guess how
other buyers are going to act.
In the bar dilemma, there are different degrees of price/penalty according to the result
of each game round: a participant gets the highest score by going to the bar and discovers that it is not crowded. He get a medium amount of points if it chooses to stay
home, and it gets a penalty (or the minimum score) in case it chooses to attend the
Irish night and finds it crowded. In our problem, this score system can be replaced by
the difference between what a single agent was expecting to spend and what actually
it spends at the end of the time interval.
Step 2: Find a solution of the previous game and see if it is adaptable
to the energy market
Find a solution of this kind of game means (usually) find if there is a strategy (or
a combination of different strategies) that can lead to a common behaviour that can
ensure the reaching of the Nash equilibrium.
For the bar game, plenty of studies have been done in order to find this point of equilibrium, for the sake of simplicity, we first analyse the most intuitive one (the stage
game solution) (21).
The number of pure strategies is finite, the number of asymmetrical mixed strategy
is countable while there is a unique symmetrical mixed strategy solution:
M −L
H−L
=
PT −1
m=0
N −1
m
pm [1 − p]N −1−m
82
7.2 Agent learning using game theory (Buyer’s side)
With p: probability to go at the bar.
Different studies (21) (22) have shown other solutions as well and especially in (22),
those approaches are reviewed according to fairness and efficiency concepts.
In the solution shown here, fairness expectations are met but according to (22) and
some tests that I performed the efficiency ratio is low (around 50%). Other solutions
proposed by cited authors, suggested the use of Fictitious play in order to propose a
more efficient outcome over a sacrificed fairness. An approach that has both fairness
and efficiency requirements satisfied is represented by a Q-Learning strategy in which
a central authority (e.g: a major) introduces entrance fees for bar attending costumers
to be distributed to the players who are staying at home.
Focussing again on the energy problem, we have to give priority to the fairness characteristic: our goal is to define a peer to peer architecture in which, by definition, there
is no space for discrimination between peer agents. In addiction to that, the third
(Q-Learning) solution provide a fair and efficient way to approach the bar dilemma,
but it increases complexity with the concept of a central authority and introducing fees
that have no correspondence to our case.
Here lies another difference: in the bar game if a number m of people is attending the
bar with m > T then m players are losing. In our problem just T − m people are
actually going to retrieve a low payoff. Using the equation shown above, we can see
that for each participant we have a certain probability that can be used to determine
whenever it is advisable to attend the Irish night. Repeating the game we can see how
every agent sooner or later will attend the bar and how most of the times, the pub will
not be that crowded.
When trying to use this solution to our problem, we can map some variables as T for
the ratio between users and amount of energy produced by a Genco, N is the total
number of buyer agents and M,H and L are intervals defined according to the expected
money spent/actual money spent. However this model lacks of a really big set of variables that we have in our system and we have to use ( e.g: time constraints, limited
prosumers supplies etc ...) .
Even if we could find a really complicate mathematical model that, using the previous
statement, can be influenced by those variables, there are plenty of special scenarios
83
7. TRADING STRATEGIES
Table 7.1: initial state 1
Action
P. has supplies
P. has no supplies
Contact Pros.
See tab3 (+2 Ip)
See tab2 (0 Ip)
Contact Genco
See tab4 (+1 Ip)
See tab4 (+1 Ip)
Table 7.2: initial state 2
Action
P. has supplies
P. has no supplies
Contact other P.
See tab3 (+2 Ip)
See tab2 (0 Ip)
Contact Genco
See tab4 (+1 Ip)
See tab4 (+1 Ip)
in which we cannot use this approach: e.g. what if due to bad weather conditions,
prosumers will not be able to produce a decent amount of energy? Then we would not
have a two-paths minority game.
Despite all these difficulties we can try approaching a solution by using several tables
in order to represent the complexity of this problem.
Tables 7.1 and 7.2 are called initial tables while tables 7.3 and 7.4 can be defined
as ending tables.
The difference is that only tables 7.3 and 7.4 show an ending of the negotiation that is
symbolized with letters H,M or L as the payoff entity inside those cells.
This whole four charts system can be also modelled using a graph like a Mealy State
Diagram in which every table is a transition state and the player can travel between
different states by satisfying the condition Jx, and therefore gaining (or losing) a certain amount of intermediate points indicated in statement Kx. The diagram is shown
in figure 7.1.
Table 7.3: final state 3
Action
Pros. accepts
Pros. refuses
Place bid
H
Stay in tab3 (-1 Ip)
Abort negotiation
-
See tab2 (0 Ip)
84
7.2 Agent learning using game theory (Buyer’s side)
Table 7.4: final state 4
Action
Genco above T.
Genco below T.
Accept genco’s offer
M
L
Refuse genco’s offer
See tab2 (-1 Ip)
See tab2 (-1 Ip)
Since we have a finite number of players, multiple game rounds, a state space, a set of
actions that can be taken by the participants, intermediate payoffs and transition rules
according to probabilistic calculations, stochastic games concepts apply.
Every buyer starts by taking a decision in the first table. The balancer agent is
the entity that knows how much energy can be produced by all the prosumers and
by using this information it can calculate a quantity that represents how many buyers
could be served by prosumers; this number can be related to our T threshold in the El
Farol game. According to that threshold we can calculate the probability to chose to
contact prosumers instead of a Genco in this stage of the negotiation (quite similar on
how it was possible to solve the El farol Bar dilemma using the unique mixed strategy
solution). However at this moment we do not have a clear vision of future payoffs, but
we can assign to those initial tables, a certain amount of fictional points that we call
intermediate points (Ip). The goal of this game is to reach a final stage in an ending
table keeping in mind that higher Ips increase the chance to have a satisfactory game
result (H or M final payoff).
So the buyer is redirected in other tables until it reaches a final cell: doing so the
number of Ips can increase in case it is a lucky choice (contacting a prosumer that for
sure has enough supplies) or decrease in the opposite scenario. In the initial tables the
buyer is redirected, in the other tables according to a previously calculated value that
is related to the amount of energy the all prosumers can produce. In the ending tables
the algorithm is different: in order to simulate the importance of the time variable,
lower Ip values means that the buyer has been travelling around different tables for
such a long time and chances to find a suitable seller or even an under threshold Genco
will be scarce. That is because in the ending tables negative values are present. When
the Ip value is very small (<< 0) then the agent is forced to get a contract with a
Genco in order to avoid wasting other time (and consequently other money).
85
7. TRADING STRATEGIES
Figure 7.1: Stochastic Game approach modeled with a Mealy state machine.
86
7.3 Bidding learning algorithm
At the end of each round the system evaluates if the single agent could be considered
as winner or loser; earlier we said that the difference between the expected money to
earn/spent and the actual money earned/spent can point out who are the winners and
who are the losers, but there is more to say about that. Let’s take this example: a
buyer is expecting to have an energy contract that wants him to pay a certain amount
(X) of money per single energy unit; at the end of the game he reaches the H ending
cell in the third table spending Y. We have now three different hypothesis:
• X=Y
• X>Y
• X<Y
In the first case, there are no adjustments to apply to our intermediate points accumulating game since a supposed high payoff (H cell) turned out to be exactly what
the buyer desired to spend.
The second scenario is a bit more complicated and will be discussed later: now we have
just to point out that even in this case the buyer is winning.
In the last hypothesis the buyer was expecting to spend a really unlikely low sum of
money. Even this case will be better explained later.
The whole game has different variables that have to be set in order to find ourselves as
close to the X=Y scenario as possible. As for the other scenarios, we have to remember
that an H ending means that the buyer won an auction handled by a prosumer: if the
auction system does not comprehend a learning algorithm then the odds to find the
buyer in the third hypothesis become high.
That is why, linked to this minority game, there is also a fuzzy logic style control system
in order to handle the bidding part of the negotiation (23).
7.3
Bidding learning algorithm
Each buyer initially has the following variables that are subject to change during the
different rounds:
87
7. TRADING STRATEGIES
• An expected amount of money it desires to spend for an energy unit
• A certain number of refuses to receive before deciding to change seller
• A certain amount of money representing the rising of the stake after each bidding
refusal
While each prosumer:
• An amount of money it desires to earn after a negotiation
• A certain number of refuses to give if the buyer bids are lower than the expected
gain.
So every buyer sends a sealed bid to the prosumer. The prosumer therefore checks for
the highest bid and refuses everything else; now if the highest bid is satisfying he may
decide to accept the contract otherwise he refuses also the highest bid and wait for
every buyer to send to him other sealed bids that will be higher than the previous ones,
in an iterative process. As said before, a certain buyer has a limited amount of times
in which he sends offers, after that the stakes are that high that he may consider other
sellers.
From the buyer’s point of view, we need an algorithm that can adjust previous variables
in order to succeed in a auction bidding system.
The more we spend, the stronger will be the autonomous reaction that will change
those variables in order to contrast the unwanted trend (a graphic representation is
given in figure 7.2).
The graph is centred in the expected money to spend. If after a complete round
the buyer spending is inside the interval shown in the graph, then he does not have to
change anything in his strategy.
If the money spent is surpassing the right interval, then the system reacts by slowly
rising the stake of every single bid.
If the money spent are much higher than the first threshold or after some time (even
with risen stakes) the buyer keeps failing, then the system slowly react by increasing
the spending expectations.
88
7.4 Test results
Figure 7.2: Fuzzy logic styled diagram for controlling bids. Buyer’s side 1) Do nothing
2) Rise stakes/bids 3) Increase expected money to spend
Additional considerations
Before the negotiation
• According to prosumers production threshold, every buyer starting from the first
initial table immediately decides to contact a Genco or a prosumer.
• The buyer follows every needed redirection across the four tables. Every choice
depends on how many intermediate points the single buyer has accumulated.
After the negotiation
• If the buyer has a supposed high number of Ips, but still he ended up in a M or
L cell and his money expectation are not met then the system has to learn from
that, changing the way the user travels across the tables according to the Ips.
After a bidding negotiation:
• The system applies the mathematical functions that we can extract from the
fuzzy styled control system graph shown in figure 7.2.
7.4
Test results
In order to test the efficiency of this whole algorithm, a simple simulation environment
has been realized featuring 100 buyer participants, 10 prosumers and 5 Gencos over
89
7. TRADING STRATEGIES
400 negotiation rounds. This algorithm presents a large number of variables that the
programmer could adjust in order to find the functioning point of work of this method.
It has to be specified that in order to use those numbers, we just simulate agent behaviours but we are using software objects (specifically designed classes) instead. This
will avoid dealing with sending/receiving messages and allow us to ”play” with higher
rounds and participants number.
Those variables are:
• Number of Ip(s) used as threshold in order to redirect the participant from one
final table to the other.
• Difference between starting prices for the two kinds of sellers.
• All the unspecified variables presented in figure 7.2
• Best way to assign values to H,M and L final payoffs.
• Price dynamics from one round to the other.
• Amount of price penalties for exceeding thresholds
• Probability for a prosumer to become more expensive than a Genco
• Taking into account that supply and demand forecasting could not be 100% accurate
The best way to give a precise number to those variables is probably to study a
mathematical formulation in which we can combine all other known values (number of
participants, amount of demands and supplies etc..) in order to retrieve the unknown
data. It is not an easy task and since it needs time and competence, a numerical approach was used instead.
At the end of each round, the program calculates the simple average expecting
budget and the average money spent, assigning to each round number those other two
values (e.g: round 1, Paid Price, Expected Price). Since we have 400 rounds, then 400
entries of this kind are stored in a log file.
90
7.4 Test results
In order to have a clearer idea of the efficiency and precision of this algorithm,
we want to show the difference between applying it and not to use strategies at all:
in this latter scenario, every buyer will contact a prosumer first and after signing a
contract, the participant does not adjust all the previous randomized values (i.e: bid
entity, stakes, number of refusals, expected prices ... ). Also in this case, a file collecting
results about average paid price for each round is needed.
Finding the working point that could demonstrate an important gap between profits
using a strategy and not using it was not an easy task, however, under the following
conditions, we arrived to the results shown in figure 7.3
• Intersection between average starting prices of the sellers should not exceed 33%
• Slow and not exaggerate price swings between each round.
• Significant price penalties for exceeding Gencos’ threshold.
• The higher is the error percentage between the forecast demand values and the
actual requested values (negative error), the better become the improvement between using and not using the learning algorithm. Positive errors may worsen
participant performances.
• Very fast reaction to follow the expected price: the intervals shown in figure 7.2
should have a very small size.
As shown in the graph in figure 7.3, if the above conditions are satisfied, the gap
between the two situations is very visible. In addition to that we can see how expected prices, starting from very low (and impossible to obtain) values tend to reach
an equilibrated amount that represent the cheapest alternative in quite all the examined
negotiation rounds.Obtained prices with the strategy, follows really close that value.
91
7. TRADING STRATEGIES
Figure 7.3: Simulated Learning Strategy results - Three functions shows the difference between no learning prices and the chasing of expected prices with obtained average
deals when using the explained strategy
92
8
Conclusions
8.1
Results’ discussion
Surprisingly, writing the conclusions of this work turned out be the most complex task
among all the others. That is because we need to analytically prove that our solution
could embrace the so called deregulation of the energy market leading to a more welfare oriented and environmental friendly way to consume and distribute energy. The
deregulation process, already presented and explained in chapters 1 and 2, could force
people thinking of a natural adjustment of overall electricity prices towards lower values due to the gradual introduction of many more seller participants in the electricity
exchange and distribution net: the equation ”more participant equals lesser prices” is
part of everyone’s beliefs since the evergreen concept of the benefits for the final users
of transforming the market from first a monopoly, then an oligopoly and eventually a
fully liberalized platform is known and accepted as a postulate also from people outside
the economics studies. Unfortunately this rule does not apply everywhere in the vast
fields of economics and talking about energy liberalization, additional considerations
and studies has to be taken in order to avoid false enthusiasms and data misunderstandings.
Deregulation enthusiasts sometimes forget to mention the costs and the difficulties
in adapting the physical transmission net in order to embrace this change; some older
studies (24) literally name as ”a bad idea”, not just because of the lack of the costs
analysis, but also for the lack of benefits obtainable in choosing the deregulation instead
93
8. CONCLUSIONS
of an improved regulation. Even if those studies are relatively old, newer data from
the American NUS Consulting Group collected and commented by Richard Soultanian
1,
show that the more we try to deregulate a market inside a single country, the more
we can expect price rises. In figure 8.1 we can see how Italy has the most expensive
price per single energy unit even if its market is deregulated at 80% of its capacity,
while South Africa, that presents a strongly regulated production controlled by just
one participant (Eskom), has the lowest prices among all the examined countries.
Figure 8.1: 2010 electricity cost comparison - Shows the price difference between
different countries with different energy market approaches
According to these considerations, our whole software architecture could be an accomplice of an indiscriminate price rise towards those governments who believed in this
liberalization, making all the efforts of every research like ours, not just useless but,
even worse, harmful if implemented in the real environment. Fortunately, those data
are telling just a part of the matter: the deregulation here, is seen as an opening of
the market to other Gencos, ignoring the important contribute that a multitude of prosumers could bring to the same market; in other words, examining the actual situation
does not give a full insight of the benefits of the regulation because (especially in Italy,
and later in this section we will explain why), it refers to a monopoly to oligopoly shift
1
Power engineering international, volume-18, issue-8, issued 1st sept 2010
94
8.1 Results’ discussion
of the market instead a full liberalization. In addition to that, the latter source forget
to mention who, among the examined countries, is the most dependant from the foreign
electric energy market and who most relies on importing raw materials used to produce
energy: needless to say, Italy is one of the first importers of raw materials and already
produced electric energy. Raw materials come basically from North African and Eastern European countries, while the flow of energy brought from France and Switzerland
reaches high percentages of the Italian needs 2 . Still referring to the Italian situation,
in that country, the deregulation is still in its early stage: even if 80% of the population
could chose between a plurality of Gencos, they all rely (mostly) on the same set of
imported raw material (not renewable sources) and the transmission service that on
the paper is unbounded from the production management, is still handled by the same
restricted set of big electric energy producers and most of the other Gencos simply rent
the same transmission lines. All of this is a clear signal of a transition that is far from
being declared completed.
As already pointed out, deregulation per se is not the definitive solution to all the
energy distribution problems addressed by the European energy commission (see chapter 2), but instead is a necessary element that we have to combine with a responsible
usage of our not renewable energy sources and, obviously, a rapid increment of all that
is involved by the renewable possibilities as well as the adoption of a right government
policy to facilitate all of that. By doing so, we can therefore see improvement environmental wise as well as electric prices stabilization towards lower levels (25), otherwise
a raw deregulation applied between mostly fuel-fossil based energy Gencos will lead to
unsatisfactory results (26).
Talking about dependency from outside sources, other studies demonstrate how a
not optimized design of inter-national electric energy transmission lines between (energy wise) heterogeneous countries could lead to some unexpected price rises after the
first stages of the liberalization (27).
2
http://energialibera.studenti.it/2010/11/22/litalia-importa-quasi-totalmente-lenergia-di-cui-habisogno/
95
8. CONCLUSIONS
The connection between all of these considerations with our architecture lies in the
fact that our architecture indeed, privileges the consumption of first renewable sources
(from the prosumers) and then tries to assign the remaining consumers to Gencos that
are adopting a model in which contracting energy above the production threshold will
result in price penalties forcing the buyer agent to find its contract in another Genco,
causing all the Genco to approach to their threshold (calculated according the forecast
supplies, but it could be also a function of the pre existing condition of privileging
internal resources instead of the foreign market imported) without exceeding it. It still
remains to numerical demonstrate the benefits of this kind of approaches, in order we
demonstrate that the implementation of our architecture will lead to:
• A more responsible energy consumption.
• Cleaner environmental emissions.
• Likely lower energy contracts’ prices.
As for the first statement, we can think about what Forbes3 says about countries’
energy production efficiency: ”a country with a very high GDP and relatively little energy consumed is likely to be a very energy-efficient economy. Conversely a country with
huge energy consumption and relatively little GDP is unlikely to be efficient”. Therefore, a responsible consumption is a way to use energy oriented to consume less (even
if GDP could allow higher expenses) and independent resources’ usage: at higher position according to this method to compile a chart between the most efficient energy wise
countries we can find Japan and Denmark. In both of those so distant nations, things
in common are the efforts (mainly high taxes) in order to prevent energy or raw material imports together with low carbon dioxide emissions. Strictly follows Switzerland
(the ”greenest” country according to Environmental Performance Index produced by
Columbia and Yale University) with the implementation of energy autonomous entities
called ”energiestadts”. So independence, greener energy sources and low consumption
are the keywords here: that can be furthermore proven by looking at figure 8.2.
Under a not responsible demand (D1) corresponds an extremely high price (P1)
according to simulated electricity market rules. In the case of a distributed consumption, and therefore more responsible (D2), a much lower price will result for the buyer
3
http://www.forbes.com/2008/07/03/energy-efficiency-japan-biz-energy cx jz 0707efficiency countries.html
96
8.1 Results’ discussion
Figure 8.2: Explanation of demand response effects on a quantity (Q) - price
(P) graph - D2 indicates a more responsible usage since it is distributed above time and
remains below the seller supply threshold that could be situated somewhere between P1
and P2. Found on wikipedia.com
97
8. CONCLUSIONS
(P2). As a more practical example, it is estimated (28) that a 5% lowering of demand
would result in a 50% price reduction during the peak hours of the California electricity
crisis in 2000/2001. The market also becomes more resilient to intentional withdrawal
of offers from the supply side.
All of the above perfectly complies to the pricing model adopted by our design:
buyer agents will concentrate their demands to a restricted set of Genco until they
reach that threshold, after that the set of ”cheapest” big energy suppliers changes becoming to indicate as ”best sellers” the other Gencos that before the reaching of the
threshold by the previous Gencos they were slightly more expensive, but now they are
most convenient due to the small numbers of supplied demand accomplished until that
moment.
As for all that concerns the environmental emissions (a concept also present in the
above responsible consumption related consideration shown just above), some agencies
that promote, install and manage wind turbines and solar panels (two methods used
and discussed in our platform to obtain energy from renewable) provide simulations on
emissions and earnings in installing those kind of devices on a domestic environment.
Basically we can use those data in order to give a numerical answer about prosumers’
gains and lowering of the average price of energy contracts inside the deregulated market
handled by our designed platform. Enel-si 4 , a subsidiary of the Enel Italian Energy
company provides very complete on-line simulation that allows us to choose sizing, type
and location among the Italian territories, outputting results about savings, earnings
etc... that will be extremely useful in our objective benefits estimation.
Areas chosen are those territories in which solar irradiation power and wind strength
present higher value in an yearly average measurement: it is known, for instance, that
the North-East part of Italy has the strongest wind among all the others (bora), but
it is a seasonal wind, while we need something more constant over the four seasons.
From the table 8.1, it is clearly visible that a wind powered house has higher expenses
and higher electricity capacity production but producing the same amount of energy
with not renewable sources could lead to emissions shown in that table. This result
was easily predictable, while it is much more interesting taking a look in how could
4
http://www.enelgreenpower.com/enelsi/it-IT/offerta/simulazioni/index.aspx
98
Sant’Antioco,
Carbonia Iglesias (CI)
Favignana,
Trapani (TP)
Solar Panel
City
Wind Turbine
Device
Installation: 60000
maintenance: 600/year
Installation: 8160
maintenance: 82/year
13.8 m2 Polycrystals
S-oriented 1.7 Kwp
Costs (e)
18 m High
Max Pow gen. 20KW
Tech. Specs
99
1260/year
22058/year
Earnings (e)
Emissions
Not emitted CO2
1485 Kg/year
Not emitted CO2
40439 Kg/year
Table 8.1: Wind and Solar power in the Italian biggest islands (Sardegna, Sicilia) domestic environment
8.1 Results’ discussion
8. CONCLUSIONS
a prosumer earn from all of that, and consequently offer lower starting price to an
interested buyer: in figures 8.3 and 8.4, we can find how many years are needed for a
prosumer to amortize all the installation costs in the two scenarios.
Figure 8.3: Amortizing time and predicted earnings: solar device - The prosumer
starts earning from the 3rd year
All those data are calculated with the actual forced scenario in which the prosumer
can only give back to the net the surplus power: obviously the amount of money earned
is a fixed price per kWh decided by the Genco (Enel in this case). The prosumer inserted in our fully liberalized system, could think about starting their auction from
that price since it is lower than the average price asked by a Genco to buyers:
Price for a single energy unit supplied by a prosumer (starting auction price):
GencoP rice + CT SO + (A/t) = StartingP rice
with CT SO costs due to TSOs’ services.
GencoPrice is the price given by the Genco to the re-emission of a single energy unit
inside the Genco’s net.
and A is a constant related to the amortizing costs with t the time elapsed since the
first run of the device: the term A/t is supposed to become null after the amortizing
100
8.1 Results’ discussion
Figure 8.4: Amortizing time and predicted earnings: wind device - The prosumer
starts earning from the 6th year
time shown in the previous figures (8.3 and 8.4).
As for comparing to a Genco, a big energy producer does not have the first term,
but amortization costs and CT SO are supposed to be higher due to the higher values of
production costs and infrastructures rented from a TSO. Also, if we apply the winning
policy adopted by greener countries mentioned above in this section (e.g: Switzerland
and Denmark), Gencos using not renewable energy sources should face gas emissions
taxes. The only taxes that a prosumer should pay are related for auto-generating
energy (still present in Italy, but doomed to disappear due to new European policies as
explained in chapter 1 and 2, and be substituted with the introduction of government
incentives). As a result of all of that, a fully deregulated market with an appropriate
tax and incentives policies should help the introduction in the field by small energy
producers able to fulfil the needs of their immediate neighbourhood: this will allow
buyers to chose between different sellers with different prices causing average contract
costs to stabilize to a slightly swinging value but still towards a convergence to a value
that is supposed to take place in middle of prosumers’ starting price and Gencos’ fixed
price below the production threshold. This convergence (already shown in chapter 7,
101
8. CONCLUSIONS
figure 7.3) is already been studied by the economists who investigated the economic
implication of the energy deregulation (29). As we can see, almost all the assumptions
and results outputted by our designed platform are reflected to real actual data and
future’s trends: also the dependency of prosumers’ supplies to the weather conditions
are visible in Enel-si final graph simulations and hereby shown in figure 8.5.
Figure 8.5: Graph showing the dependency from the weather to the supply
capacity of a solar panel - Not surprisingly spring and summer periods are the most
prolific for a solar panel
8.2
Improvements and future implementations
The work related to this platform is not 100% complete yet. First of all, this thesis
lacks of a complete discussion on how distribute and replicate balancer agents in order
to ensure reliability on one hand and on the other, allowing the net to be compliant to
the requirements needed in order to be called p2p. Secondly, no tests relating a distributed environment has been made and even if that does not represent an issue due
to the ease of distributing instances in JADE, the coded part of the project suffers this
limitation, creating problem in sending and receiving messages inside a local platform.
In addition, as already explained in chapter 7, our learning negotiation algorithm has
102
8.2 Improvements and future implementations
still a wide improvement margin: furthermore tests should be taken in order to determine the correct point of functioning according to platforms values like number of
agent participants, prices, peer distances etc... , as well as the implementation of some
additional strategy used by sellers.
Other serious open issues lie in the fact that as already introduced in chapter 5,
JADE-S does not provide a 100% secure solution due to its vulnerability to DoS attacks
and compromised libraries. As for the last matter, according to the open source released
JADE code, we can calculate a checksum of the JadeSecurity.jar files and force every
agent to check the hash function output of the *.jar in use files, with the checksum
related to the official released ones in order to find out in advance if the examined host
is compromised (30).
As for future improvements and implementations, since the subject is vast and most
of its aspects are still in their early stages, an open mind approach should be taken in
order to let our system be ready for the most futuristic scenarios. Studies hypothesized
the possibility to exchange energy contract with a very high time granularity should
open the market grid also to electric vehicles (31),(32).
As for the p2p net management, we can think about a feedback system between
peers or designing intermediate peers with purpose to sell prosumers’ unsold energy
units. In addition, in the future our platform could handle more than just electric
energy: we can therefore think about a system in which every peer could bargain every
form of energy as well as additional (ancillary) services like autonomous searches for the
cheapest TSO contracts, backup guarantees services etc... relying on already proposed
studies about software agents negotiating multiples goods with a simultaneous auction
based system (33).
103
8. CONCLUSIONS
104
[11] Brazier Padget Riat Warniner and Natarajan.
An
Agent-Based Infrastructure for Energy Profile
Capture and Management. Agent technology for energy systems ATES 2010, 2010. 8
[12] Chen-Ching Liu and Nanpeng yu. Multi-agent systems
and electricity markets: State-of-the-art and the
future. Power and Energy Society General Meeting - Conversion and Delivery of Electrical Energy in the 21st Century, 2008 IEEE, July 2004. 8
References
[13] Fichter Veit Czernohous and Weinhardt. Management
decision support using long-term market simulation. Power and Energy Society General Meeting - Conversion and Delivery of Electrical Energy in the 21st Century, 2008 IEEE, 2003. 8
[1] Gomez T. Cossent R. and Frias P. Towards with large
penetration of distributed regulation: is the current regulation of electricity distribution ready?
Regulatory recommendations under a European
perspective. Energy Policy 37, March 2009. 6
[14] Nguyen G. Dang T.T Hluchy L. Laclavik M. Balogh Z. and
Budinska I.
Agent platform evaluation and comparison.
PELLUCID project related research paper, Jun, 2002. 12,
13
[2] Fekete K. Nikolovski S. Puzak D. Slipac G. and Keko
H. Agent-based modelling application possibilities
for Croatian electricity market simulation. Electricity Market, 2008. EEM 2008. 5th International Conference on European, 25 july 2008. 6
[15] David Bonnefoy Steven Willmott Nicolas Lhuillier Ion
Constantinescu Jerome Picault and Yann Camenen.
Agentcities Platform Interoperability Test Suite.
Centre de Recherche de Motorola Paris and Ecole Polytechniques Federale de Lausanne, Version 1.1. 18
[3] L. Tesfatsion D. Aliprantis and H. Zhao. An AgentBased Test Bed for the Integrated Study of Retail
and Wholesale Power System Operations. Agent
technology for energy systems ATES 2010, 2010. 6
[16] James Keirstead and Koen H. van Dam.
A comparison of two ontologies for agent-based
modeling of energy systems.
Proceedings of the First International Workshop on Agent
Technologies for Energy Systems, 2010. 19
[4] Astrid Zeman Ying Guo and Rongxin Li. Utility Simulation Tool For Automated Energy Demand Side
Management.
Agent technology for energy systems
ATES 2010, -. 6, 8
[17] Jenermann Reiner and Walter Lang.
Mobile Java Code for Embedded Transport Monitoring Systems.
Proceedings embedded world conference, 2006. 44
[5] Marta Marmiroli and Hiroshi Suzuki.
Web-based Framework for Electricity Market.
Power System and Transmission Eng. Center, Mitsubishi
Electric Corporation. 7, 8, 74
[18] A. Schuster X. Vila and A. Riera.
Security for a Multi-Agent System based on
JADE.
Department of Electronics and Computer Science, University of Santiago de Compostela, 2006. 62
[6] N. Lai L.L. Motshegwa T. Suasinghe H. Rajkumar and
Blach R. Feasibility study with agents on energy
trading. Advances in Power System Control, Operation
and Management, 2000. APSCOM-00. 2000 International
Conference, 2000. 7
[19] Regine Endsuleit and Jacques Calmet.
A Security Analysis on JADE(-S) V. 3.2.
Universität Karlsruhe (TH), -. 63, 65, 66
[20] Shu Fan and Luonan Chen.
Short-Term Load Forecasting Based on an Adaptive Hybrid Method.
Osaka Sangyo University, June 20, 2005. 70
[7] Wollenberg Harp, Brignone and Samad. SEPIA: A simulator for electric power industry Agents. IEEE
Control Systems Magazine, aug. 2000. 7
[21] Duncan Whitehead.
The El Farol Bar Problem Revisited: Reinforcement Learning in a Potential Game.
-, sept 17, 2008. 82, 83
[8] Isabel Praa Carlos Ramos Zita Vale and Manuel
Cordeiro. MASCEM: a multiagent system that
simulates competitive electricity markets. Intelligent Systems, IEEE volume 18 issue n. 6, nov./dec. 2003.
7
[22] Duncan Whitehead.
Fair and Efficient solutions to the Santa Fe Bar
Problem.
Proceedings of the Grace Hopper: Celebration of Women
in Computing, 2002. 83
[9] Hiroshi Takamori and Ken Nagasaka.
Toward Designing Value Supportive Infrastructure for Electricity Trading.
Eiroku Go, CSD corporation, 2007. 8, 70
[23] Yu Shun-kun Yuan Jia-hai and Hu Zhao-guangt.
A Multi-agent Trading Platform for Electricity
Contract Market.
Power Engineering Conference, 2005. IPEC 2005. The 7th
International, 2005. 87
[10] Warmer Kamphuis Hermans J van der Bosch Frunt and Jokic. Balancing Services in Smart Electricity Grids
Enabled by Market-Driven Software Agents. Agent
technology for energy systems ATES 2010, 2010. 8, 70
105
REFERENCES
[24] Jack Casazza.
Electric Power Deregulation - A Bad Idea?
Jack Casazza is an IEEE Life Fellow and a member of
IEEE-USA’s Energy Policy Committe, may 2005. 93
[29] Georg Zachmann.
Convergence of Electricity Wholesale Prices in
Europe? A Kalman Filter Approach.
DIW Berlin, German Institute for Economic Research,
sept 2005. 102
[25] Pallab Mozumder and Achla Marathe.
Gains from an integrated market for tradable renewable energy credits.
Department of Economics, University of New Mexico and
Los Alamos National Laboratory, Computer and Computational Science, 2004. 95
[30] Charles P. Wright Gopalan Sivathanu and Erez Zadok.
Enhancing File System Integrity Through Checksums.
Stony Brook University - Technical Report FSL-04-04,
2004. 103
[31] Keith Decker Keith Trnka Sachin Kamboj Nathaniel
Pearre Willett Kempton and Colin Kern.
Exploring the formation of Electric Vehicle Coalitions for Vehicle-To-Grid Power Regulation.
College of Earth, Ocean and Environment, Dept. of Computer and Information Sciences, University of Delaware,
2010. 103
[26] Alexander Ritschel and Greg P. Smestad.
Energy subsidies in Californias electricity market
deregulation.
International Environmental Policy Program, Monterey
Institute of International Studies, Monterey, CA 93940,
USA, 2004. 95
[27] Derek Bunn and Georg Zachmann.
Inefficient arbitrage in inter-regional electricity
transmission.
Journal of regulatory economics Volume 37, Number 3,
243-265, july 2009. 95
[32] Nelis Boucké Tom Holvoet Stijn Vandael and Geert Deconinck.
Decentralized demand side management of plugin hybrid vehicles in a Smart Grid.
DistriNet and ELECTA, Katholieke Universiteit Leuven,
Belgium, 2010. 103
[28] IEA: International Energy Agency.
Demand Responsein Liberalised Electricity Markets.
The power to choose, july 2003. 98
[33] Amy Greenwald and Justin Boyan.
Bidding Algorithms for Simultaneous Auctions:
A Case Study.
Brown University and ITA software, jan 10, 2002. 103
106