Title (Please note: This is a field. Update via `Update Ericsson

Ericsson Internal
1 (13)
Prepared (Subject resp)
Approved (Document resp)
No.
Checked
Date
Rev
2013-02-28
PA1
Reference
BUTLER Renewable Energies
maintenance guide
Ericsson Internal
2 (13)
Prepared (Subject resp)
No.
Approved (Document resp)
Checked
Date
Rev
2013-02-28
PA1
Reference
Contents
1
2
3
4
5
5.1
5.2
6
7
8
9
10
11
12
13
14
Scope ................................................................................................ 3
Pursued goals ................................................................................... 3
Overall view ....................................................................................... 3
The project and the server ................................................................. 4
Simulators ......................................................................................... 5
Smart meters simulator ..................................................................... 5
Weather conditions simulator ............................................................ 5
DDX integration ................................................................................. 6
CEP engine ....................................................................................... 7
Relational database ........................................................................... 8
Update manager................................................................................ 8
Forecaster module............................................................................. 8
SOAP web service............................................................................. 9
Chart GUI ........................................................................................ 12
Administration GUI .......................................................................... 12
Workflow ......................................................................................... 12
Ericsson Internal
3 (13)
Prepared (Subject resp)
No.
Approved (Document resp)
1
Checked
Date
Rev
2013-02-28
PA1
Reference
Scope
This guide provides basic knowledge to be able to understand, install and maintain the BUTLER
Renewable Energies proof of concept integrated with the DDX proof of concept.
2
Pursued goals
What this proof of concept tries to provide are two different forecasts: power consumption and power
generation. Several data sources are used as input:





Current weather conditions
Weather forecast
Power consumption records
Number of available wind mills
Solar panel available surface
In order to forecast power consumption predictions, linear regression technique is used. On the other hand,
to calculate power generation forecasts, simple multiplications are enough.
3
Overall view
The following picture shows all the main parts of the Renewable Energies proof of concept (the integration
with DDX will be given from granted from now on) and the data flow among them:
Ericsson Internal
4 (13)
Prepared (Subject resp)
No.
Approved (Document resp)
Checked
Date
Rev
2013-02-28
PA1
Reference
The list below provides a brief description regarding what is each element task about:









Simulators: Java threads that periodically send data items to DDX. There are two kinds: power
consumption and weather conditions.
DDX: Device Data eXchange. Technology and Innovation (TB) unit can provide documentation and
support.
CEP engine: Java thread based on the Esper1 Complex Event Processing engine. It aggregates
different data flows, perform simple calculations and stores their results in the database.
Relational database:PostgreSQL2 is the solution chosen in order to store all the data generated by
the proof of concept.
Update manager: Java thread that periodically pulls JSON weather data reports from
Wunderground®3 using their API and update the database according to them.
Forecaster module: Java thread that periodically performs the linear regression calculations.
Based on the Java library OpenForecast4.
SOAP web service: Interface to query programmatically the forecasts results.
Chart GUI: Web interface to watch graphically the forecasts results.
Administration GUI: Web interface to orderly start and stop every Java thread supported by
Comet5 technology. Stop functionalityis not guaranteed.
4
The project and the server
The JavaEE project is called RenewableEnergiesForecasting and it has been developed with Eclipse and
Java6. Since the server that is going to host this PoC (195.53.58.184) is the same that hosts the DDX PoC
and it already had a Tomcat 7.0.26 installed and running, the developments have been optimized for that
version. The same reason applies for the Java6 usage. The source code is located at
/home/ericsson/Environment/Source/RenewableEnergiesForecasting. It is needed to have a
Tomcat 7 instance attached to Eclipse in order to compile the project.
There are two changes that are needed to be performed in a new installation of the PoC:


At Tomcat’s server.xml: The tag <Connector port="8080" protocol="HTTP/1.1"
connectionTimeout="20000" redirectPort="8443"/> must be replaced by <Connector
connectionTimeout="20000" port="8080"
protocol="org.apache.coyote.http11.Http11NioProtocol" redirectPort="8443"/>. This
enables the Comet technology.
New environment variable: A persistent environment variable called CATALINA_HOME pointing to
the Tomcat non-binary files folder must be created. Edit the file /etc/environment and add a new
line with the text CATALINA_HOME=”/home/ericsson/Environment/tomcat”. It is necessary to
make the log work.
1http://esper.codehaus.org/
2
http://www.postgresql.org/
3http://www.wunderground.com
4
5
http://www.stevengould.org/software/openforecast/index.shtml
http://en.wikipedia.org/wiki/Comet_%28programming%29
Ericsson Internal
5 (13)
Prepared (Subject resp)
Approved (Document resp)
No.
Checked
Date
Rev
2013-02-28
PA1
Reference
The following table provides the different duos user/passwords needed:
Element
ssh server access
Tomcat manager
PostgreSQL (port 5432)
User
ericsson
gui
eric
Password
ddxbutler
pinosventanalpizarra
pinosventanalpizarra
Besides, the Owncloud service also located in the server is accessible by using ddx / ericsson
5
Simulators
The class com.ericsson.energies.simulator.SimulatorManager is the responsible for controlling both
kinds of simulators instances (com.ericsson.energies.simulator.ConsumptionSimulators and
com.ericsson.energies.simulator.WeatherSimulators). This class is a thread that is managed by the
class com.ericsson.energies.Launcher.
First of all it is needed to know that the BUTLER consortium decided to work simulating ten cities: Bremen,
Brussels, Grenoble, Lecce, Luxembourg, Madrid, Milan, Santander, Oulu and Zurich.
5.1
Smart meters simulator
These
elements
are
represented
by
the
class
com.ericsson.energies.simulator.ConsumptionSimulator.
There
is
a
properties
file,
consumption.properties, that configures their behaviour, located at WebContent/WEB-INF/conf. The
following parameters are the only ones that can be modified:






streamServerIp: The CEP engine IP address.
streamServerPort: The CEP engine input socket.
firstDeviceIdN: Since it is necessary to register each city simulators group sequentially in the DDX
PoC, the first device id must be here. See section 6.
firstSensorIdN: Since it is necessary to register each city simulators group sequentially in the DDX
PoC, the first sensor id must be here. See section 6.
numOfMetersN: The number of smart meters per city.
intervalMeterTime: The time between each data sending. It is set to 30 minutes because of the
Wunderground® API constraints. See section 5.2.
The remaining ones must not be modified.
5.2
Weather conditions simulator
These elements are represented by the class com.ericsson.energies.simulator.WeatherSimulator.
There is a properties file, weather.properties, that configures their behaviour, located at
WebContent/WEB-INF/conf. The following parameters are the only ones that can be modified:
Ericsson Internal
6 (13)
Prepared (Subject resp)
Approved (Document resp)












No.
Checked
Date
Rev
2013-02-28
PA1
Reference
streamServerIp: The CEP engine IP address.
streamServerPort: The CEP engine input socket.
downloadRate: The time between each weather conditions JSON file download from
Wunderground®. It is set to 30 minutes because of the Wunderground® API constraints.
updateRate: The time between each database update process with weather forecast data from the
JSON files. It is set 1440 minutes (1 day) because that’s the period Wunderground® API provides a
new day forecast data within their files. See section 9.
wundergroundKey: Since it is necessary to register in Wunderground® in order to access their API,
a license key is provided.
numOfLocations: Number of cities. It is set to 10.
locationN: Each location in the format Wunderground® API understands them (country,city)
deviceN:The DeviceId of the weather simulator from the N location
anemometerN: The SensorId of the anemometer sensor belonging to the previous device from the
location N
thermometerN: The SensorId of the thermometer sensor belonging to the device from the location N
hygrometerN: The SensorId of the hygrometer sensor belonging to the device from the location N
conditionsN: The SensorId of the sky conditions sensor belonging to the device from the location N
The Wunderground® API restrictions for developers allow a maximum of 500 downloads per day and 10
downloads per minute. Since the PoC needs data from 10 cities, that gives a minimum period of half an
hour in order to do not violate those constraints.
500 𝑑𝑜𝑤𝑛𝑙𝑜𝑎𝑑𝑠
= 2,08 𝑑𝑜𝑤𝑛𝑙𝑜𝑎𝑑𝑠 𝑝𝑒𝑟 ℎ𝑜𝑢𝑟
24 ℎ𝑜𝑢𝑟𝑠 × 10 𝑐𝑖𝑡𝑖𝑒𝑠
Each Wunderground® key is attached to an email address and warnings or information messages are sent
there, so it is recommendable to get a new one each time the person in charge of this PoC is replaced.
6
DDX integration
Both smart meters simulators and weather conditions simulators must be registered in the DDX PoC. Each
smart meter simulator must be a single device with a unique sensor attached. It is necessary that each city
simulators group are registered sequentially so their IDs are consecutive. This can be done easily using the
DDX XML batch load option. The sample files can be found at the Owncloud service.
Weather conditions simulators don’t have that restriction. Their relationship between devices and sensors is
one device per four sensors: anemometer (wind speed), thermometer (temperature), hygrometer (relative
humidity) and conditions (sky conditions: sunny, cloudy, rainy, etc). Every data is represented by a double
type number. Regarding sky conditions, it works as follows:
1
0.75
0.5
clear, sunny
partlysunny, mostlysunny, partlycloudy
chancerain, chancestorms, chanceflurries, chancesleet, chancesnow, cloudy,
fog, hazy, mostlycloudy
Ericsson Internal
7 (13)
Prepared (Subject resp)
Approved (Document resp)
0.25
0
No.
Checked
Date
Rev
2013-02-28
PA1
Reference
storms, flurries, sleet, snow
night
These data are useful in order to calculate the solar power generation.
Once smart meters and weather devices are registered in the Data Owner Portal, it is necessary to create
data packages and perform a subscription from the Dynamic Data Mall. Then the credentialsFile must be
saved toWebContent/WEB-INF/conf.
7
CEP engine
The class com.ericsson.energies.esper.Esper is a thread that is managed by the class
com.ericsson.energies.Launcher. It runs the Complex Event Processing engine that aggregates both
input data (power consumption and weather conditions) it is able to receive through a socket, performs
simple calculations and inserts the results in the database. Those calculations are:



Adds all power consumption data coming from the same city.
Calculates wind power generation in each city taking into account wind speed, temperature,
humidity and number of mills deployed.
Calculates solar power generation in each city taking into account sky conditions and solar panels
surface deployed.
CEP engine only works with real-time data, it does not perform any prediction.
This class is a thread that is managed by the class Launcher.
There are several files that Esper needs. The first one is the properties file called esper.properties
located at WebContent/WEB-INF/conf. Some of its fields can be modified:





esperPort: The input socket.
ddxMQTThost: Whether the DDX PoC is installed in the same server, it is set to 195.53.58.184.
ddxMQTTport: The Mosquitto MQTT broker socket, 1883 by default.
numOfLocations: number of cities
locationN: name and two coordinates that describes an imaginary square (top left and bottom right
corners) that approximately covers the city area. Smart meters and weather sensors belonging to
this city must be located inside this square.
The remaining fields must not be modified. Three other files are important in order to recognize the smart
meters (CRM.csv) and to perform power generation calculations (WindMill_meters.csv and
SolarSurface.csv), all of them located at WebContent/WEB-INF/CSV_Files.
CRM.csv file must contain every smart meter SensorId and its corresponding city.
Ericsson Internal
8 (13)
Prepared (Subject resp)
Approved (Document resp)
No.
Checked
Date
Rev
2013-02-28
PA1
Reference
Finally, data contained by WindMill_meters.csv and SolarSurface.csv have been totally made up.
They keep the number of mills per city and the deployed solar panels surface.
8
Relational database
The database this PoC uses is called poc_er_pc. Its credentials are DBA / sql. The schema DBA
contains several tables.






ConsPredict and ConsPredictLinear: they store calculations performed by the forecaster module
regarding power consumption forecast. See section 10.
ConsReal: it stores power consumption calculations coming from the CEP engine. See section 7.
ForecastCoeff and ForecastCoeffLinear: the forecaster module uses them to store temporary
coefficients it needs to perform its calculations.
GenPredict: it stores calculations performed by the forecaster module related power generation
forecast. See section 10.
GenReal: it stores power generation calculations coming from the CEP engine.See section 7.
WeatherFor: it stores weather forecast data from the JSON files provided by Wunderground®. See
section 9.
9
Update manager
The class com.ericsson.energies.database.UpdateManager is a thread that is managed by the class
com.ericsson.energies.Launcher. It inserts into the database the weather forecast data coming from
Wunderground® via JSON files. It not only stores those data into the table WeatherFor, but also creates
new rows within ConsReal with the column Consumption set to 0 because the Forecaster module needs
them.
This class shares the properties file weather.properties with the WeatherSimulator class. Despite all the
specifications about the Wunderground® API are in section 5.2, this class manages the one that actually
downloads the JSON files (com.ericsson.energies.weather.JSONDownloader). The other class
managed by the Update manager, com.ericsson.energies.weather.DBUpdater, is the real one in
charge of the database update.
10
Forecaster module
The class com.ericsson.energies.forecast.MultilinearRegression is a thread that is managed by
the class com.ericsson.energies.Launcher. It forecasts future power consumption and generation:

It takes from the database training data from the table ConsReal and, using the linear regression
technique; it forecasts future data and stores them in the table ConsPredictLinear. Independent
values are temperature and relative humidity.
Ericsson Internal
9 (13)
Prepared (Subject resp)
No.
Approved (Document resp)


Checked
Date
Rev
2013-02-28
PA1
Reference
Using a variation of the previous technique, it performs the same calculation and stores it in the
table ConsPredict.
At the same time, it performs simple multiplications to calculate future solar and wind power
generation.
It checks a properties file called forecast.properties that is located at WebContent/WEB-INF/conf. It
contains the following fields:





sleepTime: The time between each forecasting batch.
minusHours: The number of hours backwards from the current moment to take as training data
plusHours: The number of hours forward from the current moment to forecast
numOfLocations: number of cities
locationN: name of each city
Remaining fields must not be modified.
11
SOAP web service
This is the first interface the PoC had. Since it is a different project, its source code is
at/home/ericsson/Environment/Source/SmartHome. It contains web services related to solar power and
wind power generation: SolarEnergy and WindEnergy classes.
Both classes contain the same number of functions: 3. The following table depicts a brief description of
those six functions.
Operation
Get Time of
Maximum Wind
Energy Price
Purpose
It returns the time for maximal €price in the interval of X hours from
the current time for the energy
produced by the wind farms in a
specific location
Get Time of
Minimum Wind
Energy Price
It returns the time for minimal €-price
in the interval of X hours from the
current time for the energy produced
by the wind farms in a specific
location
Get Wind
Forecast
Confidence
It returns the forecasting %confidence of energy price produced
by the wind farms in a specific
location based in the data stored into
the database during the last X hours.
If the result is 0, try again with a
Arguments
Input arguments:
 Length of the forecasting
time window in hours (max
48h): integer
 Location: string
Return type: Date/time
Input arguments:
 Length of the forecasting
time window in hours (max
48h): integer
 Location: string
Return type: Date/time
Input arguments:
 Length of the forecasting
time window in hours: integer
 Location: string
Return type: double
Ericsson Internal
10 (13)
Prepared (Subject resp)
Approved (Document resp)
Get Time Of
Maximum Solar
Energy Price
No.
Checked
Date
Rev
2013-02-28
PA1
smaller amount of hours.
It returns the time for maximal €price in the interval of X hours from
the current time for the energy
produced by the solar farms in a
specific location.
Get Time Of
Minimum Solar
Energy Price
It returns the time for minimal €-price
in the interval of X hours from the
current time for the energy produced
by the solar farms in a specific
location.
Get Solar
Forecast
Confidence
It returns the forecasting %confidence of energy price produced
by the solar farms in a specific
location based in the data stored into
the database during the last X hours.
If the result is 0, try again with a
smaller amount of hours.
Reference
Input arguments:
 Length of the forecasting
time window in hours (max
48h): integer
 Location: string
Return type: Date/time
Input arguments:
 Length of the forecasting
time window in hours (max
48h): integer
 Location: string
Return type: Date/time
Input arguments:
 Length of the forecasting
time window in hours: integer
 Location: string
Return type: double
Both classes retrieve data from the table ConsPredictLinear.
By using the Eclipse wizard it is possible to export them to .WAR files:



First of all create four empty Dynamic Web Projects: SolarEnergyWebService,
SolarEnergyWebServiceClient, WindEnergyWebService and WindEnergyWebServiceClient.
Copy the .JAR files joda-time-2.2.jar and postgresql-9.2-1003.jdbc4.jar to
SolarEnergyWebServiceClient\WebContent\WEB-INF\lib and
WindEnergyWebServiceClient\WebContent\WEB-INF\lib; and add them to the build paths.
Right click on either the classes SolarEnergy or WindEnergy → New → Other → Web Services →
Web Service.
Ericsson Internal
11 (13)
Prepared (Subject resp)
Approved (Document resp)




No.
Checked
Date
Rev
2013-02-28
PA1
Reference
Follow the steps without changing any option
The project Solar|WindEnergyWebService and be exported as a Tomcat-deployable .WAR file.
The project Solar|WindEnergyWebServiceClient can invoke the web service from the class
Solar|WindEnergyProxy.
Ericsson Internal
12 (13)
Prepared (Subject resp)
No.
Approved (Document resp)
12
Checked
Date
Rev
2013-02-28
PA1
Reference
Chart GUI
Once the RenewableEnergiesForecasting.war file has been deployed by Tomcat, this GUI should be
available at http://195.53.58.184:8080/RenewableEnergiesForecasting/
By selecting a city and an amount of hours a chart with the forecasted wind power generation, solar power
generation and power consumption at the same time is shown. Its HTML and Javascript code is at
index.html and its servlet is com.ericsson.energies.services.QueryFactory.
13
Administration GUI
Once the RenewableEnergiesForecasting.war file has been deployed by Tomcat, this GUI should be
available at http://195.53.58.184:8080/RenewableEnergiesForecasting/admin.html
Its
HTML
and
Javascript
code
is
at
com.ericsson.energies.services.Administration.
admin.html
and
its
servlet
is
Password: ericsson
Once the CONNECT button is clicked, a Comet connection is performed between the browser (client) and
the servlet (server). Two new buttons are offered: START and DISCONNECT.



START: It starts all the threads that make up the whole project. Once every process is running, this
button will change to STOP.
DISCONNECT: Disables the Comet connection between the browser and the servlet.
STOP: Unfortunately this functionality has not been debugged, so it does not work properly.
Currently the only way to properly stop the service is from Tomcat manager.
Do not close the browser tab containing this GUI without clicking on DISCONNECT button. Otherwise the
service should be restarted from Tomcat manager because it shall become unreachable from this GUI.
14
Workflow
After pushing the START button, the following sequence is executed
com.ericsson.energies.Launcher whether the STOP button is not pushed:
by
the
class
1. Database pool manager starts
2. DDX pull client performs the subscriptions to every simulator (after launching this thread, the mother
thread Launcher will be slept for 5 seconds because this operation takes time)
3. CEP engine starts
4. Update manager starts the JSON download and, once finished, the database update process (after
launching this thread, the mother thread Launcher will be slept for 5 minutes because this operation
takes a long time)
Ericsson Internal
13 (13)
Prepared (Subject resp)
Approved (Document resp)
No.
Checked
Date
Rev
2013-02-28
PA1
Reference
5. Up to this point it is called phase 1. From now on phase 2 starts. Simulator manager starts. It
creates and launches all the simulator instances of both kinds (after launching this thread, the
mother thread will be slept for 3 days before the next step)
6. Forecaster module starts. Since it needs training input data, up to this point they would have been
populated in the database by the remaining working threads.
After at least one execution of the Forecaster module, both SOAP web services and the Chart web GUI
shall be able to provide results.