OAF
Jean-Jacques Gras
Stephen Jackson
Blazej Kolad
7/31/2017
1
OAF
Problem:
To identify (and report) problems and anomalies
of BI instruments.
Challenges:
-
-
We need to analyze thousands
variables for hundreds of devices.
of
Computations are demanding in terms of database
access and processing power, can take several days.
7/31/2017
2
Solutions:
OAF
1) A lot of ad-hoc scripts hacked by
different people (often short-term
at CERN).
7/31/2017
2) Unified approach
-
Extract common functionality
-
Automatize as much as possible
3
OAF
Timber-Snapshots
“The Logging Service persists data coming from pre-defined signals
into Oracle databases, and provides an API and a generic GUI
(TIMBER)”
7/31/2017
4
ARCHITECTURE
Excel
configuration files
User
Interface
Report pdf
triggers
produce
produce
triggers
sDaily
Execution
Supplies
Summary
EMAIL
Logging DB
Optional
Expert
Code
7/31/2017
5
FRAMEWORK SEQUENCE
1)
2)
3)
4)
5)
6)
7)
8)
Extract Data
Compute extra data
Make summary page
Analyze alarms
Make default plots
Make extra plots
Run expert code
Inform
7/31/2017
Basic analysis (default plots
of variables) can be made
just by creating timber
snapshot.
6
The OAF Program
One single program will for any type of our instruments and
solely based on the data published in the logging DB and a
simple configuration excel file:
•
Compute new data sets from the one recorded (relative
values, grouping per CYCLE or BMODE, statistics, FFTs…)
•
Detect (and alert you) warning signs encountered in your
logging (missing data, wrong status, wrong settings…)
•
Plot this data the way you want it
7/31/2017
7
Compute new data sets from the one recorded (relative
values, grouping per CYCLE or BMODE, statistics, FFTs…)
Configuration
Tab
7/31/2017
8
Compute new data sets from the one recorded (relative
values, grouping per CYCLE or BMODE, statistics, FFTs…)
BSRA data limited to stable
beam.
7/31/2017
9
Compute new data sets from the one recorded (relative
values, grouping per CYCLE or BMODE, statistics, FFTs…)
BLM DAB’s
temperature
statistics
7/31/2017
10
Detect (and alert you) warning signs encountered in your
logging (missing data, wrong status, wrong settings…)
7/31/2017
11
Plot this data the way you want it
7/31/2017
12
Plot data the way you want it
7/31/2017
13
Plot data the way you want it
7/31/2017
14
Plot data the way you want it
PS BWS Profile and
fitting survey
7/31/2017
15
Summary email
Report
7/31/2017
16
Error on measurement
7/31/2017
17
Report repository
7/31/2017
18
Report Generic part – first page
7/31/2017
19
Report Generic part
7/31/2017
20
Report Generic part
Red – Alarm was raised
Yellow – Data not consistent
Brown – No alarms set
7/31/2017
21
Potential Extra Usage
We could try also to use this tool ‘ONLINE’
We could define a short time (ie 15 mn) dynamic
DB snapshot to look at the last 15 mn DB
recording to check if things are going well
7/31/2017
22
Main and Common Framework:
The GUI
1. extract_data()
# Get the snapshot from DB
2. compute_extra_data() # Compute extra data if any as
conf.
3. analyse_alarms()
# Analyse alarms as config.
4. make_plots()
# makes extra plots if any as conf.
5. close_report()
# close and save the report file
6. inform()
# send info mail to client as conf.
7/31/2017
• Green: Based on config file <inst>_<dataset>
• Orange: Based on DB Public snapshot
23
Technical Solutions

125+ of the most popular Python packages for science, math, engineering, data
analysis ( NumPy, SciPy, Pandas, IPython, Matplotlib, Numba, Blaze, Bokeh..)

Completely free - including for commercial use

Cross platform on Linux, Windows, Mac

Installs into a single directory and doesn't affect other Python installations on
your system. Doesn't require root or local administrator privileges

Spyder, Ipython

We use Jpypy for java – python integration

Installed on bdidev2
7/31/2017
24
Technical Solutions
7/31/2017
25
Conclusions
•
OAF is now ready and operational
•
Reports will be produced for several instruments
•
Highlights and observations regularly presented and
discussed as recurrent topics of our technical boards.
•
We are also currently developing with Ana’s collaboration
dedicated reports to help our machine coordinators in their
activity
7/31/2017
26
Thank you
7/31/2017
27
7/31/2017
28