Monitoring the software quality in
FairRoot
Florian Uhlig
Mohammad Al-Turany
Denis Bertini
Radek Karabowicz
What is FairRoot?
• Simulation and analysis framework for the experiments at the upcoming
accelerator center FAIR.
• Base for three experiments at GSI/FAIR (CBM, Panda, R3B) and some
other users (Multi Purpose Detector @ NICA)
03.05.2010
ALICE FAIR Meeting 2010 Groningen
2
What is FairRoot?
• Simulation and analysis framework for the experiments at the
upcoming accelerator center FAIR.
detector data
Analysis chain:
loop over events
loop over raw data
calibration
apply calibration
get hits
loop over hits
do tracking
get tracks
hits
tracking
tracks
loop over tracks
do pid
pid
particles
get particles
analysis
do physics analysis
physics
publish
03.05.2010
ALICE FAIR Meeting 2010 Groningen
3
What is FairRoot?
• Simulation and analysis framework for the experiments at the
upcoming accelerator center FAIR.
detector
detector data
geometry
calibration
Analysis chain:
loop over events
loop over raw data
same
geometry
simulation
apply calibration
get hits
loop over hits
do tracking
get tracks
IO
hits
tracking
magnetic
field tracks
detector
parameters
loop over tracks
do pid
pid
event
display
particles
analysis
C++ OO
structure
physics
03.05.2010
build and
Test
system
get particles
do physics analysis
publish
ALICE FAIR Meeting 2010 Groningen
4
Why to have an automatic QA?
• Know the status of the project at any time.
– Program still works but is much slower !
• Find errors/problems in the code as early as possible, latest when
they enter into the repository.
– Produce QA information at different stages of process.
• Find errors /problems with external dependencies when there is a
change of these dependencies.
– Hard to find problems (I did not change anything. Yesterday it worked.)
• Find errors/problems on all supported platforms.
– Users can have a complete zoo of different systems.
– How to handle all these different platforms.
• Collect the QA information at one central place
– Easy to find and easy to use.
If the QA is not automatic it does not scale
03.05.2010
ALICE FAIR Meeting 2010 Groningen
5
Which tools do we use?
• Start with self written Makefiles.
– Hard to maintain.
– No support for testing.
• Used Autotools for some time.
– Same problems.
• Tested CMake and were happy.
– Create “Makefiles” for different platforms.
– Test support.
– Large user base assures support.
• CDash to handle data created with CMake
– PHP framework
– MySQL database
• Both tools are open source.
03.05.2010
ALICE FAIR Meeting 2010 Groningen
6
When to create QA information?
• At any time the information could be useful.
• In FairRoot
– If someone experiments with new features in his local
working copy and want to test them.
– If new code enters the central code base.
– Time to time a full check on all supported platforms should
be done.
03.05.2010
ALICE FAIR Meeting 2010 Groningen
7
Experimental test
Central SVN repository
1. Update (optional)
4. Dashboard prepare and display results
2. Configure, build and test
on local machine
3. Send results automatically
to central web page
5. Developer check results
03.05.2010
ALICE FAIR Meeting 2010 Groningen
8
Continuous test
Central SVN repository
Dedicated test server
2. SVN triggers test server
3a. Update of local copy
3b. Configure, build and test
on local machine
4. Send results automatically
to central web page
5. Dashboard prepare and
display results
1. Developer commit
code
6. In case of problems
Dashboard sends
an E-mail to Developer
and Administrator
7. Developer check results
03.05.2010
ALICE FAIR Meeting 2010 Groningen
9
Nightly test
Central SVN repository
1a. Update
5. In the morning Developers and Administrators check
their mails and the dashboard. And the development
cycle starts again
4. In case of problems
Dashboard sends
an E-mail to Developer
and Administrator
3. Dashboard prepare and display results
2. Send results automatically
to central web page
1b. Configure, build and test
on any test machine
which was setup
03.05.2010
ALICE FAIR Meeting 2010 Groningen
10
Which QA information is created?
1. On the test machines.
– Output from the generation of the Makefiles
– Errors and warnings from the build process
– The result (passed/failed/timeout), the execution time and in
case of failure the full output of the tests.
– Detailed information about the code coverage of the tests.
– Output of the memory process
– Compliance with the coding conventions using the IRST
RuleChecker.
03.05.2010
ALICE FAIR Meeting 2010 Groningen
11
Which QA information is created?
1. On the central server.
– Linked web pages to display the results from all test
machines
– Mean runtime of the tests.
– Different histograms.
•
•
Execution times for a given time period.
Test status for a given time period.
– E-mails when expected test machines did not deliver
expected results
– E-mails to developers who commit erroneous code.
– ......
03.05.2010
ALICE FAIR Meeting 2010 Groningen
12
How does the dashboard looks like?
03.05.2010
ALICE FAIR Meeting 2010 Groningen
13
Summary
• Perfect tools for us.
–
–
–
–
–
–
Easy to setup.
Easy to use.
Good documentation.
Good support.
Well maintained.
Make our lives much more easy!!!!!!
• You don’t have to buy it, it’s for free.
• Information
– FairRoot http://fairroot.gsi.de
– DashBoard http://fairroot.gsi.de/CDash
03.05.2010
ALICE FAIR Meeting 2010 Groningen
14