
Automatic test
analysis framework
Jacek Suchowski
TE-MPE
On behalf of the TE-MPE-MS Software Team:
M.Audrain, K.Fuchsberger, J.-C.Garnier, A.Gorzawski,
A.Moscatelli, I. Romera Ramirez, P.C.Turcu, M. Zerlauth
Content
 Motivation & Overview
 Possibilities of use
 How to use
 Plans & Improvements
2
Manual test analysis
3
Motivation & Overview
Make the test analysis:
 Automatic
 Re-usable
 Repeatable
 Trustable
 Simple
4
Possibilities to use
 Post Mortem
 BIS
 AccTesting
and more…
5
Accelerator Testing
 Two sprints so far (cycle of 3 weeks of work) in
cooperation with R. Gorbonosov and A. Jalal
 Executed from AccTesting Server – triggered on
demand
 In near future triggered after test execution is
finished
 Signals data retrieved from
 Post Mortem
 Logging database
6
Assertion language

Comparing signal value with a scalar constant
analysis.assertThat(V_MEAS).isEqualTo(5.0, VOLT)
.withinAbs(0.5, MILLI(VOLT)).at(PM_EVENT_TRIGGER);
7
Assertion language – time definitions
.at(PM_EVENT);
.at(1, SECOND).before(PM_EVENT);
.at(1, SECOND).after(PM_EVENT);
analysis.assertThat(V_MEAS)
.isEqualTo(5.0, VOLT)
.withinAbs(0.5, VOLT)
…
.startingFrom(PM_EVENT)
.endingAt(PM_EVENT)
.starting(2,SECOND).before(PM_EVENT)
.endingAt(PM_EVENT);
.during(PLATEAU);
8
Assertion language – time definitions
.at(PM_EVENT);
.at(1, SECOND).before(PM_EVENT);
.at(1, SECOND).after(PM_EVENT);
analysis.assertThat(V_MEAS)
.isEqualTo(5.0, VOLT)
.withinAbs(0.5, VOLT)
…
.startingFrom(PM_EVENT)
.endingAt(PM_EVENT)
.starting(2,SECOND).before(PM_EVENT)
.endingAt(PM_EVENT);
.during(PLATEAU);
9
Assertion language

Checking state change of a boolean signal
analysis.assertThat(BEAM_PRESENCE).changesTo(false)
.starting(4, SECOND).before(PM_EVENT_TRIGGER)
.ending(8, SECOND).after(PM_EVENT_TRIGGER);
10
Assertion language – boolean signals
 Signals
 … isEqualTo(true) …
 … changesTo(false) …
 … changesOnceTo(true) …
 Bits


… forBit(VS_STATE) …
… forBits(VS_EXTINTLOCK, VS_FAULT, VS_STATE) …
11
Assertion language

Comparing signal with a function
analysis.assertThat(V_MEAS).isEqualTo(EXP).withinAbs(0.0, VOLT)
.starting(3, MINUTE).before(PM_EVENT_TRIGGER)
.ending(1, MINUTE).before(PM_EVENT_TRIGGER);
12
Units
 All signals have assigned units / Dimensionless
 Units have to match to each other
Ampere ≠ Volt
 Can be easily converted
1000 mV = 1V
60 s = 1 min
 Support for operations on different units
Volt / Ampere = Ohm
13
Eclipse

The easiest way to write the module
14
How to write module?
 Eclipse – Integrated Development Enviroment
 Templates for creating modules
 Autocomplete syntax
 Checks syntax
 Compile the modules
 Execute the analysis module with fake signals
 Debug the module to evaluate operation
 Shows the results of analysis
15
Developing module from eclipse
 Write the assertions for analysis
 Create the analysis context
 Specify circuit name
 Specify start date
 Specify end date
 Test with the context
 Store in SVN when ready to be released
16
Example – PnoD1On60A
public class PnoD1On60A implements AnalysisModule {
@Override
public void describeAnalysis(AnalysisDescription analysis) {
analysis.assertThat(I_MEAS).isEqualTo(8.0, AMPERE).withinAbs(2.0, AMPERE)
.at(120, SECOND).after(PM_EVENT_TRIGGER);
analysis.assertThat(I_MEAS).isEqualTo(EXP).withinAbs(2.0, AMPERE).starting(10, MILLI(SECOND))
.after(PM_EVENT_TRIGGER).ending(2, MINUTE).after(PM_EVENT_TRIGGER);
analysis.assertThat(V_MEAS).isEqualTo(EXP).withinAbs(2.0, VOLT).starting(10, MILLI(SECOND))
.after(PM_EVENT_TRIGGER).ending(2, MINUTE).after(PM_EVENT_TRIGGER);
...
}
}
17
Detecting special points – to be done
 Determine reference point between signals
18
Testing the module in eclipse
public class PnoD1On60AModuleTryInGui {
public static void main(String[] args) {
AnalysisRunningUtils.runInGui(new PnoD1On60A(),
createTestAnalysisContext("RQ6.L7B2",
date("05/02/2013 09:56:52"),
date("05/02/2013 10:30:12")));
}
}
19
Analysis result
 Binary result of the analysis
 Result of evaluation of every assertion
 Graphical representation
 Shows all assertions
 Shows all analysed signals
 Shows the time(range) covered by the analysis
module
 Highlight the failures
 Documentation
20
Analysis result - example

Analysis failed
21
Analysis result - example

Analysis success
22
Plans & Improvements
 Verifications analysis modules with old data
 More types of assertions - e.g. isGreaterThan()
 Time range assertions - e.g. during(PLATEAU)
 Detecting special points - e.g. start of decay
 Using test parameters
 Documentations
 What else?
23
Questions?
Thanks for your attention!
