Practical Course
Evolutionary Generation of Test Scenarios
for Autonomous Driving
Preliminary Meeting
Tim Rohlfs
June 30, 2016
Tim Rohlfs
Evolutionary Generation of Test Scenarios
Organization
Who we are:
Chair 22 of Software Engineering
Prof. Alexander Pretschner
Research areas:
Security
Testing
Model-based software development
Responsible for this course:
Tim Rohlfs
[email protected]
Target audience:
Master students with good MATLAB skills
Tim Rohlfs
Evolutionary Generation of Test Scenarios
The parking assistant
Figure: Buehler & Wegener 2003
Tim Rohlfs
Evolutionary Generation of Test Scenarios
Testing
Figure: Pretschner, Proz. u. Meth. b. Testen v. SW
Tim Rohlfs
Evolutionary Generation of Test Scenarios
Model-based testing
Figure: Pretschner, Proz. u. Meth. b. Testen v. SW
Tim Rohlfs
Evolutionary Generation of Test Scenarios
Difficult questions
Figure: Pretschner, Proz. u. Meth. b. Testen v. SW
Tim Rohlfs
Evolutionary Generation of Test Scenarios
Evolutionary algorithms: The principle
Evolution
Tim Rohlfs
Evolutionary Generation of Test Scenarios
Evolutionary algorithms: The principle
Evolution
“Given a population of individuals,
Tim Rohlfs
Evolutionary Generation of Test Scenarios
Evolutionary algorithms: The principle
Evolution
“Given a population of individuals,
the environmental pressure causes natural selection,
Tim Rohlfs
Evolutionary Generation of Test Scenarios
Evolutionary algorithms: The principle
Evolution
“Given a population of individuals,
the environmental pressure causes natural selection,
which causes a rise in the fitness of the population.”
Tim Rohlfs
Evolutionary Generation of Test Scenarios
Evolutionary algorithms: The principle
Evolution
“Given a population of individuals,
the environmental pressure causes natural selection,
which causes a rise in the fitness of the population.”
Evolutionary algorithms
“Given a quality function to be maximised,
we can randomly create a set of candidate solutions,
i.e., elements of the function’s domain,
and apply the quality function as an abstract fitness measure.
Based on this fitness,
some of the better candidates are chosen
to seed the next generation
by applying recombination or and/or mutation to them.”
A. E. Eiben et al., Introduction to Evolutionary Computing, Springer 2003
Tim Rohlfs
Evolutionary Generation of Test Scenarios
Evolutionary algorithms: The framework
INITIALIZE population with random candidate solutions
EVALUATE each candidate
repeat until termination condition is reached:
SELECT parents
RECOMBINE pairs of parents
MUTATE resulting offspring
EVALUATE new candidates
SELECT individuals for next generation
end repeat
A. E. Eiben et al., Introduction to Evolutionary Computing, Springer 2003
Tim Rohlfs
Evolutionary Generation of Test Scenarios
Approach of Buehler and Wegener: System setup
Figure: Buehler & Wegener 2003
Tim Rohlfs
Evolutionary Generation of Test Scenarios
Approach of Buehler and Wegener: Test case
Figure: Buehler & Wegener 2003
Tim Rohlfs
Evolutionary Generation of Test Scenarios
Results of Buehler and Wegener
Figure: Buehler & Wegener 2003
Tim Rohlfs
Evolutionary Generation of Test Scenarios
Objectives of the course
We want to
learn about evolutionary algorithms
learn about the possibilities and limitations of the approach
Our focus is not to build a “cool parking assistant”!
Course has an experimental character:
our goal is not to implement a feature-rich, production-ready
system,
but to experiment and evaluate
=⇒ Let’s see what we can achieve!
Tim Rohlfs
Evolutionary Generation of Test Scenarios
Tasks
Work packages:
Build a simulation of the parking assistant
Build a simulation of the environment
Apply the tbxmpga function of the GEA toolbox
Output/visualize the results
Organization:
Complexity is difficult to estimate =⇒
Tim Rohlfs
Evolutionary Generation of Test Scenarios
Tasks
Work packages:
Build a simulation of the parking assistant
Build a simulation of the environment
Apply the tbxmpga function of the GEA toolbox
Output/visualize the results
Organization:
Complexity is difficult to estimate =⇒ iterative and
incremental process
Tim Rohlfs
Evolutionary Generation of Test Scenarios
Tasks
Work packages:
Build a simulation of the parking assistant
Build a simulation of the environment
Apply the tbxmpga function of the GEA toolbox
Output/visualize the results
Organization:
Complexity is difficult to estimate =⇒ iterative and
incremental process
We work in groups of two or three, each group works all
packages
Tim Rohlfs
Evolutionary Generation of Test Scenarios
Assessment (preliminary)
For each increment:
From each group a participant is chosen who presents what
the group has done (attestation)
Each group submits a documentation of its work
At the end of the course:
Each participant presents (part of) his/her contribution,
answers questions (presentation)
Your performance will be graded based on
the code quality and functionality of your implementation
(20 %, group-wise)
your submitted documentation (30 %, group-wise)
your attestation (25 %, individually)
your presentation (25 %, individually)
Tim Rohlfs
Evolutionary Generation of Test Scenarios
Prerequisites
Required:
Working knowledge of MATLAB
Willingness to independently research into additional
literature, if necessary
Nice-to-have:
Working knowledge of software development
Basic experience in graphics programming
Tim Rohlfs
Evolutionary Generation of Test Scenarios