Centre for Research in
Evolution, Search & Testing
Welcome to CREST
King’s College London
Mark Harman
Centre for Research in
Evolution, Search & Testing
Centre for Research in
Evolution, Search & Testing
CREST
Engineering: Better, faster and cheaper
– through automation and
– Improved insight
World leading in
– Program Slicing
– Search Based Software Engineering
– Software Testing
Also widely known for work in
– Service-Oriented Software Engineering
– Foundations of Source Code Analysis
http://crest.dcs.kcl.ac.uk/
Centre for Research in
Evolution, Search & Testing
CREST: Where we are
CREST
King’s College London
http://crest.dcs.kcl.ac.uk/
Centre for Research in
Evolution, Search & Testing
CREST: Where we are
20 minutes walk
CREST
King’s College London
http://crest.dcs.kcl.ac.uk/
Centre for Research in
Evolution, Search & Testing
CREST Funding
http://crest.dcs.kcl.ac.uk/
Centre for Research in
Evolution, Search & Testing
Industrial Links
http://crest.dcs.kcl.ac.uk/
Centre for Research in
Evolution, Search & Testing
Industrial Links
http://crest.dcs.kcl.ac.uk/
Centre for Research in
Evolution, Search & Testing
What is SBSE?
In SBSE we apply search techniques to search large
search spaces, guided by a fitness function that
captures properties of the acceptable software
artefacts we seek.
Genetic Algorithms, Hill climbing, Simulated
Annealing, Random, Tabu Search, Estimation of
Distribution Algorithms, Particle Swarm
Optimization
http://crest.dcs.kcl.ac.uk/
Centre for Research in
Evolution, Search & Testing
Why is SBSE?
http://crest.dcs.kcl.ac.uk/
Centre for Research in
Evolution, Search & Testing
Why is SBSE?
http://crest.dcs.kcl.ac.uk/
Centre for Research in
Evolution, Search & Testing
Why not SBSE?
?
http://crest.dcs.kcl.ac.uk/
Centre for Research in
Evolution, Search & Testing
Why not SBSE?
EPSRC network
1999 – 2002
Laid foundation for SBSE
http://crest.dcs.kcl.ac.uk/
Centre for Research in
Evolution, Search & Testing
Current state of SBSE research
18 countries
20 in UK alone
CREST is undisputed world leader
http://crest.dcs.kcl.ac.uk/
Centre for Research in
Evolution, Search & Testing
Current state of SBSE research
18 countries
20 in UK alone
CREST is undisputed world leader
Mark Harman
The Current State and Future of Search Based Software Engineering
29th International Conference on Software Engineering (ICSE 2007), Future of Software Engineering
(FoSE). Minneapolis, USA, May 2007
http://crest.dcs.kcl.ac.uk/
Centre for Research in
Evolution, Search & Testing
SEBASE Project Overview
Overall: £2.7M
KCL: £1.2M
Aim:
To provide an entirely new way of understanding and practicing software
engineering
Software Engineering as search and optimize
http://crest.dcs.kcl.ac.uk/
Centre for Research in
Evolution, Search & Testing
… plus EvoTest
Overall: £2.1M
KCL: £250k
Aim:
To provide evolutionary search for software test automation
Software testing as search and optimize
http://crest.dcs.kcl.ac.uk/
Centre for Research in
Evolution, Search & Testing
The Eight Queens Problem
http://crest.dcs.kcl.ac.uk/
Centre for Research in
Evolution, Search & Testing
The Eight Queens Problem
http://crest.dcs.kcl.ac.uk/
Centre for Research in
Evolution, Search & Testing
The Eight Queens Problem
Perfect
http://crest.dcs.kcl.ac.uk/
Centre for Research in
Evolution, Search & Testing
The Eight Queens Problem
Score 0
http://crest.dcs.kcl.ac.uk/
Centre for Research in
Evolution, Search & Testing
The Eight Queens Problem
http://crest.dcs.kcl.ac.uk/
Centre for Research in
Evolution, Search & Testing
The Eight Queens Problem
http://crest.dcs.kcl.ac.uk/
Centre for Research in
Evolution, Search & Testing
The Eight Queens Problem
Two
Attacks
http://crest.dcs.kcl.ac.uk/
Centre for Research in
Evolution, Search & Testing
The Eight Queens Problem
Score -2
http://crest.dcs.kcl.ac.uk/
Centre for Research in
Evolution, Search & Testing
The Eight Queens Problem
http://crest.dcs.kcl.ac.uk/
Centre for Research in
Evolution, Search & Testing
The Eight Queens Problem
http://crest.dcs.kcl.ac.uk/
Centre for Research in
Evolution, Search & Testing
The Eight Queens Problem
Three
Attacks
http://crest.dcs.kcl.ac.uk/
Centre for Research in
Evolution, Search & Testing
The Eight Queens Problem
Score -3
http://crest.dcs.kcl.ac.uk/
Centre for Research in
Evolution, Search & Testing
That was easy
http://crest.dcs.kcl.ac.uk/
Centre for Research in
Evolution, Search & Testing
Generate a solution
Place
8
Queens
With no
attacks
http://crest.dcs.kcl.ac.uk/
Centre for Research in
Evolution, Search & Testing
Scale up: Generate a solution
http://crest.dcs.kcl.ac.uk/
Centre for Research in
Evolution, Search & Testing
Scale up: Generate a solution
http://crest.dcs.kcl.ac.uk/
Evolution beats random
Execution Time (in cycles)
Evolutionary Test
Frequency
Centre for Research in
Evolution, Search & Testing
Execution Time (cycles)
Frequency
Random Test
http://crest.dcs.kcl.ac.uk/
Generation
Execution Time (cycles)
Centre for Research in
Evolution, Search & Testing
SBSE is so generic
Testing
Fitness function: coverage, time, …
Representation: input vector
Restructuring
Fitness function: cohesion and coupling
Representation: mapping nodes to clusters
http://crest.dcs.kcl.ac.uk/
Centre for Research in
Evolution, Search & Testing
SBSE is so generic
Testing
Fitness function: coverage, time, …
Representation: input vector
Restructuring
Fitness function: cohesion and coupling
Representation: mapping nodes to clusters
http://crest.dcs.kcl.ac.uk/
Centre for Research in
Evolution, Search & Testing
SBSE is so generic
Testing is a search problem
Fitness function: coverage, time, …
Representation: input vector
Requirements is a search problem
Fitness function: cost, value, dependence constraints
Representation: bitset
http://crest.dcs.kcl.ac.uk/
Centre for Research in
Evolution, Search & Testing
SBSE is so generic
Testing is a search problem
Fitness function: coverage, time, …
Representation: input vector
Regression is a search problem
Fitness function: testing goals
Representation: retained test set
http://crest.dcs.kcl.ac.uk/
Centre for Research in
Evolution, Search & Testing
SBSE is so generic
Testing is a search problem
Fitness function: coverage, time, …
Representation: input vector
Planning is a search problem
Fitness function: time, schedule robustness
Representation: work package allocation
http://crest.dcs.kcl.ac.uk/
Centre for Research in
Evolution, Search & Testing
SBSE is so generic
Testing is a search problem
Fitness function: coverage, time, …
Representation: input vector
Planning is a search problem
Fitness function: time, schedule robustness
Representation: work package allocation
http://crest.dcs.kcl.ac.uk/
Centre for Research in
Evolution, Search & Testing
SBSE Applications
Transformation
Requirements
Effort prediction
Management
Heap allocation
Regression test
SOA
Refactoring
Test Generation
Maintenance
Model checking
Probe dist’ion
UIOs
Comprehension
Protocols
Component sel
Agent Oriented
http://crest.dcs.kcl.ac.uk/
Cooper, Ryan, Schielke, Subramanian, Fatiregun, Williams
Bagnall, Mansouri, Zhang
Aguilar-Ruiz, Burgess, Dolado, Lefley, Shepperd
Alba, Antoniol, Chicano, Di Pentam Greer, Ruhe
Cohen, Kooi, Srisa-an
Li, Yoo, Elbaum, Rothermel, Walcott, Soffa, Kampfhamer
Canfora, Di Penta, Esposito, Villani
Antoniol, Briand, Cinneide, O’Keeffe, Merlo, Seng, Tratt
Alba, Binkley, Bottaci, Briand, Chicano, Clark, Cohen,
Gutjahr, Harrold, Holcombe, Jones, Korel, Pargass,
Reformat, Roper, McMinn, Michael, Sthamer, Tracy,
Tonella,Xanthakis, Xiao, Wegener, Wilkins
Antoniol, Lutz, Di Penta, Madhavi, Mancoridis, Mitchell,
Swift
Alba, Chicano, Godefroid
Cohen, Elbaum
Derderian, Guo, Hierons
Gold, Li, Mahdavi
Alba, Clark, Jacob, Troya
Baker, Skaliotis, Steinhofel, Yoo
Haas, Peysakhov, Sinclair, Shami, Mancoridis
Centre for Research in
Evolution, Search & Testing
SBSE Applications in which SEBASE is active
Transformation
Requirements
Effort prediction
Management
Heap allocation
Regression test
SOA
Refactoring
Test Generation
Maintenance
Model checking
Probe dist’ion
UIOs
Comprehension
Protocols
Component sel
Agent Oriented
http://crest.dcs.kcl.ac.uk/
Cooper, Ryan, Schielke, Subramanian, Fatiregun, Williams
Bagnall, Mansouri, Zhang
Aguilar-Ruiz, Burgess, Dolado, Lefley, Shepperd
Alba, Antoniol, Chicano, Di Pentam Greer, Ruhe
Cohen, Kooi, Srisa-an
Li, Yoo, Elbaum, Rothermel, Walcott, Soffa, Kampfhamer
Canfora, Di Penta, Esposito, Villani
Antoniol, Briand, Cinneide, O’Keeffe, Merlo, Seng, Tratt
Alba, Binkley, Bottaci, Briand, Chicano, Clark, Cohen,
Gutjahr, Harrold, Holcombe, Jones, Korel, Pargass,
Reformat, Roper, McMinn, Michael, Sthamer, Tracy,
Tonella,Xanthakis, Xiao, Wegener, Wilkins
Antoniol, Lutz, Di Penta, Madhavi, Mancoridis, Mitchell,
Swift
Alba, Chicano, Godefroid
Cohen, Elbaum
Derderian, Guo, Hierons
Gold, Li, Mahdavi
Alba, Clark, Jacob, Troya
Baker, Skaliotis, Steinhofel, Yoo
Haas, Peysakhov, Sinclair, Shami, Mancoridis
Centre for Research in
Evolution, Search & Testing
Regression
Companies have their act together
We have large pools of regression test data
They are getting too large
We need to Select and Prioritize
http://crest.dcs.kcl.ac.uk/
Centre for Research in
Evolution, Search & Testing
Regression Selection
http://crest.dcs.kcl.ac.uk/
Centre for Research in
Evolution, Search & Testing
Regression Selection
http://crest.dcs.kcl.ac.uk/
Centre for Research in
Evolution, Search & Testing
Regression Prioritization
The optimal test case orderings:
http://crest.dcs.kcl.ac.uk/
C
D
A
B
D
C
A
B
Centre for Research in
Evolution, Search & Testing
Regression Prioritization
http://crest.dcs.kcl.ac.uk/
Centre for Research in
Evolution, Search & Testing
Requirements Optimization
Most expensive mistakes and misunderstandings
Many customer Many requirements Many releases
Fairness, Approximation
http://crest.dcs.kcl.ac.uk/
Centre for Research in
Evolution, Search & Testing
Requirements optimization
http://crest.dcs.kcl.ac.uk/
Centre for Research in
Evolution, Search & Testing
Dependence Analysis
Some case studies in dependence analysis
Work we are doing with Prof. Binkley for
DaimlerChrysler
IBM
Motorola
Case studies are based on open source
Prof. Binkley’s visit is supported by EPSRC
http://crest.dcs.kcl.ac.uk/
Centre for Research in
Evolution, Search & Testing
Bubble charts
replace
http://crest.dcs.kcl.ac.uk/
prepro
Centre for Research in
Evolution, Search & Testing
Flex evolution
http://crest.dcs.kcl.ac.uk/
Centre for Research in
Evolution, Search & Testing
Profiling
sendmail
http://crest.dcs.kcl.ac.uk/
findutils
Centre for Research in
Evolution, Search & Testing
Monotone Slice Size Graphs
Example Dependence Profile
X axis: program points
Y axis: normalised dependence level
http://crest.dcs.kcl.ac.uk/
Centre for Research in
Evolution, Search & Testing
What is going on here?
http://crest.dcs.kcl.ac.uk/
Centre for Research in
Evolution, Search & Testing
No clusters

http://crest.dcs.kcl.ac.uk/
Centre for Research in
Evolution, Search & Testing
Enormous
clusters

http://crest.dcs.kcl.ac.uk/
Centre for Research in
Evolution, Search & Testing
Refactor to remove
http://crest.dcs.kcl.ac.uk/
Centre for Research in
Evolution, Search & Testing
Summary
World leading Centre in
Testing
Search Based Software Engineering
Slicing
Centre hub of National and International Networks
Service Oriented Software Engineering
Source Code Analysis
Search Based Software Engineering
Funded by
EPSRC
EU
Industry
http://crest.dcs.kcl.ac.uk/