How complete are your tests?
Code Coverage in Eclipse
Joachim Hofer
09/10/10
Code Coverage in Eclipse
1 / 10
Outline
●
Motivation
●
Types of Code Coverage
●
Case Study:
●
09/10/10
●
Testing TestBench
●
Tool Requirements
Code Coverage Tools
Code Coverage in Eclipse
2 / 10
Why Code Coverage? (1)
●
Tests need goals!
●
Black Box:
–
–
●
White Box:
–
–
09/10/10
Requirement Coverage
System Tests
Code Coverage
Integration/Unit Tests
Code Coverage in Eclipse
3 / 10
Why Code Coverage? (2)
●
●
Not for optimizing yet another code metric!
●
Easy to increase in all the wrong ways...
●
False sense of security
What can go wrong with developer tests?
●
… don't test what they are intended to
●
… won't get executed
●
… can be elaborate in the wrong places
→ Analyze their coverage!
09/10/10
Code Coverage in Eclipse
4 / 10
Types of Code Coverage
●
●
Coverage Types
●
Statement Coverage
●
Branch Coverage
●
Condition Coverage
●
Path Coverage
Not good enough!
Often good enough...
Difficult...
Usually impractical.
Instrumentation Types
●
●
09/10/10
Based on Source Code
Based on Bytecode
Usually recommended, but...
JVM multi-language support!
Code Coverage in Eclipse
5 / 10
Case Study: Testing TestBench
●
System Test (QF-Test, TestBench)
●
~16k tests run every night
●
Takes 10 hours
●
Video capturing
(due to popular request: done via vnc und vnc2flv)
●
Integration/Unit Test (TestNG)
●
> 1000 tests run after each commit
●
Takes 10 minutes
●
Complemented by FindBugs
09/10/10
Code Coverage in Eclipse
6 / 10
Case Study: Tool Requirements
●
Measures Branch Coverage
●
Available via Command Line for Build
●
Integration into CI Server (Hudson et al)
●
Eclipse Plug-in Available
●
Based on Bytecode
●
Free if possible...
09/10/10
Code Coverage in Eclipse
7 / 10
Code Coverage Tools
looks good
●
CodeCover
●
Emma / EclEmma
●
Clover
●
Cobertura
looks good
not bytecode-based
no branch coverage
not free
no Eclipse plug-in
→ eCobertura Eclipse plug-in
09/10/10
no CI integration
Code Coverage in Eclipse
yay!
still pretty basic
8 / 10
Links
●
●
Eclipse Plug-ins
●
eCobertura - http://ecobertura.johoop.de
●
EclEmma - http://eclemma.org/
●
CodeCover - http://codecover.org/
●
Clover - http://www.atlassian.com/software/clover/
Command Line / Ant
●
Cobertura - http://cobertura.sourceforge.net/
●
Emma - http://emma.sourceforge.net/
●
JaCoCo - http://www.eclemma.org/jacoco/
09/10/10
Code Coverage in Eclipse
9 / 10
Thanks!
Any questions?
09/10/10
Code Coverage in Eclipse
10 / 10