Code Coverage Analysis for Eclipse
Code Coverage Analysis for Eclipse
EclipseCon 2008, Santa Clara
Marc R. Hoffmann, [email protected]
Gilles Iachelini, [email protected]
17.03.2008
© 10.03.2006
2008 Marc R. Hoffmann, made available under EPL v1.0
Code Coverage Analysis for Eclipse
Focus
Technical
Java
Eclipse
© 2008 Marc R. Hoffmann, made available under EPL v1.0
2
Code Coverage Analysis for Eclipse
Topics
ƒ Principles and Techniques
ƒ Code Coverage Tools in Eclipse
ƒ Coverage Analysis of Eclipse Apps
© 2008 Marc R. Hoffmann, made available under EPL v1.0
3
Code Coverage Analysis for Eclipse
Tutorial Prerequisites
ƒ Eclipse 3.3.x
ƒ Tutorial Data
Tutorial
USB Sticks
http://www.eclemma.org/research
© 2008 Marc R. Hoffmann, made available under EPL v1.0
4
Code Coverage Analysis for Eclipse
Usage Scenario
System Test
Unit Tests
Automated UI Test
Manual Execution
INDIRECT METRIC
White Box Testing
© 2008 Marc R. Hoffmann, made available under EPL v1.0
Black Box Testing
5
Code Coverage Analysis for Eclipse
Coverage Units
ƒ Control Flow Coverage
JClasses
JMethods
JLines
JStatements
JBranches
JPaths
Covered Units
Coverage Ratio =
© 2008 Marc R. Hoffmann, made available under EPL v1.0
Total Units
6
Code Coverage Analysis for Eclipse
Statement Coverage
public int clip(int lower, int upper, int x) {
if (x < lower) {
x = lower;
}
if (x > upper) {
x = upper;
}
return x;
}
Test Set for Full Statement Coverage:
© 2008 Marc R. Hoffmann, made available under EPL v1.0
clip(1, 9, 0)
clip(1, 9, 10)
7
Code Coverage Analysis for Eclipse
Branch Coverage
public int clip(int lower, int upper, int x) {
if (x < lower) {
x = lower;
}
if (x > upper) {
x = upper;
}
return x;
}
Test Set for Full Branch Coverage:
© 2008 Marc R. Hoffmann, made available under EPL v1.0
clip(1, 9, 0)
clip(1, 9, 10)
8
Code Coverage Analysis for Eclipse
Path Coverage
public int clip(int lower, int upper, int x) {
if (x < lower) {
x = lower;
}
if (x > upper) {
x = upper;
}
return x;
}
Test Set for Full Path Coverage:
© 2008 Marc R. Hoffmann, made available under EPL v1.0
clip(1,
clip(1,
clip(1,
clip(9,
9,
9,
9,
1,
0)
10)
5)
5)
9
Code Coverage Analysis for Eclipse
Granularity
© 2008 Marc R. Hoffmann, made available under EPL v1.0
10
Code Coverage Analysis for Eclipse
Tool Matrix
Cmd
Clover
Ant
Eclipse
9
9
9
Coverlipse
Cobertura
9
9
EMMA/EclEmma
9
9
TPTP
© 2008 Marc R. Hoffmann, made available under EPL v1.0
9
9
11
Code Coverage Analysis for Eclipse
Coverage While You Work
ƒ Install EclEmma
02_EclEmma_1.3.1/eclemma-1.3.1.zip
ƒ Import sample application
03_Exercise_1/...simplemath.zip
ƒ Run FormulasTest as JUnit test
ƒ Identify lines in class Formulas not
covered by the test cases.
ƒ Add test cases to get full coverage
© 2008 Marc R. Hoffmann, made available under EPL v1.0
12
Code Coverage Analysis for Eclipse
Coverage Data Analysis
ƒ Import Apache Collections Library
04_Exercise_2/...collections.zip
ƒ Run TestAllPackes as JUnit test
ƒ Explore Results in Coverage view
© 2008 Marc R. Hoffmann, made available under EPL v1.0
13
Code Coverage Analysis for Eclipse
Feedback Loop
JU
Write Tests
© 2008 Marc R. Hoffmann, made available under EPL v1.0
Fix It
14
Code Coverage Analysis for Eclipse
Implementation Strategies
JVM
J
Compiler
010
Instrumentation
Java Source
Runtime Profiling
Byte Code
Offline
© 2008 Marc R. Hoffmann, made available under EPL v1.0
Class Loader
On-the-Fly
15
Code Coverage Analysis for Eclipse
Why Byte Code Instrumentation?
ƒ
ƒ
ƒ
ƒ
Performance Issues with JVM Profilers
No Source Required
Works on 3rd party JAR‘s
Platform independent
© 2008 Marc R. Hoffmann, made available under EPL v1.0
16
Code Coverage Analysis for Eclipse
Byte Code Instrumentation
JVM
010
Instrumentation
Coverage
Runtime
Structure
Information
© 2008 Marc R. Hoffmann, made available under EPL v1.0
17
Code Coverage Analysis for Eclipse
Byte Code Probes
// access flags 9
public static clip(III)I
L0
LINENUMBER 6 L0
ILOAD 2
ILOAD 0
IF_ICMPGE L1
L2
L0
LINENUMBER 7 L2
ILOAD 0
LINENUMBER
6 L0
ISTORE22
ILOAD
L1
ILOAD 0
LINENUMBER 9 L1
ILOAD 2
ILOAD 1
IF_ICMPLE L3
L4
LINENUMBERL1
IF_ICMPGE
10 L4
ILOAD 1
L2
ISTORE 2
LINENUMBER
7 L2
L3
ILOAD 0
LINENUMBER
12 L3
ISTORE
2
ILOAD 2
IRETURN
[…]
© 2008 Marc R. Hoffmann, made available under EPL v1.0
// access flags 9
public static clip(III)I
L0
GETSTATIC Formulas.$VRc : [[Z
L1
ICONST_1
AALOAD
ASTORE 3
L2
LINENUMBER 6 L2
ILOAD 2
ILOAD 0
L3
ALOAD 3
ICONST_0
ICONST_1
BASTORE
IF_ICMPGE L4
L5
LINENUMBER 7 L5
ILOAD 0
ISTORE 2
ALOAD 3
ICONST_1
ICONST_1
BASTORE
[…]
18
Code Coverage Analysis for Eclipse
Code Coverage for Eclipse/OSGi Bundles
© 2008 Marc R. Hoffmann, made available under EPL v1.0
19
Code Coverage Analysis for Eclipse
Code Coverage for JUnit Plug-In Tests
ƒ Import EclEmma Plug-ins
05_Exercise_3/eclemma_core.zip
ƒ Run AllEclEmmaCoreTests JUnit
plug-in test
© 2008 Marc R. Hoffmann, made available under EPL v1.0
20
Code Coverage Analysis for Eclipse
Headless?
Eclipse/OSGi
IDE
© 2008 Marc R. Hoffmann, made available under EPL v1.0
Target
21
Code Coverage Analysis for Eclipse
Class Loading via OSGi
JVM
J
Compiler
010
Class Loader
OSGi Framework
Implementation
© 2008 Marc R. Hoffmann, made available under EPL v1.0
22
Code Coverage Analysis for Eclipse
Equinox OSGi Runtime
© 2008 Marc R. Hoffmann, made available under EPL v1.0
Bundle
Bundle
Bundle
Bundle
Bundle
Equinox Adapter Hooks
Framework
Instrumentation
Extension
23
Code Coverage Analysis for Eclipse
Code Coverage for Equinox
ƒ Copy to your plugins folder
06_Exercise_4/…osgihook
ƒ Insert into config.ini
osgi.framework.extensions=
com.mountainminds.eclemma.osgihook
ƒ Start/Stop Eclipse
© 2008 Marc R. Hoffmann, made available under EPL v1.0
24
Code Coverage Analysis for Eclipse
TDD/Test First ↔ Code Coverage
superseeds
TDD
Test First
Code
Coverage
verifies
© 2008 Marc R. Hoffmann, made available under EPL v1.0
25
Code Coverage Analysis for Eclipse
Thank You!
ƒ Use Code Coverage
ƒ It‘s Easy and Helpful
ƒ OSGi/Equinox is Code Coverage Ready
ƒ Makes Writing Unit Tests Fun
© 2008 Marc R. Hoffmann, made available under EPL v1.0
26
Code Coverage Analysis for Eclipse
References
ƒ Code Coverage Tools
J Clover, http://www.atlassian.com/software/clover/
J Coverlipse, http://coverlipse.sourceforge.net/
J Cobertura, http://cobertura.sourceforge.net/
J CodeCover, http://codecover.org/
J EclEmma, http://www.eclemma.org/
J EMMA, http://emma.sourceforge.net/
J TPTP, http://www.eclipse.org/tptp/
ƒ Byte Code Library and Outline Plug-in:
J ASM, http://asm.objectweb.org/
ƒ Code Coverage for Equinox
J http://www.eclemma.org/research/instrumentingosgi/index.html
© 2008 Marc R. Hoffmann, made available under EPL v1.0
27
Code Coverage Analysis for Eclipse
Copy Right Info
ƒ This presentation is contributed by Marc R.
Hoffmann, Mountainminds GmbH & Co. KG,
made available under EPL 1.0
ƒ Some example code taken from the Apache
Jakarta Commons project, provided under
Apache License Version 2.0.
ƒ All pictures in this presentation taken from
stock.xching, http://sxc.hu/
© 2008 Marc R. Hoffmann, made available under EPL v1.0
28