Automated Unit Testing with
AgitarOne
Presented by Eamon McCormick
Senior Solutions Consultant, Agitar Software Inc.
© Agitar Software, 2007
1
Presentation Outline

Unit testing introduction

Good unit test properties

Mock Objects

Continuous Integration and unit tests

Automated Unit Testing with AgitarOne
© Agitar Software, 2007
2
Unit Testing Introduction

Unit testing is a procedure used to validate that individual
units of source code work properly. A unit is the smallest
testable part of an application. In object-oriented
programming, the smallest unit is a class and the methods
within that class.

Unit tests benefit organizations by:
– Improving design
• In TDD, unit testing is a design activity!
– Eliminating the unknown
– Speeding development
– Reducing bugs in new and existing features
– Allowing safe refactoring of source code
© Agitar Software, 2007
3
Get It Right:
Prevention Saves Money
$16,000
% Defects
introduced in
this phase
85%
Percentage
of Bugs
% Defects
found in
this phase
$1000
$ Cost to
repair defect
in this phase
$250
$25
Coding
$100
Unit
Test
Function
Test
System
Test
After
Release
Source: Applied Software Measurement, Capers Jones, 1996
© Agitar Software, 2007
4
Keep It Right:
Detect Changes in Code Behavior
Impacts
Code B
Change
Code A
Continuous
Integration

“Sensors” show the
impact of code changes

High test density brings
confidence and agility

Continuous integration &
test brings the best impact
Impacts
Code C
© Agitar Software, 2007
5
Properties of Good Tests

Automatic
–
–
–
–

Should
Should
Should
Should
be easy to run
not require manual intervention
run quickly
be integrated into automated build
Thorough
– Consider setting coverage targets
– Use judgment as to what should be tested

Repeatable
– Minimize dependencies on external factors by using mock objects
etc.

Independent
– Tests should not depend on each other
– Test one thing at a time (however a single test can have multiple
asserts)
© Agitar Software, 2007
6
Mock Objects Enable Unit Testing

What are they
– Objects that simulate the behavior of real objects
– Created by developers to help with testing

When To Use
– Need to isolate the code under test from difficult
resources (unavailable or non-deterministic)
– Behavior is difficult to trigger (e.g. network error)
– Real object is very slow
– Real object requires user intervention
– Real object does not yet exist
© Agitar Software, 2007
7
Continuous Integration Enhances Unit
Testing

Continuous Integration mechanics
– On
1.
2.
3.
4.
at least a nightly basis, have a process to:
Get the latest version of code from your repository
Compile the code
Execute all unit tests written for the code
Create a report with build and test results
“Imperfect tests, run frequently, are much better than
perfect tests that are never written at all” – Martin Fowler
© Agitar Software, 2007
8
Unit Testing Utopia
Our Classes
Our Unit Test Classes
C15
C13
C9
C1
C14
C10
C2
C3
C15
C13
C11
C4
C5
C12
C6
C7
C8
C9
C1
C14
C10
C2
C3
C11
C4
C5
C12
C6
C7
C8

Every class has unit tests

The tests are executed many times each day

The tests are thorough, up to date, independent and
easy to maintain and analyze
© Agitar Software, 2007
9
AgitarOne is a Complete Unit Test
Solution
Automated
Detect Changes
JUnit
in
Generation
Behavior
Get Immediate
CruiseControl
Feedback
Management
Report on Progress
Dashboard
© Agitar Software, 2007
AgitarOne™
Test
Software
for the
Unexpected
Agitation
Code-Rule
Find
Anti-Patterns
Enforcement
10
Test for the Unexpected:
Software Agitation
10101
01010
10101
Analyze
Code
Summarize
AgitarOne
Agitation
Fix Bug
Observations of
Code Behavior, and
Review
Coverage Data
If observation
reveals a bug,
fix it!
Create a JUnit
Developer

Use automated JUnit generation to create
your regression test
Validate intended code behavior, interactively



© Agitar Software, 2007
Discover invariants (or specify and validate them)
Achieve high data and state coverage without manual setup
Test without needing application server, database, etc.
11
Detect Changes in Behavior:
Automated JUnit Test Generation
10101
01010
10101
Analyze
Code
Filter & Verify
AgitarOne
Agitation
Standard
JUnit Tests
Review or Accept

Developer
Check in
Your New Tests
Create a useful set of JUnit tests, automatically



© Agitar Software, 2007
Generate good static tests to detect code behavior changes
Achieve high data and state coverage without manual setup
Reduce the work by 80%+ for thorough JUnit coverage
12
Avoid Bug-Prone Coding Constructs:
Flexible Code Rule Enforcement
© Agitar Software, 2007
13
Measure, Manage, and Report:
Agitar Management Dashboard
What code hasn’t been tested
?
How rigorous are my tests
?
Do I have objective data on quality from outsource partners
Am I focusing on the most critical code
What bugs have been found
© Agitar Software, 2007
?
?
?
14
Get Immediate Feedback:
Continuous Integration and Test
Store
24x7
Code & Tests
Configuration
Management
Status
Dashboard
E-Mail
Rapid Notification
of Changes
Build
Processes
Proven best-practice for development productivity



© Agitar Software, 2007
High assertion density  rapid problem detection
AgitarOne has 50,000 tests running constantly
Minutes to feedback, not hours, days, or weeks
15
Useful Resources

Agitar website: www.agitar.com

Agitar’s Developer Testing Blog: www.developertesting.com

Agitar’s discussion site: http://forums.agitar.com

JUnit Factory – free generated JUnits: www.junitfactory.com

JUnit Homepage: www.junit.org

Test Drive Development Site: www.testdriven.com
© Agitar Software, 2007
16
Thank You
© Agitar Software, 2007
17