Agile Test
Automation
Created and Modified By:
Nishant Awasthi
January 13, 2009
Property of Progressive Insurance &
Casualty Company
1
Agile Test Automation
Treat testware as software
 Application of agile development principles to
the test automation project.
Requirements modeling.
Automation architecture modeling.
Acceptance testing of automated scripts.
Time Boxed iterations with frequent delivery
of testware.

Property of Progressive Insurance &
Casualty Company
2
Agile Automation Process…
contd..

Continuous integration of testware
 Running

tests with the latest development build
Continuous integrated testing
 Execute
tests daily in the Development and QA
environment
 Provide continuous feedback

Lock-step with the development and the QA
team for iterative releases of testware
Property of Progressive Insurance &
Casualty Company
3
To Prod
Release
Check-in
Build
Source
Control
Build Proc
Dev Env
QA Env
Nightly and Ad Hoc
Nightly and Ad Hoc
Story Card
Assignment
Unit Test and
Code
Requirements
Synch
Embedded
Testing
Acceptance
Testing
Development Cycle
Manual Testing
Test Plan
Creation
System
Testing
QA Cycle
Notes DB
Automation
Prioritization
and Analysis
Automation
Assignment
Automation
Development
TestDirector
Lab Machines
Automation
Acceptance
The Automation Cycle
Automation
Tracker
Property of Progressive Insurance &
Casualty Company
Automation
Script Review
Automation
Release
4
To Prod
Release
Check-in
Build
Source
Control
Build Proc
Dev Env
QA Env
Nightly and Ad Hoc
Nightly and Ad Hoc
Story Card
Assignment
Unit Test and
Code
Requirements
Synch
Embedded
Testing
Acceptance
Testing
Manual Testing
Development Cycle
Test Plan
Creation
Notes DB
System
Testing
Automation
Prioritization
and Analysis
Automation
Assignment
Automation
Development
TestDirector
Lab Machines
Automation
Acceptance
The Automation Cycle
Development IPM
Development Iteration
Planning Meeting
for the current iteration
Automation
Tracker
Property of Progressive Insurance &
Casualty Company
Automation
Script Review
Automation
Release
5
To Prod
Release
Check-in
Source
Control
Build Proc
Build
Dev Env
QA Env
Nightly and Ad Hoc
Nightly and Ad Hoc
Story Card
Assignment
Unit Test and
Code
Requirements
Synch
Embedded
Testing
Acceptance
Testing
Manual Testing
Development Cycle
Test Plan
Creation
Notes DB
System
Testing
Automation
Prioritization
and Analysis
Automation
Assignment
Automation
Development
TestDirector
Lab Machines
Automation
Acceptance
The Automation Cycle
Automation
Tracker
Property of Progressive Insurance &
Casualty Company
Automation
Script Review
Automation
Release
6
To Prod
Release
Check-in
Source
Control
Build Proc
Build
Dev Env
QA Env
Nightly and Ad Hoc
Nightly and Ad Hoc
Story Card
Assignment
Unit Test and
Code
Requirements
Synch
Embedded
Testing
Acceptance
Testing
Manual Testing
Development Cycle
Test Plan
Creation
Notes DB
System
Testing
Automation
Prioritization
and Analysis
Automation
Assignment
Automation
Development
TestDirector
Lab Machines
Automation
Acceptance
The Automation Cycle
Automation
Tracker
Property of Progressive Insurance &
Casualty Company
Automation
Script Review
Automation
Release
7
Automation Ceremonies
Name
Automation Planning
Meeting (APM)
Frequency
Agenda
Every other week
Following
Development
Iteration planning
meeting.
1)
2)
3)
Review automation progress
Reprioritize any existing test cases as needed
Review test cases prioritized for the current iteration
Automation Release
meeting (ARM)
Every other week
Following
Development
iteration releases.
1)
2)
3)
Determine velocity and burn down.
Determine current workload.
Synchronize scripts for release.
Automation Acceptance
Ad Hoc, as and
when required.
1)
Synch up with QA analyst and get feedback on the behavior and
functionality of the script.
Mark the script for elevation in the daily run of automated
regression Testware.
a. Overview of functionality
b. Determine automatability
c. Determine dependencies (script modification, new
development etc.)
d. Change Automation Status as needed
e. Assign automation developer to analyze the test plan and
create a task in Automation Tracker.
2)
Property of Progressive Insurance &
Casualty Company
8
To Prod
Release
Check-in
Source
Control
Build Proc
Build
Dev Env
QA Env
Nightly and Ad Hoc
Nightly and Ad Hoc
Story Card
Assignment
Unit Test and
Code
Requirements
Synch
Embedded
Testing
Acceptance
Testing
Manual Testing
Development Cycle
Test Plan
Creation
Notes DB
System
Testing
Automation
Prioritization
and Analysis
Automation
Development
Automation
Acceptance
The Automation Cycle
APM
Automation Planning Meeting
for the current iteration
Automation
Assignment
TestDirector
Lab Machines
Automation
Tracker
Property of Progressive Insurance &
Casualty Company
Automation
Script Review
Automation
Release
9
Automation Prioritization
Attribute
Description
Business Risk of Not Automating Test
If one script will be testing a higher-risk feature, it should be assigned a higher
priority.
Number of Planned Executions
If a script will be executed more times, it should be assigned a higher priority.
Efficiency (Automated vs. Manual)
If one script will eliminate more manual testing it should have a higher priority.
Volatility of Features Being Tested
If one script is testing a more volatile feature, it should be assigned a lower
priority, since it will require more revisions due to the volatility.
Dependency of Other Tests
If a high priority task depends on the completion of the candidate task, then the
candidate task should also be assigned a high priority.
Reuse Potential
Scripts which can be reused should have higher priority than one-off scripts.
Open Defects
Scripts which test an open defect should be prioritized according to the
severity/priority of the defect.
Automatibility
If a test cannot be automated it should receive a low priority (at best). If it will
take significant effort to automate, especially in relation to the benefit to the
automation, it should also receive a low priority.
Usability
If a script tests usability as opposed to proper functioning of the application, it
should receive a lower priority.
Property of Progressive Insurance &
Casualty Company
10
To Prod
Release
Check-in
Source
Control
Build Proc
Build
Dev Env
QA Env
Nightly and Ad Hoc
Manual Testing
Nightly and Ad Hoc
Story Card
Assignment
Unit Test and
Code
Requirements
Synch
Embedded
Testing
Acceptance
Testing
Test Plan
Creation
Notes DB
System
Testing
Automation
Prioritization
and Analysis
Automation
Assignment
Automation
Development
TestDirector
Lab Machines
Automation
Acceptance
The Automation Cycle
Automation
Tracker
Property of Progressive Insurance &
Casualty Company
Automation
Script Review
Automation
Release
11
Model based test script design
Model : description of a system’s behavior
Simpler than the actual system
State modeling is a common practice in
embedded systems world
 Finite State Model : consists of a set of states,
a set of input events and the relations between
them.
 Generate Sequences of Test Actions from the
FSM

Property of Progressive Insurance &
Casualty Company
12
Model based test script …contd…
Application behavior model
 Exploratory modeling
 Walkthroughs
 Paired exploration
Capturing states of the application
 When is this action possible?
 What is the outcome of this action?
 Automated test action sequences

Property of Progressive Insurance &
Casualty Company
13
Automation Steps
1)
Test Plan Analysis


2)
Script design and development








3)

Use the data table of QTP for positive data parameterization.
Repeat the parameterization for negative data.
Checkpoints

5)
Modularize your script based on the different states or state transitions your application
Keep a single flow either positive or negative in each action that you record.
Return back to the initial state in your script.
Record the flow you are automating.
Try to add all the objects on a particular page from the active screen.
Use smart identification or vary the object recognition properties if required.
Playback your script
Repeat these steps for the negative flow
Parameterization

4)
Analyze and refactor the manual test plan.
Group similar test conditions.
Use custom and built in checkpoints
Script Elevation


Get your script acceptance tested and reviewed
Property of Progressive Insurance &
Elevate it to Test Director Casualty Company
14
To Prod
Release
Check-in
Source
Control
Build Proc
Build
Dev Env
QA Env
Nightly and Ad Hoc
Manual Testing
Nightly and Ad Hoc
Story Card
Assignment
Unit Test and
Code
Requirements
Synch
Embedded
Testing
Acceptance
Testing
Test Plan
Creation
Notes DB
System
Testing
Automation
Prioritization
and Analysis
Automation
Development
Automation
Acceptance
The Automation Cycle
Acceptance
Automation acceptance
between QA and automation
developer
Automation
Assignment
TestDirector
Lab Machines
Automation
Tracker
Property of Progressive Insurance &
Casualty Company
Automation
Script Review
Automation
Release
15
To Prod
Release
Check-in
Source
Control
Build Proc
Build
Dev Env
QA Env
Nightly and Ad Hoc
Manual Testing
Nightly and Ad Hoc
Story Card
Assignment
Unit Test and
Code
Requirements
Synch
Embedded
Testing
Acceptance
Testing
Test Plan
Creation
Notes DB
System
Testing
Automation
Prioritization
and Analysis
Automation
Development
Automation
Acceptance
The Automation Cycle
ARM
Automation release meeting
between automation
developers
Automation
Assignment
TestDirector
Lab Machines
Automation
Tracker
Property of Progressive Insurance &
Casualty Company
Automation
Script Review
Automation
Release
16
To Prod
Release
Check-in
Source
Control
Build Proc
Build
Dev Env
QA Env
Nightly and Ad Hoc
Manual Testing
Nightly and Ad Hoc
Story Card
Assignment
Unit Test and
Code
Requirements
Synch
Embedded
Testing
Acceptance
Testing
System
Testing
Test Plan
Creation
Notes DB
Automation
Prioritization
and Analysis
Automation
Assignment
Automation
Development
TestDirector
Lab Machines
Automation
Acceptance
The Automation Cycle
Automation
Tracker
Property of Progressive Insurance &
Casualty Company
Automation
Script Review
Automation
Release
17
Roles and Responsibilities
QA Analyst
1)Mark automation status pending after
the test plan is created.
2)Prioritize the test plan with respect to
need for automation
3)Review and walkthrough the
prioritized test plan to the automation
developer.
4)Notify the automation developer
working on the test plan if there are any
changes.
5)Provide feedback to the automation
developer after the automation status is
“Ready for Release”
Automation Developer
1)Pick up test plans that says “Creation
complete” prioritized and assigned.
2)After picking up the test plan change
Automation Status to In Process and
Update Automation Comments field with
name/time stamp.
3)Estimate and define task(s) in
Automation tracker (task planning
system)
4)Create automation script.
5)Synch up with QA analyst and get
feedback on the behavior and
functionality of the script after
developing the script.
6)Mark the test plan as ready for release
after getting it reviewed.
7)Release the test script into the daily run
of “Automated Regression Testware” and
mark automation status as completed.
8)Facilitate automation ceremonies.
Property of Progressive Insurance &
Casualty Company
Application Developer
1)Keep automation team in loop while
making UI changes.
2)Help automation team with controls
and technical details of the app as and
when required.
18