Making UAT More
Effective
Jim Trentadue
Software Quality Consulting Director
Original Software
[email protected]
www.origsoft.com | © Original Software
1
Agenda
1
UAT in IT
2
Defining UAT within the overall test effort
3
Supporting UAT from an IT perspective
4
Enabling UAT Planning
5
Enabling UAT Execution
6
Session Recap
www.origsoft.com | © Original Software
2
UAT in IT
www.origsoft.com | © Original Software
3
UAT in the IT Industry
Role in the various SDLC methodologies
Waterfall
Agile
www.softwaretestingsoftware.com
Plan UAT
• Define Scope
of UAT
Identify
UAT
Scenarios
Execute
UAT Tests
• Review
• Execute &
Business Reqs, Document
Use Cases and Results
UI Design
• Log UAT
• Document UAT Defects
Test Scenarios
Verify UAT
defect fixes
Report UAT
results
UAT
Signoff
• Rerun impacted • Go / No Go Call
UAT Scenarios
Test Cases?
www.origsoft.com | © Original Software
4
Automation for everyone – even UAT
Shift-Left to Reduce Errors, Shift-Right for Better Quality
Shift Right!
Skillset Spectrum
More Technical,
More Specialists
Pro’s
Con’s
Cheaper or
free software
Significant
investment to
implement
Better-known
in market place
Requires
specialized
resources
Project
Team Goal:
Do Testing
Better!
Dev, DBA, QA Specialist
No Coding Skills
Required
Pro’s
Con’s
Increased
utilization of
automation
Higher entry
point cost to
buy or build
Automation
done earlier by
everyone
Potentially
longer to
implement
BA, Functional QA, UAT
www.origsoft.com | © Original Software
5
Defining UAT
www.origsoft.com | © Original Software
6
Defining UAT – Testing Assessment
Risk Assessment
First, understand the risk and strike a proper balance
Is there an SQA team?
Independent QA team
executes system tests?
Yes
No
Is UAT the only active
testing group?
Ensuring we have
different roles!
www.origsoft.com | © Original Software
7
Defining UAT - Dividing Responsibilities
Understanding Breadth vs. Depth testing approaches
LOGIN
MENU
ORDER
INVENTORY
PROCESSING
CONFIRMATION
LOGIN
• Login as Manager
• Login as Supervisor
• Login as User
(Full access)
(Administrative access)
(Limited access)
MENU
• Access order A
• Access order A
• Access order A
(Requires full or admin access)
(Should be able to access)
(Negative test – should not be able to access)
ORDER
• Process order A
• Process order A
• Process order B
(Does not require full or admin access)
www.origsoft.com | © Original Software
REPORTS
8
Supporting UAT
www.origsoft.com | © Original Software
9
Responsible parties for UAT
Who is responsible for a successful UAT initiative
Successful UAT with minimal disruption to the business
www.origsoft.com | © Original Software
$$$
Business Impact
External Customers
UAT Lead
Environments
IT Operations
Testing Department
Developers
Coordination
and Training
Business Users
EVERYTHING!
Usability
Business Analysts
Project Managers
Business
Schedules
Business knowledge
and Training Guides
10
Testing Department support for UAT
EVERYTHING!
Requirements
Stories
Design
Prototype / Demo
Test Plans / Cases
Test Summary
Retrospectives
Defects
Planning
Test Strategy – UAT part
Acceptance Test assist
Test Data approach
Training outline
Execution
Strategy
Backlog
Status
Test Results
Defects
Product Readiness
Scenario sharing
Environment preparation
Test Data setup
Defect retest
www.origsoft.com | © Original Software
11
Key questions & concerns
Is this just an IT problem?
Enough business
interest to allot
time?
Who is
accountable for
success?
How can UAT
improve?
Technical
Enablement
www.origsoft.com | © Original Software
12
Enabling UAT Planning
www.origsoft.com | © Original Software
13
Enabling UAT Planning: Estimation
How should UAT be estimated for planning & execution?
Estimation
Current Situation
Fraction of
time taken
from QA
estimate
Plan lacking
for other
business
deadlines
Improvement
Systematically
from UAT
regression
test runs
www.origsoft.com | © Original Software
14
Enabling UAT Planning: Test Assets
How much document assistance does SQA provide?
Test Assets
Current Situation
Re-create
UAT scripts or
take existing
SQA tests
Commitment
from UAT staff
on completing
testing tasks
Improvement
Maintain test
repository of
UAT-created
artifacts
www.origsoft.com | © Original Software
15
Enabling UAT Planning: Test Mgmt. access
Who coordinates what UAT needs / does in TM solution?
Test Management
Current Situation
Unclear what
access UAT
needs into TM
repository
System tests /
scenarios are
given to UAT
to modify
Improvement
UAT specific
view & access
for relevant
information
www.origsoft.com | © Original Software
16
Enabling UAT Planning: Test Automation use
Who creates / runs the automation and for what purpose?
Test Automation
Current Situation
Test engineer
created tests
based from
system tests
Automation is
executed by
system tester;
validation only
Improvement
Biz process
steps capture
done by UAT,
used by UAT
www.origsoft.com | © Original Software
17
Enabling UAT Planning: Training artifacts
Does UAT have to re-execute to produce training materials?
Training
Current Situation
UAT focuses
on executing
test, not for
training doc
No schedule
allocation for
training docs
production
Improvement
Enable UAT
with markup
feature during
test execution
www.origsoft.com | © Original Software
18
Enabling UAT Execution
www.origsoft.com | © Original Software
19
Enabling UAT Execution: Accountability
Who has it from IT and Business?
Accountability
Current Situation
IT plans 50%+
resource time
for UAT
Coordination
Biz dedicates
colleague for
strictly UAT
Coordination
Improvement
Direct input:
What is done
Who did it
When did they
Where was it
www.origsoft.com | © Original Software
20
Enabling UAT Execution: Testing Status report
Who did it? What did they do? Where did they test?
Testing Status
Current Situation
Frequent
checkpoint
and status
updates
Lack of time,
priority shift;
defect ties
back to SQA
Improvement
Clear visibility
to UAT results
www.origsoft.com | © Original Software
21
Enabling UAT Execution: Continuous Testing
How can we turn this into automatic maintainability?
Productivity
Current Situation
Constantly
re-engineer
test cases
for accuracy
File location
holds test that
are outdated;
maintenance!
Improvement
Continuous
updates to
UAT artifacts;
Shift Right!
www.origsoft.com | © Original Software
22
Test Management access
Test Automation through
Test Management access
Test Management access
EST
Solution Implementation /
Framework Modification
TC
TM
TA
TR
www.origsoft.com | © Original Software
Solution Implementation /
Framework Modification
ACC
Test Management access
Process Review for UAT Planning & Execution
Recap of Better UAT procedures
REP
CT
23
Session Recap
www.origsoft.com | © Original Software
24
Presentation Recap
Key takeaways for UAT effectiveness
Understand where UAT falls within the SDLC and where they are
positioned for Test Automation
Define where UAT is within the Testing organization and how
responsibilities are divided
Outline who are the main parties supporting UAT, especially those
roles within the Testing department
Review the means which UAT Planning can be more efficient,
through solutions or framework build in Test Mgmt. system
Review the means which UAT Execution can be more effective,
through dynamic result reporting and test case maintenance
www.origsoft.com | © Original Software
25
Thank you for attending
this presentation!
Jim Trentadue
Software Quality Consulting Director
Original Software
[email protected]
www.origsoft.com | © Original Software
26