EFFECTIVE TEST AUTOMATION
DEBT STRATEGY AND
PAGE/SCREEN DESIGN PATTERN
USING DATA-DRIVEN TESTING
PARADIGM FOR AUTOMATED
WEB/MOBILE APP TESTING
Michael Burnside
Tw itter : @ SQ AE vangel ist
B l og: http:/ / sqaevangel ist.bl ogspo t.c o m /
Website: http:/ / sqaevangel ist.c om /
Softw are Qu ali ty A ssu rance, Qu ali t y Engi neeri ng, and Web and
Mob i le Test A u tomati on Exp ert
02 May 2014
Effective Test Automation Debt Strategy and
Page/Screen Design Pattern
Table of Contents:
1. Mission Statement
2. Automated testing innovations and pros/cons
3. Decisions, Decisions….Which Technology to choose for
automated testing investment?
4. Example of simple mobile app testing for iOS/Android apps.
5. Architecture
6. Test Technology Stack
7. Class diagrams
8. Run the tests demo
9. Q & A, Feedback
Effective Test Automation Debt Strategy and
Page/Screen Design Pattern
Mission Statement
The purpose of this presentation to identify test automation
concepts, technology and design options while offering a learning
example using a mobile application.
Effective Test Automation Debt Strategy and
Page/Screen Design Pattern
Automated Testing Innovations: Pros/Cons (mobile)
1.
•
•
Native language: ie, using the platform’s recommended
programming language and test harness.
Pros: less dependency on third party libraries and support for
each API release immediately. Minimal refactoring.
Cons: less flexibility and options possibly for test result reporting,
trend analysis, and usually needs more “testability” from the
actual app to be interacted with from the test app., requires test
automation developers to proficient with code used (how hard
would it be to find a QA person with Objective-C skills for iOS
apps testing?)
Effective Test Automation Debt Strategy and
Page/Screen Design Pattern
Automated Testing Innovations: Pros/Cons (mobile) – Part 2
2. Expressive language/keyword-driven: ie, using a proprietary
expressive language to send directives to a mechanism that can
interact with cross-platform UI’s and simulate human interaction use
cases and use assertions.
• Pros: Usually a small amount of training to learn and use
language for testing. Possible contribution of non-technical
resources to add tests. Some ability to test cross-platform. No
testability changes on the app needed.
• Cons: dependency of technology on software development API’s
and OS’s which change rapidly and can break existing test
automation. Example: Calabash
Effective Test Automation Debt Strategy and
Page/Screen Design Pattern
Automated Testing Innovations: Pros/Cons (mobile) – Part 3
3. Data-Driven: ie, using many popular programming languages to
interact with apps rendered on multiple platforms with lightweight
software objects that interact with single mobile app screens and can test
using simple workflows from files like Excel, java properties (.ini files), CSV.
• Pros: tests. Ability to test cross-platform using the same code with
framework supporting simple abstractions. Minimal effort to test
another app on a different platform. Non-technical resources can
create their own test testes for their test environments with their own
accounts. No testability changes needed.
• Cons: dependency of technology on software development API’s and
OS’s which change rapidly and can break existing test automation.
Usually need at least one skilled programmer on the QA team to
facilitate the testing and have some object-oriented design knowledge.
• Example: Appium
Effective Test Automation Debt Strategy and
Page/Screen Design Pattern
Decisions, Decisions….Which Technology to choose for automated testing
investment?
What are the best decisions for my company organization considering the following information:
1)
2)
3)
4)
5)
6)
Do I want to pay for a product that allows for test automation for my app?
What are the investment risks?
When is my timeframe for X amount of code coverage?
What is the acceptable time/money investment for maintenance?
Am I comfortable with the dependencies on OS/API/Test harness dependencies?
Do I have enough technically capable resources on staff/available to invest a strategy for long term maintenance investment?
There are no right or wrong answers. Make the decision based on what you want to accomplish for
short/long terms goals, or both. Regardless of what you choose, there is ALWAYS time/money
investment in test automation.
Effective Test Automation Debt Strategy and
Page/Screen Design Pattern
Example of simple mobile app testing for iOS/Android apps
Let’s investigate a simple login application that is identical for iOS
and Android mobile devices using the screen object design pattern
using an entirely data-driven set of tests (no hard-coded tests).
Desired:
1) Design and implement a set of automated tests using simple
object-oriented design patterns that are directed and validated
for acceptance criteria using data-driven files.
2) Easy capability to extend capability of testing the same app on a
different mobile device OS.
3) Easy maintainability for lowest future cost and time effort.
Effective Test Automation Debt Strategy and
Page/Screen Design Pattern
Architecture:
There are 4 basic components to the automated testing solution:
1) Screen interfaces and implementation classes – interaction with the actual mobile app
happens here. 3 rd party api calls happen here.
2) Workflow classes – navigate from point A to point B in the application, execute a
consistent set of operations
3) Base Screen and Test classes – utility encapsulating many widely-used methods in the
language of choice and 3 rd party method simplification.
4) Base functional test and screen classes per feature. Allows for customization of testing
using separate accounts/users/data environment setup.
5) Test implementation classes using data-driven framework to parse files in CSV, Pipedelimited, CSV
Effective Test Automation Debt Strategy and
Page/Screen Design Pattern
Test Automation Technology Stack:
1.
Java
2. Junit
3. TestNG
4. Appium
5. Web/Mobile Test automation framework (can be downloaded from SQAEvangelist.com –
open source)
6. Xcode (iOS)
7. Android Studio ADT
Effective Test Automation Debt Strategy and
Page/Screen Design Pattern
1) Create a single interface class for each screen that can be used
for both iOS app and Android applications.
2) Create separate implementation classes (one for ios and other
for android)
3) Create a single workflow that takes a parameter that indicates
which screen implementation class to use. Both testing classes
use the same workflow.
4) Data files drive the tests and can test positive and negative test
cases.
Effective Test Automation Debt Strategy and
Page/Screen Design Pattern
Example/Demonstration
Effective Test Automation Debt Strategy and
Page/Screen Design Pattern
Thanks!
I sincerely hope this presentation was useful to you and that your
company or organization implement the information and processes
in this presentation to the fullest extent.
If you have any feedback, please feel free to get in touch with me
(via Twitter) at @SQAEvangelist. I would love to hear very
constructive comments on how to improve this presentation and
make it more effective.