View Point
Developing a successful Point-of-Sale
(POS) test automation strategy
- Sujith George
Abstract
While Test Automation has been around for a while, QA teams in the retail industry are still struggling to extend it successfully
to test POS systems because of certain inherent characteristics of these systems. The following document attempts to help QA
teams understand the challenges involved in automation of POS testing and how they can be overcome by developing effective
comprehensive strategy.
www.infosys.com
Organizations view test automation
as a strategy to reduce timelines,
cut costs and improve quality.
While this is true, it is also important
to note that before we reap the
benefits of automation we need to
make significant investments and
not everything can be automated.
ROI from automation depends
on Automation Coverage, i.e.,
percentage of test scenarios that have
been automated. With POS Systems
accounting for a major portion of a
Retailer’s IT spends, automating POS
testing is something that cannot be
ignored. It is something that most
Retailers want to undertake, but
hesitantly, owing to the ambiguities
surrounding POS Test Automation.
Testing of POS applications is
unlike other types of testing.
While most applications can be
tested using a desktop, testing of
POS applications requires a CashRegister which usually comes with
a set of other peripheral devices
such as receipt-printers, bar-code
scanners, cash-drawers, pin-pad
devices, swipe-readers etc.. Usage
of these peripherals requires human
intervention and this necessitates
the physical presence of the tester.
The POS devices come in a widerange of shapes and sizes: legacy
POS devices with limited/no-display,
to modern POS devices with touchscreen UI; stationary terminals to
wireless handheld devices; StorePOS to Restaurant-POS. It, therefore,
naturally follows that automation
of POS testing too, is different from
traditional automation. Based on
our extensive experience from
engagements involving testing of
POS applications, we have identified
a few factors which make POS testing
a challenge.
2 | Infosys – View Point
1
Human and Peripheral Device
Interaction
The majority of the POS scenarios such as scanning a bar-code,
swiping cards, pin-pad-entry, opening and closing of cashdrawer etc. involve peripheral devices, which require human
intervention. Such scenarios are typically difficult to automate
using a tool and hence the extent of automation is lowered to
15-20% of the test scenarios. Consequently realizing ROI takes
much longer.
2
Non-Standard UI Objects
In order to provide users with a vastly improved user experience,
today’s POS systems come with thick-client applications and
touch-screen UI. The systems are further customized extensively
through use of non-standard objects. In fact certain objects
are created specifically for the application, e.g. screen-shots
of receipts. However, the flip-side is that most of these nonstandard objects cannot be recognized by standard automation
tools, thereby making it difficult to automate the POS system.
3
Dynamic UI
POS software, whether custom-built or bought off-the-shelf,
invariably ends up being customized to suit the retailer’s specific
needs. The UI is often highly dynamic to allow it to cater to
ever-changing business needs. Further business processes are
frequently modified and the cost and time required to maintain
an automated regression test suite increases steeply and in some
cases makes it infeasible to maintain one.
4
Multiple Interfaces and Complex
Configurations
POS applications typically interface with a number of
external systems, both front-end and back-end (such as CRM,
Merchandising systems, Sales Audit, E-Commerce etc.). These
external systems in turn could also be third-party systems
built on a wide range of platforms. Most POS vendors end
up having multiple versions/formats of POS hardware and
software. Maintaining multiple versions of automated scripts for
incremental changes in code and configuration is infeasible.
Strategy to build
a Robust, Scalable and Re-usable
POS Test Automation Suite
Feasibility Analysis
TECHNICAL FEASIBILITY
Carrying out a Proof of Concept (PoC) on the application
architecture under consideration helps determine the feasibility of
automation. The PoC should focus on the following aspects:
• Technical architecture of the POS Application: understanding
the technical architecture of the POS application is crucial
at the PoC and tool selection stage. The applicability of an
automation tool and even the choice of tool adapters and
add-ins depends on the underlying technical architecture
and communication protocols of the application under test.
• Evaluation and comparative analysis of automation tools:
Tools differ in their features, advantages and disadvantages.
If time permits a comparative analysis of two or more tools
should be carried out to evaluate the tools for fitment. A
scorecard based comparative analysis ensures short-listed
tools are quantitatively evaluated against criteria which
carry specific weightages. POS systems, by virtue of being
thick-client applications, are typically customized a lot.
Therefore the evaluation process should also include a check
of whether the tool is able to recognize non-standard UI
objects employed by the POS system under test.
• Understanding of the application: A high level understanding
of the application not only helps create robust automation
scripts but also modularize them. Commonly used actions/
functions can be scripted as Keywords and re-usable
Business Process Components.
• Selection of scenarios for PoC: During the PoC scenarios
selected for automation should be chosen such that they
are representative of possible technical challenges that the
QA team may encounter during the automation of the POS
testing.
FINANCIAL FEASIBILITY
Before we undertake automation we must assess the financial
feasibility of the initiative. While assessing the financial feasibility
of the automation initiative we need to consider ROI (Number of
regression test cycles which need to be executed to break even)
and include investments which would need to be made as part of
the long term strategy.
The ROI we are able to extract depends on the extent to which we
can automate the test processes. A traditional approach to POS
automation, sees QA teams typically realize ROI in ~20 releases.
Infosys – View Point | 3
However, this can be accelerated by extending the automation
coverage and adopting a ready-made POS automation framework.
In fact the automation framework should be developed in such
a manner that it includes reusable business process for POS
systems. In this manner we can not only eliminate the time and
effort required to build a framework from scratch but also reduce
significantly the time required to develop scripts for functional
scenarios.
and re-usable automation suite. Investment considerations should
include:
The financial feasibility analysis should include long term
investments which need to be made to ensure a robust, scalable
• Simulators: Most POS scenarios involve humans interacting
with peripheral devices. Wherever possible such actions/calls
should be simulated to ensure scenarios are tested end-to-end.
• Procurement of the right automation tool with the required
add-ins. The add-ins help overcome technological obstacles
and ensure compatibility between the automation tool and
the POS applications. Upgrading POS Registers: the memory
of POS Registers, which need to host the automation tool,
needs to be upgraded regularly
Process & Methodology
KEYWORD DRIVEN FRAMEWORK
Advanced techniques such as a Keyword driven approach, rather
than the traditional Record & Replay approach, enable high degree
of modularization and promotes reuse. Common POS functions
such as ‘item lookup’, ‘price lookup’, ‘credit-authorization’, ‘void’ etc.
can be scripted as re-usable functions / keywords and invoked by
multiple scripts addressing different test scenarios. Importantly
modularization also enhances the maintainability of test scripts.
DATA PARAMETERIZATION
Hard-coding of data into scripts makes them difficult to maintain a
robust automation script should be data driven and designed such
that test data can be supplied and manipulated at run time. This
not only ensures flexibility at the time of executing test scripts but
also ensures better quality of testing.
EARLY AUTOMATION
Choosing an automation framework which supports early
automation helps improve the ROI. Traditional automation
frameworks require the application-under-test to be fully
developed and stable for scripting to start, and therefore only
regression testing can be targeted for automation. On the other
hand, a framework built on the principles of Early-Automation
can enhance the scope of automation and in turn help realize
ROI faster. For example, a framework can be used to define
Business Processes and validation rules in terms of models that
are abstracted from the underlying scripting technology. Such
4 | Infosys – View Point
models can be created before the application is functionally
stable. Once the application stabilizes and is available, automation
scripts can be generated from these models. Effectively this
allows test automation to begin as early as the application build
stage itself.
SCOPE OF AUTOMATION
It is important to identify the right scenarios for automation. It
would be a waste of time, effort and money to automate scenarios
which will not be executed often during successive test cycles. A
Scorecard based approach helps identify the right scenarios for
automation. The approach, assigns scores) to scenarios (on a scale
of 1-n) against pre-determined evaluation criteria. The criteria
themselves carry differing weightages. Scenarios/transactions that
score higher than a threshold value are selected for automation.
Typical transactions that can be automated are:
• Sales / Return / Exchange for different tender types like Cash,
Debit, Credit, Gift and Retailer Cards
• Price change, Discounts, Promotions, Capture of Customer
Information
• Merchandise Locator, Price Lookup etc
TEST MANAGEMENT
Integrating the automation suite with test management tools such
as HP Quality Center, IBM Rational Test Manager etc. allows us to
leverage the UI features of the test management tool. This makes it
easy to understand and report the test execution status.
Environment & Infrastructure
The test environment, data and infrastructure used for POS Test
automation play an important role in ensuring the quality of
testing and should ideally be a scaled down version of a real-time
store. Unlike a manual tester, an automation script does not have
the intelligence to verify the validity of the test processes. The test
data setup for automation too should be absolutely error free. The
following points should be given due importance to ensure what is
being automated is a true reflection of real-life business scenarios.
TEST ENVIRONMENT
Ensure same hardware, technology, platform used in the production
environment is made available for testing activities too. Reserving
a few registers exclusively for developing the automation suite
helps expedite the process and improve the quality of scripts. It
is also difficult to utilize the same set of registers for both manual
and automation testing due to the limited memory of the registers
which is used up during the installation of the automation tool.
DATABASE BACKUP
Over the course of multiple test cycles data integrity is affected
adversely and it is necessary to restore the database to a previous
baseline. Hence maintaining a backup of the database at regular
checkpoints is essential.
TEST DATA
Manual testing, which includes exploratory testing, usually affects
test data which gets altered during the course of the test execution.
Hence it is important to maintain separate test databases for
manual and automated testing.
CODE VERSIONING AND RELEASE MANAGEMENT
To ensure right scripts are executed on right environment & right
version of the application code
Tools & Accelerators
TOOLS EVALUATION AND SELECTION
Automation tools constitute a significant portion of the
investments and one needs to ensure that the tool being procured
is compatible with existing technology of POS systems. Hence it
is important to evaluate the automation tool vis-à-vis application
Architecture and, Technology spectrum.
The automation tool needs to be compatible with the technology
platform on which the POS system is running`. We need to pay
special attention to the following:
• 32-bit or 64-bit: Automation tools for Windows are generally
built to work with either 32-bit or 64-bit applications. Some
are even designed to work with both. It is important to
understand whether the POS Application-Under-Test (AUT)
is a 32-bit or a 64-bit application and accordingly determine
the suitability of a tool. For example, earlier versions of QTP
are known to support only 32-bit applications, whereas
newer versions support 64-bit applications too (with some
limitations).
• Operating Systems (OS): OS manufacturers are known to
supply versions of OS that are tailor-made to support a
specific POS system (e.g. Windows Embedded OS for POS).
Such customized OS incorporate drivers and libraries specific
to the POS hardware. While selecting an automation tool we
need to ensure it is compatible with the specific OS.
• Touchscreen UI: Automation tools sometimes fail to recognize
objects used in modern POS systems with touchscreen UI
features. For example, touchscreen UI (developed using
Java Apps Support Framework), when spied using QTP, is
displayed as default WinObject with ‘Standard Properties’ set
to Null (including x- and y-coordinates). Such touch screen
UI can be automated using tools which simulate object
methodology (object region is mapped to standard objects
like Button, Textbox, Checkbox, Radio etc.).
Infosys – View Point | 5
ACCELERATORS
peripheral devices attached to the POS terminal. Alternatively
we would use ready-made platform-independent POS
automation frameworks too. Besides increasing automation
coverage in general, these workarounds can save time, cost
and effort.
The process of automation itself can be expedited using
accelerators.
• Technology Accelerators: These can either be technical
workarounds to help the automation tool recognize nonstandard UI objects, or approaches/methodologies to simulate
• Domain Accelerators: Pre-built knowledge of POS domain in
the form of libraries of critical POS business processes.
Conclusion
A one size fits all approach prevents successful automation of POS systems. Not only is testing
POS systems different from other types of testing, it also varies from one POS system to the
other. Hence to successfully automate the testing of POS systems one requires a deeper
understanding of the POS-specific challenges which QA teams may encounter and the
factors which need to be considered while automating the test process. These challenges
can be overcome by adopting a comprehensive strategy which focuses on selecting the
appropriate tools, accelerators and a structured process.
6 | Infosys – View Point
About the Author
Sujith George, Senior Project Manager, Infosys
Sujith George is a Senior Project Manager at Infosys’ Independent
Validation and Testing Services practice. With over 12 years of
experience, he has managed large test engagements for clients
from the retail industry and has helped them define and implement
productivity improvement initiatives. Sujith has a keen interest in
Point-of-Sale (POS) Testing and has developed the test strategy for
several POS Testing engagements.
Infosys – View Point | 7
About Infosys
Many of the world's most successful organizations rely on Infosys to
deliver measurable business value. Infosys provides business consulting,
technology, engineering and outsourcing services to help clients in over
30 countries build tomorrow's enterprise.
For more information, contact [email protected]
www.infosys.com
© 2012 Infosys Limited, Bangalore, India. Infosys believes the information in this publication is accurate as of its publication date; suchinformation is subject to change without notice. Infosys acknowledges
the proprietary rights of the trademarks and product names of other companies mentioned in this document.