Tool Look
This article is provided courtesy of STQE,
the software testing and quality engineering magazine.
QUICK LOOK
■ Good at capture/playback
and scripting automated tests
■ Useful features that
aren’t documented
N
A Look at
Rational SQA Robot
by Noel Nyman
oel Nyman continues sharing his experiences of working in the
Microsoft WindowsNT Group, where he evaluated several
automation tools for the Applications Test team. That team uses
retail workstation applications to test
the version of WindowsNT under
development. He used the tools to
create tests that ran across a variety
of hardware and system configurations, while ignoring features that
weren’t useful to the team. The opinions expressed here are his alone, and
they are neither an endorsement nor
critique of the products by Microsoft.
This is the second installment in a
series.
Product Summary
Rational SQA Robot (formerly SQA
Robot, Rational Software, http://www.
rational.com/) combines capture/playback script generation with a scripting
language. Rational SQA Robot is part
of the Rational Suite of products. Oth-
er Suite tools manage the common
database, create reports, create and
track bugs, and load test client/server
applications. I tested Rational SQA
Robot version 6.1.
Capture/Playback
vs. Scripting
Like most GUI automation tools,
Robot is oriented toward recording
keyboard and mouse movements to
create tests. The application being
tested must run with the same initial
conditions to replay recorded tests
successfully. NT Applications Test
runs automation on many machine
platforms with different screen resolutions. Although Robot documentation indicates that scripts will play
back successfully regardless of
screen resolution, we had to add code
to recorded tests so they could run on
all those platforms without reporting
false errors. So, my evaluation looked
at both Robot’s recording features
and scripting capabilities.
FIGURE 1 SQA Test
Log Viewer
63
www.stqemagazine.com
Software Testing & Quality Engineering
March/April 1999
This article is provided courtesy of STQE, the software testing and quality engineering magazine.
Recording a Test
Robot’s test cases are comparisons on windows, objects,
the contents of the Windows clipboard, menus, file existence, file contents, or even user-defined .dll and .exe calls.
The test cases are recorded with the properties of the
objects such as size, position, state text, and class as baseline data. The baseline and the data from later test runs can
be examined in the Test Log Viewer in case of failures.
Robot tries to identify controls as objects and
records mouse click coordinates only as a last resort.
When the tester clicks on an object that Robot doesn’t
recognize, the Define Object dialog appears. Visual
SlickEdit (MicroEdge, http://www.slickedit.com/) uses a
custom “vs_control” class for controls that look like Windows check boxes. When I clicked on one of them, Robot
did not recognize it as a “check box,” although Robot correctly associated the adjacent text label with the control. I
used Define Object to tell Robot that this “vs_control” is a
check box.
SlickEdit also uses the “vs_control” class for its option
buttons and command buttons. Other automation tools try
to “Check” or “UnCheck” those controls once the class is
associated with check boxes. Those commands are meaningless to SlickEdit’s option and command buttons and can
cause errors. Robot doesn’t have that problem because its
only CheckBox command is Click. I recorded a test in a dialog containing a check box, an option button named Landscape, and the OK button. Here’s the script Robot created:
The editor is simple and effective with no frills. There
is an Undo command (missing in some tools), but no color coding for keywords, no syntax checking for errors
(until compile time), and no context (right click) menu.
The scripting language has many commands, some of
which seem unnecessary. How often will you use arc tangent in a GUI test script? There’s a full set of looping
structures, file control and I/O, some DDE and ODBC
commands, and even financial functions. Yet the Windows
control functions, the commands we’d use most often for
GUI testing, are sparse. Although Robot can identify the
state of a check box, there’s no way to set the state,
except the catchall Click command. A tester could use
If/Then statements to Click on the box only if it’s
UnChecked. That takes three lines of code, compared to
the single command available in other tools.
Robot has no “menu exists” function. I use that capability in other automation tools to test apps that change
their menus in different environments. In some tools I
can simulate “menu exists” by creating a special error
handler routine. If my tests try to execute a menu that’s
not available, they trigger the error handler. It checks for
the “couldn’t select that menu” error message and handles
it instead of generating a failure. In Robot, menus can
only be examined as Robot test cases, and test case failures can’t be trapped by an error handling routine. So, my
usual work around can’t be implemented in Robot.
Compiled Include Libraries
CheckBox Click, "Text=Print Color Coding"
CheckBox Click, "Text=Landscape"
CheckBox Click, "Text=OK"
The right things happen during playback, even though
the option and command buttons aren’t check boxes. The
lack of control methods can be a problem in scripting (see
“Creating a Test Script” later).
I created tests for the WinEdit (Wilson WindowWare,
http://www.windowware.com/) toolbar, which can be “torn
off” the parent to float. Most of the automation tools I evaluated could float the toolbar, but they either lost track of it
at that point or were unsuccessful at docking it. Robot
recorded tear-off and docking, and replayed them successfully.
Test Logs
After a test runs, Robot displays an error log. The “Pass”
and “Fail” markers identify test cases. The tester can create the optional Log Messages while recording tests. If an
unexpected dialog appears during replay, Robot logs a
“Warning” and tries to close the dialog using the Escape
key (see Figure 1 on previous page).
Creating a Test Script
“From Scratch”
There’s no “New” option on Robot’s File menu. All scripts
must start from the recorder. Robot neither encourages
nor enforces a structure on tests. All executable code is
recorded in a single subroutine called “Main,” which gets
long and complex if you record long test sequences. You
can manually make parts of Main into functions and subroutines to help control the clutter.
Robot contains a mechanism for “including” code and
variable definitions from common files. Each include file
must be prototyped in a header file. It can be implemented in SQABasic or as a Windows DLL. This feature makes
Robot potentially very useful because testers can use it to
call Win32 API functions from Robot scripts. Unlike most
of the other tools I’ve evaluated, Robot supports structures. So, you can call the really interesting Win32 functions.
Significant as it is, this feature is not described anywhere in the Robot documentation. The only clue about
how to use it comes from SQAUtil.sbl, an include file supplied with Robot. It’s used to make calls into a special
Windows DLL to get the machine name and other information from the operating system (OS). Using that SBL
file as a guide, I created a simple Robot script that calls
the Win32 function GetVersionEx to fill a structure with
OS version information. Robot passes values by reference
by default (not documented), so no cast to a pointer is
needed. The syntax for declaring a fixed-length string is
also undocumented.
Type VersionInfo
dwOSVersionInfoSize
As Long
dwMajorVersion
As Long
dwMinorVersion
As Long
dwBuildNumber
As Long
dwPlatformID
As Long
sCSDVersion
End Type
As String*128
Declare Function GetVersionExA Lib "kernel32.dll"
(OSVersionInfo as VersionInfo) as Long
64
March/April 1999
Software Testing & Quality Engineering
www.stqemagazine.com
This article is provided courtesy of STQE, the software testing and quality engineering magazine.
This Robot subroutine logs the values for OSVersionInfo
using my custom LogOut function.
Sub Main
Dim Result As Long
Dim OSVersionInfo As VersionInfo
OSVersionInfo.sCSDVersion = String(128, Asc("*"))
OSVersionInfo.dwOSVersionInfoSize = Len(OSVersionInfo)
Result = GetVersionExA(OSVersionInfo)
LogOut "MajorVersion:
LogOut "MinorVersion:
LogOut "BuildNumber:
LogOut "PlatformID:
LogOut "CSDVersion:
" & Str(OSVersionInfo.dwMajorVersion)
" & Str(OSVersionInfo.dwMinorVersion)
" & Str(OSVersionInfo.dwBuildNumber)
" & Str(OSVersionInfo.dwPlatformID)
" & Str(OSVersionInfo.sCSDVersion)
End Sub
Calling Win32 API functions is valuable, but not cheap
to implement in Robot. There’s no equivalent to Rational
Visual Test’s winapi.inc, which prototypes most of the
Win32 functions. You’d have to make and maintain your
own header file.
Final Analysis
Rational Robot does a good job for both recording and
scripting most test automation. It works with several other
Rational products that provide load testing, bug tracking,
and test case planning. As with most tool suites, you get the
best results if you use all of the features. If you don’t need
some parts of the Rational tool suites, the integration is less
important to you, as it was for us. Then the strengths and
weaknesses of the individual tools, like Robot, are more significant. STQE
Noel Nyman has worked in software product development
and testing for over 20 years on an eclectic project mix
including embedded controllers, shrink-wrap applications, and operating systems. “He tests, therefore he is.”
STQE magazine is produced by STQE Publishing, a division of
Software Quality Engineering.
65
www.stqemagazine.com
Software Testing & Quality Engineering
March/April 1999