Ben-Gurion University of the Negev
The Software Engineering Program
Access Layers
Automatic Software Testing Tool
for Computer Networks
Testing Document
Team Members
Adi Shachar
Yaniv Cohen
Dudi Patimer
Academic Advisor
Technical Advisor
Dr. Mayer Goldberg
Dan Zamir
Automatic Software Testing Tool for Computer Networks
Testing Document
Table of Contents
1.Introduction
........................................................................................................ 3
2.Functional Requirements ......................................................................................... 4
3.Non-Functional Requirements ................................................................................. 9
4.TDD Development .................................................................................................. 11
5.Random & Automatically Tests ............................................................................... 12
6.Testing the User Interface ...................................................................................... 13
7.Testing build, Integration & Deployment ............................................................... 14
2
Automatic Software Testing Tool for Computer Networks
Testing Document
1. Introduction
This document details our efforts towards testing and debugging the Automatic
Software Testing Tool.
In this document you will find descriptions on how tests is being performed and
executed and how we test the requirements that were presented in the ARD
document.
The document will describe how to test the functional and non-functional
requirements, as well as UI tests and automated & random tests.
In each part there will be the subject we wish to test, the description of the test
(what actions we perform in order to test that subject), and the desired outcome of
the test execution.
3
Automatic Software Testing Tool for Computer Networks
Testing Document
2. Functional Requirements
In this section we specify the tests that verify the function requirements of the
system. These functional requirements are defined in the Application Requirements
Document (ARD).
Functional
Requirement
Test Description
Expected result
Creating a new additional action with
'command line' type and valid attributes.
Create
Additional
Action
Creating a new additional action with
'Script' type, existing script file and valid
attributes.
Creating a new additional action with
'Test Script' type, existing script file and
valid attributes.
Creating a new additional action with
'Batch File' type, existing batch file and
valid attributes.
Creating a new additional action with
existing name
Creating a new additional action with
missing fields (Creator name or Action
name).
Creating a new additional action with
invalid OS Type.
Creating a new additional action without
choosing type.
Creating a new additional action with
invalid timeout (negative number).
Creating a parameter with 'Option' type
and valid attributes.
Creating a parameter with 'Input' type
and valid attributes.
Creating a parameter with 'Option' and
'Input' type and valid attributes.
Creating a parameter with existing name
in this action.
Creating a parameter with missing fields
(parameter name).
Creating a parameter with invalid OS
Type.
Creating a parameter without choosing
type.
Create Test
Scenario
Creating a new test scenario with at least
one action.
4
A new additional action was
created and stored properly in the
database.
A new additional action was
created and stored properly in the
database.
A new additional action was
created and stored properly in the
database.
A new additional action was
created and stored properly in the
database.
A message will be shown to the
user notifying name already exist.
A message will be shown to the
user about the missing fields
needed to be filled
A message will be shown to the
user notifying the error.
A message will be shown to the
user about the missing type.
A message will be shown to the
user notifying the error.
A new parameter was added to
the additional action and stored
properly in the database.
A new parameter was added to
the additional action and stored
properly in the database.
A new parameter was added to
the additional action and stored
properly in the database.
A message will be shown to the
user notifying name already exist.
A message will be shown to the
user about the missing fields
needed to be filled
A message will be shown to the
user notifying the error.
A message will be shown to the
user about the missing type.
A new test scenario was created
and stored properly in the
database.
ARD
Reference
2.1
2.2
Automatic Software Testing Tool for Computer Networks
Creating a new test scenario without
actions.
Creating a new test scenario with existing
name.
Creating a new test scenario with missing
fields (TSC name, Creator name).
Create a new test scenario, select an
action and select parameters for this
action.
Create a new test scenario, select an
action and select end- stations for this
action.
Create a new test scenario, select an
action and select a valid delay for this
action.
Create a new test scenario, select an
action and select a valid duration for this
action.
Create a new test scenario, select an
action and select an invalid delay for this
action.
Create a new test scenario, select an
action and select an invalid duration for
this action.
Create a new test scenario, and select
end-stations for this test scenario.
Creating a new test plan with at least one
TSC.
Creating a new test plan without TSC's.
Create Test
Plan
Creating a new test plan with existing
name.
Creating a new test plan with missing
fields (TP name, Creator name).
Create a new test plan, select a TSC and
select end- stations for this TSC.
Create a new test plan, and select endstations for this test plan.
Edit additional action and saving with a
new name.
Edit
Additional
Action
Edit additional action and saving with
existing name.
Selecting a basic action to edit.
Edit Test
Scenario
Edit test scenario and saving with a new
name.
Edit test scenario and saving with existing
name.
5
Testing Document
A message will be shown to the
user notifying that no action was
selected.
A message will be shown to the
user notifying name already exist
A message will be shown to the
user about the missing fields
needed to be filled
A new test scenario was created
and stored properly in the
database.
A new test scenario was created
and stored properly in the
database
A new test scenario was created
and stored properly in the
database
A new test scenario was created
and stored properly in the
database
A message will be shown to the
user notifying the error.
A message will be shown to the
user notifying the error.
A new test scenario was created
and stored properly in the
database
A new test plan was created and
stored properly in the database.
A message will be shown to the
user notifying that no TSC was
selected.
A message will be shown to the
user notifying name already exist.
A message will be shown to the
user about the missing fields
needed to be filled
A new test plan was created and
stored properly in the database
A new test plan was created and
stored properly in the database
A new additional action was
created and stored in the
database.
The additional action was updated
and stored in the database.
A message will be shown to the
user that a basic action can't be
modified.
A new test scenario was created
and stored in the database.
The test scenario was updated and
stored in the database.
2.3
2.1
2.2
Automatic Software Testing Tool for Computer Networks
Edit Test Plan
Edit test plan and saving with a new
name.
Edit test plan and saving with existing
name.
Delete additional action that doesn't
appear in any TSC.
Delete
Additional
Action
Selecting a basic action to delete.
Delete additional action that appears in a
TSC.
Delete Test
Scenario
Delete Test
Plan
Delete test scenario that doesn't appear
in any TP.
Delete test scenario that appears in a TP.
Delete test plan.
Execute command line single action
without validity string and select exist
end-stations.
Execute script single action without
validity string and select exist endstations.
Execute batch file single action without
validity string and select exist endstations.
Execute test script single action without
validity string.
Executing
Single Action
Execute single action with validity string
that is actually placed in the action
output and select exist end-stations.
Execute single action with validity string
that isn't actually placed in the action
output and select exist end-stations.
Execute single action with delay of 60
seconds and select exist end-stations.
Execute single action on non-exist endstation.
Run a single action with a small
connection timeout.
Run a single action and configure the
PSTools folder to non-exist folder.
6
Testing Document
A new test plan was created and
stored in the database.
The test plan was updated and
stored in the database.
The additional action was
removed from the database
successfully.
A message will be shown to the
user that a basic action can't be
deleted.
A message will be shown to the
user that the action can't be
deleted.
The test scenario was removed
from the database successfully.
A message will be shown to the
user that the test scenario can't be
deleted.
The test plan was removed from
the database successfully.
The action executes successfully
on all the selected end-stations
and the result of the execution for
each end-station is success.
The action executes successfully
on all the selected end-stations
and the result of the execution for
each end-station is success.
The action executes successfully
on all the selected end-stations
and the result of the execution for
each end-station is success.
The action executes successfully
on the local machine and the
result of the execution is success.
The action executes successfully
on all the selected end-stations
and the result of the execution for
each end-station is success.
The action executes successfully
on all the selected end-stations
and the result of the execution for
each end-station is failure.
The action executes successfully
on all the selected end-stations
and waited 60 seconds.
A message will be displayed telling
that the end-station is not
reachable and the result of the
execution is failure.
The tool should display failure
result for this action specifying the
connection timeout has reached.
The tool should display failure
result for this action specifying the
2.3
2.1
2.2
2.3
2.4
Automatic Software Testing Tool for Computer Networks
Run a single action with non-exist
script/batch file.
Execute single action on end-station that
configured with the wrong OS type, i.e.
Unix instead of Windows.
Execute single action without any
content of the selected end-station OS
type.
Execute single action without selecting
end-stations.
Execute TSC/TP and select exist endstations.
Execute TSC/TP with delay of 60 seconds
between each action and select exist
end-stations.
Execute TSC/TP on non-exist end-station.
Executing
TSC/TP
Run a TSC and configure the PSTools
folder to non-exist folder.
Run a TSC that compose of actions that
should success and one action that
should fail and select to stop if this action
fails.
Run a TSC that compose of actions that
should success and one action that
should fail only on one end-station and
select to stop if this action fails.
Execute a single action and select a valid
report name.
Generating
Report
Execute an action and select an invalid
report name.
Execute a single action and without
selecting a report name.
Execute a test scenario and select a valid
report name.
7
Testing Document
missing file.
The tool should display failure
result for this action specifying the
missing file.
A message will be displayed telling
that the action can't be executed
on the end-station and the result
of the execution is failure.
A message will be displayed telling
that the action can't be executed
on the end-station and the result
of the execution is failure.
A message will be displayed telling
that he need to choose endstations.
The actions execute successfully
on all the selected end-stations
and the result of the execution for
each end-station and action is
success.
The actions execute successfully
on all the selected end-stations
and waited between each
execution 60 seconds.
A message will be displayed telling
that the end-station is not
reachable after the first action and
the result of the execution is
failure, the execution should
continue on all exist end-stations.
The tool should display failure
result for this after the first action
specifying the missing file and stop
executing the TSC.
The tool should execute the
actions in the TSC and stop the
execution after the action fails.
The tool should execute the
actions in the TSC and stop the
execution after the action fails
only in the end-station that
encounters the failure.
A report file is built with correct
data and stored in the configured
report directory under the
selected name.
A message will be shown to the
user that the report name isn't
valid.
A report file is built with correct
data and stored in the configured
report directory under the name:
<Action Name> <Date> <Time>.
A report file is built with correct
data and stored in the configured
2.4
2.5
Automatic Software Testing Tool for Computer Networks
Execute a test scenario and without
selecting a report name.
Execute a test plan and select a valid
report name.
Execute a test plan and without selecting
a report name.
Create new end-station with valid
parameters.
Create EndStation
Delete EndStation
Create new end-station with invalid OS
Type.
Create new end-station with invalid IP
address.
Delete end-station that doesn't appear in
any TSC/TP.
Delete end-station that appear in TSC/TP.
8
Testing Document
report directory under the
selected name.
A report file is built with correct
data and stored in the configured
report directory under the name:
<TSC Name> <Date> <Time>.
A report file is built with correct
data and stored in the configured
report directory under the
selected name.
A report file is built with correct
data and stored in the configured
report directory under the name:
<TP Name> <Date> <Time>.
A new end-station added to the
system and stored in the
database.
A message will be shown to the
user about the invalid OS Type.
A message will be shown to the
user about the invalid IP address.
The end-station was removed
from the database successfully.
A message will be shown to the
user that the end-station can't be
deleted.
-
-
Automatic Software Testing Tool for Computer Networks
Testing Document
3. Non-Functional Requirements
In this section we define the tests for the non functional requirements as
specified in the Application Requirements Document.
Test type
Speed
Capacity &
Throughput
Test Description
Run several single action/TSC/TP on a large
group of existing end-stations with
connection timeout defined for each action.
Testing all GUI interactions.
Execute single action/TSC/TP and select to
open a report file.
Run a single action with a properly validity
string (exist string after success executing the
action).
Run a single action with an improperly
validity string (a non-exist string after success
executing the action).
Run a single action with a valid duration.
Run a TSC with actions that should all
success.
Run a TSC that compose of actions that
should success and one action that should fail
and select to stop if this action fails.
Reliability
Run a TSC that compose of actions that
should success and one action that should fail
only on one end-station and select to stop if
this action fails.
Run a single action with non-exist
script/batch file.
Run a single action and configure the PSTools
folder to non-exist folder.
Run a TSC and configure the PSTools folder to
non-exist folder.
Run a single action with a small connection
timeout.
Run a single action/TSC/TP on a non-exist
end-station.
Run a single action/TSC/TP on an end-station
9
Expected result
ARD
Reference
The software will handle
successfully all the results from
the end-stations and will
complete the execution process
successfully.
All GUI interactions should take
at most 1 second.
Displaying the report file should
take at most 1 second.
3.1.1
The tool should display success
result for this action.
The tool should display failure
result for this action.
The tool should execute the
action on the selected endstations and stop the execution
after the selected duration.
The tool should display success
result for each action in the TSC.
The tool should execute the
actions in the TSC and stop the
execution after the action fails.
The tool should execute the
actions in the TSC and stop the
execution after the action fails
only in the end-station that
encounters the failure.
The tool should display failure
result for this action specifying
the missing file.
The tool should display failure
result for this action specifying
the missing file.
The tool should display failure
result for this after the first
action specifying the missing file
and stop executing the TSC.
The tool should display failure
result for this action specifying
the connection timeout has
reached.
A message will be displayed
telling that the end-station is not
reachable.
A message will be displayed
3.1.2
Automatic Software Testing Tool for Computer Networks
Portability
and change the end-station IP in the middle
or disconnect from the network.
The utility should run on all versions of
Microsoft Windows (2000, NT, XP, 2003,
Vista…)
10
Testing Document
telling that the end-station is not
reachable.
The program will work on all
Microsoft Windows versions.
3.1.4
Automatic Software Testing Tool for Computer Networks
Testing Document
4. TDD Development
We didn’t use TDD to develop our project because the course enforce us to use
the waterfall method, which means that when we get to the programming we’ve
got all of the design ready and we just need to implement it.
11
Automatic Software Testing Tool for Computer Networks
Testing Document
5. Random & Automatically Tests
We will use automatic and random tests for several reasons (check both functional
and non-functional requirements).
 For checking non-functional requirements we will write a test that randomly
selects actions and end-stations and execute the actions on the selected endstations till we stop the test (this will test if the system can hold under a
massive actions being performed on a large group of end-station for a long
period of time.
 The previous test will also keep in the report file the start-time and the endtime for each action (this way we will test if the action is being performed in
the selected order, timeout period and the delay that was defined in the
action).
12
Automatic Software Testing Tool for Computer Networks
Testing Document
6. Testing the User Interface
Test type
Test
Insert illegal value.
Text Fields
Insert value out of range (number)
Radio button
Menu Navigation
Double Click on
action
Double Click on
TSC/TP
Tree View
Switch between the buttons of a
radio button group.
Selecting an action to perform from
the menu.
Selection an action to execute from
the execution screen and double
click on it.
Selection TSC/TP from the execution.
Selection an action from the tree
view of the execution screen and
click on it.
13
Expected result
A message will be shown to the user about
the illegal value when he selects to save the
changes.
A message will be shown to the user that the
value is out of range and display the range of
legal values.
Only one radio button is selected.
The screen of the selected menu item is
displayed.
The action details, default parameters and
end-stations will be selected and displayed to
the user.
The actions in the TSC/TP will be shown in a
tree-view.
The action details, default parameters and
end-stations will be selected and displayed to
the user.
Automatic Software Testing Tool for Computer Networks
Testing Document
7. Testing build, Integration & Deployment
The system is designed specifically for deployment in Access Layers, and it doesn't
intended to be installed at different locations (but it can be installed).
This process of installing includes:
 Copying the execution file to the target computer at the desired directory.
 Copy the PSTools directory to the target computer at the desired directory.
 Placing the configuration file in the same directory (and updating it if
necessary) or act as specified at the user manual.
 Installing the database on the target computer (as mentioned in the user
manual and in the readme.txt file).
The process of uninstalling includes:
 Deleting the directory from the target computer.
 Removing the database from the database server.
If the application was not installed correctly then it will display a message to the user
at the startup of the tool stating which part is missing or not installed correctly
(missing configuration file, database …).
14