CSIS3600 Systems Analysis and
Design
System Implementation
and Testing
Testing
Once coding has begun, the testing process can also
begin.
Testing is an important part of development.
Initial testing plans are usually devised during the
analysis stage to be carried out during the
implementation phase.
Besides testing the software for errors and defects,
testing serves as a way to improve communications.
Analyst serves as an advisor and coordinator for
program testing.
Quality Assurance and Testing
QA activities are often shortchanged during
design and especially during
implementation.
Schedule pressures can build with each
successive phase.

QA and testing activities may be bypassed in an
ill-fated attempt to speed up the project.
QA activities require development
personnel to open their work to thorough
examination and criticism by others.

Many people are reluctant to do this.
Many people view testing and test
personnel as the bearers of bad news.

They mistakenly believe that no news is good
news and that bypassing testing is a way of
avoiding bad news.
How to Overcome the Attitude
Towards QA
Integrate QA into the project and schedule it
from the beginning and never abandon it.
What Must Be Tested?
Code (application programs, etc.)
Procedural manuals
New Hardware
System Interfaces
When should testing really
begin?
Ideas?
When Should Testing be Done?
Done throughout the systems development
process
Best to test beforehand than to fail after
installation
 Testing can be done on 'parts' and then in
totality

Categories of Tests
Testing can be divided several ways.
Generally there are two categories of tests.

Static testing in which the code being tested is
not executed


Technical Reviews, Walkthroughs, Inspections and
Desk checking are static
Dynamic testing in which the software code is
executed.
Formal but less Intimidating
Quality Assurance Procedures
Technical review
Formal or informal review of design or
construction details by a group of developers
 Sometimes it just helps for someone else to
‘look at it’

Walkthrough
A review by two or more people of
accuracy and completeness.
Tend to be informal
The focus is on what the code does
Most often used during analysis and design
but can also be used during implementation
If done during analysis, done without actual
code (can be done using models)
[somewhat what I do when I review your
group deliverables]
DeskChecking
Testing technique in which the program
code is sequentially executed manually by
the reviewer
Usually someone who understands program
logic reviews the code with a pencil and
paper
Inspection



More formal version of a walkthrough
Participants review and analyze materials before they
meet as a group
When group meets, participants play specific roles






Presenter – summarized material being inspected
Critics – describe errors or concerns found
Secretary – records notes
Additional errors or problems may be uncovered during
the discussion
Usually the goal is to examine program code for
predictable language-specific errors
Participants discuss possible solution strategies and,
before the meeting is over, agree on approaches to
problem resolution.
Formal QA Procedures
Testing usually follows four phases:
Modeling the software's environment
 Selecting test scenarios
 Running and evaluating test scenarios
 Measuring testing progress

Test Cases
A test case is a formal description of
a starting state
 one or more events to which the software must
respond
 the expected response or ending state

Starting state and ending state are
represented by a set of test data.
Starting state may represent a particular set of
database contents
 existence of a particular customer and order for
that customer
 Event may be represented by a set of input data
 customer account number and order number
used to query order status
 Expected response may be a described behavior
(display of certain information) or a specific
state of stored data (canceled order)

More on Test Cases
Preparing test cases and test data is tedious
and time-consuming.
At the program or module level, every
instruction must be executed at least once.
There are automated tools based on proven
mathematical techniques that can be used to
generate a complete set of test cases.
What Tests to Select
Set of tests that cause each source statement to be
executed at least once
Set of tests that cause each branching structure (If,
CASE, While, etc) to be evaluated with each of its
possible actions.
Set of tests that cause each data structure to be
initialized and then subsequently used
Set of tests that expose the seeded faults.

Seeded faults are errors intentionally inserted into the
source code. Ideally by finding seeded errors, the tester
will also find real errors.
Tests
Unit Testing
Integration Testing
Stub Testing
System Testing
Build and Smoke Test
Regression Testing
Unit Testing
Testing of each module in isolation in an attempt
to discover any errors in its code.
Programmers are generally responsible for unittesting their own code prior to integrating it with
modules written by other programmers.
Some organizations assign “testing buddies” to
help programmers test their own code.
Having a different programmer test the code
usually finds more errors.
Integration Testing
Process of bringing together modules for test
purposes
Objective is to identify errors that were not or
could not be detected by unit-testing such as:




Interface compatibility
Parameter values
Run-time exception
Unexpected state interactions
It is a gradual process,

You being with the root and add coordinating modules
one at a time
Stub Testing
Process used in testing modules
Stubs are 2-3 lines of code that stand in for
a missing module
Effective where modules operate top-down
During testing the coordinating module
calls a stub who ‘stands’ in for the missing
module.
System Testing
Similar to integration testing but extends the bringing
together of program modules to bringing together
components.
Follows the same incremental logic as integration testing.
More than simply expanding integration testing
 attempts to demonstrate whether the system has met its
objective
Involves the use of non-live test data in a non-live testing
environment that simulates the real environment as closely
as possible.
Usually test case scenarios are developed to form the basis
of system testing
Build and Smoke Test
System test that is performed daily
The system is completely compiled and linked and a
battery of tests is performed to see whether anything
malfunctions in an obvious way (“smokes”)
Commonly associated with interactive or rapid
development
Identify problems results from code modification
Daily testing ensure that errors are found quickly and
can be easily tracked to their source
Regression Testing
After testers submit successfully reproduced
failures to development, developers generally
create a new version of the software (in which the
bug[s] have been removed).
Testing progresses through subsequent software
versions until one is determined to be fit for
release.
This retesting is called regression testing.
Acceptance Test
Performed to determine whether the system
fulfills user requirements
Last round of testing before system is
handed over to users
Formal activity
Versioning
Alpha
Beta
Limited Availability
General Availability
Alpha
Test version that is incomplete but ready for some
level of rigorous testing.
Tests performed during alpha include:




Recovery testing – force software to fail to insure it can
recover
Security testing – verify protection mechanisms
Stress testing – tries to break the system
Performance testing – determines how the system
performs under a range of possible environments.
Beta
A system that is stable enough to be tested
by end users
Run in an end users environment using their
own data
Sometimes thought of as a rehearsal of the
installation phase
Production version
Limited Availability – A production version
considered final but is released to a limited
number of end users for additional user
acceptance testing
General Availability – A production version
is considered a final product that can be
formally distributed to general population of
users
Numbering of Versions
Alpha usually starts at 0.#
 Beta is also numbered beginning with 0.#
 First Production version starts with 1.0
 Versions progress from here sometimes with
alpha and beta versions of major upgrades

Example Timeline – Test and Production
Date
January 2000
Alpha version 0.1
April 2000
Alpha version 0.2
July 2000
Beta version 0.3
September 2000
Beta version 0.4
January 2001
Production version 1.0
April 2001
September 2001
Planned version 2.0
Production version 1.1
January 2002
Alpha version 1.91.
April 2002
Beta version 1.9.2
January 2003
Production version 2.0
Complexity of Testing OO
Methods can be called by many other methods and the
calling methods may be distributed across many classes
Classes may inherit methods and state variables from other
classes
The specific method to be called is dynamically
determined at run time based on the number and type of
message parameters
Objects can retain internal variable values (that is, the
object state) between calls.
 The response to two identical calls may be different due
to state changes that result from the first call or occur
between calls.
Testing Summation
Testing and Technical reviews find between
50 and 75 percent of errors.
Some errors are more easily detected using
one method while other by using a different
method.
Thus, the two techniques are more effective
used jointly rather than individually.
References:
Types of Testing Coverage:
http://www.kaner.com/coverage.htm
Step-by-Step Test Design
http://www.stqemagazine.com/
Number of testers to developers:
http://www.jrothman.com/Papers/ItDepends.html
Testing and Java Technology
http://developer.java.sun.com/developer/technicalArticles/I
nnerWorkings/testing/
Site for Testers to Share Techniquies
http://www.testingcraft.com/exploratory-robinson.html
Great Bugs Site
http://www.bugnet.com/