Cultural Heritage in REGional NETworks
REGNET
Part 2:
Functional Testing
Project Meeting Content Group
2002-06-24 - 2002-06-26
2
Functional Tests - An introduction
Methodology
Goals
Based upon black box techniques:
Functional tests are based on use
cases. The goals of these tests are
to verify:
- to ensure proper system
functionality, including navigation,
data entry, processing, and retrieval
Goals are not to define additional
requirements (=> usability tests,
task-oriented).
June 2002
Verifying the application and its
internal processes by interacting
with the application via the
Graphical User Interface (GUI)
and analysing the output or
results
Based on the “Use Cases” already
prepared during the requirements
phase
REGNET Project Team Meeting Content Group
3
Functional Tests - An introduction
Use Case => Test Case (An Example)
UC 1.1.3 - Import / export existing data from local system
Upload and conversion for different data management systems
Test Case 1.1.3 - Import / export existing data from local system
Action
Data Input
Expected Output
OK
Expected Results
or reported
as Bug No.
Import data
Click on
„Import data“
Request to determine (= place, type, format,
name) data
Determine data,
Describe data in
provided Fill inForm OR field/s
Dataset 1,
Click on „ok“
Systems lists data to be imported, request to
confirm, modify or cancel procedure
Confirm import
Click on
„confirm“
System confirms import of data
June 2002
REGNET Project Team Meeting Content Group
Rechecked
4
Functional Tests - Status
Test cases available
72 Test cases for data generation, search & retrieval completed
High level test cases for the other components (56)
June 2002
REGNET Project Team Meeting Content Group
5
Functional Tests - Problems
Problems & Open Issues
Problems:
Test cases are all not complete:
- it is not defined how the output
should look like
- it is not defined how the input
should look like
- false and right inputs are not
specified (actions which occure
when false input was made)
Problem: Use cases are not
very specified!
Suggestion (original draft):
Carry out the tests for the available
test cases at first (content
provider). 3 content partners, effort
estimation: 2 days per partner!
Report not only incidents but also
further specifications, e.g. if the
output is not formatted
appropriately it could be defined
now.
Could be a possibility to detect
white spots in requirement
definition
June 2002
REGNET Project Team Meeting Content Group
?
6
Functional Tests - Schedule draft
Validation schedule: Start on 1st of July 2002 ! Step-by-Step approach
06/28/02
07/05/02
07/19/02
15/08/2002
Sofia Meeting
Test finalising
Functional tests
Usability tests (heuristic, scenarios)
Content (integrity and quality) tests
Which functionality is available ?
Red light:
Orange light:
Green light:
June 2002
external solution
improvements necessary/possible before
carrying out further tests
go for usability tests
REGNET Project Team Meeting Content Group
7
Functional Tests - Strategy
 Strategy for pragmatic testing!?
High-level testing - functionality available or not?
Detailed testing could be carried out by using the test cases
already worked out and/or the pure use case list !? The modelling
of further test cases is very time-intensive ...
- Estimation: 300 test cases for all use cases quality level 5
until End of June (VALT), Modelling: 10 days, testing: 25 days
- Responsiblities (CP/TP) for modelling and testing
Must be carried out in the first 2 weeks of July!
June 2002
REGNET Project Team Meeting Content Group
8
Functional Tests - Discussion
To be done in this session/working groups
Working groups: Practical exercises ?
- Reponsible content partners
- Test cases for the other functionalities
vs. pragmatic approach
- Agreement on reporting workflow
E-Shop/Catalogue Management
Auction System
E-Publishing
Topic Map (Generator), Viewer?
Data Generation
Search (Multi-Site search?)
Portal (Heuristic Evaluation)
After Sofia: Creation of test manual, Carrying out tests, Reporting
and monitoring, Functional tests QA
Reporting
Workflow
June 2002
REGNET Project Team Meeting Content Group
Functional Tests - Incident Reporting and
Change Management
Why?
- To define communication worflows during
(also after ?!) the validation phase
- To enable tracking of all incidents
Decision already
made: no automatic
tool
- To enable tracking of tests carried out
- To enable priorisation of work to be done
- To enable the consortium to define
dates for the release of the system
June 2002
REGNET Project Team Meeting Content Group
9
10
Functional Tests - Incident Reporting
Tester
• Tests with test cases
• Submit one report per incident to tech-partner
Validation PM
• Adds incident to open incidents list
• Keeps track of reported incidents
• Keeps track of timelines
Tech Partner
• Receives incidents
June 2002
Tester
Incident Reports
Validation PM
REGNET Project Team Meeting Content Group
Technical Partner
11
Functional Tests - Incident Solving
Tech Partner
• Resolves incidents
• Reports resolved incidents to tester and
Validation PM
Validation PM
• Sets status of incidents in open incidents
list to “resolved”
Technical
Partner
Reports on
Resolved Incidents
Validation PM
Tester
• Receives reports on resolved incidents
June 2002
REGNET Project Team Meeting Content Group
Tester
12
Functional Tests - Closing Incidents
Tester
• Tests resolved incidents with test cases
• Submits reports on resolved incidents to
Validation PM
Validation PM
• Sets status of incidents to “closed”
Tester
Reports Incidents
to be closed
Validation PM
June 2002
REGNET Project Team Meeting Content Group
13
Functional Tests - Re-opening Incidents
Tester
• Tests with test cases
• Re-opens incidents (if necessary)
Tester
Validation PM
• Sets status of incident to “re-opened”
• Keeps track of reported incidents
• Keeps track of timelines
Re-opened
Incident Reports
Validation PM
Tech Partner
• Receives re-opened incident reports for
resolving
June 2002
REGNET Project Team Meeting Content Group
Technical Partner
14
Functional Tests - Weekly Reporting
Validation PM
• Sends weekly status reports on
incidents to Technical Partners and
all Testers
• Reminds Technical Partners on all
important pending issues
• Reminds Testers on all outstanding
tests to be carried out
June 2002
Validation PM
Weekly Report
on Open Incidents
Technical Partner
REGNET Project Team Meeting Content Group
Tester
Tester
Tester
15
Reporting Form
June 2002
The next page is for developers
comments
REGNET Project Team Meeting Content Group
16
Reporting Log
June 2002
REGNET Project Team Meeting Content Group