Towards Sound Development of PIXITP, Conformance Test Suites, and Conforming
Implementations For Various Formal Description Techniques
Hazem El-Gendy, Ph. D., P. Eng.,
Dr. Nabil El Kadhi
Ministry of Endowments of Egypt, Cairo, Egypt
Prof. Comp. Science, EpiTech
Voice Tel. & Fax: +20 2 23936088,
14/16 Rue Voltaire 94270 Kremelin Bicetre, France
e-mail: [email protected]
email: [email protected]
and Narayan Debnath, D. Sc.
Dept. of Computer Sc., Winnona State University, Winnona, USA
[email protected]
Abstract: The testability of a protocol standard
is a property of the standard that reflects the
degree of precision in developing universal
interpretation of what it means for an
implementation to conform to the standard, and
the ability to exercise the implementations of
the protocol standard for purposes of assessing
their conformance to the standard [3]. In this
paper, we investigate the Conformance Testing
Requirements (CTRs) of a protocol standard
that result from the formal specification of the
allowed (dynamic) behavior of the protocol
standard. We focus on those CTRs that are
applicable to all the international Formal
Description
Techniques
(FDTs);
more
specifically, SDL, Lotos, Estelle, and ASN.l.
We develop a distinction among Implemenlor's
options, Implementer's Choice, and NonDeterministic Choices as well as testing
semantics for each of them. Then, we develop
rules for the explicit and precise specification of
these CTRs, and for developing a precise and
explicit P1X1TP (Protocol Implementation
eXtra Information for Testing Proforma). These
rules are essential for facilitating the
controllability and propagation of observability;
consequently, enhancing the testability of the
protocol standard.
Rende Vous as in Lotos.
In this paper, we investigate the CTRs of a
protocol standard that result from the formal
specification of the allowed (dynamic) behavior
of the protocol standard. We focus on those
CTRs that are applicable to various international
FDTs: SDL, Lotos, Estelle, and ASN.l. We
formalize several FDT-based CTRs. This
formalization includes developing rules for the
explicit and precise specification of these CTRs
and developing CT semantics for them. The
former facilitates automated derivation of these
CTRs while the latter establishes universal
understanding of these CTRs and their
implications on implementations, networks, and
testers. Then, we develop rules for developing a
precise and explicit PIXITP (Protocol
Implementation eXtra Information for Testing
Proforma). These rules are proven to be very
essential for enhancing the controllability
(allowing the tester to exercise the transitions)
and propagation of observability (allowing the
tester to observe the lUT's (Implementation
Under Test) behavior. This enhances the
testability of the protocol standard. This also
facilitates the design of telecommunications
systems for testability and interoperability, and
the automated derivation of the CTRs [3].
I.
Introduction
Conformance Testing (CT) plays an essential
role in providing confidence that conforming
implementations correctly support what they are
claimed to support. It also increases the
confidence
that
different
conforming
implementations
will
interoperate.
The
effectiveness of CT relies heavily on the
universal derivation of CTRs. These CTRs
should be commonly recognized by different
parties: the developers of the standard, the
vendors, the customers, and the test centers.
Consequently, explicit and precise formalization
of CTRs in a protocol standard is essential for
the development of a sound CT Theory.
Many of these CTRs result from the particular
way the protocol is specified using an
international FDT. Some of these CTRs may be
FDT-dependent that result from specific
mechanisms of the FDT such as the use of FIFO
Queue, as in SDL and Estellc, or the use of
II. FDT-Bascd CTRs
In a protocol standard, typically, rhere are CTRs
that are specified in the standard's PICSP; these
are called PICSP-based CTRs. These
requirements include, for example, what
capabilities are mandatory or conditional or
optional [5]. The standard includes also CTRs
that result from the specific design of a
capability (specification of the Externally
Observable Behavior of the communicating
entity); these are called FDT-based CTRs [3].
The specification of this specific design is
usually formalized by using an international
FDT such as SDL, Lotos, Estelle, or ASN.l.
The following example illustrates the
differences between the PICSP-based CTRs and
the FDT-based CTRs.
F.xample 1:
Case 1: Consider a transport protocol that has
the capabilities:
- 1C: Initiating establishment of a connection.
- RC: Responding to a request to initiate a
connection.
- DT: Transferring Data (send and receive),
- DC: Disconnection of the connection.
The modular standard can be given by ModStdl:
______________________________________
ModSldl := (PICSPl, BehavSpecl)
P1CSP1 (PICSP 1)
Identifiers:
Supplier
Standard Version
First Edition
Date of Statement
Implementation Name and Version
Extra Information for Testing
#
Capabilities:
Capability
Ref Slat
Al
Initiate establishment of a 1C
connection
O.1
A2
Respond to a request to RC
initiate a connection
O.1
A3
Transferring data
DT
M
A4
Disconnection
connection
a DC
M
of
Su
Legend: Ref, Slat, and Su stand for Reference,
Status, and Support respectively. The letter "M"
in the status column stands for Mandatory, i.e.,
support of the capability is always required for
conformance. Character string "O.I", stands for
Optional provided that at least one of the
capabilities identified by these characters is
supported; i.e., support of one of the capabilities
1C and RC is mandatory for conformance and
the support of both capabilities is optional.
END-PICSP1
Figure 1: SDL-Based modular specification
END-ModStdl
The standard, as indicated in the PICS
Proforma, requires support of at least one of the
two capabilities 1C and RC, but not necessarily
both capabilities, then an implementation that
faithfully supports "ModSS1 := RC U DT U DC;
ModSS1))" is a conforming implementation to
ModStdl. Such an implementation does not
support some transitions in "ModSS2 := IC U
DT U DC; ModSS2))". Both ModSS1 and
ModSS2 have the same initial state (idle) and
Branch off it. In this case the branching off the
"idle" state is interpreted, based on the PICSP,
as "an implementer's option". Consequently,
conforming implementations do not have to
support expressions on both sides of such a
branching off a state. This indicates that, for the
purpose of conformance testing, the PICSP
takes precedence, over the semantics of the
FDT, in deriving the CTRs and in identifying
the conformance implications of the use of the
FDT constructs -in the Behavior Specification.
Consequently, we get the first rule for deriving
CTRs as follows:
Rule 1: For the purpose of conformance
testing, the PICSP takes precedence, over the
semantics of the FDT, in deriving the CTRs
and in identifying the conformance
implications of the use of the FDT constructs
-in the Behavior Specification.
Rule 2: Conforming implementations do not
have to support expressions on both sides of a
branching off a state; this is the case where
there are optional capabilities as indicated by
the protocol PICSP.
This results in some form of "module
(capability) - based conformance-requirements
analysis" first before analyzing the CTRs
resulting from every capability [3]. It provides a
semantic to what should constitute a module (a
capability). This has also a lot of applications in
verification and validation [8].
But, if it were that the PICSP of the standard
requires support of all the capabilities, then
support of both ModSSj and ModSS2 would be
mandatory for conformance.
Case 2:
Consider the specification of the DT capability
as given in Figure 1.
An 1UT, to conform to DT, the IUT must
support all the transitions specified by DT. This
means that the "branching off a stale" construct
is interpreted, in this particular case, as that
conformance to all the expressions linked by the
construct is implementer's mandatory. This
indicates that, within the behavior specification
of a capability, the occurrence of the FDT
constructs such as the SDL Branching construct
or the Lotos Choice construct must not be
interpreted as "implementer's option" because
the support of the expressions on both sides of
the construct is mandatory for conformance to
the capability.
Consequently, we get the
following Rule.
Rule 3: Within a capability, support of all
transitions is implementer's mandatory. This
is the case of SDL branching constructs or
Estelle transitions or the Lotos Choice
construct.
But, support of all the transitions does not
necessarily require support of all the allowed
sequences. This is illustrated by Case 3 given
below.
Case 3:
Consider the DT capability that is capable of
four events:
- receiving from the transport service user a
transport data request service primitive
(T_Dat_Req),
- receiving
from the network service
provider a transport transferring data PDU (dt),
- transmitting to the network service provider
any
queued
transport
data
request
(Remove_Buf_5),
- transmitting to the transport service user any
queued transport transferring data PDU
(Remove_Buf_3).
The first two transitions can occur at any time
while the last two transitions can occur only
when the associated predicates hold. Now,
consider the following scenario:
Predicates of the last two transitions hold.
Hence, the implementation has the choice
between two input events and two output
events. The order of executing these four events
is left as "Implementer's choice" to the
implementer where every manufacturer has the
freedom to choose the order that it believes to
most suit the manufacturer's systems or that
enhances the performance or that provides
better customization of the manufacturer's
product to certain market needs. The nonmandatory of any particular sequence is implied
by the semantics of FDTs such as SDL.
Consequently, the conformance testing must not
require the implementation to have all the
possible sequences nor to have any particular
sequence. This leads to Rule IV as follows:
Rule 4: For an IUT to conform to a
capability (faithfully supports the capability),
it does not have to support all possible
sequences of events/interactions nor any
specific sequence.
EndExamplel
III. Formalization of FDT-Based CTRs
In this section, we develop rules for systematic
derivation of CTRs for those aspects of the
standard that are FDT based. Some of the FDTbased CTRs are FDT specific; for example, the
CTRs that deal with testing the underlying
communications mechanism (e.g., the Rendez
Vous or the FIFO Queue). However, other
CTRs are applicable to the three international
FDTs: SDL, Loios, and Estelle; this is
surprising since the FDTs use different
mechanisms for specification. Here, we focus
on developing rules for those CTRs thal are
applicable to all the three FDTs. To make these
rules FDT-independent, we base them on
Specification Constraints.
III.1. Implementer's Choice
A typical standard does not enforce any
particular order for checking the various
buffers; the order (priority) is left as an
"implemenwr's choice". For example, consider
case 3 of Example 1. If the implementation of
the protocol entity has received both T_Dat_Req
PDU and dt PDU, and the service user has data
to send then, the order of executing these events
is not mandated by the specification (this is
implied by the semantic of the FDT). This way,
the implementations of the standard arc not
restricted to follow any specific internal
implementation style.
The manufacturer to decide on these
Implementer's Choices makes decisions such as
"Should the input queues and output queues be
scanned in a round-robin basis or should a
particular queue (e.g., a queue that is important
for presenting a quality service to the user) be
given a higher priority (e.g., always scanned
first or the number of PDUs in it is limited
below a particular number)?
Also, Implementer's Choices covers issues such
as resolving non-determinism in the Behavior
Specification by augmenting this behavior
specification
with
system-dependent
parameters. An example of this, from the
transport layer protocol, is resolving of the nondeterminism in the RC capability regarding
whether or not to accept a request to establish a
new connection. The standard indicates that
such a decision is based on the system's internal
status and it is left to the manufacturer to map
this to the parameters of its system (e.g., how
much free resources are required and how
availability of sufficient resources is determined
and other factors such as "Should these
available resources be used to establish a new
connection or to offer more features over the
already established connections?".).
Implementer's Choice (ImpChoice): Those
aspects of a protocol standard that the details of
how to support them are left to the
manufacturer. This is in order to not restrict the
implementation and to facilitate various creative
ways of supporting them, without affecting the
functionality of the specified capability.
Typically, these various ways for support suit
the various manufacturers' systems, their
architectures, and internal parameters. These
Implementer's Choices are not reflected in the
PICSP but are typically implied by the FDT
used. Typically, these covers the following
cases:
1. Selecting a particular sequence for executing
a list of allowed events (selecting a particular
sequence to scan multiple queues).
2. Resolving
non-determinism
in
the
specification of the protocol.
The identification of the places of Implementer's
Choices in a standard provides several items of
thc standard's PIXITP.
The way a
manufacturer resolves these
Implementer's
Choices provides a lot of the information
needed for the PIXIT. For example, it provides
information about the maximum number of
established connections that can exist
simultaneously as well as information about
how to bring the IUT to a state where it rejects
every new request to establish a new
connection; the latter one is very important for
the correct execution of essential test cases
that exercise specific transitions.
Rule 5: For every non-determinism, in the
behavior specification of a supported
capability,
use
externally
observable
parameters to specify, in the PIXITP and the
PIXIT, the implications of the way of
resolving this non-determinism on the
externally observable behavior of the IUT
that affect testing of the IUT.
Rule 6: For every specification parameter/
predicate, that represents an internal status
of the system, in the Behavior Specification,
use externally observable parameters to
specify, in the PIXITP and the IUT's PIXIT,
the externally observable implications of the
way that this parameter/predicate was
mapped to the system's internal parameters
and, that affect the testing of the IUT.
Rule 5 is important for controllability; that is
allowing the tester to exercise the transitions.
Rules 5 and 6 are important for the propagation
of observability; that is the ability of tester,
depending on the abstract test architecture, to
observe the Implementation Under Test (IUT).
III.2. Non-Deterministic Choice
A formal specification of the observable
behavior of a communications protocol typically
includes one or more non-deterministic choices
in any of the following forms:
1) non-deterministic choice that results from
having, with the same event, two transitions
leading to two different behavior
expressions. Example 2 illustrates this.
Example 2:
Consider the RC specification as in Figure 2.
Figure 2: SDL Behavior Specification of RC
where on receiving a CR, the implementation,
based on its internal status, may move to where
it is ready to only accept the connection request
by sending a CC PDU or may move to where it
is ready to only reject the connection request by
sending DR PDU.
This non-determinism results mainly from
omitting some of the details (e.g., the triggering
conditions of the transitions) of the behavior
specification. In the example of the RC
capability, what are omitted are two predicates:
a predicate OK, that indicates availability of
resources to establish one more connection, and
its complement predicate NOTOK. The
complete behavior specification of the RC
capability can be given as in Figure 3.
Of course, there is not any sense in allowing an
implementation to support only one branch of
the behavior (e.g., always send DR PDU) and
yet
be
classified
as
a
conforming
implementation (a conforming implementation
must be able to successfully establish a
connection at least some of the time).
Consequently, this occurrence of nondeterminism must not be interpreted as
implmentor's option.
Figure 3: SDL Behavior Specification
2) non-deterministic choice that results from
having
internal
(un-observable)
events.
Example 3 illustrates this.
Example 3:
RC is as given in Figure 1.
Same as the first type, this type of nondeterminism results mainly from omitting some
of the details of the behavior specification.
Typically, behavior specifications that have this
type of non-determinism can be transformed to
equivalent behavior specifications that have
instead the first type. The first style indicates
that On-accepting a CR PDU, the
implementation will make, based on its internal
status, a decision to whether. it will accept or
reject the connection request. While the second
style indicates that the implementation After
receiving CR PDU will make the decision.
Consequently, these two styles model
the
behavior
from
the
environment's
perspective
and
the implementation's
perspective respectively.
This non-determinism is useful in verification,
testing [ELG96b, ELG04, ELG05] and in
allowing the specification of capabilities in an
abstract level that does not force the specifier to
have to specify any mapping of predicates to
internal parameters [ELG96b]. This type of nondeterminism can be detected by intelligent
syntax analyzers.
3) a non-deterministic choice that results from
the standard having implementer's choices.
For example, the standard behavior
specification of the DT capability of
Example 1 does not dictate any particular
sequence for scanning the buffers which
makes the sequence of executing the four
events, from the perspective of the
environment or the tester, non-deterministic.
In such cases, the other communicating
entities must be ready to respond properly to
any and to every valid sequence. This affects
the design of the Conformance Testers and
the Conformance Test Suites because the
conformance testers must not pre-assume
any specific sequence and the verdict of the
test case must be pass as long as the IUT
responds with any of the allowed sequences.
This type of non-determinism can also be
detected by intelligent syntax analyzers.
All these forms of non-determinism are not
reflected in the PICSP but are implied by the
semantic of the FDT. Consequently, we have
the following two new rules:
Rule 7:
Within the specifications of a
capability,
the
occurrence
in
FDT
specification of non-determinism of the
following two forms must not be interpreted
as implement or's choice:
 due to having two possible transitions out
of the same state on receiving the same
input event; or
 due to having internal events.
Rule 8:
For the case of having nondeterminism due to having implementer's
choices, the tester must not impose on the
IUT or pre-assume any particular sequence
and the verdict of the test case must be
"pass" as long as any of the possible
sequences is supported.
III.3. Other CTRs
The following CTRs are also based on the
Behavior Specification of the communicating
entities:
1) Every implementation, that faithfully
supports a capability, must support all the
states of the capability.
2) Every implementation, that faithfully
supports a capability, must support all the
transitions of the capability including those
that correspond to non-determinism.
3) Every implementation, that faithfully
supports a capability must support the
repetitiveness of the test result.
4) Every implementation, that faithfully
supports a capability, needs not to support all
the sequences allowed by the "BehavSpec" of
the capability in the places of "Implementer's
Choices".
El-Gendy's method [4] can be used to derive test
sequences for the requirements indicated in 1
and 2 above.
IV. Summary
The process of implementing a standard starts
with the manufacturer resolving the following
types of choices:
1) "Implementer's
options"
where
the
manufacturer decides (based mainly on
marketing requirements) what optional
capabilities to support based on the
Testability Directed PICS Proforma.
2) "non-deterministic choices" that results
from omitting some of the details (e.g.,
predicates) where the
manufacturer
explicitly includes, in its internal
specification, these predicates and maps
them to the internal parameters of the
manufacturer.
3) "Implementation choices" in the behavior
specification
where
any
particular
implementation has to be deterministic.
These decisions arc based mainly on the
internal design and architecture of the
manufacturer's system.
V. Conclusions
In this paper, we have developed a foundation
and a testing semantics for "implementer's
choice", and "non-deterministic choice" as well
as 8 rules for automated development of sound
PIXITP, conformance test suites, testers, and
conforming implementations. This facilitates
automated derivation of the places of
implementer's choices and/or non-determinism
in a standard.
Guidelines for the implementers to resolve these
non-determinism and/or implementer's choices
have also been developed and illustrated by
examples.
The rules developed for the construction of
explicit and precise PIXITP enhance both the
controllability and the observability of IUT.
Consequently, these rules enhance the testability
of the standard.
In complement to this work is the development
of methods for automated derivation of test
sequences [8]. Further research is still required
to study the coverage of these methods and to
develop a foundation and framework for other
CTRs.
References
1. [CCITT] CC1TT Recommendation Z.100,
"Specification and Description Language
SDL", 1992.
2. R. Dssouli and G. Bochmann, "Conformance
Testing
with
Multiple
Observers",
Proceeding of the 6th IF1P International
Workshop on Protocol Specification, Testing,
and Verification, 1986, pp. 217-229.
3. Hazem El-Gendy and R. Probert, "Relevant
Testability Aspects of a Communications
Standard", IEEE 4th MICC, Langkawi
Island, Malaysia, November 20-23, 1995,
pp.14.2.1-14.2.8.
4. Hazem El-Gendy and Robert Probert,
"Relevant
Testability Aspects
of a
Communications Standard", Proceedings of
the 4th IEEE Malaysia International
Conference on Communications, Langkawi
Island, Malaysia, November 20-23, 1995,
pp.14.2.1-14.2.6.
5. ISO/IEC
9646, "Information Retrieval,
Transfer and Management for OSI, OSI
Conformance Testing Methodology and
Framework", Parts 1 to 5. Also, in CCITT
Recommendations X.290 to X.294.
6. ISO/IEC
JTC1, "Formal Methods in
Conformance Testing (Working Document)",
November 2002.
7. Hazem El-Gendy and Nabil El Kadhi, “Using
Formal
Methods:
Importance
and
Experience”, International Journal on
Computing Methods in Science and
Engineering, Published in Greece, 2005.
8. Hazem El-Gendy and Nabil El Kadhi,
“Formal
Method
for
Automated
Transformation of Lotos Specifications to
Estelle Specifications”, International Journal
of Software Engineering & Knowledge
Engineering, USA, Vol. 15, No. 5, October
2005, pp. 1-19.
9. Hazem El-Gendy, “Formal Automated
Transformation of Lotos Specifications into
SDL Specifications”, Proceedings of the 8th
World Multi-conference on Systemics,
Cybernetics, and Informatics sponsored by
the International Institute of Informatics and
Systemics (IIIS), Orlando – Florida, USA,
July 18-21, 2004.
10. Hazem
El-Gendy,
"Study
of
the
Characteristics of CT-Equivalence with
Proves", Journal of Computational Methods
in Sciences and Engineering, Volume 6,
Numbers 5, 6, 2006, pp. 171-179.
11. Chih-Yung Chang and Shin-Chih Tu,
“Active Route-Maintenance Protocol for
Signal-Based Communication Path in ad hoc
Networks”, Journal of Network and
Computer Applications, Vol. 25, Issue 3, July
2002, Academic Press, pp. 161-177.
12. S. Farahvash, K. Akhavan, and M.
Kavehrad“ ,Packet Transmission Over a Fixed
Wireless Loop Using Adaptive Rate
Techniques”, International
Journal
of
Wireless Information Networks, Vol. 9, No.
3, July 2002, pp. 165-178.
13. J. Q. Bao and L. Tong“ ,Protocol-Aided
Channel Equalization in Wireless ATM”,
IEEE Journal on Selected Areas in
Communications, Vol. 18, No. 3, March
2000, pp. 418-435.
14. D. P. A. Greenwood and R. A. Carrasco,
“Neural Networks for the Adaptive Control of
Disruptive Non-Linear Network Traffic”, IEE
Proceedings Communications, Vol. 147, No.
5, October 2000, pp. 285-291.