Formal Methods and Testing:
Possible Attributes for Success
A. J. Cowling
Department of Computer Science
University of Sheffield
Rationale
The Number of Different Methods
 Many formal and semi-formal methods exist
 New ones are still being created
Their Usefulness
 Determining which are useful requires empirical work
 The number of methods would imply a lot of work
 Therefore priorities need to be set
Technical Features
 Some methods appear to be more successful than others
 Possibly because of their technical features
 These would be more appropriate candidates for study
CSEE&T 2001
University of Sheffield
Department of Computer Science
Role of Models
Requirements
Model
Development
Process
Specification
Formal
steps
Model
Design
Model
Testing
Implementation
Model
Required V & V
CSEE&T 2001
University of Sheffield
Department of Computer Science
Testing and Models
Basic Testing Methods
 Base the generation of test cases on one of these models
– Black-box testing uses the specification model
– White-box testing uses the implementation model
Hybrid Testing Methods
 Combine the approaches – eg:
– Black-box methods to generate the test sets
– White-box methods to measure their coverage
 May provide more effective testing than individual basic methods
– At least, according to some papers
State-based Testing
 Uses state-machine models for specification and implementation
 Extended models (eg the X-machine) allow powerful results:
– absence of faults up to some bounds,
– under some assumptions, complete absence of faults
CSEE&T 2001
University of Sheffield
Department of Computer Science
Key Formal Methods
Model Checking
 Requires state-based specification models
 Shows whether required properties hold for the models
 Can handle very large systems (10^20 states)
Machine Model Verification
 Uses state-based specification and implementation models (eg B)
 Can verify that implementation is consistent with specification
Refinement
 Typically uses relational models (eg Z, VDM)
 Refinement steps produce correct-by-construction implementations
 Discontinuities in the models need to be accommodated
– Retrenchment has been proposed for this
CSEE&T 2001
University of Sheffield
Department of Computer Science
Attributes for Success
Role of Models
 Successful approaches appear to all be model-based
 State machine models are particularly successful
 Extended state-machine models even more so
Differences between Models
 The different stages require different models for one system
 Any form of V & V must accommodate these differences
– ie must represent design transformations
– currently an interest within model-driven architecture
 It appears that successful methods:
– explicitly handle multiple models, and
– explicitly represent the differences between them.
CSEE&T 2001
University of Sheffield
Department of Computer Science