Producing Better Output From Finite State Verification Systems

By Steve Kassof
Advisor: Aaron Cass

Ways to get it wrong
◦ Between Floors
◦ Doors open while moving
◦ People left stranded

Problems with testing
◦ Time
◦ Thoroughness
Open Doors
First Floor
Doors
Open
First Floor
Doors
Closed
Second
Floor
Doors
Open
Second
Floor
Doors
Closed
Open Doors
Up One Floor
Close
Doors
Down One Floor
Close
Doors
Open
Doors
Close
Doors
Close
Doors
Move
Car

Creating execution paths
◦ Model how a program runs
◦ Faster and less expensive than test running
Open Doors
First Floor
Doors
Open
First Floor
Doors
Closed
Second
Floor
Doors
Open
Second
Floor
Doors
Closed
Open Doors
Up One Floor
Close
Doors
Down One Floor
Close
Doors
Open
Doors
Close
Doors
Close
Doors
Move
Car

Give user violating paths
◦ State with incorrect system properties
◦ Something went wrong
Open Doors
Close
Doors
Close
Doors
Second
Floor
Doors
Open
Second
Floor
Doors
Closed
Open Doors
Up One Floor
Down One Floor
First Floor
Doors
Closed
Down One Floor
Up One Floor
First Floor
Doors
Open
Open
Doors
Close
Doors
Close
Doors
Move
Car


No standard output designed to be userfriendly
Different output for different FSVs
◦ As many as it can find
 Makes it harder for the user
◦ First path found
 May not have enough information

FSVs don’t know which path to show
◦ Could be too long, too complicated, or too short

FSV finds many violating paths
◦ Must have passed through some error

Small and representative subset of violating paths
◦ Small
 Easy to comprehend
◦ Representative
 Shows various ways a problem occurs
◦ Given paths from an FSV, take 5 paths least alike
 Reducing commonality makes the subset more representative
Open
Doors
Close
Doors
Close
Doors
Move
Car

Take the 1000 shortest paths from the FSV
◦ Find number of common nodes between two paths
◦ Store number to avoid multiple calculations

Find the five paths with lowest commonality
◦ Worst case: Θ(n5)

Built simple elevator program
◦ Two floors
◦ People pushed call button at random times
◦ Known error
 Elevator will instantly respond to call buttons

Not fully implemented yet
◦ Features missing
◦ Wrote to see if we could get violating paths to use

Commonality
◦ Change how difference between paths is calculated
◦ Every node is involved in each trace



Evaluate Efficiency
Full Elevator Controller
User Study
◦ Requires test subjects to languages not taught at
Union