SoftwareDesignandTesting
CSE592
Advanced Topics in Computer Science – Software Engineering
Test Generation from
Finite-State Models
Lecture 8
Dr. Ilchul Yoon ([email protected])
Sources: (1) Foundations of Software Testing (textbook), (2) slides by Prof. Mathur
Finite-StateMachines
l
l
Anabstractrepresentationofbehaviorexhibitedbysome
systems
Derived from application requirements.
l
l
l
Mealyvs.MooreMachine
Afinitestatemachine(FSM)
l
2
Source: “State machines and Statecharts” by Bruce Powel Douglass
Mealy
But,notallaspectsofrequirementsarespecifiedbyanFSM.(e.g.,
realtimerequirements,performancerequirements.
ApartofUML2.x
Example(Mooremachine- actionsareassociatedwithstates)
Moore
3
4
EmbeddedSystemsandFSMs
l
Anembeddedsystemoften
l
l
l
l
Receivesinputsfromitsenvironmentand
Respondswithappropriateactions.
Whiledoingso,itmovesfromonestatetoanother.
l
l
Finite-StateMachines
Examplewithinitialstate,finalstate,andaction
l
Mealymachine(actionsareassociatedwithtransitions)
Theresponsedependsonitscurrentstate.
Behaviorofanembeddedsystemisoftenmodeledby
afinitestatemachine(FSM).
l
IsFSMrequirementsspecordesignspec?
l
l
Canbeusefulforboth
Applicationdomains?
l
GUIs,networkprotocols,pacemakers,Tellermachines,
WEBapplications,safetysoftwaremodelinginnuclear
plants,andmanymore.
5
FSMandStatecharts
l
FSM(Mealymachine):Formaldefinition
NotethatFSMsaredifferentfromstatecharts.
l
l
l
6
Deterministic
WhileFSMscanbemodeledusingstatecharts,thereverse
isnottrue.
TechniquesforgeneratingtestsfromFSMsaredifferent
fromthoseforgeneratingtestsfromstatecharts.
Theterm“statediagram”,adirectedgraph,isoftenusedto
denoteagraphicalrepresentationofanFSMora
statechart.
Non-deterministic
d: Q x X® 2Q
7
FormalDescriptionExamples
TestGenerationfromFSMs
Blue:Generateddata
Requirements
Ourfocus
Testgenerationalgorithm
FSM
Testgenerationforapplication
ApplicationTestinputs
FSMbased
Testinputs
Testinputs
Application
Testdriver
9
TabularRepresentation
Pass/fail
Oracle
©AdityaP.Mathur 2009
10
FSMProperties
l
Completelyspecified
l
l
AnFSMMissaidtobecompletelyspecifiediffromeach
stateinMthereexistsatransitionforeachinputsymbol.
Stronglyconnected
l
11
Observedbehavior
AnFSMMisconsideredstronglyconnectedifforeachpair
ofstates(qiqj)thereexistsaninputsequence thattakesM
fromstateqi toqj.
12
FSMProperties
l
FSMProperties
V-Equivalence
l
l
l
l
LetM1=(X,Y,Q1,m10,T1,O1)andM2=(X,Y,Q2,m20,T2,O2)
betwoFSMs.
LetVdenoteasetofnon-emptystringsovertheinput
alphabetX.i.e.,VÍ X+.
Letqi andqj,betwodifferentstatesofmachinesM1 andM2,
respectively.
k-equivalence
l
l
LetM1=(X,Y,Q1,m10,T1,O1)andM2=(X,Y,Q2,m20,T2,O2)
betwoFSMs.
StatesqiÎQ1 andqjÎQ2 areconsideredk-equivalent if,when
excitedbyanyinputoflengthk,yieldidenticaloutput
sequences.
l
l
l
l
qi andqi areconsideredV-equivalent ifO1(qi,s)=O2(qj ,s)for
allsÎ V.
qi andqi areconsideredequivalent ifO1(qi,s)=O2(qj ,s)for
anysetV.
l
l
l
Statesthatarenotk-equivalentareconsideredk-distinguishable.
Itisalsoeasytoseethatiftwostatesarek-distinguishableforany
k>0thentheyarealsodistinguishableforanyn³ k.
IfM1 andM2 arenotk-distinguishablethentheyaresaidtobekequivalent.
Inotherwords,thereisnosÎ Xk suchthatO(qi,s)≠Q(qj,s)
Otherwise,theyaredistinguishable.
13
FSMProperties
l
l
l
l
FaultModel
Machineequivalence
l
MachinesM1 andM2 aresaidtobeequivalentif
(a)foreachstates inM1thereexistsastates'inM2 such
thats ands 'areequivalentand,
(b)foreachstates inM2 thereexistsastates 'inM1 such
thats ands 'areequivalent.
Minimalmachine
l
14
AnFSMMisconsideredminimalifthenumberofstatesin
MislessthanorequaltoanyotherFSMequivalenttoM.
15
l
AnFSMservestospecifythecorrectrequirementor
designofanapplication.Hencetestsgeneratedfrom
anFSMtargetfaultsrelatedtotheFSMitself.
l
Faultmodeldefinesasmallsetofpossiblefaultsthat
canoccurintheimplementation.
WhatfaultsaretargetedbythetestsgeneratedusinganFSM?
16
FaultModel(SequencingFaults)
a/1
a/0
q0
q0
a/1
b/1
a/1
q1
b/0
b/0
b/0
b/0
Operationerror
17
Transfererror
a/0
q0
a/1
q2
a/1
q1
b/0
Correctdesign
Extrastateerror
©Aditya P. Mathur 2009
Missingstateerror
18
MutantsofFSMs
MutantsofthemodelMd
l
Possibleimplementationsofthemodel
Obtainedbyintroducing1+ faults1+ times
l
b/0
q0
b/1
a/1
b/1
q1
MutantsofFSMs
l
b/1
q1
©Aditya P. Mathur 2009
l
a/1
a/1
q0
q1
Correctdesign
l
a/1
q0
a/1
b/1
FaultModel(SequencingFaults)
usethefaultmodelintroduced
SomemutantsmaybeequivalenttoMd.
l
l
Notethatthisisdifferenttoprogrammutation
l
Examplefirst-order
mutants
l
l
19
OutputbehaviorsofMd andthemutantsareidenticalonall
possibleinputs
Findatestthat
distinguishMandM1
Findatestthat
distinguishMandM2
Transfererror
20
Operationerror
FaultCoverage
l
TestGenerationusingChow’sMethod
Howtoevaluatethegoodnessofatestset?
l
l
CounthowmanyfaultsitrevealsinanimplementationMi
Assumptions
l
l
l
Minimal,connected,deterministic
Completelyspecified
Formaldefinitionoffaultcoverage,givenfaultmodel
l
Nt:totalnumberoffirst-ordermutantsofthemachineM
l
l
l
l
i.e.,thenumberofallpossibleimplementations
l
AlgorithmSketch(derivetestsetfromagivenFSMM)
l
Ne:numberofmutantsthatareequivalenttoM
Nf:numberofmutantsdistinguishedbyatestsetT
generatedusingatest-generationmethod.
Nl:numberofmutantsnotdistinguishedbyT
l
l
l
l
Estimatethemax.numberofstates(m)inthecorrect
implementationofM.
ConstructthecharacterizationsetWforM.
ConstructthetestingtreeforMandgeneratethetransition
coversetP fromthetestingtree.
ConstructsetZfromWandm.
DesiredtestsetisP·Z
21
Step1:Estimationofm
l
l
Wedonothaveaccesstothecorrectdesignorthe
correctimplementation.Soweneedtoestimate.
Thisisbasedonaknowledgeoftheimplementation.
Intheabsenceofanysuchknowledge,letm=|Q|.
22
Step2:ConstructionofW.WhatisW?
l
l
l
l
LetM=(X,Y,Q,q1,d,O)beaminimalandcomplete
FSM.
W isafinitesetofinputsequences thatdistinguishthe
behaviorofanypairofstatesinM.
Givenstatesqi andqj inQ,Wcontainsastringssuch
that:O(qi,s)¹O(qj,s)
Example
l
l
23
W={baaa,aa,aaa}
e.g.,baaadistinguishes
stateq1fromq2as
O(baaa,q1)¹ O(baaa,q2)
©Aditya P. Mathur 2009
24
StepstoConstructW
k-equivalencePartitionofQ?
l
Step1:Constructasequenceofk-equivalence
partitionsofQdenotedasP1,P2,…,Pm,m>0.
l
Step2:Traversethek-equivalencepartitionsin
reverseordertoobtaindistinguishingsequencefor
eachpairofstates.
©Aditya P. Mathur 2009
Ak-equivalencepartitionofQ,denotedasPk,isa
collectionofnfinitesetsSk1,Sk2…Skn suchthat
l
l
l
∪ni=1 Ski =Q
StatesinSki arek-equivalent.
IfstatevisinSki andvinSkj fori¹j,thenuandvarekdistinguishable.
25
HowtoConstructak-equivalencePartition?
l
l
GivenanFSMM,constructa1-equivalencepartition,
startwithatabularrepresentationofM.
26
Construct1-equivalencePartition
l
GroupstatesidenticalintheirOutputentries.This
givesus1-partitionP1 consistingofS1={q1,q2,q3}
andS2={q4,q5}.
S
1
2
27
©Aditya P. Mathur 2009
Current
state
a
Output
b
a
b
q1
0
1
q1
q4
q2
0
1
q1
q5
q3
0
1
q5
q1
q4
1
1
q3
q4
q5
1
1
q2
q5
28
Nextstate
Construct2-equivalencePartition
l
Construct2-equivalencePartition(cont.)
RewriteP1table.
l
l
l
Removetheoutputcolumns.
Replaceastateentryqi byqij wherejisthegroupnumber
inwhichliesstateqi.
S
Current
state
1
2
a
b
q1
q11
q42
q2
q11
q52
q3
q52
q11
q4
q31
q42
q5
q21
q52
©Aditya P. Mathur 2009
P3 Table
Nextstate
l
a
b
1
q1
q11
q43
q2
q11
q53
2
q3
q53
q11
3
q4
q32
q43
q5
q21
q53
30
l
Continuetheregroupingandrelabeling
Repeatconstructingk-eq.partitionsuntilconverge
S
a
b
q1
q11
q43
1
q2
q11
q54
2
q3
q54
3
q4
4
q5
1
Nextstate
ConstructP4 table
Current
state
S
Current
state
Construct4-equivalencePartition(cont.)
l
Groupallentrieswithidenticalsecondsubscripts under
thenextstatecolumn.
Relabelthegroups
P2 Table
S
©Aditya P. Mathur 2009
29
ConstructP3 table
l
l
Groupallentrieswithidenticalsecondsubscripts under
thenextstatecolumn.
Relabelthegroups
Group number
Construct3-equivalencePartition(cont.)
l
l
Nextstate
©Aditya P. Mathur 2009
l
ConstructP2 table
Current
state
Next state
a
b
q1
q11
q44
2
q2
q11
q55
q11
3
q3
q55
q11
q32
q43
4
q4
q33
q44
q21
q54
5
q5
q22
q55
31
©Aditya P. Mathur 2009
32
P4 Table
k-EquivalencePartition:Convergence
l
Theprocessisguaranteedtoconverge.(Pn =Pn+1)
l
Whentheprocessconverges,andthemachineis
minimal,eachstatewillbeinaseparategroup.
l
Thenextstepistoobtainthedistinguishingstrings
foreachstate.
©Aditya P. Mathur 2009
33