-1
{it
lI L;:l
tr f
J*JL
f}\
u'r ;*
t\--iy **
TalkingModel-Checking
Technology
A conuersationwith the2007ACMA.M. Turing Award winners.
rr,i!turi r.i. r,i.&{r, E. Allen
Emerson,and JosephSifakis
werehonored for their role in
developing Model-Checking
into a highly effective verification technology,widely adopted in
the hardwareand softwareindustries.
evenless receptive.Researchin the fbrmal-methodscommunig' in the 1980s
usually consisted of designing ancl
verifting tricky programs with fewer
than 50lines using only pen and paper.
If anyone asked how such a program
worked in practiceon a real computer,
it would havebeeninterpretedas an insult or perhapssimply as irrelevant.
EAEThe idea behind model checking was to avoid having humans construct proofs. It tLlrns out that many
important programs,suchas operating
systems,have ongoing behavior and
ideallyrun forever;tl-reydon't just start
and stop.ln I977,An-rirPnuelisuggested that temporal logic could be a good
wayto describeand reasonabout these
programs. Now, if a program can be
specifiedin temporal logic,then it can
be realizedas a finite stateprogram
a program with just a finite nurnber
of different configurations.This suggestedthe idea of model checking-to
checkwhether a finite stategraph is a
model of a temporal logic specification. Then one can develop efficient
algorithms to check whether the temporal-logicspecificationis true of the
state graph by searching through the
stategraph for certain patterns.
EMcYes, Allen and I noticed that
many concurrent programs had what
we called "finite state synchror-rizatior-r
skeletons." fioseph Sifakis and J,P.
Queillemade the sameobservation,independently.)For example,the part of
a mutual-exclusionprogram that handles synchronizationdoes not depencl
Let's talk about the history of formal
software verification.
E.ALLEN
EMERSoN
By the late 1960s,
we recognized that a program should
be viewed as a mathematical object.
It has a syntaxand semantics and formally defined behaviorengenderedby
that syntax and semantics.The idea
was to give a mathematical proof that
a p r o g r a m m e t a c e r t a i nc o r r e c t n e s s
specification.So one would havesome
axiomscharacterizingthe way the program worked for such-and-such an
instruction and some inferencerules,
and one would construct a fonnal
proof of the svstem,like philosophers
do sometimes.
But it neverreallyseemedto scaleup
to large programs. You ended up with
somethinglike 15-pagepapersproving
that a half-pageprogram was correct.It
was a greatideabut didn't seemto pan
out in practice.
What about the history of model
checking?
EDMUND
M.CLARKE
ThCbirth of model
checking was quite painful at times.
Like most researchon the boundarybetweentheoryand practice,theoreticians
thought the idea was trivial, and system
builders thought it was too theoretical.
Researchersin formal methods were
112
c o u u u r r c A T r o N s o F T H Ea c M
J U L y2 0 0 8
IcoNTTNUED ON P.1101
v0L.51
No 7
a:l!:ii t-i,/l"f
don't know how to veri$'verv complex
systelns.
EMcWe're aiwavs plar,ir-rg
a catchup galttei se're :llnitls behind. We,ve
developedntore pon'erfirltechniques,
but it's still difficuit to l<eepup with the
advanceof technologyand the comp l e x i n o f n e \ \ s' - v s t e n t s .
Canwe use model checkingto check
concurrent programs?
eas Arguably,model checking is a
very natural fit for parallel programming. Typically,we treat parallelism
as a nondetern-rinistic-or,informall,v,
random-choice, so,in a waya parallel
program is a more cornplexsequential
program,with n-ranynondeterministic
behaviors.Model checkingis verywell
suited to describing and reasoning
about the associatedcoordinationand
synchronizationpropertiesof parallel
programs.
EMCConcurrentprogramsare much
more difficr-rltto debug because it's
EAEIn hardwareveriflcation,Verilog
difficult for humans to keep track of a and VHDL are widely used design delot of things that are happening all at scription languages.Many industrial
once.Model checkingis ideal for that.
model checkers typically accept deJs But if you have progralls that signsdescribedin theselanguages.
interact with the physical environIs model checking something curment, tinte becomesven' in-rportant. rently taught to undergraduates?
For thesesystents,verificatiouis n-ruch
Js Formal verification is definitelv
more con.rplicatecl.
t a u g h ti n E u r o p e E
. u r o p eh a st r a d i r i o n Do we have any algorithms that can ally had a stronger community in foroperate directly on implementable mal methods,and I'd like to sayit has
code?
also traditionally had a strongercomEMcTo verify the processof trans- munity in semanticsand languages.
lating a design to code, or to verify
EMcYes,
there'salwavsbeenmore inthe code itself, is much more diffi- terestin verificationin Europethan in
cult. Some sr,rccessful
model checkers the U.S.Most of the major universities
use tl.risapproach,however.The Java here-CMU, Stanford,UC Berkeley,U.
Pathlinder n-rodelchecker cleveloped Texas,and so on-do offer coursesin
at NASAAntesgeneratesbvte code for model checking at both undergradua Javaprogram and sir-r.rulates
the byte ate and graduatelevels,but it hasn'tfilcodeto find errors.
tered down to schoolswhere no one is
.ls The best availabletechnologyis doing researchin the topic. part of that
proprietary technology that was de- has to do rvith the availabilityof approveioped by U.S.companies.But most priate textbooks;good books are just
of the code-levelmodel checl<ersare beginningto come out.
used to verify sequentiai software.If
EAE Formal methods are being
you want to verily concurrent software, taught with some frequency
fin the
then you needto be verycareful.
U.S.l,but they are not broadly incorEMcThe SLAM model checker de- porated into the core undergraduate
velopedat MicrosoftResearchfor find- curriculum as required courses to
ing errorsin Windows devicedriversis the extent that operatingsystemsand
probably the most successfulsoftware data structuresare.It is probablymore
model checker.It is now distributedto prevalentat the graduatelevel.But the
people who want to write device driv- distinction between undergraduate
ers for Windows. However,it is hardly and graduateis not clear-cut.At many
a general-purpose software model schools advancedundergrad and bechecker.
ginning grad overlap.
What's in store for model checking
and formal verification?
EMcI intend to continue looking at
waysof making model checkingmore
powerful.The stateexplosionphenomenon is still a difficult problem. I have
worked on it for 27 yearsand probably
will continue to do so.Another thing I
want to do is focus on embeddedsoftware systemsin automotiveand avionics applications.These programs are
often safety-critical.
For example,in a
few years, cars will be "drive-bv-wire";
there will be no mechanical linkage
between the steering wheel and tl"re
tires. The software will definitely need
to be verified.Fortunately,embedded
softwareis usually somewhatsimpler
in structure, without complex pointers;I think it may be more amenableto
model checkingtechniquesthar.rgeneral software.
us Personally,I believewe should
look into techniquesthat allow some
sort of compositionalreasoning,where
we infer global properties from local
properties of the system, becauseof
the inherent limitations of technioues
b a s e do n t h e a r r a lsl i so f a g l o b a lm o d e l .
I'm working on this, aswell as on theories of how to build systemsout of components,component-based
systems.
sae Model checking has caused a
sea changein the wav we think about
establishing program correctness,
from proof-theoretic(deductiveproof)
to model-theoretic(graph search). I
think we will continue to make more
or lessstead-y
progress,but the paceof
developmentof hardwareand software
is going to accelerate.
Whetherwe ever
catch up I don't know. Systemsthat
a r e b e i n g d e s i g n e da r e g e t t i n g b i g g e r
and messier.The seat-of-the-pants
approachwill no longerwork. We'll have
to get better at doing things modularly,
and we'll have to have better abstractions.
L e a h H o f f m a nw r r t c sa b o u ts c i c n c ea n d t e c h n o l o q y
f r o m B r o o k t y nN Y
o 2 0 0 8A C I \ 4
C 0 1 , 0 7 8 2 , , 0 8 , ' 0 750500 0
.r..1-':008 voL 51
N0 7
c o M M U N r c a r t o N so F T H E A c M
111
;**l by|r
IcoNTTNUED FROM P.112]
on the data being exchangedin the
criticalsections.Many communication
protocols had the same properry. We
decidedto seeifwe could analyzefinitestateprograms by algorithmic means.
How exactly does thatwork?
EAEYou havea program describedby
its text and its specificationdescribed
by its text in some logic.It's either true
or false that the program satisfiesthe
specification,and one wants to determine that.
JoSEPHsrFAKlsRight. You build a
mathematicalmodel [of the program],
and on this model, you check some
properties,which are also mathematically specified.To check the property,
you need a model-checkingalgorithm
that tal<esas input the mathematical
model you've constructed and then
givesan answer:"yes,""no," or "I don't
know." If the property is not verified,
you get diagnostics.
And to formalize those specifications, those properties...
eeeWhat people really want is the
program they desire,an inherently preformal notion. They have some vag'ue
idea about what sort of program they
want, or perhaps they have some sort
of committee that came up with an
Englishprose descriptionof what they
want the program to do, but it's not a
mathematicalproblem.
So one benefit of model checking
is that it forces you to preciselyspeciS
your design requirements.
EMcYes. But for many people, the
most important benefit is that if the
specificationisn't satisfied,the model
checker provides a counterexample
executiontrace.In other words, it provides a trace that shows you exactly
howyou get to an error that invalidates
110 coulauHrcATroN
o Fs T H EA c M
your specification,and often you can
use that to find really subtle errors in
design.
How have model-checking algorithms evolved over the years?
EMc Model-checking algorithms
have evolvedsignificantly over the past
27 years.The first algorithm for model
checking,developedby Allen and myself, and independently by Queille
and Sifakis, was a fixpoint algorithm,
and running time increasedwith the
squareof the number of states.I doubt
if it could havehandled a systemwith
a thousand states. The first implementation, the EMC Model Checl<er
(EMC stands for "Extended Model
Checker"), was based on efficient
graph algorithms, developedtogether
with Allen and PrasadSistla,another
student of mine, and achievedlinear
time complexityin the sizeof the state
space.We were able to veriS' designs
with about 40,000 states.Becauseof
the state-explosionproblem, this was
not sufficient in many cases;we were
still not able to handle industrial designs.My student Ken McMillan then
proposeda much more powerful technique calledsymbolicmodel checking.
We were able to check some examples
with 10 to the one-hundredth power
states(1 with a hundred zerosafter it).
This was a dramatic breakthrough but
was still unable to handle the state-explosion problem in many cases.In the
late 1990s,my group developeda technique called bounded model checking, which enabled us to find errors
in many designswith 10 to the 10,000
power states.
eae These advancesdocument the
basic contribution of model checking.
For the first time, industrial designs
are being verifled on a routine basis.
Organizations,such as IBM, Intel, Microsoft, and NASA, have key applications where model checkingis useful.
Moreover, there is now a large model-checking community, including
model-checkingusersand researchers
contributing to the advanceof modelcheckingtechnology.
What are the limitations of model
checking?
Js You have two basic problems:
how to build a mathematical model
of the systemand then how to check a
property, a requirement, on that mathematicalmodel.
/01 51
N0 7
First of all, it can be very challenging to construct faithful mathematical
models of complex systems.For hardware, it's relativelyeasyto extractmathematicalmodels,and we'vemade a lot
ofprogress.For software,the problem
is quite a bit more difficult. It depends
on how the software is written, but we
can veriS, a lot of complex software.
But for systemsconsistingof software
running on hardware,we don't know
how to construct faithful mathematical models for their verificatior.r.
The other limitation is in the complexity of the checking algorithm, and
here we have a problem called the
state-explosionproblem (that Clarke
referred to earlier), which means that
the number of the statesmay go exponentially high with the number of componentsof the system.
EMcSoftwareverification is a Grand
Challenge.By combining model checking with static analysistechniques,it
is possibleto find errors but not give
a correctness proof. As for the stateexplosionproblem, dependingon the
logic and model of computation,you
can prove theoretically that it is inevitable.But we'vedevelopeda number of
t e c h n i q u etso d e a lw i t h i t .
Such as?
ruc The most important technique
is abstraction.The basic idea is that
part of the progran, or the protocol
you'reverifting doesr-r't
reallvl.ravean1'
effecton the particularpropertiesthat
you're checking.So what you car-rdo is
simply eliminatethoseparticularparts
from the design.
You can alsocombine model checl<ing with compositional reasoning,
where you take a complex design and
break it up into smaller components.
Then you check those smaller components to deducethe correctnessof the
entire system.
How large are the programs we can
currently veri$ with model checking?
EMcWell, first of all, there's not
always a natural correspondencebetween a program's size and its cornplexity. But I would say we can often
check circuits with around 10 to the
100th power states(1 with a hundred
zerosafter it).
.ts Right. We know how to verifl' systems of medium complexity todayit's difficult to say but perhaps a program of around 10,000lines. But we