Automated Extraction of Inductive Invariants
to Aid Model Checking
Mike Case
DES/CHESS Seminar
EECS Department, UC Berkeley
April 10, 2007
Motivation
• Want to build the fastest unbounded model
checker
– Can get significant speedup from knowledge of a few
local properties (inductive invariants)
• Want to find and prove inductive invariants
– Not all invariants are useful for model checking
• In previous work, we have a way to quickly
prove many local properties
– Can these be limited to properties that are useful for
verification?
– Is there an easy way to incorporate this information
into a model checker?
April 10, 2007
Mike Case, DES Seminar
2
•
Approximate
Reachable States •
•
Sequential
Synthesis
•
•
•
Interpolation
Known-unreachable states
are sequential don’t-cares
IWLS ’06
Upcoming IBM Internship
•
Better Verification
•
•
•
•
Graph Theory
•
Synthesis
Verification •
•
Build reachability approximation
Interpolation explores a smaller
state space
EECS 219C
•
Quickly refute
candidate properties
• Under investigation
Only help where reachability info
needed most
Motivated by work at Calypto
IWLS ’07
Finding Inductive
Invariants
•
EECS 219C
Sequential
Simulation
Efficiently store candidate
properties
IWLS ’06, WG ’06, WG ’07
April 10, 2007
Quickly find and prove small
properties
Approximation to reachability
EECS 290a, IWLS ’06
Mike Case, DES Seminar
3
Outline
Approximate
Reachable States
Sequential
Synthesis
Interpolation
Better Verification
Sequential
Simulation
Graph Theory
Synthesis
Verification
April 10, 2007
Finding Inductive
Invariants
Mike Case, DES Seminar
4
Approximating the Reachable
States
• Prove local properties hold  reachable states
• Conjunction gives reachability approximation
I
April 10, 2007
Mike Case, DES Seminar
5
Quickly Proving Local Properties
• Use simple induction to prove the properties
– “Sequential Equivalence Checking without State Space
Traversal,” van Eijk, DATE ‘98
• Biggest obstacle is an overly large set of candidate
properties
– Candidates discovered through random simulation, but
inadequate for large designs
– Candidates are Boolean implications
• Lots of expressive power
• Can minimize the number of implications under test by applying a
reduction technique on the implication graph
– Can also window the candidate set to only prove small subsets
at a time
• Can hurt results because sometimes we need multiple implications
to be proved in parallel
April 10, 2007
Mike Case, DES Seminar
6
Outline
Approximate
Reachable States
Sequential
Synthesis
Interpolation
Better Verification
Sequential
Simulation
Graph Theory
Synthesis
Verification
April 10, 2007
Finding Inductive
Invariants
Mike Case, DES Seminar
7
Motivation for interpolation
• Desire to experiment with a state-of-the-art
model checker
• Chose interpolation because it is complete
and fast
– “Interpolation and SAT-Based Model
Checking,” McMillan, CAV ’03
– “An Analysis of SAT-based Model Checking
Techniques in an Industrial Environment,”
Amla, CHARME ‘05
April 10, 2007
Mike Case, DES Seminar
8
Reachability-Based Verification
frontier := initial states
Image 2
Bad state reached?
Image 1
sat
I
B
unsat
frontier += image(frontier)
no
Fixed Point?
yes
April 10,Property
2007
Verified
Property Falsified
Mike Case, DES Seminar
9
Interpolation
Initialize approximation
parameters
Reachability:
Tighten approximation
parameters
Image 2
Image 1
frontier := initial states
Bad state reached?
B
I
sat
Interpolation:
unsat
Image 2
frontier +=
approxImage(frontier)
no
Fixed Point?
yes
April 10,Property
2007
Verified
Cex reached
on a BMC from the
initial state?
Image 1
no
I
B
S
yes
Property Falsified
Mike Case, DES Seminar
10
Problems With Interpolation
• Can explore unreachable states
– No control over the approximate image and can
contain unreachable states
– If an unreachable state enters start, many other
unreachables will follow
– Can lead to an unreachable bad state being explored
• Requires frequent model refinements
– Refining the approximation parameters and restarting
is the most expensive operation
– Discards all prior work
April 10, 2007
Mike Case, DES Seminar
11
Outline
Approximate
Reachable States
Sequential
Synthesis
Interpolation
Better Verification
Sequential
Simulation
Graph Theory
Synthesis
Verification
April 10, 2007
Finding Inductive
Invariants
Mike Case, DES Seminar
12
Enhanced Interpolation
• Interpolation may explore unreachable
states
• Approximate reachable states to help
bound number of interpolation iterations
Quickly approximate
reachable states
Interpolate
Property Verified
April 10, 2007
Property Falsified
Mike Case, DES Seminar
13
Key Observations From
Experimental Results
• Preprocessing imposes a runtime penalty
– Can be minimized by resource thresholding
– Need to give it sufficient time to prove properties
• Ignoring overhead…
– Faster in 92% of designs, can solve 4% that
previously timed out
– Slower in 4% of the benchmarks
• Not all invariants help the model checker
• Adding a constraint to a SAT solver might slow it down
• We can do better!
April 10, 2007
Mike Case, DES Seminar
14
Outline
Approximate
Reachable States
Sequential
Synthesis
Interpolation
Better Verification
Sequential
Simulation
Graph Theory
Synthesis
Verification
April 10, 2007
Finding Inductive
Invariants
Mike Case, DES Seminar
15
A Better Way to Enhance Interpolation
• Abstraction refinement most expensive
• Show either S or B unreachable
– No other constraints matter
• Suppose we had a tool to find invariants to
do this
Image 2
Image 1
I
April 10, 2007
S
Mike Case, DES Seminar
B
16
Targetted Invariant Tool
• Given a state S that we want to prove
unreachable
• Find {P} such that
– Implies that S is unreachable
– Can be proved with simple induction
April 10, 2007
Mike Case, DES Seminar
17
Initialize approximation
parameters
Tighten approximation
parameters
frontier := initial states
Bad state reached?
no
yes
Can we
find invariants?
sat
unsat
frontier +=
approxImage(frontier)
Cex reached
on a BMC from the
initial state?
no
Fixed Point?
yes
no
yes
Property Falsified
Property Verified
April 10, 2007
Mike Case, DES Seminar
18
Another Application
• We’ve helped interpolation
– Short-circuited expensive refinement
• Can we help other applications?
– Consider simple induction
– Technique used to prove properties
– Is often incomplete. Can we fix this?
April 10, 2007
Mike Case, DES Seminar
19
Simple Induction Can Also Be Helped
Obtain a set of
candidate properties
Remove violated
candidates
yes
Is there a way to
violate the
base case?
Base Case
Do all properties hold in all
initial states?
no
Remove violated
candidates
yes
Is there a way to
violate the
inductive step?
Inductive Step
For all states where the
properties hold, do they
hold in all next states also?
no
Remaining candidates hold
 reachable states
April 10, 2007
Mike Case, DES Seminar
“Sequential Equivalence
Checking without State Space
Traversal,” van Eijk, DATE ‘98
20
Enhancing Simple Induction
• Simple induction is fast, but often fails to
prove properties that are true
S
p
XS
p
¬p
• If can show that S or XS unreachable,
then this structure can’t disrupt the proof
April 10, 2007
Mike Case, DES Seminar
21
Proving Properties by Induction
Obtain a set of
candidate properties
Remove violated
candidates
yes
yes
Is there a way to
violate the
base case?
no
no
Remove violated
candidates
Remove violated
candidates
yes
Is there a way to
violate the
inductive step?
yes
no
Remaining candidates hold
 reachable states
April 10, 2007
Can we
find invariants?
Mike Case, DES Seminar
… with possibly reachable
counterexample states
22
Proving That A State Is Unreachable
• Multiple areas could benefit from a tool
that could prove 1 state unreachable
– Interpolation
– Simple Induction
• Previous work proves a large set of states
unreachable
– Proves many small properties
– Can we limit the properties to target states of
interest?
April 10, 2007
Mike Case, DES Seminar
23
The Proof Graph
S
{ P}
{ P}
(a state)
(a set of
properties)
S
(a set of
properties)
(a state)
• Every property in the set is • S is the reason the inductive
proof of the properties does not
violated in S
succeed
• Proving any such property
– S is the counterexample in the
implies that S is
inductive step of the proof
unreachable
• Proving S unreachable is a
• {P} are how we will prove S
necessary condition for proving
unreachable
any property in the set
• S is why we can’t prove {P}
April 10, 2007
Mike Case, DES Seminar
24
Proof Graph Example
S0
{ P0 1 }
{ P0 2 }
{ P0 3 }
S1
S2
S3
{ P2 }
{ P3 }
{ P1 }
April 10, 2007
• Input S0
• Find properties
violated in S0
• Prove {P0}
• Cover the new states
with properties
• Prove {P3}
• Prove {P03}
Mike Case, DES Seminar
25
Proof Graph Notes
S0
{ P0 1 }
{ P0 2 }
{ P0 3 }
S2
S3
{ P2 }
{ P3 }
S1
{ P1 }
April 10, 2007
• Proof of a property set
implies that all parent
states are unreachable
• Proof attempt on leaves
only
• Leaves can be proved
independently
• Select shallowest leaf for
next proof
Mike Case, DES Seminar
26
Special Case: Cycles
S0
{ P0 }
S1
{ P1 }
• If a cycle develops…
• Cannot prove either
property set independently
• Might be able to prove
them together
• Successful proof implies
{ P } = { P }  { P }
both states unreachable
April 10, 2007
2
0
1
Mike Case, DES Seminar
27
Initialize approximation
parameters
Tighten approximation
parameters
frontier := initial states
Bad state reached?
no
yes
Can we
find invariants?
sat
unsat
frontier +=
approxImage(frontier)
Cex reached
on a BMC from the
initial state?
no
Fixed Point?
yes
no
yes
Property Falsified
Property Verified
April 10, 2007
Mike Case, DES Seminar
28
Interpolation Results
10000
• Solves some
problems that
previously timed
out
• Needs work
Assisted Time (sec)
1000
100
10
1
0.1
0.1
1
10
100
1000
10000
Interpolation Time (sec)
April 10, 2007
Mike Case, DES Seminar
29
Notes on Performance
• What is “good performance” for a verification
tool?
– Only meaningful statistics are time, memory, and
whether or not verification completed
– Industry very concerned with completion on a large
set of problems
– Slower on average is ok, if we complete on most
benchmarks
• We verify a few designs that previously timed
out, but not enough
– Lack a powerful simulator
– Lots of false properties in the candidate set
– This is a work in progress
April 10, 2007
Mike Case, DES Seminar
30
Outline
Approximate
Reachable States
Sequential
Synthesis
Interpolation
Better Verification
Sequential
Simulation
Graph Theory
Synthesis
Verification
April 10, 2007
Finding Inductive
Invariants
Mike Case, DES Seminar
31
Simulation Motivation
70000
• Safety property in
S420
60000
– “Small” design
– Timeout for me
Candidate Properties
50000
40000
• 50% of candidates
remain after 10
minute sim
30000
20000
10000
0
0
100
200
300
400
500
600
Seconds
April 10, 2007
Mike Case, DES Seminar
– Initial candidate set
is quite poor
– Can we refine it in
30 seconds?
32
Future Work
• Continue improving my implementation
– Refine candidates with more/better simulation
– Try other property domains – maybe
implications are the wrong choice
• Apply my tool in other applications
– Could synthesis benefit from knowing that 1
interesting state is unreachable?
– Can another state of the art model checker
(UCSB) be fitted to use my tool?
April 10, 2007
Mike Case, DES Seminar
33
Questions?
April 10, 2007
Mike Case, DES Seminar
34
Backup
April 10, 2007
Mike Case, DES Seminar
35
Outline
Approximate
Reachable States
Sequential
Synthesis
Interpolation
Better Verification
Sequential
Simulation
Graph Theory
Synthesis
Verification
April 10, 2007
Finding Inductive
Invariants
Mike Case, DES Seminar
36
Sequential Synthesis
•
•
•
•
Over-approximate reachable states
Under-approximate unreachables
Safe to use as sequential don’t cares
Sequential don’t cares from 2 minutes of
pre-processing give same synthesis
results as exact seqdc set
April 10, 2007
Mike Case, DES Seminar
37
Outline
Approximate
Reachable States
Sequential
Synthesis
Interpolation
Better Verification
Sequential
Simulation
Graph Theory
Synthesis
Verification
April 10, 2007
Finding Inductive
Invariants
Mike Case, DES Seminar
38
Why Graph Theory
• motivation
April 10, 2007
Mike Case, DES Seminar
39
Minimum Equivalent Graph
A
C
April 10, 2007
D
B
F
Mike Case, DES Seminar
E
G
40
Sequential Simulation
• Many parallel• Identify “interesting”
random walks states
• Re-start random walk
from each interesting
state
• Alan Mishchenko
April 10, 2007
Mike Case, DES Seminar
• BMC from the
interesting states
• Jason Baumgartner,
IBM
41