Property Directed Reachability for
Proving Absence of Concurrent
Modification Errors
Asya Frumkin
Yotam Feldman
Ondřej Lhoták
Oded Padon
Mooly Sagiv
Sharon Shoham
VMCAI 15.01.17
Main Contributions
• Abstraction of Java programs for the Concurrent
Modification Error (CME) using decidable logic
• Interprocedural PDR
• Universally quantified procedure summaries
• Property guided
• Implementation and initial evaluation
Concurrent Modification Exception (CME)
Concurrent Modification Exception (CME)
Concurrent Modification Exception (CME)
Illustrative Example
in
addList
out
flatten
Illustrative Example
in
out
Illustrative Example
in
out
Illustrative Example
in
out
Potential bug 1: in = out
CME
Illustrative Example
in
out
Potential bug 2: out  in
Illustrative Example
in
out
CME
Potential bug 2: out  in
The Concurrent Modification Problem
• Can reveal logical bugs in the program
• Interprocedural
func() {
…
// (subl  in)
flatten(in, subl)
…
}
CME
Challenges
• Complex call-graph:
 Deep call graph
 Recursive procedures
 Loops
• Infinite state space
 Unbounded collections
 Unbounded number of collections/iterators
 Undecidability
Our Approach
Java
program:
concrete
transitions
Abstraction
Abstract
transitions:
(V,V’)
in
decidable
logic
Safe
Interprocedural
PDR
Potentially
not safe
+
Universally
quantified
procedure
summaries
SP(V,V’)
Effectively Propositional Logic (EPR)
• A decidable fragment of first-order logic
• Formulas of the form **, where  satisfies the
following:
 Quantifier-free formula
 Can contain relation symbols and equality
 Stratified function symbols
• Example: x. y. r(x,y)  t(y)
Abstraction Relations and Symbols
Name
Parameters
Description
cnt(I)
I: iterator  C: Container Function which maps
iterators to their
underlying containers
member(C,O)
C: container, O: obj
Object is included in
container
stale(I)
I: iterator
Iterator is invalid
cme(I)
I: iterator
Concurrent modification
occurred
Abstraction Relations and Symbols
Name
Parameters
Description
cnt(I)
I: iterator  C: Container Function which maps
iterators to their
underlying containers
member(C,O)
C: container, O: obj
Object is included in
container
stale(I)
I: iterator
Iterator is invalid
cme(I)
I: iterator
Concurrent modification
occurred
Abstraction Relations and Symbols
Name
Parameters
Description
cnt(I)
I: iterator  C: Container Function which maps
iterators to their
underlying containers
member(C,O)
C: container, O: obj
Object is included in
container
stale(I)
I: iterator
Iterator is invalid
cme(I)
I: iterator
Concurrent modification
occurred
Abstraction Relations and Symbols
Name
Parameters
Description
cnt(I)
I: iterator  C: Container Function which maps
iterators to their
underlying containers
member(C,O)
C: container, O: obj
Object is included in
container
stale(I)
I: iterator
Iterator is invalid
cme(I)
I: iterator
Concurrent modification
occurred
Abstraction Relations and Symbols
Name
Parameters
Description
cnt(I)
I: iterator  C: Container Function which maps
iterators to their
underlying containers
member(C,O)
C: container, O: obj
Object is included in
container
stale(I)
I: iterator
Iterator is invalid
cme(I)
I: iterator
Concurrent modification
occurred
Abstraction of Add Operation
c.add(o)
 C,O,I. (member'(C, O)  member(C, O)  (C = c  O = o))
 (stale'(I)  stale(I)  cnt(I) = c)
in
out
cnt
member
c
o
cnt
it2
cnt
it1
(C – container, O – object, I – iterator)
o
it2
stale
c
cnt
it1
stale
Our Abstraction of Programs in EPR
• Pros:
 Captures essence of collections, iterators and CME
 Decidability of EPR
• Cons:
 Potential loss of precision:
• Not modeling order in collections
• Arithmetic
• Treating conditions as non-deterministic transitions
Our Approach
Java
program:
concrete
transitions
Abstraction
Abstract
transitions:
(V,V’)
in
decidable
logic
Safe
Interprocedural
PDR
Potentially
not safe
+
Universally
quantified
procedure
summaries
SP(V,V’)
Procedure Summary
A two-vocabulary formula that overapproximates the
input/output relation of the procedure.
• Globals
• Input parameters
in
P
Concrete
Summary
• Changed globals
• Output parameters
out
Procedure Summary
A two-vocabulary formula that overapproximates the
input/output relation of the procedure.
• Globals
• Input parameters
P
• Changed globals
• Output parameters
Our algorithm computes
universally quantified procedure
summaries that are sufficient to
prove the safety property:
I. cme(I) (I – iterator)
Summary for Flatten Procedure
S(flatten) =  I. in  out  member(in, out)  cme’(I)
(I – iterator)
Essence of Computing Procedure
Summaries - Example
{ true }
S(genLists)();
SAT
S(addList)();
{ I. cme(I) }
YES
Summaries
S(addTwo) = true
S(genLists) = true
?
Essence of Computing Procedure
Summaries - Example
Model
< a=lst1, b=lst2, cme={}, cnt(it1)=lst1 >
S(genLists)();
< a=lst1, b=lst2, cme={}, cnt(it1)=lst1 >
S(addTwo)();
< a=lst1, b=lst2, cme={it1}, cnt(it1)=lst1 >
Summaries
S(addTwo) = true
S(genLists) = true
Essence of Computing Procedure
Summaries - Example
< a=lst1, b=lst2, cme={}, cnt(it1)=lst1 >
Iterator it = a.iterator();
SAT
?
b.add(it.next());
b.add(it.next());
< a=lst1, b=lst2, cme={it1}, cnt(it1)=lst1 >
Summaries
S(addTwo) = true
S(genLists) = true
Diagram
An existential formula that provides abstraction for the
finite transition s = (in, out).
U = {lst1, lst2, it1}
in
out
lst2
b
lst1
a
cnt
it1
lst2
b
lst1
a
cnt
it1
CME
[CAV 2015] Karbyshev, A., Bjorner, N., Itzhaky, S., Rinetzky, N., Shoham, S.: Property directed inference
of universal invariants or proving their absence.
Diagram
An existential formula that provides abstraction for the
finite transition s = (in, out).
U = {lst1, lst2, it1}
in
cme = { }
cnt(it1) = lst1
a = lst1
b = lst2
out
cme’ = {it1}
cnt’(it1) = lst1
a’ = lst1
b’ = lst2
𝐷 𝑠 = ∃𝑥: 𝐶, 𝑦: 𝐶, 𝑧: 𝐼. 𝑥 ≠ 𝑦 ∧ 𝑥 = 𝑎 ∧ 𝑦 = 𝑏 ∧ 𝑥 = 𝑎′ ∧
𝑦 = 𝑏 ′ ∧ cme(𝑧) ∧ cnt(𝑧) = 𝑥 ∧ cnt′(𝑧) = 𝑥 ∧ cme′(𝑧)
[CAV 2015] Karbyshev, A., Bjorner, N., Itzhaky, S., Rinetzky, N., Shoham, S.: Property directed inference
of universal invariants or proving their absence.
Diagram
An existential formula that provides abstraction for the
finite transition s = (in, out).
U = {lst1, lst2, it1}
in
cme = { }
cnt(it1) = lst1
a = lst1
b = lst2
out
cme’ = {it1}
cnt’(it1) = lst1
a’ = lst1
b’ = lst2
𝐷 𝑠 = ∃𝑧: 𝐼. 𝑎 ≠ 𝑏 ∧ 𝑎′ = 𝑎 ∧ 𝑏 ′ = 𝑏 ∧
cme(𝑧) ∧ cnt(𝑧) = 𝑎 ∧ cnt′(𝑧) = 𝑎 ∧ cme′(𝑧)
[CAV 2015] Karbyshev, A., Bjorner, N., Itzhaky, S., Rinetzky, N., Shoham, S.: Property directed inference
of universal invariants or proving their absence.
Essence of Computing Procedure
Summaries - Example
D(s)
BodyaddTwo
SAT
Iterator it = a.iterator();
b.add(it.next());
b.add(it.next());
∃𝑧: 𝐼. 𝑎 ≠ 𝑏 ∧ 𝑎′ = 𝑎 ∧
∧
𝑏 ′ = 𝑏 ∧ cme(𝑧) ∧
cnt(𝑧) = 𝑎 ∧
cnt′(𝑧) = 𝑎 ∧ cme′(𝑧)
NO
s –concrete counterexample
transition
Summaries
S(addTwo) = true
S(genLists) = true
?
Essence of Computing Procedure
Summaries - Example
UNSATCORE:
Iterator it = a.iterator();
b.add(it.next());
∧
∃𝑧: 𝐼. cme′(𝑧)  𝑎  𝑏
b.add(it.next());
Iterator it = a.iterator();
b.add(it.next());
b.add(it.next());
⇒ ∀𝑧: 𝐼. ¬cme′(𝑧) ∨ 𝑎 = 𝑏
Essence of Computing Procedure
Summaries - Example
UNSATCORE:
Iterator it = a.iterator();
b.add(it.next());
∧
∃𝑧: 𝐼. cme′(𝑧)  𝑎  𝑏
b.add(it.next());
Iterator it = a.iterator();
b.add(it.next());
b.add(it.next());
⇒ ∀𝐼. 𝑎 ≠ 𝑏  ¬cme′(I)
Essence of Computing Procedure
Summaries - Example
{ true }
S(genLists)();
SAT
S(addList)();
{ I. cme(I) }
YES
Summaries
S(addTwo) = a  b  cme’(I)
S(genLists) = true
?
Essence of Computing Procedure
Summaries - Example
Model
< a=lst1, b=lst1, cme={}, cnt(it1)=lst1 >
S(genLists)();
< a=lst1, b=lst1, cme={}, cnt(it1)=lst1 >
S(addToList)();
< a=lst1, b=lst1, cme={it1}, cnt(it1)=lst1 >
Summaries
S(addTwo) = a  b  cme’(I)
S(genLists) = true
Diagram
An existential formula that provides abstraction for the
finite transition s = (in, out).
U = {lst1, it1}
in
out
lst1
a
b
lst1
a
b
cnt
it1
cnt
it1
Diagram
An existential formula that provides abstraction for the
finite transition s = (in, out).
U = {lst1, it1}
in
cme = { }
cnt(it1) = lst1
a = lst1
b = lst1
out
cme’ = { }
cnt’(it1) = lst1
a’ = lst1
b’ = lst1
𝐷 𝑠 = ∃𝑧: 𝐼. 𝑎 = 𝑏 ∧ 𝑎′ = 𝑏 ′ ∧ 𝑎′ = 𝑎 ∧
cme(𝑧) ∧ cnt(𝑧) = 𝑎 ∧ cnt′(𝑧) = 𝑎 ∧ ¬cme′(𝑧)
Essence of Computing Procedure
Summaries - Example
D(s)
BodygenLists
SAT
a = <init a>;
b = <init b>;
∃𝑧: 𝐼. 𝑎 = 𝑏 ∧ 𝑎′ = 𝑏′ ∧
∧
𝑎′ = 𝑎 ∧ cme(𝑧) ∧
cnt(𝑧) = 𝑎 ∧
cnt′(𝑧) = 𝑎 ∧ ¬cme′(𝑧)
NO
s –concrete counterexample
transition
Summaries
S(addTwo) = a  b  cme’(I)
S(genLists) = true
?
Essence of Computing Procedure
Summaries - Example
UNSATCORE:
a = <init a>;
b = <init b>;
a = <init a>;
b = <init b>;
∧
⇒ 𝑎′ ≠ 𝑏′
𝑎′ = 𝑏′
Essence of Computing Procedure
Summaries - Example
Summaries
S(addTwo) = a  b  cme’(I)
S(genLists) = a’  b’
Essence of Computing Procedure
Summaries - Example
{ true }
S(genLists)();
SAT
S(addList)();
{ I. cme(I) }
NO
Summaries
S(addTwo) = a  b  cme’(I)
S(genLists) = a’  b’
?
Dealing with Recursion
Interprocedural PDR
The algorithm maintains a sequence of frames F0, F1,
F2,...
Fi:   Si()
 - program procedures
Si - summary of procedure behavior up to call-stack depth i
[SAT 2012] Hoder, K., Bjorner, N.: Generalized property directed reachability.
]CAV 2014] Komuravelli A., Gurnkel A., Chaki S.: Smt-based model checking for recursive programs.
Frame Properties
1. Fi(main)  Safe
Bad 
ꓱ i. cme(i)
in
Fi(main)
out
Frame Properties
1. Fi(main)  Safe
2. P. Fi(P)  Fi+1(P)
in
Fi(P)
Fi+1(P)
out
Frame Properties
1. Fi(main)  Safe
2. P. Fi(P)  Fi+1(P)
3. P. BodyP[𝐹𝑖 (𝑄)ൗ𝑄]  Fi+1(P)
P
in
out
Fi(Q)
Q
Frame Properties
1. Fi(main)  Safe
2. P. Fi(P)  Fi+1(P)
3. P. BodyP[𝐹𝑖 (𝑄)ൗ𝑄]  Fi+1(P)
P
in
out
Fi+1(P)
Q
Algorithm Termination
The algorithm terminates when:
• Frame sequence converges, i.e Fi+1  Fi
↳ Program is safe
• Abstract counterexample is found
↳ Program is possibly not safe
Algorithm Termination
• Counterexample represents an abstract trace
• Perform bounded model checking to find a concrete
counterexample:
• Concrete counterexample is found
↳ Program is not safe
• No counterexample found
↳ No universal summaries exist for program
Experimental Results
Program
#Lines
#Methods
Time
(s)
#Summaries
Max
frame
#Z3
Max
summary
sm
53
6
514
8
11
5263
14
div
27
2
291
7
11
2767
15
worklist
26
5
577
8
17
7475
41
map_test
40
3
306
10
14
4653
16
c
18
3
33
8
6
791
8
flatten
45
3
800
14
18
10412
39
c_error
18
3
35

5
662

flatten_error
45
3
790

15
8746

Main Contributions
• Abstraction of Java programs for the Concurrent
Modification Error (CME) using decidable logic
• Interprocedural PDR
• Universally quantified procedure summaries
• Property guided
• Implementation and initial evaluation
Thank You!