DIAGNOSING
ABSTRACTION FAILURE
IN SEPARATION LOGIC-BASED ANALYSES
Arlen Cox
Josh Berdine
Samin Ishtiaq
Christoph Wintersteiger
The Abstraction Refinement Dream
Start
Verification
Pick Abstraction
Attempt
Proof
Pick New
Abstraction
Failure
Diagnose Failure
Find
Counterexample
Fix Bug
Success
Success
State of the Art
Separation Logic Analysis
Start
Verification
Pick Abstraction
Attempt
Proof
Pick New
Abstraction
Failure
2 Diagnose Failure
Find
1 Counterexample
Fix Bug
Success
Success
Traditional Abstraction
Refinement
Not Our
Contribution
Pick Abstract/Attempt Proof
Proof Fails
Diagnosing Abstraction Failure
1. An Abstract State
2. Concrete State
• Unreachable
• Reaches Error
• Contained in Abstract
State
Weakest
Precondition
Partition the Abstract State
No WP() in Separation Logic
Weakest
Precondition
No WP() in Separation Logic
int* p;
…
*p = 17;
PSPACE-complete*
due to aliasing
* Calcagno, C., Yang, H., O’Hearn, P.W.: Computability and complexity results for a spatial
assertion language for data structures. In: FSTTCS (2001)
Separation Logic-based Analyses
 Restricted Logic
• Does not support separating implication (
),
general negation (
), general conjunction (
)
 Do not support backward reasoning
• No weakest precondition
 Contribution: A method to use forward analysis
to diagnose failures
 Contribution: A method for efficiently
performing forward counterexample searches
Example
l
…
l = new ListNode(new Obj(), NULL);
while(*) {
l = new ListNode(new Obj(), l);
}
while(l != NULL) {
n = l->next;
free(l->data);
free(l);
l = n;
}
l
NULL
Background: Pick Abstraction
Start
Verification
Pick Abstraction
Attempt
Proof
Pick New
Abstraction
Failure
Diagnose Failure
Find
Counterexample
Fix Bug
Success
Success
Pattern-Based Abstraction
l
…
NULL
Pattern-Based Abstraction
l
…
NULL
Pattern-Based Abstraction
l
…
NULL
Background: Proof Attempt
Start
Verification
Pick Abstraction
Attempt
Proof
Pick New
Abstraction
Failure
Diagnose Failure
Find
Counterexample
Fix Bug
Success
Success
Proof Search (SLL)
l = new ListNode(new Obj(), NULL);
while(*) {
l = new ListNode(new Obj(), l);
}
while(l != NULL) {
n = l->next;
free(l->data);
free(l);
l = n;
}
Proof Search (SLL)
l = new ListNode(new Obj(), NULL);
l = new ListNode(new Obj(), NULL);
while(*) {
l = new ListNode(new Obj(), l);
}
while(l != NULL) {
n = l->next;
free(l->data);
free(l);
l = n;
}
Proof Search (SLL)
l = new ListNode(new Obj(), NULL);
l = new ListNode(new Obj(), l);
l = new ListNode(new Obj(), NULL);
while(*) {
l = new ListNode(new Obj(), l);
}
while(l != NULL) {
n = l->next;
free(l->data);
free(l);
l = n;
}
Proof Search (SLL)
l = new ListNode(new Obj(), NULL);
l = new ListNode(new Obj(), l);
assume(l != NULL)
n = l->next;
free(l->data);
l = new ListNode(new Obj(), NULL);
while(*) {
l = new ListNode(new Obj(), l);
}
while(l != NULL) {
n = l->next;
free(l->data);
free(l);
l = n;
}
Counterexamples
Start
Verification
Pick Abstraction
Attempt
Proof
Pick New
Abstraction
Failure
Diagnose Failure
Find
Counterexample
Fix Bug
Success
Success
Traditional Bounded Model
Checking
Not Our
Contribution
l = new ListNode(new Obj(), NULL);
l = new ListNode(new Obj(), l);
assume(l != NULL)
n = l->next;
free(l->data);
free(l);
l = n;
assume(l == NULL)
1. Unroll Transition
System
2. Check Property
3. Repeat
- Can explode for
deep properties
- Doesn’t help
proof process
Contribution: BMC Over Abstract
Transition System
l = new ListNode(new Obj(), NULL);
l = new ListNode(new Obj(), l);
assume(l != NULL)
n = l->next;
free(l->data);
1. Unroll Abstract
Transition System
2. Check Property
3. Repeat
+ Restricted search
space
+ Finds counterexamples that
caused this proof
failure
Contribution: BMC Over Abstract
Transition System
Contribution: BMC Over Abstract
Transition System
Contribution: BMC Over Abstract
Transition System
Must End
in Error
Contribution: BMC Over Abstract
Transition System
Contribution: BMC Over Abstract
Transition System
Unroll up to a
bound
Contribution: BMC Over Abstract
Transition System
Stay in Error
Contribution: BMC Over Abstract
Transition System
Otherwise Transition
According to
Program
Contribution: BMC Over Abstract
Transition System
Send to SMT solver; quantifiers and all.
Contribution: BMC Over Abstract
Transition System
Encoding of
Address
Data
Allocated
Siz
e
p = malloc(size);
Encoding of
Address
Data
Allocated
Siz
e
p = malloc(size);
q = malloc(size);
Encoding of
Address
Data
Allocated
Siz
e
p = malloc(size);
q = malloc(size);
Encoding of
Address
Data
Allocated
Siz
e
p = malloc(size);
q = malloc(size);
r = p + size;
*r = 3; //(no error)
Encoding of
Address
Data
Allocated
Siz
e
p = malloc(size);
q = malloc(size);
r = p + size;
*r = 3; //(error)
Counterexample Search
l = new ListNode(new Obj(), NULL);
l = new ListNode(new Obj(), l);
assume(l != NULL)
n = l->next;
free(l->data);
Counterexample Search
l = new ListNode(new Obj(), NULL);
l = new ListNode(new Obj(), l);
Just need structure.
assume(l != NULL)
n = l->next;
free(l->data);
Don’t need separation
logic formulas
Counterexample Search
No Error
Counterexample Search
l
l = new ListNode(new Obj(), NULL);
NULL
No Error
Counterexample Search
l
l = new ListNode(new Obj(), NULL);
assume(l != NULL)
n = l->next;
free(l->data);
l = new ListNode(new Obj(), l);
NULL
Error
Unreachable
No Error
Counterexample Search
l
l = new ListNode(new Obj(), NULL);
assume(l != NULL)
n = l->next;
free(l->data);
Error
Unreachable
l = new ListNode(new Obj(), l);
NULL
No Error
Counterexample Search
l
l = new ListNode(new Obj(), NULL);
assume(l != NULL)
n = l->next;
free(l->data);
Error
Unreachable
l = new ListNode(new Obj(), l);
l = new ListNode(new Obj(), l);
assume(l != NULL)
n = l->next;
free(l->data);
Error
Unreachable
NULL
No Error
Counterexample Search
 Produces concrete counterexamples
 Contribution: Only explores failed proof
• Finds counterexamples that would cause this
particular proof failure
 Contribution: Relies on SMT solver for
unrolling
• Property-guided, intelligent backtracking
 Bit-precise memory model
Contribution: Diagnosing Failure
Start
Verification
Pick Abstraction
Attempt
Proof
Pick New
Abstraction
Failure
Diagnose Failure
Find
Counterexample
Fix Bug
Success
Success
Diagnosing the Failure
Was the abstraction
here responsible for
failure?
l = new ListNode(new Obj(), NULL);
l = new ListNode(new Obj(), l);
assume(l != NULL)
n = l->next;
free(l->data);
Diagnosing the Failure
Delete
Program
Before Join
Point
l = new ListNode(new Obj(), l);
assume(l != NULL)
n = l->next;
free(l->data);
Diagnosing
the Failure
Synthesize
Program
Prefix that Creates
Abstract State
Precisely
l = NULL
l = new ListNode(*, l);
Re-run
Non-deterministic
Counterexample
l = new ListNode(new Obj(), l);
data field Search
assume(l != NULL)
n = l->next;
free(l->data);
Error Found!
Diagnosing the Failure
Diagnosing the Failure
Diagnosing the Failure
Diagnosing the Failure
Diagnosing the Failure
for p in Join_Points(ATS) {
ATS’ = Synthesize_Prefix(p, ATS)
CEx = Find_Counterexample(ATS’)
if(exists CEx) {
ATS = Refine(ATS, p, CEx);
}
}
Picking New Abstraction
Start
Verification
Pick Abstraction
Attempt
Proof
Pick New
Abstraction
Failure
Diagnose Failure
Find
Counterexample
Fix Bug
Success
Success
Picking New Abstraction
 Partial order of abstractions
 Pick next best abstraction
Proof Search with SLL_OBJ
l = new ListNode(new Obj(), NULL);
l = new ListNode(new Obj(), l);
assume(l != NULL)
n = l->next;
free(l->data);
free(l);
l = n;
assume(l == NULL)
Conclusions
Start
Verification
Pick Abstraction
Attempt
Proof
Pick New
Abstraction
Failure
Diagnose Failure
Find
Counterexample
Fix Bug
Success
Success
Conclusions
Start
Verification
Pick Abstraction
Pick New
Abstraction
Failure
Diagnose Failure
New BMC Approach
• Search abstract transition
system instead of program
Success
Attempt
• Only finds
causes for
Proof
proof failure
• Use monolithic encoding
• Take advantage of
intelligent backtracking
Find
✔Counterexample
Fix Bug
Success
Conclusions
Start
Verification
Pick Abstraction
Pick New
Abstraction
Failure
✔Diagnose Failure
New Approach to Diagnosis
• Synthesize program prefix
• Attempt
Use guidedSuccess
counterexample
Proof to diagnose
search
• Find failing abstraction
• Find failing concrete
value contained by
abstraction
Find
✔Counterexample
Fix Bug
Success
Conclusions
Start
Verification
Pick Abstraction
Attempt
Proof
-
Pick New
Abstraction
Failure
✔Diagnose Failure
Find
✔Counterexample
Fix Bug
Success
Success
Questions?