AstréeA
A Static Analyzer for Large Embedded Multi-Thread Software
Antoine Miné
CNRS & École normale supérieure
Paris, France
Verification, Model Checking, and Abstract Interpretation
12 January 2015
Mumbai
Introduction
Introduction
Goal:
static analysis of concurrent programs
Discover properties of the dynamic behaviors of programs:
directly on the source code
(not a model)
in an automated way
(not interactive)
in a terminating and efficient way
with approximations
(computability and efficiency)
soundly
(full coverage of all behaviors)
=⇒ usable for certification in the context of critical industry
We use the theory of abstract interpretation [Cousot and Cousot]
We developed the AstréeA static analyzer
(an extension of the Astrée analyzer to concurrent embedded software)
VMCAI’15 – 12 Jan. 2015
The AstréeA static analyzer
Antoine Miné
p. 2 / 65 (1/1)
Introduction
The Astrée static analyzer
Analyse Statique Temps-RÉEl
check statically for the absence of run-time errors (RTE)
on synchronous reactive codes, in a large subset of C
fast ('50 KLoc/h), sound and precise (aims at 0 alarm)
Development team: ÉNS, Paris
(B. Blanchet), P. Cousot, R. Cousot, L. Mauborgne,
(D. Monniaux), J. Feret, A. Miné, X. Rival
Time-line
2001
2003
2005
2009
VMCAI’15 – 12 Jan. 2015
Astrée project starts
0 alarm on A340 primary control software
0 alarm on A380 primary control software
industrialization by AbsInt
The AstréeA static analyzer
Antoine Miné
p. 3 / 65 (1/1)
Introduction
AstréeA = Astrée + A
AstréeA:
Analyse statique de programmes temps-réels asynchrones
static analyzer for concurrent embedded C codes
checks for run-time errors and data-races
fork of the Astrée analyzer (around 2007)
reuses Astrée’s iterator and abstract domains
builds on them a thread-modular analysis
adds new abstract domains
as Astrée, aims towards high precision by specialization
unlike Astrée, still many false alarms on target code
but already usable (industrialization in progress)
VMCAI’15 – 12 Jan. 2015
The AstréeA static analyzer
Antoine Miné
p. 4 / 65 (1/1)
Introduction
Classic and specialized analyzers
Classic and specialized analyzers
VMCAI’15 – 12 Jan. 2015
The AstréeA static analyzer
Antoine Miné
p. 5 / 65 (1/1)
Introduction
Classic and specialized analyzers
“Classic” abstract interpreter design
1
State the concrete semantics
2
State the class of properties of interest
3
Fix the class of properties that actually need to be inferred
4
Design an analyzer over a computable abstract semantics
VMCAI’15 – 12 Jan. 2015
The AstréeA static analyzer
Antoine Miné
p. 6 / 65 (2/5)
Introduction
Classic and specialized analyzers
“Classic” abstract interpreter design
1
State the concrete semantics
function from programs to a (rich) mathematical world
formalization of the language specification
ground truth, immutable
not computable!
2
State the class of properties of interest
3
Fix the class of properties that actually need to be inferred
4
Design an analyzer over a computable abstract semantics
VMCAI’15 – 12 Jan. 2015
The AstréeA static analyzer
Antoine Miné
p. 6 / 65 (3/5)
Introduction
Classic and specialized analyzers
“Classic” abstract interpreter design
1
State the concrete semantics
2
State the class of properties of interest
e.g.: variable bounds X ∈ [a, b]
3
Fix the class of properties that actually need to be inferred
4
Design an analyzer over a computable abstract semantics
VMCAI’15 – 12 Jan. 2015
The AstréeA static analyzer
Antoine Miné
p. 6 / 65 (4/5)
Introduction
Classic and specialized analyzers
“Classic” abstract interpreter design
1
State the concrete semantics
2
State the class of properties of interest
3
Fix the class of properties that actually need to be inferred
e.g.: linear constraints αX + βY ≤ γ
generally richer than the properties of interest
need to represent intermittent assertions, inductive loop invariants
may depend on the class of analyzed programs
4
Design an analyzer over a computable abstract semantics
VMCAI’15 – 12 Jan. 2015
The AstréeA static analyzer
Antoine Miné
p. 6 / 65 (5/5)
Introduction
Classic and specialized analyzers
“Classic” abstract interpreter design
1
State the concrete semantics
2
State the class of properties of interest
3
Fix the class of properties that actually need to be inferred
4
Design an analyzer over a computable abstract semantics
derive or invent abstract operators (e.g., interval arithmetic)
invent acceleration operators O
abstract domain: data-structures and algorithms
abstract composition and O accumulate precision loss
=⇒ we will not find the most precise property in the class
VMCAI’15 – 12 Jan. 2015
The AstréeA static analyzer
Antoine Miné
p. 6 / 65 (5/5)
Introduction
Classic and specialized analyzers
Specialized abstract interpreter
Refine the abstraction
1
Start with a simple and fast analyzer (intervals)
and a representative program in a class of programs of interest
2
Refine by hand the analyzer until 0 false alarm
determine which intermittent properties are missed
add a new domain
(if the property is not expressible)
employ fast transfer functions, if possible
limit the activation scope (variables, program part) to keep scalability
connect to existing domains through partial reductions
refine transfer functions
add communications (reductions)
adjust precision parameters
activation scope, iteration parameters, . . .
(available to end-users)
VMCAI’15 – 12 Jan. 2015
The AstréeA static analyzer
Antoine Miné
p. 7 / 65 (1/1)
Introduction
Classic and specialized analyzers
Specialized abstract interpreter
Refine the abstraction
Result
sound by construction
efficient, by parsimony
0 false alarm on the target program
encourages modular design, reusable abstractions
Rationale:
For each program and property, an adequate domain exists
but its construction is generally not mechanizable
Any combination of domains fails on infinitely many programs
A domain succeeds on infinitely many programs
In practice, the analyzer is precise on a whole class of programs
False alarm reduction require pre-program tuning of parameters
(possible by end-users)
VMCAI’15 – 12 Jan. 2015
The AstréeA static analyzer
Antoine Miné
p. 8 / 65 (1/1)
Introduction
Classic and specialized analyzers
Specialized abstract interpreter
Reinvent the concrete semantics
We may also need to change the concrete semantics!
Reasons:
it abstracts away platform details too much
arithmetic overflows: non-deterministic
modular
(more precise analysis)
ill-typed dereferences: program halt
bit-level type-punning
(the analysis covers more behaviors)
necessary when analyzing non-portable programs
it does not support some constructions
e.g., concurrency: Astrée
AstréeA
it is incomplete
(hard limit on the precision of any abstraction)
e.g., AstréeA’s first interference semantics
it is too difficult to abstract efficiently
e.g., state-based semantics
trace-based semantics with partitionning
Main focus of this talk!
VMCAI’15 – 12 Jan. 2015
The AstréeA static analyzer
Antoine Miné
p. 9 / 65 (1/1)
Introduction
Classic and specialized analyzers
Specialized abstract interpreter
Reinvent the concrete semantics
original
DO
α
/ D]
α
/ F ] (D ] )
α
refined
D0
Even though the concrete semantics has changed
the abstracts domains can be reused
abstractions may still be sound
e.g., non-deterministic overflow
modular overflow
completed by new abstractions sound only in the refined semantics
abstract domains as parameters of the new semantics
e.g., partitioned trace semantics, parameterized by a state abstraction
thread semantics as sequential program semantics
(slightly modified with interferences)
VMCAI’15 – 12 Jan. 2015
The AstréeA static analyzer
Antoine Miné
p. 10 / 65 (1/1)
Introduction
Classic and specialized analyzers
Overview
Generalities on Astrée(A)
From sequential (Astrée) to concurrent programs (AstréeA)
iterated sequential analysis with simple interference
Abstract rely-guarantee
a complete concrete semantics
retrieving simple interferences by abstraction
novel abstractions of interferences
Memory semantics
support for type-punning
abstract “domainlets” to analyze idiomatic code
variable and array summarization
Experiments
on a large industrial application (ARINC 653)
work-in-progress on a small embedded OS
VMCAI’15 – 12 Jan. 2015
The AstréeA static analyzer
Antoine Miné
p. 11 / 65 (1/1)
Generalities on Astrée(A)
Generalities on Astrée(A)
VMCAI’15 – 12 Jan. 2015
The AstréeA static analyzer
Antoine Miné
p. 12 / 65 (1/1)
Generalities on Astrée(A)
Sources for Astrée’s concrete semantics
Concrete semantics:
C99 norm
defined through
(portable programs)
IEEE 754-1985 norm
(floating-point arithmetic)
architecture parameters
(sizeof, endianess, struct, etc.)
compiler and linker parameters
Properties of interest:
(initialization, etc.)
absence of run-time error
no integer nor float numeric overflow
no invalid arithmetic operation
no invalid memory access
(/0, << 33)
(arrays [], pointers *)
respect the constraints put by the programmer
(assert)
i.e., reachability of bad memory states
VMCAI’15 – 12 Jan. 2015
The AstréeA static analyzer
Antoine Miné
p. 13 / 65 (1/1)
Generalities on Astrée(A)
Astrée(A)’s targets
Analyzed programs: embedded critical C codes
no dynamic memory allocation
no recursivity
Astrée:
no concurrency
tuned for synchronous control/command software
(numeric & boolean; no string, list, etc.)
but sound on all accepted programs
AstréeA:
support shared-memory concurrency
support operating system
(statically allocated threads)
(locks) through externally provided models
on-going support for data-structures
(strings, arrays; static allocation but dynamic usage)
VMCAI’15 – 12 Jan. 2015
The AstréeA static analyzer
Antoine Miné
p. 14 / 65 (1/1)
Generalities on Astrée(A)
A few abstract domains used in Astrée(A)
octagons
±X ± Y ≤ c
[Miné 2006]
congruences
X ≡ a [b]
[Granger 1989]
boolean decision trees
ellipsoids
digital filters
[Feret 2005]
exponentials
X ≤ (1 + α)βt
[Feret 2005]
trace partitions
[Mauborgne]
[Mauborgne Rival 2005]
relational domains are often required to find inductive invariants
VMCAI’15 – 12 Jan. 2015
The AstréeA static analyzer
Antoine Miné
p. 15 / 65 (1/1)
Generalities on Astrée(A)
Abstract interpretation of sequential programs
Two approaches
1
while random do
2 if x < y then
3 x ←x +1
Equation solving
Interpretation by induction
X1 = I
X2 = X1 ∪ J x ← x + 1 KX3 ∪ J x ≥ y KX2
X3 = J x < y KX2
def
J while random do S KX =
lfp λY. X ∪ J S KY
def
J if x < y then S KX =
J S K(J x < y KX ) ∪ J x ≥ y KX
linear memory in program length
linear memory in program depth
flexible solving strategy
fixed iteration strategy
(follows the program structure)
Astrée uses an interpreter by induction
(scales in memory to large programs)
VMCAI’15 – 12 Jan. 2015
The AstréeA static analyzer
Antoine Miné
p. 16 / 65 (1/1)
Generalities on Astrée(A)
Astrée’s interpreter
↓
syntax iterator
↓
trace partitioning domain
l
memory domain
l
pointer domain
l
(reduced product of) numerical abstract domains
..
l
l
l
l
.
intervals
VMCAI’15 – 12 Jan. 2015
octagons
decision trees
The AstréeA static analyzer
filters
...
Antoine Miné
p. 17 / 65 (2/14)
Generalities on Astrée(A)
Astrée’s interpreter
↓
simplified C code
syntax iterator
↓
side-effect-free C assignments & tests
trace partitioning domain
l
memory domain
l
scalar assignments & tests
pointer domain
l
numeric assignments & tests
(reduced product of) numerical abstract domains
..
l
l
l
l
.
intervals
VMCAI’15 – 12 Jan. 2015
octagons
decision trees
The AstréeA static analyzer
filters
...
Antoine Miné
p. 17 / 65 (3/14)
Generalities on Astrée(A)
Astrée’s interpreter
↓
for (i=0;...)
a[i] = *p;
syntax iterator
↓
trace partitioning domain
l
memory domain
l
pointer domain
l
(reduced product of) numerical abstract domains
..
l
l
l
l
.
intervals
VMCAI’15 – 12 Jan. 2015
octagons
decision trees
The AstréeA static analyzer
filters
...
Antoine Miné
p. 17 / 65 (4/14)
Generalities on Astrée(A)
Astrée’s interpreter
↓
for (i=0;...)
a[i] = *p;
syntax iterator
↓
i = 0, a[i] = *p, . . .
trace partitioning domain
l
memory domain
l
pointer domain
l
(reduced product of) numerical abstract domains
..
l
l
l
l
.
intervals
octagons
decision trees
filters
...
−→ induction on the syntax
VMCAI’15 – 12 Jan. 2015
The AstréeA static analyzer
Antoine Miné
p. 17 / 65 (5/14)
Generalities on Astrée(A)
Astrée’s interpreter
↓
for (i=0;...)
a[i] = *p;
syntax iterator
↓
i = 0, a[i] = *p, . . .
trace partitioning domain
l
memory domain
l
pointer domain
l
(reduced product of) numerical abstract domains
..
l
l
l
l
.
intervals
VMCAI’15 – 12 Jan. 2015
octagons
decision trees
The AstréeA static analyzer
filters
...
Antoine Miné
p. 17 / 65 (6/14)
Generalities on Astrée(A)
Astrée’s interpreter
↓
for (i=0;...)
a[i] = *p;
syntax iterator
↓
i = 0, a[i] = *p, . . .
trace partitioning domain
i : 0 7→X ] , 1 7→ Y ] , . . .
l
memory domain
l
pointer domain
l
(reduced product of) numerical abstract domains
..
l
l
l
l
.
intervals
octagons
decision trees
filters
...
−→ iterate on each partition
VMCAI’15 – 12 Jan. 2015
The AstréeA static analyzer
Antoine Miné
p. 17 / 65 (7/14)
Generalities on Astrée(A)
Astrée’s interpreter
↓
for (i=0;...)
a[i] = *p;
syntax iterator
↓
i = 0, a[i] = *p, . . .
trace partitioning domain
i : 0 7→X ] , 1 7→ Y ] , . . .
l
a[0] = *p, a[1] = *p, . . .
memory domain
l
pointer domain
l
(reduced product of) numerical abstract domains
..
l
l
l
l
.
intervals
octagons
decision trees
filters
...
−→ iterate on each partition
VMCAI’15 – 12 Jan. 2015
The AstréeA static analyzer
Antoine Miné
p. 17 / 65 (8/14)
Generalities on Astrée(A)
Astrée’s interpreter
↓
for (i=0;...)
a[i] = *p;
syntax iterator
↓
i = 0, a[i] = *p, . . .
trace partitioning domain
i : 0 7→X ] , 1 7→ Y ] , . . .
l
a[0] = *p, a[1] = *p, . . .
memory domain
l
pointer domain
l
(reduced product of) numerical abstract domains
..
l
l
l
l
.
intervals
VMCAI’15 – 12 Jan. 2015
octagons
decision trees
The AstréeA static analyzer
filters
...
Antoine Miné
p. 17 / 65 (9/14)
Generalities on Astrée(A)
Astrée’s interpreter
↓
for (i=0;...)
a[i] = *p;
syntax iterator
↓
i = 0, a[i] = *p, . . .
trace partitioning domain
i : 0 7→X ] , 1 7→ Y ] , . . .
l
a[0] = *p, a[1] = *p, . . .
memory domain
p?
l
pointer domain
l
(reduced product of) numerical abstract domains
..
l
l
l
l
.
intervals
octagons
decision trees
filters
...
−→ resolve pointers to lvalues
VMCAI’15 – 12 Jan. 2015
The AstréeA static analyzer
Antoine Miné
p. 17 / 65 (10/14)
Generalities on Astrée(A)
Astrée’s interpreter
↓
for (i=0;...)
a[i] = *p;
syntax iterator
↓
i = 0, a[i] = *p, . . .
trace partitioning domain
i : 0 7→X ] , 1 7→ Y ] , . . .
l
a[0] = *p, a[1] = *p, . . .
memory domain
p?
l
p?
pointer domain
l
(reduced product of) numerical abstract domains
..
l
l
l
l
.
intervals
octagons
decision trees
filters
...
−→ resolve pointers to lvalues
VMCAI’15 – 12 Jan. 2015
The AstréeA static analyzer
Antoine Miné
p. 17 / 65 (11/14)
Generalities on Astrée(A)
Astrée’s interpreter
↓
for (i=0;...)
a[i] = *p;
syntax iterator
↓
i = 0, a[i] = *p, . . .
trace partitioning domain
i : 0 7→X ] , 1 7→ Y ] , . . .
l
a[0] = *p, a[1] = *p, . . .
memory domain
p?
l
p?
pointer domain
p 7→ x
l
(reduced product of) numerical abstract domains
..
l
l
l
l
.
intervals
octagons
decision trees
filters
...
−→ resolve pointers to lvalues
VMCAI’15 – 12 Jan. 2015
The AstréeA static analyzer
Antoine Miné
p. 17 / 65 (12/14)
Generalities on Astrée(A)
Astrée’s interpreter
↓
for (i=0;...)
a[i] = *p;
syntax iterator
↓
i = 0, a[i] = *p, . . .
trace partitioning domain
i : 0 7→X ] , 1 7→ Y ] , . . .
l
a[0] = *p, a[1] = *p, . . .
memory domain
l
a@0 = x
pointer domain
l
(reduced product of) numerical abstract domains
..
l
l
l
l
.
intervals
octagons
decision trees
filters
...
−→ resolve lvalues to scalar cells
VMCAI’15 – 12 Jan. 2015
The AstréeA static analyzer
Antoine Miné
p. 17 / 65 (13/14)
Generalities on Astrée(A)
Astrée’s interpreter
↓
for (i=0;...)
a[i] = *p;
syntax iterator
↓
i = 0, a[i] = *p, . . .
trace partitioning domain
i : 0 7→X ] , 1 7→ Y ] , . . .
l
a[0] = *p, a[1] = *p, . . .
memory domain
l
a@0 = x
pointer domain
l
a@0 = x
(reduced product of) numerical abstract domains
..
l
l
l
l
.
intervals
octagons
decision trees
filters
...
−→ pointer arithmetic to integer arithmetic
VMCAI’15 – 12 Jan. 2015
The AstréeA static analyzer
Antoine Miné
p. 17 / 65 (14/14)
Generalities on Astrée(A)
Astrée’s interpreter
↓
for (i=0;...)
a[i] = *p;
syntax iterator
↓
i = 0, a[i] = *p, . . .
trace partitioning domain
i : 0 7→X ] , 1 7→ Y ] , . . .
l
a[0] = *p, a[1] = *p, . . .
memory domain
l
a@0 = x
pointer domain
l
a@0 = x
(reduced product of) numerical abstract domains
..
l
l
l
l
.
intervals
octagons
decision trees
filters
a@0 = x
...
−→ execute numeric abstract operations
domains communicate by (partial) reduction
VMCAI’15 – 12 Jan. 2015
The AstréeA static analyzer
Antoine Miné
p. 17 / 65 (14/14)
Generalities on Astrée(A)
Acceleration trick #1: Alternate loop fixpoint
Standard fixpoint
Alternate fixpoint
λX . lfp λY. X ∪ J S KY
λX . lfpX λY. Y ∪ J S KY
Y0 = ⊥
Y1 = X
Y2 = X ∪ J S K Y1
Yn+1 = X ∪ J S K Yn
the fixpoints
Y1 = X
Y2 = Y0 ∪ J S K Y0
Yn+1 = Yn ∪ J S K Yn
(and iterates) are
(they may differ in the abstract!)
equal in the concrete
in the abstract, the join of two elements that are similar
is more efficient than for two very different elements
Astrée uses maps with sharing across abstract elements
also useful with memory abstractions: avoids refolding arrays (see later)
=⇒ ∪ is more efficient than ∪
VMCAI’15 – 12 Jan. 2015
The AstréeA static analyzer
Antoine Miné
p. 18 / 65 (1/1)
Generalities on Astrée(A)
Acceleration trick #2: Fixpoint bootstrapping
Nested loops
while random do
S1 ;
while random do S2
Interpreter by induction:
computes nested fixpoints
def
P = lfpX λY. Y ∪ (lfpJ S1 KY λZ. Z ∪ J S2 KZ)
=⇒ P must recompute completely the inner fixpoint
for each iteration of the outer fixpoint!
=⇒ very expensive
Equation-system:
more flexibility
No need to wait for the inner fixpoint to stabilize
before propagating abstract states outside the loop
VMCAI’15 – 12 Jan. 2015
The AstréeA static analyzer
Antoine Miné
p. 19 / 65 (2/3)
Generalities on Astrée(A)
Acceleration trick #2: Fixpoint bootstrapping
Nested loops
while random do
S1 ;
while random do S2
Solution:
fixpoint bootstrapping
[Mauborgne]
def
P = lfpX λY. Y ∪ (lfpJ S1 KY λZ. Z ∪ J S2 KZ)
remember the result of the last inner fixpoint:
def
Yi0 = lfpJ S1 KYi λZ. Z ∪ J S2 KZ
when recomputing the inner fixpoint for the next outer iteration
given the new inner loop entry J S1 KYi+1
start iterating from Yi0 ∪ J S1 KYi+1 instead
VMCAI’15 – 12 Jan. 2015
The AstréeA static analyzer
Antoine Miné
p. 19 / 65 (3/3)
Generalities on Astrée(A)
Acceleration trick #2: Fixpoint bootstrapping
Nested loops
while random do
S1 ;
while random do S2
Result:
huge speed improvements
×6.5 on a matrix multiplication loop
also very useful for thread-modular analyses (see later)
moderate increase in memory consumption
(can be handled as a cache, enforce memory bound)
gives incomparable results in theory
no precision loss in practice
(non-monotonicity of O)
(rationale: we start below but nearer the fixpoint)
VMCAI’15 – 12 Jan. 2015
The AstréeA static analyzer
Antoine Miné
p. 19 / 65 (3/3)
From sequential to parallel programs
From sequential to parallel programs
VMCAI’15 – 12 Jan. 2015
The AstréeA static analyzer
Antoine Miné
p. 20 / 65 (1/1)
From sequential to parallel programs
Multi-thread program execution
t1
1a
while random do
if x < y then
3a
x ←x +1
2a
t2
1b
while random do
if y < 100 then
3b
y ← y + [1, 3]
2b
Execution model:
finite number of threads
the memory is shared (x ,y )
each thread has its own program counter
execution interleaves steps from threads t1 and t2
(assignments and tests are supposed atomic)
=⇒ we have the global invariant 0 ≤ x ≤ y ≤ 102
VMCAI’15 – 12 Jan. 2015
The AstréeA static analyzer
Antoine Miné
p. 21 / 65 (1/1)
From sequential to parallel programs
Failed attempt: Separate analysis
t1
1a
while random do
if x < y then
3a
x ←x +1
2a
t2
1b
while random do
if y < 100 then
3b
y ← y + [1, 3]
2b
X` ⊆ Z2 , ` ∈ {1a, 2a, 3a}
X`0 ⊆ Z2 , ` ∈ {1b, 2b, 3b}
X1a = I
X2a = X1a ∪ J x ← x + 1 KX3a ∪
J x ≥ y KX2a
X3a = J x < y KX2
0 =I
X1b
0 = X ∪ J y ← y + [1, 3] KX
X2b
1a
3b ∪
J y ≥ 100 KX2b
0 = J y < 100 KX
X3b
2b
=⇒ x = y = 0
=⇒ x = 0, y ∈ [0, 102]
Separate analysis: unsound
VMCAI’15 – 12 Jan. 2015
The AstréeA static analyzer
Antoine Miné
p. 22 / 65 (1/1)
From sequential to parallel programs
Failed attempt: Product of equations
t1
1a
while random do
if x < y then
3a
x ←x +1
t2
1b
2a
while random do
if y < 100 then
3b
y ← y + [1, 3]
2b
Interleaving semantics:
X`,`0 ⊆ Z2 , ` ∈ {1a, 2a, 3a}, `0 ∈ {1b, 2b, 3b}
X1a,1b
X2a,1b
X3a,1b
X2a,2b
X3a,2b
X2a,3b
X3a,3b
=I
= X1a,1b ∪ J x ≥ y KX2a,1b ∪ J x ← x + 1 KX3a,1b
= J x < y KX2a,1b
= X1a,2b ∪ J x ≥ y KX2a,2b ∪ J x ← x + 1 KX3a,2b ∪
X2a,1b ∪ J y ≥ 100 KX2a,2b ∪ J y ← y + [1, 3] KX2a,3b
= J x < y KX2a,2b ∪ X3a,1b ∪ J y ≥ 100 KX3a,2b ∪ J y ← y + [1, 3] KX3a,3b
= X1a,3b ∪ J x ≥ y KX2a,3b ∪ J x ← x + 1 KX3a,3b ∪ J y < 100 KX2a,2b
= J x < y KX2a,3b ∪ J y < 100 KX3a,2b
large number of variables and equations, large equations
=⇒ impractical
VMCAI’15 – 12 Jan. 2015
The AstréeA static analyzer
Antoine Miné
p. 23 / 65 (1/1)
From sequential to parallel programs
Simple interference-based concrete semantic
t2
t1
1a
while random do
if x < y then
3a
x ←x +1
2a
Principle:
1b
while random do
if y < 100 then
3b
y ← y + [1, 3]
2b
[Miné 2010, Carré & Hymans 2009]
analyze each thread in isolation
and gather interferences
(abstraction of) the values stored into each variable by each thread
re-analyze the threads taking interferences into account
(variable read returns the last value written, or an interference)
gather new sets of interferences
iterate until stabilization
VMCAI’15 – 12 Jan. 2015
The AstréeA static analyzer
Antoine Miné
p. 24 / 65 (2/5)
From sequential to parallel programs
Simple interference-based concrete semantic
t2
t1
1a
while random do
if x < y then
3a
x ←x +1
2a
1b
while random do
if y < 100 then
3b
y ← y + [1, 3]
2b
Analysis of t1 in isolation
(1a): x
(2a): x
=y =0
=y =0
(3a): ⊥
VMCAI’15 – 12 Jan. 2015
X1a = I
X2a = X1a ∪ J x ← x + 1 KX3a ∪ J x ≥ y KX2a
X3a = J x < y KX2a
The AstréeA static analyzer
Antoine Miné
p. 24 / 65 (3/5)
From sequential to parallel programs
Simple interference-based concrete semantic
t2
t1
1a
while random do
if x < y then
3a
x ←x +1
2a
1b
while random do
if y < 100 then
3b
y ← y + [1, 3]
2b
Analysis of t2 in isolation
(1b): x
=y =0
= 0, y ∈ [0, 102]
(3b): x = 0, y ∈ [0, 99]
(2b): x
X1b = I
X2b = X1b ∪ J y ← y + [1, 3] KX3b ∪ J y ≥ 100 KX2b
X3b = J y < 100 KX2b
output interferences: y ← [1, 102]
VMCAI’15 – 12 Jan. 2015
The AstréeA static analyzer
Antoine Miné
p. 24 / 65 (4/5)
From sequential to parallel programs
Simple interference-based concrete semantic
t2
t1
1a
while random do
if x < y then
3a
x ←x +1
2a
1b
while random do
if y < 100 then
3b
y ← y + [1, 3]
2b
Re-analysis of t1 with interferences from t2
input interferences: y ← [1, 102]
(1a): x
=y =0
∈ [0, 102], y = 0
(3a): x ∈ [0, 102], y = 0
(2a): x
X1a = I
X2a = X1a ∪ J x ← x + 1 KX3a ∪ J x ≥ (y | [1, 102]) KX2a
X3a = J x < (y | [1, 102]) KX2a
output interferences: x ← [1, 102]
subsequent re-analyses are identical (fixpoint reached)
VMCAI’15 – 12 Jan. 2015
The AstréeA static analyzer
Antoine Miné
p. 24 / 65 (5/5)
From sequential to parallel programs
Simple interference-based concrete semantic
t2
t1
1a
while random do
if x < y then
3a
x ←x +1
1b
2a
while random do
if y < 100 then
3b
y ← y + [1, 3]
2b
Concrete invariants:
we get x , y ∈ [0, 102]
we don’t get that x ≤ y
Derived abstract analysis:
similar to a sequential program analysis, but iterated
can be parameterized by arbitrary abstract domains
efficient
(few reanalyses are required in practice, loop bootstrapping helps)
interferences are non-relational and flow-insensitive
limit inherited from the concrete semantics
VMCAI’15 – 12 Jan. 2015
The AstréeA static analyzer
Antoine Miné
p. 24 / 65 (5/5)
From sequential to parallel programs
Remark: Weakly consistent memories
The interleaving semantics (sequential consistency) is not realistic
actual languages and CPU obey relaxed memory models due to
CPU-level optimizations
(memory buffers, instruction reordering)
compiler-level optimizations
(allowed by language specifications)
=⇒ an analysis sound only for sequential consistency
may not be sound for the actual memory model
(example: y ← 1; if x = 0 then · · · || x ← 1; if y = 0 then · · · )
Result: The abstract analysis is sound
wrt. a large set of weak memory models
Rationale:
flow-insensitive non-relational interferences are insensitive
to the reordering of reads and writes
[Miné 11] and [Alglave et al. 2011]
VMCAI’15 – 12 Jan. 2015
The AstréeA static analyzer
Antoine Miné
p. 25 / 65 (1/1)
From sequential to parallel programs
Mutual exclusion locks
lock
t1
unlock
W
W
W
t2
R
R
W
lock
R
unlock
Mutexes:
ensure mutual exclusion
at each time, each mutex can be locked by a single thread
enforce memory consistency and atomicity
no optimization or reordering across lock and unlock
=⇒ we need to discard spurious interferences, to improve the precision
we assume a fixed, finite number of mutexes
VMCAI’15 – 12 Jan. 2015
The AstréeA static analyzer
Antoine Miné
p. 26 / 65 (2/3)
From sequential to parallel programs
Mutual exclusion locks
lock
t1
unlock
W
W
W
t2
R
R
W
lock
R
unlock
Data-race interferences:
across read / write not protected by a mutex
VMCAI’15 – 12 Jan. 2015
The AstréeA static analyzer
Antoine Miné
p. 26 / 65 (3/3)
From sequential to parallel programs
Mutual exclusion locks
lock
t1
unlock
W
W
W
t2
R
R
W
lock
R
unlock
Data-race interferences:
across read / write not protected by a mutex
Well-synchronized interferences:
last write before an unlock in t1
influence reads between lock and first write in t2
We partition interferences by the set of mutexes held.
[Miné 2011]
VMCAI’15 – 12 Jan. 2015
The AstréeA static analyzer
Antoine Miné
p. 26 / 65 (3/3)
From sequential to parallel programs
Example
abstract consumer/producer
N × consumer
while random do
lock(m)
if x > 0 then x ← x − 1 endif;
unlock(m)
N × producer
while random do
lock(m);
x ← x + 1;
if x > 100 then x ← 100 endif;
unlock(m)
Assuming we have several (N) producers and consumers:
no data-race interference
(proof of the absence of data-race)
well-synchronized interferences:
consumer : x ← [0, 99]
producer : x ← [1, 100]
=⇒ we get that x ∈ [0, 100]
(without locks, if N > 1, our semantics cannot bound x !)
VMCAI’15 – 12 Jan. 2015
The AstréeA static analyzer
Antoine Miné
p. 27 / 65 (1/1)
From sequential to parallel programs
AstréeA’s abstract interpreter
thread iterator
↓
syntax iterator
↓
trace partitioning domain
↓
lock partitioning domain
l
memory domain
l
↑
.
interference domain ..
l
↓
pointer domain
l
(reduced product of) numerical abstract domains
..
l
l
l
l
.
intervals
VMCAI’15 – 12 Jan. 2015
octagons
decision trees
The AstréeA static analyzer
filters
Antoine Miné
...
p. 28 / 65 (1/1)
Abstract rely–guarantee
Abstract rely–guarantee
VMCAI’15 – 12 Jan. 2015
The AstréeA static analyzer
Antoine Miné
p. 29 / 65 (1/1)
Abstract rely–guarantee
Rely–guarantee reasoning
checking t2
checking t1
t1
1a
t1
t2
while random do
2a
if y < 100 then
y ← y + [1, 3]
3b
proof method introduced by Jones in 1981
generalized Hoare logics
VMCAI’15 – 12 Jan. 2015
while random do
2b
if x < y then
3a x ← x + 1
Rely–guarantee:
t2
1b
(by structural induction ⇒ thread-modular)
The AstréeA static analyzer
Antoine Miné
p. 30 / 65 (2/6)
Abstract rely–guarantee
Rely–guarantee reasoning
checking t2
checking t1
t1
1a
t1
t2
while random do
2a
while random do
2b
if x < y then
3a x ← x + 1
if y < 100 then
y ← y + [1, 3]
3b
:x =y =0
: x , y ∈ [0, 102], x ≤ y
(3a) : x ∈ [0, 101], y ∈ [1, 102], x < y
:x =y =0
: x , y ∈ [0, 102], x ≤ y
(3b) : x , y ∈ [0, 99], x ≤ y
(1a)
(1b)
(2a)
(2b)
Rely–guarantee:
t2
1b
proof method introduced by Jones in 1981
generalized Hoare logics
(by structural induction ⇒ thread-modular)
require thread-local invariant assertions
VMCAI’15 – 12 Jan. 2015
The AstréeA static analyzer
Antoine Miné
p. 30 / 65 (3/6)
Abstract rely–guarantee
Rely–guarantee reasoning
checking t2
checking t1
t1
1a
while random do
2a
if x < y then
3a x ← x + 1
t1
t2
t2
x unchanged
y unchanged
y incremented
0 ≤ y ≤ 102
0≤x ≤y
:x =y =0
: x , y ∈ [0, 102], x ≤ y
(3a) : x ∈ [0, 101], y ∈ [1, 102], x < y
while random do
2b
if y < 100 then
y ← y + [1, 3]
3b
:x =y =0
: x , y ∈ [0, 102], x ≤ y
(3b) : x , y ∈ [0, 99], x ≤ y
(1a)
(1b)
(2a)
(2b)
Rely–guarantee:
1b
proof method introduced by Jones in 1981
generalized Hoare logics
(by structural induction ⇒ thread-modular)
require thread-local invariant assertions
require guarantees on transitions generated by other threads
VMCAI’15 – 12 Jan. 2015
The AstréeA static analyzer
Antoine Miné
p. 30 / 65 (4/6)
Abstract rely–guarantee
Rely–guarantee reasoning
checking t2
checking t1
t1
1a
while random do
2a
if x < y then
3a x ← x + 1
t1
t2
t2
x unchanged
y unchanged
y incremented
0 ≤ y ≤ 102
0≤x ≤y
:x =y =0
: x , y ∈ [0, 102], x ≤ y
(3a) : x ∈ [0, 101], y ∈ [1, 102], x < y
while random do
2b
if y < 100 then
y ← y + [1, 3]
3b
:x =y =0
: x , y ∈ [0, 102], x ≤ y
(3b) : x , y ∈ [0, 99], x ≤ y
(1a)
(1b)
(2a)
(2b)
Rely–guarantee:
1b
proof method introduced by Jones in 1981
generalized Hoare logics
(by structural induction ⇒ thread-modular)
require thread-local invariant assertions
require guarantees on transitions generated by other threads
check each thread against an abstraction of the other threads
VMCAI’15 – 12 Jan. 2015
The AstréeA static analyzer
Antoine Miné
p. 30 / 65 (5/6)
Abstract rely–guarantee
Rely–guarantee reasoning
checking t2
checking t1
t1
1a
while random do
2a
if x < y then
3a x ← x + 1
t1
t2
t2
x unchanged
y unchanged
y incremented
0 ≤ y ≤ 102
0≤x ≤y
:x =y =0
: x , y ∈ [0, 102], x ≤ y
(3a) : x ∈ [0, 101], y ∈ [1, 102], x < y
while random do
2b
if y < 100 then
y ← y + [1, 3]
3b
:x =y =0
: x , y ∈ [0, 102], x ≤ y
(3b) : x , y ∈ [0, 99], x ≤ y
(1a)
(1b)
(2a)
(2b)
Rely–guarantee:
1b
proof method introduced by Jones in 1981
generalized Hoare logics
(by structural induction ⇒ thread-modular)
require thread-local invariant assertions
require guarantees on transitions generated by other threads
check each thread against an abstraction of the other threads
allows proving that x ≤ y holds!
VMCAI’15 – 12 Jan. 2015
The AstréeA static analyzer
Antoine Miné
p. 30 / 65 (6/6)
Abstract rely–guarantee
Rely–guarantee reasoning
checking t2
checking t1
t1
1a
while random do
2a
if x < y then
3a x ← x + 1
t1
t2
t2
x unchanged
y unchanged
y incremented
0 ≤ y ≤ 102
0≤x ≤y
:x =y =0
: x , y ∈ [0, 102], x ≤ y
(3a) : x ∈ [0, 101], y ∈ [1, 102], x < y
1b
while random do
2b
if y < 100 then
y ← y + [1, 3]
3b
:x =y =0
: x , y ∈ [0, 102], x ≤ y
(3b) : x , y ∈ [0, 99], x ≤ y
(1a)
(1b)
(2a)
(2b)
Note: we look for a static analysis, not a proof method
=⇒ infer automatically invariants and guarantees
Idea:
propose a constructive version of rely–guarantee
complete concrete semantics
expressed using fixpoints
amenable to abstraction [Cousot Cousot 1984]
VMCAI’15 – 12 Jan. 2015
The AstréeA static analyzer
Antoine Miné
p. 30 / 65 (6/6)
Abstract rely–guarantee
Rely–guarantee fixpoint concrete semantics
Rely–guarantee fixpoint concrete semantics
VMCAI’15 – 12 Jan. 2015
The AstréeA static analyzer
Antoine Miné
p. 31 / 65 (1/1)
Abstract rely–guarantee
Rely–guarantee fixpoint concrete semantics
Thread-based trace decomposition
a
b
b
b
a
b
Concrete trace semantics:
a
F
partial finite trace semantics in fixpoint form:
def
F = lfp F where
ai+1
a
def
ai+1
a
F = λX . I ∪ { σ0 →1 · · · σi → σi+1 | σ0 →1 · · · σi ∈ X ∧ σi → τ σi+1 }
def
threads: T = {a, b, . . .}
def
states: Σ = C × M = {•, •, . . .}
def
control state: C = T → L
(maps threads to locations)
def
memory state: M = V → V
(maps variables to values)
a
transition relation: τ ⊆ Σ × T × Σ: σ →τ σ 0
VMCAI’15 – 12 Jan. 2015
The AstréeA static analyzer
Antoine Miné
p. 32 / 65 (2/4)
Abstract rely–guarantee
Rely–guarantee fixpoint concrete semantics
Thread-based trace decomposition
Reachable states:
a
b
b
b
a
b
R
a
(concrete semantics of interest)
Extract the states reached during executions
R = αreach (F) ⊆ Σ where
def
a
αreach (T ) = { σ | ∃σ0 →1 · · · σn ∈ T : ∃i ≤ n : σ = σi }
VMCAI’15 – 12 Jan. 2015
The AstréeA static analyzer
Antoine Miné
p. 32 / 65 (3/4)
Abstract rely–guarantee
Rely–guarantee fixpoint concrete semantics
Thread-based trace decomposition
a
b
b
b
a
b
a
a
R`(t)
Reachable states projected on thread t:
attach to thread control point L instead of control state T → L
remember other thread’s control point as “auxiliary variables”
(required for completeness)
def
def
R`(t) = πt (R) ∈ P(L × (Vt → V)), Vt = V ∪ { pc t 0 | t 0 6= t }
def
π t (hL, ρi) = hL(t), ρ [∀t 0 6= t : pc t 0 7→ L(t 0 )]i
VMCAI’15 – 12 Jan. 2015
The AstréeA static analyzer
extended element-wise
Antoine Miné
p. 32 / 65 (4/4)
Abstract rely–guarantee
Rely–guarantee fixpoint concrete semantics
Thread-based trace decomposition
a
b
b
b
a
b
a
a
I(t)
Interferences generated by t:
(' guarantees on transitions)
Extract the transitions with action t observed in F
(subset of the transition system, containing only transitions actually used in reachability)
def
I(t) = αitf (F)(t) where
def
a
a
αitf (X )(t) = { hσi , σi+1 i | ∃σ0 →1 σ1 · · · →n σn ∈ X : ai+1 = t }
VMCAI’15 – 12 Jan. 2015
The AstréeA static analyzer
Antoine Miné
p. 32 / 65 (4/4)
Abstract rely–guarantee
Rely–guarantee fixpoint concrete semantics
Computing thread traces with interferences
b
a
b
Thread
a
b
x=0
while x<y
x++;
/* bla bla */
Principle: express R`(t) and I(t) directly, without computing F
States:
R`
Interleave:
transitions from the current thread t
transitions from interferences I from other threads
R`(t) = lfp Rt (I), where
def
Rt (Y )(X ) =
t
πt (I) ∪ { πt (σ 0 ) | ∃πt (σ) ∈ X : σ →τ σ 0 } ∪
{ πt (σ 0 ) | ∃πt (σ) ∈ X : ∃t 0 6= t : hσ, σ 0 i ∈ Y (t 0 ) }
=⇒ similar to reachability for a sequential program, up to I
VMCAI’15 – 12 Jan. 2015
The AstréeA static analyzer
Antoine Miné
p. 33 / 65 (2/3)
Abstract rely–guarantee
Rely–guarantee fixpoint concrete semantics
Computing thread traces with interferences
b
a
b
a
Thread
a
x=0
while x<y
x++;
/* bla bla */
Principle: express R`(t) and I(t) directly, without computing F
Interferences:
I
Collect transitions from a thread t and reachable states R:
I(t) = B(R`)(t), where
def
t
B(Z )(t) = { hσ, σ 0 i | πt (σ) ∈ Z (t) ∧ σ →τ σ 0 }
VMCAI’15 – 12 Jan. 2015
The AstréeA static analyzer
Antoine Miné
p. 33 / 65 (3/3)
Abstract rely–guarantee
Rely–guarantee fixpoint concrete semantics
Computing thread traces with interferences
b
a
b
a
Thread
a
x=0
while x<y
x++;
/* bla bla */
Principle: express R`(t) and I(t) directly, without computing F
Recursive definition:
R`(t) = lfp Rt (I)
I(t) = B(R`)(t)
=⇒ express the most precise solution as a fixpoint:
R` = lfp λZ . λt. lfp Rt (B(Z ))
=⇒ iterate analyses with interference
Completeness:
VMCAI’15 – 12 Jan. 2015
∀t : R`(t) ' R
(πt is bijective thanks to auxiliary variables)
The AstréeA static analyzer
Antoine Miné
p. 33 / 65 (3/3)
Abstract rely–guarantee
Rely–guarantee abstractions
Rely–guarantee abstractions
VMCAI’15 – 12 Jan. 2015
The AstréeA static analyzer
Antoine Miné
p. 34 / 65 (1/1)
Abstract rely–guarantee
Rely–guarantee abstractions
Retrieving the simple interference-based analysis
Flow-insensitive abstraction:
reduce as much control information as possible
but keep flow-sensitivity on each thread’s control location
State abstraction:
def
f
αR
(X ) =
remove auxiliary variables
{ h`, ρ|V i | h`, ρi ∈ X }
Interference abstraction:
remove all control information
def
αIf (Y ) = { hρ, ρ0 i | ∃L, L0 ∈ C : hhL, ρi, hL0 , ρ0 ii ∈ Y }
Note: we lose completeness
we cannot prove that x is bounded in x ← x + 1 || x ← x + 1!
VMCAI’15 – 12 Jan. 2015
The AstréeA static analyzer
Antoine Miné
p. 35 / 65 (1/1)
Abstract rely–guarantee
Rely–guarantee abstractions
Retrieving the simple interference-based analysis
Cartesian abstraction:
on interferences
forget the relations between variables
forget the relations between values before and after transitions
(input-output relationship)
only remember which variables are modified
and their new value:
def
αIc (Y ) = λV . { x ∈ V | ∃hρ, ρ0 i ∈ Y : ρ(V ) 6= x ∧ ρ0 (V ) = x }
no modification on the state
(the analysis of each thread can still be relational)
=⇒ we get back our simple interference analysis!
VMCAI’15 – 12 Jan. 2015
The AstréeA static analyzer
Antoine Miné
p. 36 / 65 (1/1)
Abstract rely–guarantee
Rely–guarantee abstractions
From traces to thread-modular analyses
abstract states
abstract interferences
(T × L) → M]
static analyzer
T → M]
O
O
αM
non-relational interferences
αM
O
T → P(M)
projected states
αcI
flow-insensitive interferences
rely-guarantee
(without aux. variables)
O
(T × L) → P(M)
projected states
R` :
Q
t∈T
O
T → P(M × M)
αfR
αfI
interferences
rely-guarantee
(with aux. variables)
{t} → P(Σt )
O
πt
OA : T
→ P(Σ × Σ)
αitf
interleaved execution trace prefixes
test
F ∈ P(Σ∗ )
VMCAI’15 – 12 Jan. 2015
The AstréeA static analyzer
Antoine Miné
p. 37 / 65 (1/1)
Abstract rely–guarantee
Rely–guarantee abstractions
Compare with sequential analyses
abstract states
O
L → M]
static analyzer
αM
states
reachability
O
R ∈ P(Σ)
αreach
execution trace prefixes
test
F ∈ P(Σ∗ )
VMCAI’15 – 12 Jan. 2015
The AstréeA static analyzer
Antoine Miné
p. 38 / 65 (1/1)
Abstract rely–guarantee
Rely–guarantee abstractions
Beyond non-relational interference abstractions
Clock thread
Accumulator thread
106
while random do
Prec ← Clock;
···
delta ← Clock − Prec;
if random then x ← x + delta endif;
...
done
while Clock <
do
Clock ← Clock + 1;
···
done
Clock is a global, increasing clock
x accumulates periods of time
no overflow on Clock − Prec nor x ← x + delta
To prove this we need:
relational abstractions of interferences
(keep input-output relationships)
hypotheses on memory consistency
(e.g., partial store ordering)
VMCAI’15 – 12 Jan. 2015
The AstréeA static analyzer
Antoine Miné
p. 39 / 65 (1/1)
Abstract rely–guarantee
Rely–guarantee abstractions
Monotonicity abstraction
Abstraction:
map variables to 1
def
αImon (Y ) =
or >
monotonic
don’t know
0
λV . if ∀hρ, ρ i ∈ Y : ρ(V ) ≤ ρ0 (V ) then 1 else >
keep some input-output relationships
forgets all relations between variables
flow-insensitive
Inference and use
gather:
]
Imon
(t)(V ) =1 ⇐⇒
all assignments to V in t have the form V ← V + e, with e ≥ 0
use: combined with non-relational interferences
]
if ∀t : Imon
(t)(V ) =1
then any test with non-relational interference J X ≤ (V | [a, b]) K can
be strengthened into J X ≤ V K
VMCAI’15 – 12 Jan. 2015
The AstréeA static analyzer
Antoine Miné
p. 40 / 65 (1/1)
Abstract rely–guarantee
Rely–guarantee abstractions
Relational invariant interferences
Abstraction:
keep relations maintained by interferences
remove control state in interferences
keep mutex state M
forget input-output relationships
keep relationships between variables
(αfI )
(set of mutexes held)
def
0
0
0
αinv
I (Y ) = { (M, ρ) | ∃ρ : h(M, ρ), (M, ρ )i ∈ Y ∨ h(M, ρ ), (M, ρ)i ∈ Y }
inv
(M, ρ) ∈ αinv
I (Y ) =⇒ (M, ρ) ∈ αI (Y ) after any sequence of interferences from Y
Lock invariant:
{ ρ | ∃t ∈ T , M : (M, ρ) ∈ αIinv (I(t)), m ∈
/ M}
property maintained outside code protected by m
possibly broken while m is locked
restored before unlocking m
VMCAI’15 – 12 Jan. 2015
The AstréeA static analyzer
Antoine Miné
p. 41 / 65 (1/1)
Abstract rely–guarantee
Rely–guarantee abstractions
Relational lock invariants
lock
unlock
t1
t2
lock
Improved interferences:
VMCAI’15 – 12 Jan. 2015
unlock
mixing simple interferences and lock invariants
The AstréeA static analyzer
Antoine Miné
p. 42 / 65 (2/4)
Abstract rely–guarantee
Rely–guarantee abstractions
Relational lock invariants
lock
unlock
t1
non−rel
t2
lock
Improved interferences:
unlock
mixing simple interferences and lock invariants
apply non-relational data-race interferences
unless threads hold a common lock (mutual exclusion)
VMCAI’15 – 12 Jan. 2015
The AstréeA static analyzer
Antoine Miné
p. 42 / 65 (3/4)
Abstract rely–guarantee
Rely–guarantee abstractions
Relational lock invariants
lock
unlock
t1
rel
t2
lock
Improved interferences:
unlock
mixing simple interferences and lock invariants
apply non-relational data-race interferences
unless threads hold a common lock (mutual exclusion)
apply non-relational well-synchronized interferences at lock points
then intersect with the lock invariant
gather lock invariants for lock / unlock pairs
VMCAI’15 – 12 Jan. 2015
The AstréeA static analyzer
Antoine Miné
p. 42 / 65 (4/4)
Abstract rely–guarantee
Rely–guarantee abstractions
Relational lock invariants
lock
unlock
t1
rel
non−rel
t2
lock
Improved interferences:
unlock
mixing simple interferences and lock invariants
apply non-relational data-race interferences
unless threads hold a common lock (mutual exclusion)
apply non-relational well-synchronized interferences at lock points
then intersect with the lock invariant
gather lock invariants for lock / unlock pairs
Note:
still valid in the presence of weak consistent memories
VMCAI’15 – 12 Jan. 2015
The AstréeA static analyzer
Antoine Miné
p. 42 / 65 (4/4)
Abstract rely–guarantee
Rely–guarantee abstractions
Relational interference example
analyzing t1
analyzing t2
t1
t2
while random do
x unchanged
y unchanged
y incremented
0 ≤ y ≤ 102
0 ≤ x, x ≤ y
lock(m);
if x < y then
x ← x + 1;
unlock(m)
t1
t2
while random do
lock(m);
if y < 100 then
y ← y + [1, 3];
unlock(m)
Using all three interference abstractions:
non-relational interferences
(0 ≤ y ≤ 102, 0 ≤ x )
lock invariants, with the octagon domain
monotonic interferences
(x ≤ y )
(y monotonic)
we can prove automatically that x ≤ y holds.
VMCAI’15 – 12 Jan. 2015
The AstréeA static analyzer
Antoine Miné
p. 43 / 65 (1/1)
Memory abstractions
Memory abstractions
VMCAI’15 – 12 Jan. 2015
The AstréeA static analyzer
Antoine Miné
p. 44 / 65 (1/1)
Memory abstractions
Astrée’s first memory abstraction
Handle C data-types:
numeric
(integer,floats)
assign a dimension in an abstract domain
pointers
as references
composite
(set of pointed-to references)
(arrays, structures)
statically and recursively split into scalar cells
(numeric and pointers)
=⇒ full field-sensitivity
(static) folding
represent a whole array with only one of its elements (efficiency gain)
=⇒ weak update (precision loss)
unsupported: union types, pointer arithmetic
VMCAI’15 – 12 Jan. 2015
The AstréeA static analyzer
Antoine Miné
p. 45 / 65 (1/1)
Memory abstractions
Low-level memory examples
Union
Type-punning
union {
struct { uint8 al,ah,bl,bh } b;
struct { uint16 ax,bx } w;
} r;
r.w.ax = 258;
if (r.b.al==2) r.b.al++;
uint8 buf[4] = { 1,2,3,4 };
uint32 i = *((uint32*)buf);
C standard:
ill-typed programs, undefined behavior
In practice:
there is no error
the semantics is well-defined
VMCAI’15 – 12 Jan. 2015
The AstréeA static analyzer
(ABI specification)
Antoine Miné
p. 46 / 65 (1/1)
Memory abstractions
Low-level memory semantics
Dynamic decomposition of the memory into scalar cells
based on the access pattern
read materialize new cells, if needed
(use the contents of overlapping cells, if any)
write materialize new cells, if needed
(discard the contents of overlapping cells, if any)
=⇒ allow overlapping cells, with an intersection semantics
Benefit:
no change needed to numeric domains
(P(variable → value)
P(cell → value))
all correct programs remain correct
[Miné 2006]
VMCAI’15 – 12 Jan. 2015
The AstréeA static analyzer
Antoine Miné
p. 47 / 65 (1/1)
Memory abstractions
Low-level memory example
0
1
2 ...
Union
r.w.ax = 258;
if (r.b.al==2) r.b.al++;
initial state: no cell (>)
VMCAI’15 – 12 Jan. 2015
The AstréeA static analyzer
Antoine Miné
p. 48 / 65 (2/4)
Memory abstractions
Low-level memory example
0
1
2 ...
ax
258
Union
r.w.ax = 258;
if (r.b.al==2) r.b.al++;
create r.w.ax, a uint16 cell at offset 0
VMCAI’15 – 12 Jan. 2015
The AstréeA static analyzer
Antoine Miné
p. 48 / 65 (3/4)
Memory abstractions
Low-level memory example
0
1
2 ...
ax
258
Union
r.w.ax = 258;
if (r.b.al==2) r.b.al++;
al
2
create r.b.al, a uint8 cell at offset 0
initialized with: r.w.ax mod 256
VMCAI’15 – 12 Jan. 2015
The AstréeA static analyzer
Antoine Miné
p. 48 / 65 (4/4)
Memory abstractions
Low-level memory example
0
1
2 ...
Union
r.w.ax = 258;
if (r.b.al==2) r.b.al++;
al
3
modify cell r.b.al
destroy invalidated cell r.w.ax
VMCAI’15 – 12 Jan. 2015
The AstréeA static analyzer
Antoine Miné
p. 48 / 65 (4/4)
Memory abstractions
Bit-level float manipulations
?
double cast(int i) {
union { int i[2]; double d; } x, y;
x.i[0] = 0x43300000; y.i[0] = x.i[0];
x.i[1] = 0x80000000; y.i[1] = i ˆ x.i[1];
return y.d - x.d;
}
VMCAI’15 – 12 Jan. 2015
The AstréeA static analyzer
Antoine Miné
p. 49 / 65 (2/3)
Memory abstractions
Bit-level float manipulations
?
double cast(int i) {
union { int i[2]; double d; } x, y;
x.i[0] = 0x43300000; y.i[0] = x.i[0];
x.i[1] = 0x80000000; y.i[1] = i ˆ x.i[1];
return y.d - x.d;
}
IEEE Binary representation: 64-bit double
a 1-bit sign s
a 52-bit exponent e, with a bias
(e represents e − bias)
a 11-bit mantissa m= .b1 . . . b12
(f represents
s
63
VMCAI’15 – 12 Jan. 2015
11−bit e
P
i
2−i bi )
52−bit m
52
The AstréeA static analyzer
0
Antoine Miné
p. 49 / 65 (3/3)
Memory abstractions
Bit-level float manipulations
Cast
double cast(int i) {
union { int i[2]; double d; } x, y;
x.i[0] = 0x43300000; y.i[0] = x.i[0];
x.i[1] = 0x80000000; y.i[1] = i ˆ x.i[1];
return y.d - x.d;
}
0x43300000 0x80000000 represents 252 + 231
0x43300000 0x80000000
ˆi represents 252 + 231 + i
y.d - x.d equals i
=⇒ cast from 32-bit signed int to 64-bit double
Justification:
some CPUs miss the cast instruction
(PowerPC)
do not rely on the compiler to emulate it
VMCAI’15 – 12 Jan. 2015
The AstréeA static analyzer
(code traceability)
Antoine Miné
p. 49 / 65 (3/3)
Memory abstractions
Bit-level float predicate domain
Cast
double cast(int i) {
union { int i[2]; double d; } x, y;
x.i[0] = 0x43300000; y.i[0] = x.i[0];
x.i[1] = 0x80000000; y.i[1] = i ˆ x.i[1];
return y.d - x.d;
}
Analysis principle:
[Miné 2012]
memory domain: detects union usage
smart initialization at materialization (read x.d, y.d)
y.d = dbl of word(y.i[0], y.i[1])
new ad-hoc symbolic domain: maintains predicates
V = Wˆ0x80000000
V = dbl of word(0x43300000, W)
VMCAI’15 – 12 Jan. 2015
The AstréeA static analyzer
(y.i[1] = iˆx.i[1])
(y.d)
Antoine Miné
p. 50 / 65 (2/2)
Memory abstractions
Bit-level float predicate domain
Cast
double cast(int i) {
union { int i[2]; double d; } x, y;
x.i[0] = 0x43300000; y.i[0] = x.i[0];
x.i[1] = 0x80000000; y.i[1] = i ˆ x.i[1];
return y.d - x.d;
}
reduction between intervals and predicates:
predicates inferred by pattern-matching of expressions
and values provided by intervals
(0x 43300000, 0x 80000000)
symbolic rewrite rules enrich intervals
(y.d − x.d
(double)i)
“Domainlets”: very small abstract domains
adapted to idiomatic constructions
easy to extent with new predicates and propagation rules
VMCAI’15 – 12 Jan. 2015
The AstréeA static analyzer
Antoine Miné
p. 50 / 65 (2/2)
Memory abstractions
Dynamic array folding
Nested arrays
struct {
int x;
struct {
int a, b;
} y[10];
} t[20];
unfolded: t[0].x, t[0].y[0].a, . . . ,
t[0].y[9].b, t[1].x, . . . , t[19].y[9].b
(420 cells)
folded: t[*].x, t[*].y[*].a, t[*].y[*].b
(3 cells)
Abstractions:
slice folding: t[0],. . . ,t[a-1],t[a · · · b],t[b+1],. . . ,t[n-1]
a and b are integers (not symbolic)
dynamic folding:
triggered by memory access t[expr ] when | J expr K | is large
(a is decreased, b is increased)
note: no unfolding
note: ∪ unifies the folding to the least common folding
VMCAI’15 – 12 Jan. 2015
The AstréeA static analyzer
Antoine Miné
p. 51 / 65 (1/1)
Memory abstractions
Stride domain for offset analysis
Nested arrays
struct {
int x;
struct {
int a, b;
} y[10];
A pointer is abstracted as a pair:
a set of possible bases
in P(variables)
a scalar cell whose value is the offset in bytes
=⇒ very easy to model pointer arithmetic
} t[20];
Numeric domains for offset abstraction:
P
can represent α0 + i αi [0, βi ]
stride domain
e.g., offset(t[∗].y[∗].a) = 324[0, 19] + 8[0, 9] + 4
(a combination of intervals and congruences is not sufficient)
αi and βi are inferred automatically
the type of pointed-to data can be dynamic
no normal form
(ad-hoc operators)
we impose αi+1 > αi × βi
(unique decomposition of any x ∈ α0 +
VMCAI’15 – 12 Jan. 2015
P
The AstréeA static analyzer
i
αi [0, βi ])
Antoine Miné
[Frichot Miné]
p. 52 / 65 (1/1)
Memory abstractions
Dynamic variable summarization
Linked list
struct
struct
struct
...
struct
s { struct s* next; ...}
s z = { NULL };
s y = { &z };
List traversal
struct s* h = &a;
while (h) { • h = h->next; }
s a = { &b };
Linked lists, but statically allocated cell variables
Variable summarization:
a single variable a · · · z
represents a ∪ b ∪ · · · ∪ z
(weak update, increased efficiency)
dynamic summarization
=⇒ summarize together variables pointed-to by the same pointer
(once the pointer’s base set reaches a certain size)
VMCAI’15 – 12 Jan. 2015
The AstréeA static analyzer
Antoine Miné
p. 53 / 65 (2/2)
Memory abstractions
Dynamic variable summarization
Linked list
struct
struct
struct
...
struct
s { struct s* next; ...}
s z = { NULL };
s y = { &z };
List traversal
struct s* h = &a;
while (h) { • h = h->next; }
s a = { &b };
Linked lists, but statically allocated cell variables
At •: iterate from base(h) = { a } to base(h) = { a, . . . , z }
adding one pointer base at a time (very inefficient)
Solution:
pointer base widening B1 O B2
find field f “almost” invariant by iteration
B2 ' B1 ∪ { b->f | b ∈ B1 }
close by transitivity
B1 O B2 = { b->f∗ | b ∈ B1 ∪ B2 }
VMCAI’15 – 12 Jan. 2015
The AstréeA static analyzer
Antoine Miné
p. 53 / 65 (2/2)
Memory abstractions
Symbolic lvalues
Original program
Easier program
for (i=0; i<1000; i++) {
if (a[i] < 10)
a[i] = a[i] + 1;
}
for (i=0; i<1000; i++) {
tmp = a[i];
if (tmp < 10)
a[i] = tmp + 1;
tmp = >;
}
Goal: allow strong update in test tmp < 10
=⇒ prevents alarms despite folding and imprecision on i
Symbolic abstract domain:
synthesize and initialize temporary variables tmp = lvalue
replaces occurrences of lvalue with tmp in rvalues
invalidates tmp when a component of lvalue changes
dynamically translate transfer functions at the memory domain level,
transparent for other domains
VMCAI’15 – 12 Jan. 2015
The AstréeA static analyzer
Antoine Miné
p. 54 / 65 (1/1)
Memory abstractions
Going further: Segmented arrays
for (i=0; i<1000; i++) {
a[i] = a[i] + 1;
if (a[i] > 10)
a[i] = 10;
}
Symbolic lvalues are ineffective here
(need not only a strong update, but also a “strong test”)
Solution: Segmented array domain
Mauborgne]
[Cousot Logozzo
dynamic partitionning of arrays with symbolic slice bounds
e.g.: a : {0} [0, 10] {i} [11, +∞] {i + 1} [0, 10] {10000}
Challenges: specific to C
pointers and pointer arithmetic
nested heterogeneous data-structures
union types
Added benefit: handling of C strings (0-terminated)
VMCAI’15 – 12 Jan. 2015
The AstréeA static analyzer
Antoine Miné
p. 55 / 65 (1/1)
Applications
Applications
VMCAI’15 – 12 Jan. 2015
The AstréeA static analyzer
Antoine Miné
p. 56 / 65 (1/1)
Applications
Embedded parallel application
Target application family
Concurrency model:
fixed number of threads
preemptive real-time scheduling on a single processor
shared memory, locks
Target application:
embedded avionic code
1.6 Mloc of C, 15 threads
reactive code + network code + lists, strings, pointers
many variables, large arrays, many loops, shallow call graph
no dynamic memory allocation, no recursivity
VMCAI’15 – 12 Jan. 2015
The AstréeA static analyzer
Antoine Miné
p. 57 / 65 (1/1)
Applications
Embedded parallel application
Analysis context
Concrete execution context:
Target application, C
Other applications
ARINC 653 operating system, C+asm
Hardware
The target application:
runs concurrently with other applications
(memory separation)
interacts dynamically with an ARINC 653 operating system
(thread control, mutex lock and unlock, communication services)
interacts with other applications through the OS
creates system objects only during an initialization phase
(the set of objects is inferred by the analysis)
VMCAI’15 – 12 Jan. 2015
The AstréeA static analyzer
Antoine Miné
p. 58 / 65 (2/2)
Applications
Embedded parallel application
Analysis context
Abstract analysis context:
Target application, C
ARINC 653 model, C + built-ins
The target application is enriched with hand-written model of the OS
2.6 Kloc of C + low-level AstréeA built-ins
stub and simulate all OS system calls
manage (fat) OS objects, mapped to (thin) AstréeA objects
(e.g., AstréeA’s locks are simple integers, ARINC 653’s locks have a string name)
=⇒ analyze stand-alone “C” programs, with no undefined symbol
VMCAI’15 – 12 Jan. 2015
The AstréeA static analyzer
Antoine Miné
p. 58 / 65 (2/2)
Applications
Embedded parallel application
Precision and specialization
2010: 12, 257 false alarms
2015: 1, 195 false alarms
Achieved through:
enabling relational interferences and precise lock handling
adding new abstract domains
interference, offset stride, float predicate, memory domains
tuning existing domains and analysis parameters
O, reductions
VMCAI’15 – 12 Jan. 2015
The AstréeA static analyzer
Antoine Miné
p. 59 / 65 (1/1)
Applications
Embedded parallel application
Efficiency
Timing and memory:
on an intel i7 2.90 GHz workstation
(1 core used)
computation time: 24h
analysis iterations: 6
90 GB RAM
(no widening needed)
(' 77% used for lock invariants)
Critical to efficiency:
dynamic array folding and variable summarization
(starts unfolded, automatically folded after initialization)
loop bootstrapping
(iteration time drops from 9h to 5h, to 2h)
limited activation scope for relational domains
(essentially manual for now, with limited automated heuristics)
VMCAI’15 – 12 Jan. 2015
The AstréeA static analyzer
Antoine Miné
p. 60 / 65 (1/1)
Applications
Embedded parallel application
Work in progress
Improving the precision and usability:
specialize towards zero false alarm for our main application
memory domains (segmented array domain, specialization to strings)
relational interference domains
inter-thread trace semantics
(partitioning wrt. the history of threads and their interactions)
automate precision-control heuristics
apply to other software in the same family
(on-going work by our industrial partners)
Improving the scope:
alternate OS models: OSEK/AUTOSAR, Posix Threads
Industrialization project
VMCAI’15 – 12 Jan. 2015
(merging with commercial Astrée)
The AstréeA static analyzer
Antoine Miné
p. 61 / 65 (1/1)
Applications
Embedded operating system
Analyzing an embedded operating system (WIP)
Recent on-going project
using AstréeA to check the C part of a small operating system
Concrete execution context:
Application
ARINC 653 library, in C
Assembly routines
Other OS components
Hardware
ARINC 653 library: 7.5 Kloc of C + a few assembly routines
interacts with an application
(system calls)
interacts with an OS kernel and other OS services
(C, asm)
explicit task switching, manage timer and interrupts
VMCAI’15 – 12 Jan. 2015
The AstréeA static analyzer
Antoine Miné
(preemptive)
p. 62 / 65 (1/1)
Applications
Embedded operating system
Abstract analysis context
Analysis driver, C+builtins
Application model, C+builtins
ARINC 653 library, in C
Stubs, C+builtins
Hand-written sources added to complete the target library (1.5 Kloc, C + builtins)
close the sources
(=⇒ stand-alone program)
C stub of asm primitives
stub bootstrap code (calls library initialization)
simulate timer interrupt with an asynchronous process
simulate stack contexts with processes, context switch with preemption
model the effect of linking to an application
populate static configuration tables
(process tables)
simulate system calls callable by any application
heavily non-deterministic, for both data and control
to simulate all the cases allowed by the specification and system limits
VMCAI’15 – 12 Jan. 2015
The AstréeA static analyzer
Antoine Miné
p. 63 / 65 (1/1)
Applications
Embedded operating system
Very preliminary results
Analysis results:
50 Kloc of C after preprocessing and simplification
analysis time: 22 mn
analysis iterations: 5
900 MB of RAM used
a few hundred false alarms
Conclusion:
the sound static analysis of an OS fragment is possible
specialization work needed to improve the precision
in the future, consider other parts of the OS
VMCAI’15 – 12 Jan. 2015
The AstréeA static analyzer
Antoine Miné
p. 64 / 65 (1/1)
Thank you
Thank you
VMCAI’15 – 12 Jan. 2015
The AstréeA static analyzer
Antoine Miné
p. 65 / 65 (1/1)