An Integration of Program Analysis
and Automated Theorem Proving
Bill J. Ellis & Andrew Ireland
School of Mathematical & Computer Sciences
Heriot-Watt University
Edinburgh
Context
• Investigate the role of proof planning within
the SPARK approach to high integrity
software
• Funded by the EPSRC Critical Systems
programme (GR/R24081) in collaboration
with Praxis Critical Systems Ltd.
• www.macs.hw.ac.uk/~air/nuspade
Overview
• The SPARK approach
• Program verification in SPARK
• NuSPADE: Our Approach
– Proof planning
– Program analysis
– Example
• Results and future work
• Conclusions
The SPARK Approach
• Developed by Praxis Critical Systems Ltd.
• For high integrity software:
– Safety, security and mission critical software,
e.g. SHOLIS - UK MoD Def Standard 00-55
• SPARK is a programming language:
– Defined as a subset of Ada that:
• Eliminates ambiguities to make static analysis possible
• Eliminates complexities to makes static analysis feasible
• Can be compiled by any Ada compiler!
– Specification via code level annotations
The SPARK Tools
• Examiner:
– Checks conformance to the SPARK subset
– Performs information and data flow static analysis
– Generates Verification Conditions (VCs) for both:
• partial correctness (correct if program terminates)
• Exception freedom (no run time errors)
• SPADE proof tools:
– SPADE Simplifier a special purpose theorem prover
– SPADE Proof Checker an interactive theorem prover
Program Verification in SPARK
SPARK
code +
specification
Examiner
Verification
Conditions
SPADE Simplifier
Proof!
Typically 93% of exception freedom VCs are
discharged automatically following this process
Program Verification in SPARK
SPARK
code +
specification
Examiner
Verification
Conditions
SPADE Simplifier
Proof!
Remaining Verification
Conditions
But the remaining 7% still account for hundreds
of VCs ...
Why the Simplifier can Fail
• VC not provable:
– Bug in the code
– Bug in the specification
• The Simplifier is not strong enough:
– Stronger invariant required
– Stronger proof methods required
Program Verification in SPARK
SPARK
code +
specification
Examiner
Verification
Conditions
SPADE Simplifier
Strengthen
invariant
User
intervention
Discover
proof
Proof!
Remaining Verification
Conditions
Proof
Script
SPADE Proof
Checker
Proof!
NuSPADE: Our Approach
SPARK
code +
specification
Examiner
Verification
Conditions
SPADE Simplifier
Strengthen
invariant
Remaining Verification
Conditions
NuSPADE
Still remaining
Verification
Conditions!
Discover
proof
Proof!
Proof
Script
SPADE Proof
Checker
Proof!
NuSPADE
• Increase proof automation
• Integrates proof and program analysis:
– Proof planner
– Program analysis oracle
Proof Planning
• Use of high-level proof outlines, known as proof
plans, to guide proof search
• Supports middle-out reasoning, i.e. the use of meta
variables to delay choice during proof search
• Automatic proof patching via proof failure
analysis, e.g. conjecture generalization, lemma
discovery, induction revision, case splitting, loop
invariant discovery
• Inductive and non-inductive applications
Proof Planning
Conjectures
Theory
Proof planning:
Methods + Critics
Proof checking:
Tactics
Program Analysis
• Program analysis automatically finds
interesting properties about source code
• Program analysis in practice:
– Flow analysis
– Performance analysis
– Discovering constraints on variables (Abstract
Interpretation)
– Discovery code properties (invariant discovery)
NuSPADE Behaviour
SPARK
code +
specification
Examiner
Verification
Conditions
SPADE Simplifier
Strengthen
invariant
Remaining Verification
Conditions
NuSPADE
Still remaining
Verification
Conditions!
Discover
proof
Proof!
Proof
Script
SPADE Proof
Checker
Proof!
NuSPADE Behaviour
SPARK
code +
specification
Strengthen
invariant
Remaining Verification
Conditions
NuSPADE
Still remaining
Verification
Conditions!
Discover
proof
Proof
Script
NuSPADE Behaviour
Remaining Verification
Conditions
NuSPADE
Proof
planner
Discover
proof
Proof
Script
NuSPADE Behaviour
Strengthen
invariant
SPARK code +
specification
Remaining Verification
Conditions
NuSPADE
Recurrence
relation
solver
Inequality /
general
reasoning
tool
Program
analysis
oracle
Proof
planner
NuSPADE Behaviour
Remaining Verification
Conditions
NuSPADE
Proof
planner
Discover
proof
Proof
Script
NuSPADE Behaviour
Remaining Verification
Conditions
NuSPADE
Proof
planner
No proof
found
Still
remaining
Verification
Conditions!
Example
subtype Index is Integer range 0 .. 9;
type D_Type is array (Index) of Integer;
subtype R_Type is Integer;
…
R:=0;
For I in Index loop
if D(I) >= 0 and D(I) <= 100 then
R:= R+D(I);
end if;
end loop;
Example
subtype Index is Integer range 0 .. 9;
type D_Type is array (Index) of Integer;
subtype R_Type is Integer;
…
R:=0;
For I in Index loop
if D(I) >= 0 and D(I) <= 100 then
R:= R+D(I);
end if;
Integer’first  R+D(I)  Integer’last
end loop;
Exception Freedom Proof Methods
• Elementary:
Pre: Trivial goal
Effect: Assume provable in proof checker
• Fertilise:
Pre: Part of the goal matches a hypothesis
Effect: Simplify the goal
• Decomposition: Pre: The goal has a transitive relation
Effect: Decompose the transitive relation
• Transitivity:
Pre: The goal has a transitive relation and
related hypotheses
Effect: Apply transitive step
Transitivity Method and Critic
Preconditions:
there exists a conclusion of the form:
Method
Critic
Expression RelationOp Constant
For every Variablei in Expression there
exists a hypothesis of the form:
Variablei RelationOp Expresssioni
Method applicable - apply the method!
Method not applicable – critic extracts hypothesis
schemas and invokes program analysis oracle
X
Example: Proof Failure
Hypotheses: element d , i   100
Conclusion: r  element d , i   int_last
None of the methods are applicable.
However, the transitivity critic fires as: r
 element d , i 
occursr in:
but there is no hypothesis of the form: r  Expressionup
(Similarly the lower bound causes the critic to fire as
there is no hypothesis of the form: r  Expressionlow
)
Example: Program Analysis
• Discover a property that introduces hypotheses of
the form: r  Expression and
up
r  Expressionlow
• Program analysis heuristics:
–
–
–
–
Type and other immediate information
Non looping code
Looping code (Involves recurrence relations)
Exiting loops (Consider loop guards)
Example: Program Analysis
R:=0;
In 
For I in Index loop
if D(I) >= 0 and D(I) <= 100
then
R:= R+D(I);
end if;
end loop;
I I
n
Rn  Rn  1
In  1
Rn  Rn  1 

1
n 1


Rn  Rn  1  D I n  1

Example: Program Analysis
(Variable I)
R:=0;
For I in Index loop
if D(I) >= 0 and D(I) <= 100
then
R:= R+D(I);
end if;
end loop;
In  In  1  1
I n  I0  n 1
In  n
Example: Program Analysis
(Variable R, first disjunct)
R:=0;
For I in Index loop
if D(I) >= 0 and D(I) <= 100
then
R:= R+D(I);
end if;
end loop;
Rn  Rn  1
Rn  R0
Rn  0
Example: Program Analysis
(Variable R, second disjunct)
R:=0;
For I in Index loop
if D(I) >= 0 and D(I) <= 100
then
R:= R+D(I);
end if;
end loop;


Rn  Rn  1  D I n  1
Rn  Rn  1  0

Rn  Rn  1  100
Rn  R0  n  0 Rn  R0  n 100
Rn  0
Rn  n 100
Rn  0  Rn  n 100
Example: Program Analysis
(Variable R, combine disjuncts)
R:=0;
For I in Index loop
if D(I) >= 0 and D(I) <= 100
then
R:= R+D(I);
end if;
end loop;
Rn  0
Rn  0  Rn  n 100
Rn  0 Rn  0  Rn  n 100
Rn  0  Rn  n 100
Example: Program Analysis
(Eliminate n)
R:=0;
For I in Index loop
if D(I) >= 0 and D(I) <= 100
then
R:= R+D(I);
end if;
end loop;
In  n
Rn  0  Rn  n 100
Rn  0  Rn  In 100
Revised code
subtype Index is Integer range 0 .. 9;
type D_Type is array (Index) of Integer;
subtype R_Type is Integer;
…
R:=0;
For I in Index loop
--# assert R >= 0 and R <= I*100;
if D(I) >= 0 and D(I) <= 100 then
R:= R+D(I);
end if;
end loop;
Example: Proof Planning Success
Hypotheses: r  i  100  element d , i   100
Conclusion: r  element d , i   int_last
Transitivity:
r  element d , i   X 0  X 0  int_last
Decomposition:
r  X1  element d , i   X 2  X1  X 2  int_last
Fertilise:
i 100  100
Elementary:
true
 int_last {i 100 / X1,100 / X 2}
Results & Future Work
• Prototype system – Demo Available!
• Integration partial
• Development corpus includes programs with
simple loops and conditionals, i.e. typical
style of critical software applications
• Evaluation corpus supplied by Praxis Critical
Systems – evaluation phase to begin shortly
Future Work
• Strengthen recurrence relation solver and
experiment with other external reasoners,
e.g. inequality reasoning
• Explore program analysis as a basis for bug
finding (ongoing)
• Explore program analysis as a basis for
reducing VC complexity
Conclusions
• An integrated approach to software verification:
–
–
–
–
Proof planning
Program analysis oracle
Proof checking
External reasoners
• Proof planning and program analysis are generic