Putting Newton into Practice: A Solver for
Polynomial Equations over Semirings
Maximilian Schlund
joint work with
Michal Terepeta Michael Luttenberger
Department of Computer Science
TU München
October 3, 2013
1 / 14
Example: Interprocedural Counting—reentrant locks
proc AcquireRelease
Lock ! ( ) ;
if
: : t r u e => A c q u i r e R e l e a s e ( )
: : t r u e => s k i p
fi ;
Release ()
end
proc Release
if
: : t r u e => U n l o c k ! ( )
: : t r u e => R e l e a s e ( )
fi
end
p r o c main
AcquireRelease ()
end
#(lock) vs. #(unlock) at
start
• at the
of AcquireRelease() ?
end
• at the end of main() ?
2 / 14
Computing Procedure Summaries
Compute effect of procedures Aquire() and Release()
(”+” = choice, ”·” = composition):
Aq
Rel
= lock · (Aq + skip) · Rel
=
unlock + Rel
3 / 14
Computing Procedure Summaries
Compute effect of procedures Aquire() and Release()
(”+” = choice, ”·” = composition):
= lock · (Aq + skip) · Rel
=
unlock + Rel
Aq
Rel
2
Abstract domain: 2N “Counting Semiring” (commutative,
idempotent, but infinite ascending chains!)
Aq
Rel
= {(1, 0)} · (Aq + {(0, 0)}) · Rel
=
{(0, 1)} + Rel
3 / 14
Computing Procedure Summaries
Compute effect of procedures Aquire() and Release()
(”+” = choice, ”·” = composition):
= lock · (Aq + skip) · Rel
=
unlock + Rel
Aq
Rel
2
Abstract domain: 2N “Counting Semiring” (commutative,
idempotent, but infinite ascending chains!)
Aq
Rel
= {(1, 0)} · (Aq + {(0, 0)}) · Rel
=
{(0, 1)} + Rel
Solution:
Aq
Rel
= {(k, k) : k ≥ 1}
=
{(0, 1)}
3 / 14
Computing Procedure Summaries
Compute effect of procedures Aquire() and Release()
(”+” = choice, ”·” = composition):
= lock · (Aq + skip) · Rel
=
unlock + Rel
Aq
Rel
2
Abstract domain: 2N “Counting Semiring” (commutative,
idempotent, but infinite ascending chains!)
Aq
Rel
= {(1, 0)} · (Aq + {(0, 0)}) · Rel
=
{(0, 1)} + Rel
Solution:
Aq
Rel
= {(k, k) : k ≥ 1}
=
{(0, 1)}
• Standard fixed point iteration does not terminate here
(approximation needed)!
3 / 14
Computing Procedure Summaries
Compute effect of procedures Aquire() and Release()
(”+” = choice, ”·” = composition):
= lock · (Aq + skip) · Rel
=
unlock + Rel
Aq
Rel
2
Abstract domain: 2N “Counting Semiring” (commutative,
idempotent, but infinite ascending chains!)
Aq
Rel
= {(1, 0)} · (Aq + {(0, 0)}) · Rel
=
{(0, 1)} + Rel
Solution:
Aq
Rel
= {(k, k) : k ≥ 1}
=
{(0, 1)}
• Standard fixed point iteration does not terminate here
(approximation needed)!
• Newton’s method terminates and computes the exact solution!
3 / 14
Modelling analyses: ω-continuous semirings
Semiring (S, +, ·, 0, 1)
(S, +, 0) is a commutative monoid
(S, ·, 1) is a monoid
· distributes over +
a·0=0·a=0
ω-continuous
:≈ infinite sums behave like absolutely convergent series.
4 / 14
Modelling analyses: ω-continuous semirings
Semiring (S, +, ·, 0, 1)
(S, +, 0) is a commutative monoid
(S, ·, 1) is a monoid
· distributes over +
a·0=0·a=0
ω-continuous
:≈ infinite sums behave like absolutely convergent series.
KleenePstar
a∗ :=
n≥0 a
n
4 / 14
Modelling analyses: ω-continuous semirings
Semiring (S, +, ·, 0, 1)
(S, +, 0) is a commutative monoid
(S, ·, 1) is a monoid
· distributes over +
a·0=0·a=0
ω-continuous
:≈ infinite sums behave like absolutely convergent series.
KleenePstar
a∗ :=
n≥0 a
n
Examples
• (N, +, ·, 0, 1) semiring, not ω-continuous!
4 / 14
Modelling analyses: ω-continuous semirings
Semiring (S, +, ·, 0, 1)
(S, +, 0) is a commutative monoid
(S, ·, 1) is a monoid
· distributes over +
a·0=0·a=0
ω-continuous
:≈ infinite sums behave like absolutely convergent series.
KleenePstar
a∗ :=
n≥0 a
n
Examples
• (N, +, ·, 0, 1) semiring, not ω-continuous!
• (N∞ , min, +, ∞, 0) Tropical semiring (
shortest paths)
4 / 14
Modelling analyses: ω-continuous semirings
Semiring (S, +, ·, 0, 1)
(S, +, 0) is a commutative monoid
(S, ·, 1) is a monoid
· distributes over +
a·0=0·a=0
ω-continuous
:≈ infinite sums behave like absolutely convergent series.
KleenePstar
a∗ :=
n≥0 a
n
Examples
• (N, +, ·, 0, 1) semiring, not ω-continuous!
• (N∞ , min, +, ∞, 0) Tropical semiring (
• ([0, 1], max, ·, 0, 1) Viterbi semiring (
shortest paths)
maximum likelihood)
4 / 14
Modelling analyses: ω-continuous semirings
Semiring (S, +, ·, 0, 1)
(S, +, 0) is a commutative monoid
(S, ·, 1) is a monoid
· distributes over +
a·0=0·a=0
ω-continuous
:≈ infinite sums behave like absolutely convergent series.
KleenePstar
a∗ :=
n≥0 a
n
Examples
• (N, +, ·, 0, 1) semiring, not ω-continuous!
• (N∞ , min, +, ∞, 0) Tropical semiring (
• ([0, 1], max, ·, 0, 1) Viterbi semiring (
•
k
(2N , ∪, ·, ∅, {0})
Counting semiring (
shortest paths)
maximum likelihood)
counting analyses)
4 / 14
Counting Semiring
To determine relations e.g. between counts of resource uses.
k
• Carrier: 2N
• A + B := A ∪ B
• A · B := {a + b : a ∈ A, b ∈ B}
• 0 := ∅
• 1 := {0}
ω-continuous, commutative, idempotent
5 / 14
Counting Semiring
To determine relations e.g. between counts of resource uses.
k
• Carrier: 2N
• A + B := A ∪ B
• A · B := {a + b : a ∈ A, b ∈ B}
• 0 := ∅
• 1 := {0}
ω-continuous, commutative, idempotent
You can also think of the (finitely generated) elements as. . .
commutative regular expressions: e.g. accac = a2 c 3 and
(a + b)∗ = a∗ b ∗
5 / 14
Counting Semiring
To determine relations e.g. between counts of resource uses.
k
• Carrier: 2N
• A + B := A ∪ B
• A · B := {a + b : a ∈ A, b ∈ B}
• 0 := ∅
• 1 := {0}
ω-continuous, commutative, idempotent
You can also think of the (finitely generated) elements as. . .
commutative regular expressions: e.g. accac = a2 c 3 and
(a + b)∗ = a∗ b ∗
Drawback
infinite ascending chains: Ai = {(k, k) : k ≤ i} (limit: A = (1, 1)∗ )
standard fixed point iteration does not terminate!
5 / 14
Solving Equations: Newton over R
To solve fixpoint equation
X = F (X )
apply Newton’s method to G (X ) := F (X ) − X (with ν (0) = 0):
ν (i+1) = ν (i) −
G (ν (i) )
=
G 0 (ν (i) )
6 / 14
Solving Equations: Newton over R
To solve fixpoint equation
X = F (X )
apply Newton’s method to G (X ) := F (X ) − X (with ν (0) = 0):
ν (i+1) = ν (i) −
G (ν (i) )
F (ν (i) ) − ν (i)
(i)
=
ν
+
=
G 0 (ν (i) )
1 − F 0 (ν (i) )
6 / 14
Solving Equations: Newton over R
To solve fixpoint equation
X = F (X )
apply Newton’s method to G (X ) := F (X ) − X (with ν (0) = 0):
ν (i+1) = ν (i) −
G (ν (i) )
F (ν (i) ) − ν (i)
(i)
=
ν
+
= ν (i) + F 0 (ν (i) )∗ · δi
(i)
(i)
0
0
G (ν )
1 − F (ν )
6 / 14
Solving Equations: Newton over R
To solve fixpoint equation
X = F (X )
apply Newton’s method to G (X ) := F (X ) − X (with ν (0) = 0):
ν (i+1) = ν (i) −
G (ν (i) )
F (ν (i) ) − ν (i)
(i)
=
ν
+
= ν (i) + F 0 (ν (i) )∗ · δi
(i)
(i)
0
0
G (ν )
1 − F (ν )
Can be generalized to semirings, if we can compute:
• F 0 (requires some care in the non-commutative case)
• F 0 (ν (i) )∗ (in general: Kleene star of Jacobian matrix)
• δi , s.t. ν (i) + δi = F (ν (i) )
6 / 14
Combinatorial interpretation of fixed point equations
“CFG view” of equations:
Equation system
X = aX 2 + c
Least solution
,
,
,
Context-free grammar
X → aXX | c
P
deriv. tree t V (t),
with V (t) = product of terminals
7 / 14
Two approximation schemes for the least solution
Approximating the least solution of X = aX 2 + c over the
counting SR
k fixed point iters
,
eval all derivation trees with height < k
8 / 14
Two approximation schemes for the least solution
Approximating the least solution of X = aX 2 + c over the
counting SR
k fixed point iters
,
eval all derivation trees with height < k
0,
8 / 14
Two approximation schemes for the least solution
Approximating the least solution of X = aX 2 + c over the
counting SR
k fixed point iters
,
eval all derivation trees with height < k
0, c,
8 / 14
Two approximation schemes for the least solution
Approximating the least solution of X = aX 2 + c over the
counting SR
k fixed point iters
,
eval all derivation trees with height < k
0, c, ac 2 ,
8 / 14
Two approximation schemes for the least solution
Approximating the least solution of X = aX 2 + c over the
counting SR
k fixed point iters
,
eval all derivation trees with height < k
0, c, ac 2 , a2 c 3 , a3 c 4 ,
8 / 14
Two approximation schemes for the least solution
Approximating the least solution of X = aX 2 + c over the
counting SR
k fixed point iters
,
eval all derivation trees with height < k
0, c, ac 2 , a2 c 3 , a3 c 4 , . . .
8 / 14
Two approximation schemes for the least solution
Approximating the least solution of X = aX 2 + c over the
counting SR
k fixed point iters
,
k Newton iters
,
eval all derivation trees with height < k
0, c, ac 2 , a2 c 3 , a3 c 4 , . . .
eval all derivation trees with dimension < k
8 / 14
Two approximation schemes for the least solution
Approximating the least solution of X = aX 2 + c over the
counting SR
k fixed point iters
,
k Newton iters
,
eval all derivation trees with height < k
0, c, ac 2 , a2 c 3 , a3 c 4 , . . .
eval all derivation trees with dimension < k
0,
8 / 14
Two approximation schemes for the least solution
Approximating the least solution of X = aX 2 + c over the
counting SR
k fixed point iters
,
k Newton iters
,
eval all derivation trees with height < k
0, c, ac 2 , a2 c 3 , a3 c 4 , . . .
eval all derivation trees with dimension < k
0, c,
8 / 14
Two approximation schemes for the least solution
Approximating the least solution of X = aX 2 + c over the
counting SR
k fixed point iters
,
k Newton iters
,
eval all derivation trees with height < k
0, c, ac 2 , a2 c 3 , a3 c 4 , . . .
eval all derivation trees with dimension < k
0, c, (ac)∗ c
8 / 14
Tree Dimension
Newton Approximation: Evaluate derivation trees up to dimension
d.
0
d
d
d
≤d −1
d −1
d −1
9 / 14
Tree Dimension
Newton Approximation: Evaluate derivation trees up to dimension
d.
0
d
d
≤d −1
d
d −1
d −1
aXX
c
aXX
c
aXX
c
c
9 / 14
Tree Dimension
Newton Approximation: Evaluate derivation trees up to dimension
d.
0
d
d
≤d −1
d
d −1
d −1
aXX
c 0
aXX
c
0
aXX
c
c
0
0
9 / 14
Tree Dimension
Newton Approximation: Evaluate derivation trees up to dimension
d.
0
d
d
≤d −1
d
d −1
d −1
aXX
aXX
c
0
c 0
aXX 1
c
c
0
0
9 / 14
Tree Dimension
Newton Approximation: Evaluate derivation trees up to dimension
d.
0
d
d
≤d −1
d
d −1
aXX
1
aXX
c
0
d −1
c 0
aXX 1
c
c
0
0
9 / 14
Tree Dimension
Newton Approximation: Evaluate derivation trees up to dimension
d.
0
d
d
≤d −1
d
d −1
d −1
1
aXX
1
aXX
c
0
c 0
aXX 1
c
c
0
0
height: 3, dimension: 1
9 / 14
Computing δi = F (ν (i) ) − ν (i)
Reminder:
ν (i+1) = ν (i) + F 0 (ν (i) )∗ · δi
• Combinatorially: take derivation trees of dimension < i apply
F to them (get larger trees), then throw away all those for
which the dimension did not increase (not effective).
• δi can be effectively computed by unfolding the equation
system w.r.t. dimension.
Commutative case
δi can be expressed using higher-order derivatives of F evaluated at
previous Newton approximations.
10 / 14
Features of our implementation
• C++ library + solver and parser
• Heavily templated
new semirings can be easily integrated
without changing the solving algorithms
• Data structures for polynomials, matrices, several predefined
semirings
• Can be obtained freely from
https://github.com/mschlund/newton
11 / 14
Performance Experiments
Runtime normalized by n3
·10−3
Runtime for different densities
3,000
ε = 0.1
ε = 0.2
ε = 0.3
ε = 0.4
3
2
time
n3
time in ms
ε = 0.5
2,000
1,000
1
0
0
50
# of equations n
100
50
# of equations n
Random quadratic equations over R≥0 with ε
Time needed for 10 Newton iterations.
n
2
100
monomials each.
12 / 14
Demo
• Computing the commutative image of contex-free grammars.
• Computing the probability that a PCFG generates a word
starting from non-terminal A (cumulative probability of all
finite derivations).
13 / 14
Summary and Future Work
• Generic C++ library for solving polynomial equations over
semirings
• Efficient algorithms and data structures for special semirings
14 / 14
Summary and Future Work
• Generic C++ library for solving polynomial equations over
semirings
• Efficient algorithms and data structures for special semirings
TODO Symbolic representations for semilinear sets (NDDs,
Presburger)
TODO More applications (e.g. Pointer Analysis)
Thank you!
14 / 14