Lucy-n: a n-Synchronous Extension of Lustre
Louis Mandel
Florence Plateau
Marc Pouzet
Laboratoire de Recherche en Informatique
Université Paris-Sud 11
INRIA Saclay – Ile-de-France
MPC – 22/06/2010
Kahn Process Networks [Gilles Kahn, 1974]
x
i
P
y
Q
o
R
Network of processes
• concurrent execution
• communication through buffers of sufficient size
If processes are deterministic then the network is deterministic
2
Programming Kahn Process Networks
Problem: computation of sufficient buffer sizes
• risk of data loss, of deadlock
• sometimes, need of infinite buffers
Goal:
• rejection of infinite buffers
• automatic sizing of buffers
Related work:
• Synchronous Data Flow and variants [Lee et al.] [Buck]
• scheduling [Carlier, Chretienne] [Baccelli, Cohen, Quadrat]
• Network Calculus [Cruz], Real-time Calculus [Thiele et al.]
3
Dataflow Synchronous Model
x
i
P
y
Q
o
R
Programming Kahn networks without buffers:
• programming languages: Lustre, Signal, Lucid Synchrone
• instantaneous consumption of produced data
• strong guaranties: bounded memory, absence of deadlocks
But: communication without buffers sometimes too restrictive
(e.g. multimedia applications)
4
n-Synchronous Model
x
i
P
y
Q
o
R
Programming Kahn networks with bounded memory
Automatic methods at compile time to:
• reject networks needing infinite memory
• compute activation paces of computations nodes
• compute sufficient buffers sizes
More flexibility with the same guaranties
5
Overview
1. Lucy-n: a n-Synchronous Extension of Lustre
2. Adaptability Relation
3. Clock Typing
6
Lucy-n: a n-Synchronous Extension of Lustre
A Dataflow Synchronous Kernel à la Lustre
x
flow
values
x
5
7 3 6
clock
2 8 1 ...
1111111...
8
A Dataflow Synchronous Kernel
Sampling
c
x
when
flow
values
clock
x
5
c
1 0 1 0 1 0 1 ...
x when c
5
7 3 6
3
2 8 1 ...
2
1 ...
1111111...
1010101...
clock(x when c) = clock(x) on c
on operator :
0.w1 on w2
1.w1 on 1.w2
1.w1 on 0.w2
def
=
0.(w1 on w2 )
def
=
1.(w1 on w2 )
def
0.(w1 on w2 )
=
8
A Dataflow Synchronous Kernel
x
Sampling
c
d
when
when
flow
values
clock
x
5
c
1 0 1 0 1 0 1 ...
x when c
5
3
2
1 ...
d
1
0
1
1 ...
(x when c) when d
5
2
1 ...
7 3 6
2 8 1 ...
1111111...
1010101...
1000101...
clock((x when c) when d) = clock(x when c) on d
on operator :
0.w1 on w2
1.w1 on 1.w2
1.w1 on 0.w2
def
=
0.(w1 on w2 )
def
=
1.(w1 on w2 )
def
0.(w1 on w2 )
=
8
A Dataflow Synchronous Kernel
Scalar Operators
x
y
+
z
flow
values
clock
x
5
7
3
6
2
8
1
...
111111...
y
3
2
1
5
4
1
7
...
111111...
z = x + y
8
9
4
11
6
9
8
...
111111...
9
A Dataflow Synchronous Kernel
c
x
y
Scalar Operators
+
z
t
when
flow
values
clock
x
5
7
3
6
2
8
1
...
111111...
y
3
2
1
5
4
1
7
...
111111...
z = x + y
8
9
4
11
6
9
8
...
111111...
c
1
0
1
0
1
0
1
...
t = z when c
8
8
...
4
6
101010...
9
A Dataflow Synchronous Kernel
c
x
y
Scalar Operators
+
z
t
when
c
o
+
r
when
flow
values
clock
x
5
7
3
6
2
8
1
...
111111...
y
3
2
1
5
4
1
7
...
111111...
z = x + y
8
9
4
11
6
9
8
...
111111...
c
1
0
1
0
1
0
1
...
t = z when c
8
4
6
8
...
101010...
r = y when c
3
1
4
7
...
101010...
o = t + r
11
5
10
15
...
101010...
9
A Dataflow Synchronous Kernel
c
x
y
Scalar Operators
+
z
t
when
c’
o
+
r
when
flow
values
x
5
7
3
6
2
8
1
...
111111...
y
3
2
1
5
4
1
7
...
111111...
z = x + y
8
9
4
11
6
9
8
...
111111...
c
1
0
1
0
1
0
1
...
t = z when c
8
8
...
101010...
...
010101...
r = y when c’
o = t + r
clock
4
2
6
5
1
rejected
10
n-Synchronous Extension: Bufferization Operator
c
x
y
+
z
t
when
t’
c’
r
when
flow
values
t
8
= z when c
o
+
clock
4
6
8
...
101010...
t’ = buffer(t)
8
4
6
...
010101...
r
= y when c’
2
5
1
...
010101...
o
= t’ + r
10
9
7
...
010101...
• adaptability relation ⇒ communication through a bounded buffer
• example : clock(t) <: clock(t’)
11
Adaptability Relation
Number of ones
Cumulative Function
11
10
9
8
7
6
5
4
3
2
1
0
0 1 2 3 4 5
Ow1
6 7 8 9 10 11 12 13 14 15 16
Instants
w1 1 1 0 1 0 1 1 0 1 0 1 1 0 1 0 1
Ow1 : cumulative function of the word w1
13
Number of ones
Adaptability Relation
11
10
Ow1
9
Ow2
8
7
Writings in the buffer
6
5
4
3
2
Readings in the buffer
1
0
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
Instants
buffer size
size(w1 , w2 ) = maxi∈N (Ow1 (i) − Ow2 (i))
adaptability
w1 <: w2 ⇔ ∃n ∈ N, ∀i, 0 ≤ Ow1 (i) − Ow2 (i) ≤ n
def
14
Number of ones
Adaptability Relation
11
10
Ow1
9
Ow2
8
7
Writings in the buffer
6
5
4
3
2
Readings in the buffer
1
0
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
Instants
buffer size
size(w1 , w2 ) = maxi∈N (Ow1 (i) − Ow2 (i))
adaptability
w1 <: w2 ⇔ ∃n ∈ N, ∀i, 0 ≤ Ow1 (i) − Ow2 (i) ≤ n
synchronizability
w1 ⊲⊳ w2
⇔ ∃b1 , b2 ∈ Z, ∀i, b1 ≤ Ow1 (i) − Ow2 (i) ≤ b2
precedence
w1 w2
⇔ ∀i, Ow1 (i) ≥ Ow2 (i)
def
def
def
14
Number of ones
Ultimately Periodic Clocks
11
10
Ow1
9
Ow2
8
7
Writings in the buffer
6
5
4
3
2
Readings in the buffer
1
0
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
Instants
Example : 0(00111) = 0 00111 00111 ...
Checking relations on clocks
• synchronizability test: (11010) ⊲⊳ 0(00111)
• precedence test: (11010) 0(00111)
15
Number of ones
Abstract Clocks: abs (w) = hb0 , b1 i (r)
11
∆1
˙
¸
`
´
10
∆0
a1 = 0, 45 53
9
Ow1
8
7
6
5
4
3
2
1
0
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
Instants


w[i] = 1 ⇒
0 1
concr (hb , b i (r)) = w |

w[i] = 0 ⇒
∆1 : r × i + b1
∆0 : r × i + b0

Ow (i) ≤ ∆1 (i) 
Ow (i) ≥ ∆0 (i) 
16
Number of ones
Adaptability Relation on Abstract Clocks
⊲⊳∼ and ∼
11
˙ 4¸`3´
10
a1 = 0, 5 5
Ow1
9
Ow2
8
7
6
5
4
˙ 9
¸`3´
3
3
a2 = − 5 , − 5 5
2
1
0
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
Instants
def
synchronizability
hb0 1 , b1 1 i (r1 ) ⊲⊳∼ hb0 2 , b1 2 i (r2 ) ⇔ r1 = r2
precedence
hb0 1 , b1 1 i (r) ∼ hb0 2 , b1 2 i (r)
buffer size
size∼ (a1 , a2 ) = ⌊b1 1 − b0 2 ⌋
def
⇔ b1 2 − b0 1 < 1
17
Property
Property size_abs_correctness:
forall (w1:ibw) (w2:ibw),
forall a1 a2: abstraction_phd,
forall H_swf_a1: strong_well_formed_abstraction_phd a1,
forall H_swf_a2: strong_well_formed_abstraction_phd a2,
forall H_non_empty_1: non_empty a1,
forall H_non_empty_2: non_empty a2,
forall (H_a1_eq_abs_w1: in_abstraction_phd w1 a1),
forall (H_a2_eq_abs_w2: in_abstraction_phd w2 a2),
forall H_sub: adaptability_abs a1 a2,
forall s,
size w1 w2 s -> (s <= size_abs a1 a2)%Z.
Correctness properties proved in Coq: _
18
Clock Typing
Typing
plus plus
c
x
y
+
z
when
t
c’
when
4
t’
+
o
r
let node plus_plus (x,y) = o where
5
rec z = x + y
6
and t = z when c
7
and t’= buffer(t)
8
and r = y when c’
9
and o = t’ + r
val plus_plus : (int * int) -> int
val plus_plus :: forall ’a. (’a * ’a) -> ’a on c’
Buffer line 7, characters 11-21: size = 1
20
Typing
plus plus
c
x
α
+
y
z
α
α
when
t
c’
when
4
t’
+
o
r
let node plus_plus (x,y) = o where
5
rec z = x + y
6
and t = z when c
7
and t’= buffer(t)
8
and r = y when c’
9
and o = t’ + r
val plus_plus : (int * int) -> int
val plus_plus :: forall ’a. (’a * ’a) -> ’a on c’
Buffer line 7, characters 11-21: size = 1
20
Typing
plus plus
c
x
α
+
y
z
α
α
when
c’
t
α on c
when
4
t’
+
o
r
α on c′
let node plus_plus (x,y) = o where
5
rec z = x + y
6
and t = z when c
7
and t’= buffer(t)
8
and r = y when c’
9
and o = t’ + r
val plus_plus : (int * int) -> int
val plus_plus :: forall ’a. (’a * ’a) -> ’a on c’
Buffer line 7, characters 11-21: size = 1
20
Typing
plus plus
c
x
α
+
y
z
α
α
when
c’
t
α on c
when
4
t’
α on c′
+
r
α on c′
o
α on c′
let node plus_plus (x,y) = o where
5
rec z = x + y
6
and t = z when c
7
and t’= buffer(t)
8
and r = y when c’
9
and o = t’ + r
val plus_plus : (int * int) -> int
val plus_plus :: forall ’a. (’a * ’a) -> ’a on c’
Buffer line 7, characters 11-21: size = 1
20
Typing
plus plus
c
x
α
+
y
z
α
when
c’
α
when
for all α
t
α on c
t’
α on c′
+
r
α on c′
o
α on c′
such that {α on c <: α on c′ } , (α × α) → α on c′
Subtyping constraint solving
• simple case:
α on w1 <: α on w2
size(α on w1 , α on w2 )
⇔ w1 <: w2
=
size(w1 , w2 )
21
Typing
plus plus
c
x
α
+
y
z
α
when
c’
α
t
α on c
when
for all α
such that {
t’
α on c′
+
r
α on c′
c <:
o
α on c′
c′ } , (α × α) → α on c′
Subtyping constraint solving
• simple case:
α on w1 <: α on w2
size(α on w1 , α on w2 )
⇔ w1 <: w2
=
size(w1 , w2 )
21
Typing
plus plus
c
x
α
+
y
z
α
when
c’
α
when
for all α
t
α on c
t’
α on c′
+
r
α on c′
o
α on c′
such that {(10) <: (01)} , (α × α) → α on c′
Subtyping constraint solving
• simple case:
α on w1 <: α on w2
size(α on w1 , α on w2 )
⇔ w1 <: w2
=
size(w1 , w2 )
21
Typing
plus plus
c
x
α
+
y
z
α
when
c’
α
when
for all α
t
α on c
t’
α on c′
+
r
α on c′
o
α on c′
such that {abs ((10)) <:∼ abs ((01))} , (α × α) → α on c′
Subtyping constraint solving
• simple case:
α on w1 <: α on w2
size(α on w1 , α on w2 )
⇔ w1 <: w2
=
size(w1 , w2 )
21
Typing
plus plus
c
x
α
+
y
z
α
when
c’
α
when
t
α on c
t’
α on c′
+
r
α on c′
o
α on c′
(α × α) → α on c′
for all α
Subtyping constraint solving
• simple case:
α on w1 <: α on w2
size(α on w1 , α on w2 )
⇔ w1 <: w2
=
size(w1 , w2 )
21
Typing
plus plus
c
x
α
+
y
z
α
when
c’
α
when
t
α on c
t’
α on c′
+
r
α on c′
o
α on c′
(α × α) → α on c′
for all α
Subtyping constraint solving
• simple case:
α on w1 <: α on w2
⇔ w1 <: w2
size(α on w1 , α on w2 )
=
size(w1 , w2 )
• more difficult case: α1 on w1 <: α2 on w2
21
Video Application
1920
HD
1080
720
ds
480
1920
SD
HD
720
1080
480
1920
SD
mg
HD
1080
1920
HD
sp
1080
Picture in Picture:
• takes two High Definition images as input
• downscales the first one to obtain a Small Definition image
• embeds the small image in the big one
22
Video Application
picture in picture end
incrust end
downscaler
α6 on hf on not first sd line on vf
α10
incrust end
p2
α10
big
whenot
for all α6 , α10
small
on incrust end
merge
p1
α6
o
α10
α10 on not incrust end
such that
{α6 on hf on not first sd line on vf <: α10 on incrust end } ,
(α6 × α10 ) → α10
23
Video Application
picture in picture end
incrust end
downscaler
α6 on hf on not first sd line on vf
α10
small
on incrust end
incrust end
p2
α10
big
whenot
merge
p1
α6
o
α10
α10 on not incrust end
for all α6 , α10
such that
{α6 on hf on not first sd line on vf <: α10 on incrust end } ,
(α6 × α10 ) → α10
Solution:
α6 ← α
α10 ← α on 04315 (1)
∀α. (α × α on 04315 (1)) → α on 04315 (1)
23
Summary
delay
buffer size
minimal result
1 920 (≈ 1 HD line)
191 970 (≈ 266.6 SD lines)
abstract result
4 315 (≈ 2 HD lines)
192 240 (≈ 267 SD lines)
• proved correct
• incomplete by nature
• good performances
24
Conclusion and Future Work
Conclusion
• n-synchronous model:
more flexible composition of nodes without loss of guaranties
• algorithm with abstraction:
efficient and working on non periodic clocks
• algorithms implemented in Lucy-n
• coq proofs
Future work:
• integer clocks
• buffers with limited size
• code generation
26