Declarative concurrency
March 3, 2014
Overview
Overview
Declarative
programming
■
◆ what is declarativeness
Immutable data
structures
DP basics
■
Declarative
concurrency
Declarative
programming
elsewhere
Declarative programming (DP)
Immutable data structures
◆ lists, trees
■
DP basics
◆ iterative comutation
◆ recursive computation
Declarative concurrency (DC)
■ DP and DC in Haskell and other languages
■
2 / 32
Overview
Declarative
programming
Some quotes
What is
declarativeness?
Declarativeness
is important
Classification of
declarative
programming
In Clojure (and
other FP)
Declarative programming
Immutable data
structures
DP basics
Declarative
concurrency
Declarative
programming
elsewhere
3 / 32
Some quotes
Overview
Declarative
programming
Some quotes
What is
declarativeness?
Declarativeness
is important
Classification of
declarative
programming
In Clojure (and
other FP)
Immutable data
structures
DP basics
Declarative
concurrency
Declarative
programming
elsewhere
■
Recent “Effective Scala” from Twitter developers
http://twitter.github.com/effectivescala/
◆ Use Futures to manage concurrency.
◆ Futures allow the programmer to express concurrent
computation in a declarative style
◆ If an immutable collection will do, use it ...
reasoning about them in a concurrent context is
simple.
◆ The Java memory model is a subtle beast, but luckily
we can avoid all of these pitfalls by using the
declarative style
What is declarative style, future and immutable collection
and how are they special?
4 / 32
What is declarativeness?
Overview
Declarative
programming
Some quotes
What is
declarativeness?
Declarativeness
is important
Classification of
declarative
programming
In Clojure (and
other FP)
Immutable data
structures
DP basics
Declarative
concurrency
An operation with input and output is declarative
■
Same input always gives the same output
◆ independent (of outside state)
◆ stateless (no inside state)
◆ deterministic (“fixed” control-flow)
Examples
typical algebraic operations (+,-,*,/)
■ if-then-else
■ pure functions
■
Declarative
programming
elsewhere
5 / 32
Declarativeness is important
Overview
Declarative
programming
Some quotes
What is
declarativeness?
Declarativeness
is important
Classification of
declarative
programming
In Clojure (and
other FP)
■
Compositional
◆ plug declarative component
arguments
Declarative
operation
Immutable data
structures
Rest of
computation
results
DP basics
Declarative
concurrency
Declarative
programming
elsewhere
■
Reasoning is simple
◆ understand component behaviour alone
◆ no need to consider external/internal state
6 / 32
Classification of declarative programming
Overview
Declarative
programming
Some quotes
What is
declarativeness?
Declarativeness
is important
Classification of
declarative
programming
In Clojure (and
other FP)
Immutable data
structures
descriptive (XML, HTML)
■ programmable (Turing complete)
■
◆ observational (just declarative interface)
◆ definitional (declarative “internals”)
■
■
■
functional programming
logic programming
declarative model (DP in Oz)
DP basics
Declarative
concurrency
(not every functional language is declarative, e.g. Scheme)
Declarative
programming
elsewhere
7 / 32
In Clojure (and other FP)
Overview
Declarative
programming
Some quotes
What is
declarativeness?
Declarativeness
is important
Classification of
declarative
programming
In Clojure (and
other FP)
Immutable data
structures
DP basics
Declarative
concurrency
■
variables are immutable
◆ function arguments, loop variables, local (let)
variables
◆ except def, defn creates global mutable variables
■
standard data structures are immutable (persistent)
◆ list, vector, map, set
■
pure functions
◆ no side effects: i.e. no IO/shared state read or write
Declarative
programming
elsewhere
8 / 32
Overview
Declarative
programming
Immutable data
structures
Immutable
linked lists
Appending lists
Trees
Immutable data
with DV
Immutable data structures
DP basics
Declarative
concurrency
Declarative
programming
elsewhere
9 / 32
Immutable linked lists
Overview
Declarative
programming
or with quoting ’(3 4 7)
■ actual representation
■ (list 3 4 7)
Immutable data
structures
Immutable
linked lists
Appending lists
Trees
Immutable data
with DV
(cons 3 (cons 4 (cons 7 ())))
◆ empty list () ends the list
◆ “cons” pair (cons H T) consists of
■
DP basics
■
Declarative
concurrency
Declarative
programming
elsewhere
head H references one list value
tail T references the rest of the list
first/rest – accessing head and tail
■ destructuring: extract head and tail
■
(let [[H & T] ’(3 4 7)] (println T) )
◆ prints (4 7)
■
changing an element in the list is impossible
◆ adding head (cons 11 lst), also (conj lst 11)
10 / 32
Appending lists
Overview
■
Declarative
programming
Immutable data
structures
Immutable
linked lists
Appending lists
Trees
Immutable data
with DV
DP basics
Declarative
concurrency
Declarative
programming
elsewhere
Appending a=(1 2 3) b=(6 7) must result in copying
the first list (cons 1 (cons 2 (cons 3 b)))
1
2
3
1
2
3
6
7
Append one list to another by just reassigning the tail?
No!
■
the first list must stay immutable
◆ whoever has reference to it should not see changes
■
the second list may be appended to any other
◆ whoever has reference to it sees no changes
11 / 32
Trees
Overview
Declarative
programming
Immutable data
structures
Immutable
linked lists
Appending lists
Trees
Immutable data
with DV
■ (defrecord tree [value left right])
– Java class
with 3 immutable fields
■ Having (tree. 3 :leaf (tree. 33 :leaf :leaf))
◆ want to replace 3 with 45
tree 3
DP basics
Declarative
concurrency
Declarative
programming
elsewhere
leaf
tree 33
leaf
leaf
12 / 32
Trees
Overview
Declarative
programming
Immutable data
structures
Immutable
linked lists
Appending lists
Trees
Immutable data
with DV
■ (defrecord tree [value left right])
– Java class
with 3 immutable fields
■ Having (tree. 3 :leaf (tree. 33 :leaf :leaf))
◆ want to replace 3 with 45
tree 3
tree 45
DP basics
Declarative
concurrency
leaf
Declarative
programming
elsewhere
tree 33
leaf
leaf
need to create new node and point to the subtrees
■ in general, path from root to node must be reconstructed
■
■
whoever references (part of) the tree never sees any
changes
12 / 32
Immutable data with DV
Overview
Declarative
programming
Immutable data
structures
Immutable
linked lists
Appending lists
Trees
Immutable data
with DV
DP basics
Declarative
concurrency
Declarative
programming
elsewhere
■
dataflow variables may be unbound
◆ reader is suspended until bound
■
dataflow variables are single-assignment
◆ reader will never see different values
Considering all above
■
we can treat DVs as immutable
◆ unless checking with IsDet operation, which is not
part of declarative model
13 / 32
Overview
Declarative
programming
Immutable data
structures
DP basics
Iterative
computation (1)
Iterative
computation (2)
Clojure ’recur’
Recursive
computation (1)
Recursive
computation (2)
Higher-order
programming (1)
Higher-order
programming (2)
DP basics
Declarative
concurrency
Declarative
programming
elsewhere
14 / 32
Iterative computation (1)
Overview
Declarative
programming
Immutable data
structures
DP basics
Iterative
computation (1)
Iterative
computation (2)
Clojure ’recur’
Recursive
computation (1)
Recursive
computation (2)
Higher-order
programming (1)
Higher-order
programming (2)
How to iterate over values with single-assignment variable?
with recursion
■ each (recursive) call creates new variables for the
arguments
■
(defn iteration [X Sum]
(if (>= X 10)
Sum ; stop and return result
(do ; else
(println X)
(iteration (inc X) (+ Sum X)))))
(println (iteration 0 0))
Declarative
concurrency
Declarative
programming
elsewhere
15 / 32
Iterative computation (2)
Overview
Declarative
programming
Immutable data
structures
DP basics
Iterative
computation (1)
Iterative
computation (2)
Clojure ’recur’
Recursive
computation (1)
Recursive
computation (2)
Higher-order
programming (1)
Higher-order
programming (2)
Declarative
concurrency
Declarative
programming
elsewhere
■
On the third iteration
"Result"
"X"
x1 0
s1 0
"Sum"
x2 1
s2 0
x3 2
s3 1
current call
new values are created on the stack in each call
■ recursion stops at x10
■ old xi and si can safely be used by external
code/threads (unlike in imperative PLs)
■
◆ Java ’final’ keyword for closures
16 / 32
Clojure ’recur’
Overview
Declarative
programming
Immutable data
structures
DP basics
Iterative
computation (1)
Iterative
computation (2)
Clojure ’recur’
Recursive
computation (1)
Recursive
computation (2)
Higher-order
programming (1)
Higher-order
programming (2)
Declarative
concurrency
Declarative
programming
elsewhere
■
avoid stack grow with ’recur’
◆ must be in tail position
◆ calls current function with tail call optimization
(TCO)
◆ JVM does not allow TCO, Clojure has to generate
iterative code instead of recursive calls
(defn iteration [i s]
(if (>= i 10)
s
(do ; else
(println i)
(recur (inc i) (+ s i))
)))
(println (iteration 0 0))
17 / 32
Recursive computation (1)
Overview
Declarative
programming
■
◆ stack grows because of append is not in tail position
Immutable data
structures
DP basics
Iterative
computation (1)
Iterative
computation (2)
Clojure ’recur’
Recursive
computation (1)
Recursive
computation (2)
Higher-order
programming (1)
Higher-order
programming (2)
Declarative
concurrency
Declarative
programming
elsewhere
Naive implementations are often wasteful
(defn append [Ls Ms]
(if (= Ls ())
Ms
(cons (first Ls) (append (rest Ls) Ms))))
(println (append ’(1 5 3) ’(3 2 3)))
■
Naive definitions are often slow
(defn reverse [Xs]
(if (= Xs ())
’()
(append (reverse (rest Xs))
(list (first Xs)) )))
(println (reverse ’(1 2 3 4)))
18 / 32
Recursive computation (2)
Overview
Declarative
programming
Immutable data
structures
DP basics
Iterative
computation (1)
Iterative
computation (2)
Clojure ’recur’
Recursive
computation (1)
Recursive
computation (2)
Higher-order
programming (1)
Higher-order
programming (2)
Use accumulators and tail recursion
(defn- reverse2_iter [Rs Ys]
(if (= Ys ’())
Rs ; return accumulated result
(recur (cons (first Ys) Rs) (rest Ys))))
(defn reverse2 [Xs]
(reverse2_iter ’() Xs)) ; empty accumulator
(println (reverse2 ’(1 4 3 2)))
Rs is an accumulator for the new list
■ Recursive call is in tail position
■
Declarative
concurrency
Declarative
programming
elsewhere
19 / 32
Higher-order programming (1)
Overview
Declarative
programming
Reverse the list [1,2,7]
7
2
1
Immutable data
structures
DP basics
Iterative
computation (1)
Iterative
computation (2)
Clojure ’recur’
Recursive
computation (1)
Recursive
computation (2)
Higher-order
programming (1)
Higher-order
programming (2)
[]
{P .}
■
{P .}
{P .}
accumulator values from R0=[] to R3=7|2|1|[]
◆ can be also viewed as state transform from S0 to S3
◆ tansformation takes In and X and returns Out
{P In X ?Out}
Declarative
concurrency
Declarative
programming
elsewhere
7|2|1|[]
2|1|[]
1|[]
X
In
{P .}
Out
20 / 32
Higher-order programming (2)
Overview
Declarative
programming
Immutable data
structures
DP basics
Iterative
computation (1)
Iterative
computation (2)
Clojure ’recur’
Recursive
computation (1)
Recursive
computation (2)
Higher-order
programming (1)
Higher-order
programming (2)
Declarative
concurrency
Declarative
programming
elsewhere
Define the generic function
proc {ForAllAcc L P In ?Out}
case L
of nil then Out=In
[] X|L2 then Mid in % Mid connects In and Out
{P In X Mid}
{ForAllAcc L2 P Mid Out}
end
end
% element to head, accumulated to tail
proc {RCons In X ?Out} Out=X|In end
fun {Reverse3 L} {ForAllAcc L RCons nil} end
ForAllAcc is actually procedural FoldL
■ state transform view: X may be incoming message
■
21 / 32
Overview
Declarative
programming
Immutable data
structures
DP basics
Declarative
concurrency
thread block
Dataflow
between threads
Evaluation order
Determinism of
control/data flow
Limitation:
exceptions
Declarative concurrency
Declarative
programming
elsewhere
22 / 32
thread block
Overview
Declarative
programming
Immutable data
structures
DP basics
Declarative
concurrency
thread block
Dataflow
between threads
Evaluation order
Determinism of
control/data flow
Limitation:
exceptions
Declarative
programming
elsewhere
■
thread ... end block around statement (expression) runs
it in a separate thread
declare A B C
A=thread 4+5 end
thread B=5+6 end
C=thread {Abs thread ˜6+5 end} end
■
threads in Oz are green threads
◆ thousands of threads are ok
■
Mozart (Oz VM) is not able to run on multiple cores
23 / 32
Dataflow between threads
Overview
Declarative
programming
Immutable data
structures
DP basics
Declarative
concurrency
thread block
Dataflow
between threads
Evaluation order
Determinism of
control/data flow
Limitation:
exceptions
Declarative
programming
elsewhere
■
variable set in one thread can be read in another
declare A B
thread A=B+1 end
B=5
thread is suspended until B is bound
■ B is bound by the main thread
■
Thats all!
■
model is still declarative
◆ i.e. same results for the same inputs
◆ result completely deterministic
◆ evaluation order becomes non-deterministic
24 / 32
Evaluation order
Overview
Declarative
programming
■
◆ eager – arguments are evaluated before a call
Immutable data
structures
■
DP basics
Declarative
concurrency
thread block
Dataflow
between threads
Evaluation order
Determinism of
control/data flow
Limitation:
exceptions
Declarative
programming
elsewhere
control flow driven
C, Java, Oz
◆ lazy – arguments are evaluated when needed
■
■
■
Haskell
Oz – ByNeed operation
data-driven
◆ operation is executed when data is available
◆ dataflow computers
25 / 32
Determinism of control/data flow
Overview
Declarative
programming
■
◆ works indendent of evaluation order
Immutable data
structures
DP basics
Declarative
concurrency
thread block
Dataflow
between threads
Evaluation order
Determinism of
control/data flow
Limitation:
exceptions
single-assignment doing its magic
■
in an iteration state is “evolving” not changing
◆ data never changes
Declarative
programming
elsewhere
26 / 32
Limitation: exceptions
Overview
Declarative
programming
■
◆ observable non-determinism is introduced
Immutable data
structures
declare A
thread A=5 end
thread A=4 end
DP basics
Declarative
concurrency
thread block
Dataflow
between threads
Evaluation order
Determinism of
control/data flow
Limitation:
exceptions
exceptions may break declarative model
■
A is bound to 4 or 5 depending on thread scheduling
◆ exception is thrown in “unlucky” thread
Declarative
programming
elsewhere
27 / 32
Overview
Declarative
programming
Immutable data
structures
DP basics
Declarative
concurrency
Declarative
programming
elsewhere
Haskell (1)
Haskell (2)
Haskell (3)
Intel ArBB
Declarative programming elsewhere
28 / 32
Haskell (1)
Overview
Declarative
programming
Immutable data
structures
Parallel Haskell has 2 operations
■
◆ returns b
◆ sparks – adds to FIFO of planned evaluations
◆ may be evaluated on other core/processor
DP basics
Declarative
concurrency
Declarative
programming
elsewhere
Haskell (1)
Haskell (2)
Haskell (3)
Intel ArBB
par a b – “sparks” a and evaluates b
■
pseq a b – evaluates a and then b
◆ returns b
◆ the order is important, becuase generally Haskell
compiler may not respect the order
All mentioned evaluations are done up to Weak Head Normal
Form (WHNF)
29 / 32
Haskell (2)
Overview
Declarative
programming
cutoff = 35
Immutable data
structures
fib’
fib’
fib’
fib’
DP basics
Declarative
concurrency
Declarative
programming
elsewhere
Haskell (1)
Haskell (2)
Haskell (3)
Intel ArBB
:: Int -> Integer
0 = 0
1 = 1
n = fib’ (n-1) + fib’ (n-2)
fib :: Int -> Integer
fib n | n < cutoff = fib’ n
| otherwise = r ‘par‘ (l ‘pseq‘ l + r)
where
l = fib (n-1)
r = fib (n-2)
par sparks r
■ pseq makes sure that main thread continues on l
■ both return its second (right hand) argument
■
◆ first (left hand) argument just ensures some
semantics
30 / 32
Haskell (3)
Overview
Declarative
programming
Haskell can run on multiple cores
■ There are many caveats
■
Immutable data
structures
◆ par and pseq only evaluate up to WHNF
◆ it is enough for our fibonacci algorithm
◆ not enough for most other
DP basics
Declarative
concurrency
Declarative
programming
elsewhere
Haskell (1)
Haskell (2)
Haskell (3)
Intel ArBB
■
■
evaluation needs to be forced by help functions
Acheiving faster programs is possible but not easy
◆ need to clearly understand Haskell execution rules
31 / 32
Intel ArBB
Overview
Declarative
programming
Immutable data
structures
DP basics
Declarative
concurrency
ArBB is from another world – data parallel languages
its data structures are immutable
■ it aims for deterministic task parallelism
■
The lesson: immutability and determinism are the friends of
concurrency anywhere.
Declarative
programming
elsewhere
Haskell (1)
Haskell (2)
Haskell (3)
Intel ArBB
32 / 32