Deriving Probability Density Functions
from Probabilistic Functional Programs
Sooraj Bhat, Johannes Borgström,
Andrew D. Gordon, Claudio Russo
EAPLS award winner
compiler of
Bhat et
(2012) and
provide
firstingredien
proof of
damental
concept
in al.
probability
theory
and the
a key
such
as
modelling
the
global
carbon
cycle.
compiler
greatly
reduces
the
development
of domain
used machine learning methods. However, effort
the necessary
fr
demonstrate
by solving
inference
problems
from various
sci
piling
probabilistic
functional
programs
to density
function
such developed.
as modelling
cycle.the first implement
been
Inthe
thisglobal
work,carbon
we present
with discrete
type
of Bhat etreturn
al. (2012)
and provide the first proof of
flip(0.8)compiler
compiler greatly reduces the development effort of domain
by solving inference problems from various sci
random(
Bernoulli(0.8))
flip(0.8)demonstrate
such as modelling the global carbon cycle.
Probabilistic Programs
if
flip(0.8)Bernoulli(0.8))
random(
then flip(0.9)
flip
(0.8)
else
flip
(0.4)
if flip
(0.8)
then flip(0.9)
random(
Bernoulli
(0.8))
Normal(0.0))
else flip(0.4)
t
f
ifrandom(
flip
(0.8)
The
probability
mass of a value V
(0.7)Normal(0.0))
then
flip
(0.9)
random(
Normal
is the proportion
of (0.0))
successful program runs
else
flip
(0.4) Normal(4.0))
random(
if
flip
(0.7)
that return V
then random(Normal(0.0))
random(
(0.0))(4.0))
if
fliprandom(
(p) Normal
else
Normal
2
if flip(0.8)
ndom(Normal(0.0)) then flip(0.9)
else flip(0.4)
)))ip(0.7)
random(Normal(0.0))
n random(Normal(0.0))
eal(0.0))
random(Normal(4.0))
l(4.0))
if
flip(0.7)
(0.0))
Z
Z
then
random(
Normal(0.0))
f
4.0))
f
A
else random(Normal(4.0))
Density Functions 1
n
Introduction
Z
A
f
if flip(p)
A
mming aims to arm data scientists
with random(
a declarative language
then
Normalfor
(m))
babilistic models, while leaving the details of how to translate those
random(
(n))
ampling or inference algorithms else
to the compiler.
ManyNormal
widely used
chniques that might be employed by such a compiler use as input
Z
ty function (PDF) of the model. Such techniques include maximum
um a posteriori estimation, L2 estimation, importance sampling,
A
Monte Carlo (MCMC) methods.
Andy: Church useslanguage
MCMC
mming
aims
to
arm
data
scientists
with
a
declarative
lity, density functions have been largely absent from
the literature
3
Integrating the density function over an interval A
yields the
proportion aims
of successful
program
runs with a
obabilistic
programming
to arm data
scientists
that
return
a
value
in
A
cifying their probabilistic models, while leaving the details
dels to efficient sampling or inference algorithms to the com
then flip(0.9)
0.0 else
or flip
4.0(0.4)
depending
th mean either
on the result of flip
.8)
ecord
w with fields mA and mB to hold each mean. Repe
p(0.9)
random(Normal(0.0))
e(0.4)
discrete case yields the function g(x) =R P({x}), which
e look for a functionif flip
f (0.7)
such that P(A) = A f (x) dx, know
then random(Normal(0.0))
m(
Normal
(0.0))
nction (PDF) of the else
distribution.
In
other
words,
f
is
a
fun
random(Normal(4.0))
curve
on
an
interval
gives
the
probability
of
generating
.7)
if flip(program
p)
erval.
The
PDF
of
this
is pictured in Figure 1 and
ndom(Normal(0.0))
0.30
Density Functions 2
0.20
ndom(Normal(4.0))
then random(Normal(m))
else random(Normal(n))
0.10
f (x) = 0.7 · pdf Gaussian(0.0, 1.0, x) + 0.3 · pdf Ga
0.00
f (x) = 0.7 · f (x) + 0.3 · f (
)
here
pdf
Gaussian
is the
PDF
of
the
Gaussian
distributio
ndom(
Normal
(−2m))
0
2
4
6
ndom(
Normal(n))
1 Introduction
om
statistics.
The function
takes higher values where the g
Fig. 1. Density function for a mixture of Gaussians.
ove is more likely
generate
an
outcome.
f (x)to
=
0.7
·
f
(x)
+
0.3
·
f
(x
4)
Probabilistic programming aims to arm data scientist
f (x) , P({x}) and enjoys
the propertytheir
P(A) =probabilistic
A of Wwhile
. Un- leaving th
Âx2A f (x) for all subsets
specifying
models,
fortunately, this construction does not work in the continuous case. Consider a simple
4
models
efficient
sampling
inference
algorithms to
mixture of Gaussians, here
written to
in Fun
(Borgström
et al. 2011), or
a probabilistic
func-
ensities functions and MCMC. In the example above, the
random(Normal(0.0))
Densities
by
compilation
if (0.7)
flip(0.8)
flip
random(
Bernoulli
(0.8))
then
random(
Normal
(0.0))
•if flip(0.8)
Sought:
Function F from t to double
then
flip(0.9)
•
if flip(p)
Given: Program
M with result type t
else random(
Normal(4.0))
gives the density of the result of M
elsethat
flip(0.4)
then random(Normal(m))
else
random(
Normal
(n))
random(
Normal
(0.0))
to programs with free variables:
•if flipGeneralisation
(0.7)
f (x) = 0.7 · f (x) + 0.3 · f (x 4.0)
Such
parameters
are
treated
as
constants
•
then random(Normal(0.0))
The density
F(4.0))
depends on a parameter valuation
else
Normal
• random(
if flip(p)
then random(Normal(m))
else random(Normal(n))
f (x) = p · f (x
+ (1
m)
p) · f (x
f 5(x) = 0.7 · f (x) + 0.3 · f (x
n)
4)
Outline
•
• Density compiler
Experimental
results
•
Motivation
6
Motivation
•
•
•
Bayesian ML is based on probabilistic models
Conveniently written in a programming language
•
Density functions are widely used in ML
Here: Markov chain Monte Carlo (MCMC)
•
Current practice: code up both model and density
•
•
•
Do the model and the density agree?
What if you want to change one or the other?
(Non)Existence of an efficient density function
limits the class of models used in practice.
(Mostly because of “hidden variables”)
7
M, N ::=
x | clanguage,
| inlu M | inr
| (M, aN)standard
For our target
wet M
choose
fststock
M | snd
M
mented with
integration.
f (M)
Base: language
of finite
Expressions
of
the
letcomputations
x=
M Target
in N Language: E, F
(no recursion or while-loops)
match
M
with
inl
x
!
N
|
inr
x
1
1
T,U ::= int | double | unit | T ! U | T +U2 |
Source (Fun): base + random(Dist(M))
E, F ::= failt
Source and Target
•
•
•
x | c | inlU E | inrT E | (E, F)
Based on “Stochastic
fstlambda-calculus”,
E | snd E | f (E)
ToPOPL’02.
ensure that a program has at most one
by Ramsey & Pfeffer,
let
x
=
E
in
F
inr are annotated with a type (see (F UN I
match
E
with
inl
x
!
F
|
inr
x
!
F
1
1
2
with the type it is used at. We omit these2
Target: base + lterm
(x1 , (possibly
..., xn ). E with binders), we write x ,
1
E
F
We
R let op(M) range over f (M), fst M, snd
E We write observe M for if M then () els
?
T sum type, we write if M then N1 else N
has
We8write G ` M : t to mean that in typ
9
random(Normal(0.0))
TrueSkill
if flip(0.7)
then random(Normal(0.0))
Player
ranking
model,
used
in
XBox
Live.
else random(Normal(4.0))
Computes a skill distribution for each player.
if flip(p)
then random(Normal(m))
else random(Normal(n))
assume
f (x) = 0.7 · f (x) + 0.3 · f (x
To compute the marginal
probability
density
of
Alice,
f (x) = p · f (x m)
Bob
Alice
we need to integrate
+ (1 p) · f (x
over all values of Bob.
10
4.0)
n)
observe x := if x then () else fail
Density compiler
Environment, compilation rules and correctness
11
deterministic inner lets.
Compilation
Probability Context:
•
Given a list ° of
random variables
::=
•
Returns the density function F
e
and an expression E for
° the
, x random variables
the joint density of
and of being in the°current
, x = Ebranch in the program
(and deterministic variables and their definitions)
A probabilistic context ° is often us
° ; E ` dens(M) ) F
in ° ; E expression F gives the PDF of M
is
openFterm
expresse
° ; E ` marg(x1 , . . . , xwhich
in ° an
; E expression
gives the that
PDF of (x
1 , . . . , xk )
k) ) F
ables in the context and the constraint
Inductively Defined Judgments of the Compiler:
12
hold° iff
notlambda-bound
contain
of random
ensity
so“M
far.det”
Weto introduce
fresh
judgment,
but some or
so
(M) )function
F We define
; EM
` does
dens(M)
) F any occurrence
arrow.)
M det holds,
then
M
isE,
also
an expression in the target language
syntax, and we s
ow
we
assume
that
z,
w
]
°
,
M.
) either F (l . 0)
° ; E ` dens(fromL(M)) ) l z.(F (inl z))
Example rules, 1
restored the
treat it as such (in rules (L ET D ET) and (M ATCH D ET), for(JB:
example).
If symm
M d
rules at least for the te
] (Ms
(T)UPLE
L) M is constant under ° .
es: ° ; E `rands(°
dens(M)
)P°ROJ
F), then
(JB: broke
up the rule
The marg judgment
yields the)
joint
variables
in
)F
° ; E ` dens(M)
F marginal PDF of the random
groups to make them e
(VAR Rthe
NDPDF
) , we first
R substitute in the deterministic
gument. To compute
let-bound
var
grok and explain)
)
F
°
;
E
`
dens(fst
M)
)
l
z.
l
w.
F
(z,
w)
0
E ) ) F and then integrate
x 2 rands(°
) ° ; E random
` marg(x)
) F Except for rule (D ISCRE
out the remaining
variables.
low, marg(x1 , ..., xk ) is used with k 2 {0, 1, 2}; the case k = 0 is used to comp
F of (S UM C ON L), (T UPLE P°ROJ
;E `
dens(x)
)
F
L) andbranch
(FROM
omitted
probability of being in the current
of L)
the are
program.
L) states that the density
of)inl(M) is the density of M in the left
(FAIL
Marginal
Density:
°;E `
marg(x
xk )UPLE
) F VAR) com1 , ...,(T
drg(e)
0 in the
right.
Its
dual
is
(
FROM
L).
The
rule
)F
) variables. (This syntactic restriction can
nal density(M
of ARGINAL
two random
c]
·
(F
())
°
;
E
`
dens(fail)
)
l
z.
0.0
{x
,
...,
x
}
[
{y
,
...,
y
}
=
rands(°
)
x1 ,the
..., xtuple
yn distinct
n
1
1
k
k , y1 , ...,
ing dependency information
for the expressions in
(Bhat
R
E P ROJ L) marginalizes
out1the
dimension
° ; E ` marg(x
, ...,left
xk ) )
l (x1 , ..., xof
l (y1 , ..., yn ). Es°
k ).a pair.
et and match: ° ; E ` dens(let x = M in N) ) F
9 judgment gives the density F of M in the current context ° , where E
The dens
(L ET
R ND
accumulated
body
of )the density function so far. We introduce fresh lambda
det) F; below we assume
e; 1 ` dens(M)
1 M.
variables in¬(M
the result
that z, w ])
° ,FE,
ns(N) ) F
° , x; E · (F1 x) ` dens(N) ) F2
Density Compiler, base cases: ° ; E ` dens(M) ) F
M in N) ) F
(VAR D ET)
(x = E 0 ) 2 °
° ; E ` dens(let x = M in N) ) F2
13
° ; E ` dens(E 0 ) )
F
(VAR R ND)
x 2 rands(° ) ° ; E ` marg(x) )
Example rules, 2
L)
(FROM L)
In (R ANDOM C ONST), a random variable drawn from a primitive distrib
E ` dens(M) ) Fconstant argument has the °expected
; E ` dens(M)
)F
PDF (multiplied with the probability t
(inl M) ) either the
F (l
. 0) branch).
° ; E (R
` ANDOM
dens(fromL
(M))
)l
z.(Fto(inl
z)) with a rando
current
R ND
) treats
calls
random
R)
g(x, y) ) F
((x, y)) ) F
by marginalizing over the argument to the distribution.
(T UPLE P ROJ L)
In if statements, the branching expression is of type bool = unit + un
° ; E ` dens(M) ) F
make a straightforward case distinction.
R
° ; E ` dens(fst M) ) l z.
l w. F (z, w)
Derived rule for if statements
projection
a ),
pair
(M ATCH(IDFET
)ET
is)based
on (L ETfrom
D ET
and we multiply the constraint that we are i
D
versions of
(S UM
C ON( isL
L),Ms
(T UPLE
PMs
ROJ ) L)
and
(joint
FROM
L) areexpression.
omitted We also em
correct
branch
or
isR
with
the
density
°
M° det
C ON L)deterministic
states that the
density
of inl(M)
is the
density
of M calls
in theto left
functions
fromL
and
fromR
to
avoid
recursive
(M ATCH D ET) w
° ; E · [Ms° = true] ` dens(N1 ) ) F1
um, and computing
0 in the right.
Its dualofisthe
(FROM
L). The rule
(T UPLE
V(M
ARATCH
) comthe density
match-bound
variable.
The
R ND) rule is b
° ; E · [Ms° = false] ` dens(N2 ) ) F2
nt marginal
of ),
two
variables.
can
on density
(L ET R ND
andrandom
we again
multiply(This
in thesyntactic
constraintrestriction
that we are
in the left (or r
° ; E ` dens(if
M thenfor
N1the
elseexpressions
N2 ) ) l z. (F
z) +tuple
(F2 z)(Bhat
considering
dependency
information
in1the
branch
of the match.
) (T UPLE P ROJ L) marginalizes
out
the left dimension of a pair.
prob. of being in the current
arguments
M
are
a
deterministic
numericrandom
operations
on real: °
numbers
we mimic
the change
of variabl
Density For
Compiler,
variables
;
E
`
dens(M)
)
F
branch
function
ofdens(let
the parameters
mpiler, let and match:
°
;
E
`
x = Masin
N)=)dx
F
tegration (often summarized
“dx
dy dy”), multiplying the density of t
(R ANDOM
C
ONST )
with the derivative
of
the
inverse operation. This is exemplified by the foll
(L
ET
R
ND
)
M det rands(° ) ] (Ms° ) ° ; E ` marg(e) ) F
M det
¬(M det)
e; 1 on
` dens(M)
)
F
Density
compiler,
numeric
operations
reals
:
°
;
E
`
dens(
f
(M))
)
F
1
° ; E ` dens(random(Dist(M))) ) l z. (pdfDist(Ms° ) z) · (F ())
; E ` dens(N) ) F
° , x; E · (F1 x)
` dens(N)
(N EG)
(I NVERSE
) ) F2
(R ANDOM R ND)
(let x = M in N) ) F ° ; E ` dens(M)
° ; E `)dens(let
x = M in N) )° F
F
; E2 ` dens(M) ) F
Standard
its probability
function) F
¬(M
det ^distribution
rands(° ) and
] (Ms
` dens(M)
° )) ° ; Edensity
°
R
dens
judgments.
Lemma
1
(Derived
Judgments).
° ; E ` relate
dens( the
f (M))
) of
l z.the various
l y. [z =terms
f (Msoccurring
)]
·
(F
y)
These derived judgments
types
in
the
marg and
°
G ,G° ` ° wf and dom(G° ) = rands(° ) [ dom(s° ) and G ,GLemma
then
densIfjudgments.
° ` E : 1double
(Derived Ju
Correctness
If G ,G°in`the
° wf
andand
do
These
derived
judgments
relate
the
types
of
the
various
terms
occurring
marg
Lemma
1
(Derived
Judgments).
(1) If ° ; E ` marg(x1 , . . . , xn ) ) F and G° ` (x1 , . . . , xn ) : (t1 ⇤ · · · ⇤ tn )
dens
judgments.
If G ,G
° wf
) [ dom(s° ) andTypes
G ,G°of`variables
E : (1)
double
then
G and
` F dom(G
: (t1 ⇤°· ·)·=
⇤ trands(°
° `
in ° ;then
n ) ! double.
If
E ` marg(x1 ,
(2)
If ° ;1E (Derived
` dens(M)
) F and G ,G° ` M : t then G ` F : t ! double.
then G ` F : (t1 ⇤ ·
Lemma
Judgments).
(1) If ° ; E ` marg(x1 , . . . , xn ) ) F and G° ` (x1 , . . . , xn ) : (t1 ⇤ · · · ⇤ tn )
(2)
Ifdouble
° ; E ` then
dens(M)
If
G
,G
`
°
wf
and
dom(G
)
=
rands(°
)
[
dom(s
)
and
G
,G
`
E
:
°
°
°
°
state that
then G ` F : (t ⇤ · · · ⇤ t ) ! double.
tions are
s, ie, its a(2)
that
darecode
its a
n
1
The soundness
theorem
asserts that, for all closed expressions M, the density f
Andy:
weGshould
also
state
that
If
°
;
E
`
dens(M)
)
F
and
G
,G
`
M
:
t
then
`
F
:
t
!
double.
°
soundness theo
tion
given
the density
indeed
(via
the
(1) If
° ; E by
` marg(x
G° dens
` characterizes
(x
. . . ,relations
xn ) : (tare
· ⇤ tnThe
) integration)
the
and
n ) ) F and
1 , . . . , xcompiler
1 ,marg
1 ⇤ · · stock
in fact partial functions, ie, its a
tion given by the densi
then
G
`
F
:
(t
⇤
·
·
·
⇤
t
)
!
double.
tribution
of
M
given
by
the
monadic
semantics:
n
1
The soundness theorem asserts that, for all
closed compiler.
expressions M, the density funcdeterministic
tribution of M given by
(2)
If
°
;
E
`
dens(M)
)
F
and
G
,G
`
M
:
t
then
G
`
F
:
t
!
double.
°
Andy: can the compiled
code integration) the distion given by the density compiler indeed characterizes
(via stock
fail at run-time?
Theorem
1 (Soundness).
If e;semantics:
1 ` dens(M)
) F and e ` M : t then
o state that
tribution
of
M
given
by
the
monadic
Theorem
1
(Soundne
The
soundness
theorem
asserts
that,
for
all
closed
expressions
M,
the
density
funcde
ations are
Z
ns, ie, its a
tion given
by the density
compiler
indeed
characterizes
(via
stock integration) the disTheorem
1
(Soundness).
If
e;
1
`
dens(M)
)
F
and
e
`
M
:
t
then
(P[[M]] e) A = F
r.
tribution of M given by the monadic semantics:
A
led code
Z
M : t then
(P[[M]]
e) A = )semantics
F
Theorem
(Soundness).
e;The
1the
` probabilistic
dens(M)
Fofand
e of
`M
Proof:
By1 joint
inductionIf on
derivations
dens(M)
andProof:
M : t,By
using
the
fol
joint
induc
A
(Ramsey & Pfeffer ’02, Gordon et al. ’13)
ing induction hypothesis: if G ,G° ` ° wf and Z° ; E ` dens(M)
F and Ghypothes
,G° `
ing)
induction
Proof:
joint
induction
on and
the derivations
ofdom(G
dens(M)
M : t, )and
using
the
followA
= ° )F=and
G ,G
: doubl
and By
G ,G
E : double
G (P[[M]]
` r ande)
rands(°
[ dom(s
) and
|µ|
° `
° `R
° RE
A
Inductively
Defined
Judgments
of
the
Compiler:
ing induction
hypothesis:
if
G
,G
`
°
wf
and
°
;
E
`
dens(M)
and
Gand
,G° s
:t 0 !
°
and
=`° M
and µ(B) = B l (rands(° )).Er, and (8x 2 dom(s° )8r 0)
. G°F `
r 0µ(B)
(x)rr
B l (rands
0 · !⇤
and implies
G
,G
`
E
:
double
and
G
`
r
and
dom(G
)
=
rands(°
)
[
dom(s
)
and
|µ|

1
0
⇤
implies
that
Err
°
°
°
°
;
E
`
dens(M)
)
F
in
°
;
E
expression
F
gives
the
PDF
of
M
that
Errinduction
· ! 0.0)onthen
R joint
Proof: By
the derivations of dens(M)
and
M : t, using 0 the⇤follow0
0
and µ(B)
)).Er,
and
(8x
2; Edom(s
)8r
.`Ggives
sF°of(x)rr
!
?M : t
°°
° ` rtheand
° ; Einduction
`=marg(x
FG
in
°
expression
F
PDF
(x
,
.
.
.
,
x
)
B l (rands(°
1 , . . . , xk ) ) if
1
k
ing
hypothesis:
,G
`
°
wf
and
;
E
dens(M)
)
and
G
,G
`
Z
°
°
0
⇤
implies
·!
0.0) then
(µ>>
andthat
G ,GErr
`
E
:
double
and
G
`
r
and
dom(G
)
=
rands(°
)
[
dom(s
)
and
|µ|
1
°
°
°
(µ
>
>=
(l
(rands(°
)).
(P[[M]]
(s
r))))
A
=
Fr
R
°
and µ(B) = B l (rands(° )).Er, and 15
(8x 2 dom(s° )8r 0 . G°Z ` r 0 andA s° (x)rr 0 !⇤ ?
if flip(0.7)
eoremThe
asserts
soundness
that, for
theorem
all
closed
asserts
expressions
that,
for
M,
all
the
closed
density
expre
f
then random(Normal(0.0))
nsity
tioncompiler
given byindeed
the density
characterizes
compiler
indeed
stockcharacterizes
integration) (via
the
else
random(
Normal(via
(4.0))
Example
bytribution
the monadic
of Msemantics:
given by the monadic semantics:
let b = random(Bernoulli(p)) in
if b
(modulo
ness).
Theorem
If e; 1 1` dens(M)
(Soundness).
F If
and
e;
1e `(dens(M)
M)) : t then) beta-eq)
F and
then)
random(
Normal
m
else random(
Z Normal(n))
Z
e `M
f· (x)
=e)
0.7
·+
f=
(x; 0.0)F+ 0.3 · f (
A
=
F
(P[[M]]
A
l (P[[M]]
x.Sb2{t,f} Pe)
(b)
·
[b
=
t]
f
(x
m
)
Bernoulli(p)
A
l x.Sb2{t,f} PBernoulli(pA
(b)
·
[b
=
t]
·
f
(x
m
)
+
Sb2{t,f} PBernoulli(p)) (b) · [b = f] · f (x n)
Sb2{t,f}induction
PBernoulli
· [b =
f] · f (x
nt,
) =
pon
) (b)the
pdens(M)
· f (x;the
m) foll
the
By derivations
joint
of (dens(M)
derivations
and
M f:(x)
of
using
a
uction
Proof:
on
· dens(
f°(x;`n)M
esis:
ing ifinduction
G ,G° ` °hypothesis:
wf and ° ;ifEG`,G
dens(M)
)and
F +and
°(1; EGp`),G
° ` ° wf
bleand
andG G
,G°` `r REand
:ldouble
dom(G
)(p=
Grands(°
r and
dom(s
) )=and
rands(°
|µ|
x.P
·`f (x
m)) [
+dom(G
°and
°°
Bernoulli
) (t)
0
0
0
0
l
x.P
(t)
·
f
(x
m
)
+
observe
x
:=
if
x
then
()
else
ds(°
and)).Er,
µ(B) =
andB(8x
l (rands(°
2
dom(s
)).Er,
)8r
and
.
G
(8x
`
r
2
and
dom(s
s
(x)rr
)8r
.
G
!
Bernoulli
(
p
)
°
°
°
°
°
PBernoulli(p) (f) · f (x n)
⇤
0
⇤
! implies
0.0) then
that Err ·P!
0.0)(pthen
Bernoulli
) (f) · f (x n)
ntroduction
16
Z
Implementation
•
Direct implementation of the compilation rules
•
•
•
•
In F#, operating on a subset of (quoted) F#
Operates on log-probabilities
Uses let-expansion in the Marginal rule
Parametric in integration function
(currently a simple Riemann sum)
17
Evaluation
•
Synthetic models, and ecological systems models
from Computational Sciences, MSR Cambridge
Orig. model+density
Example
mixture of Gaussians
linear regression
species distribution
net primary productivity
global carbon cycle
orig
F#
F#
C#
C#
C#
Fun model
LOC ,
orig
32
27
173
82
1532
LOC ,
Fun
20
18
37
39
402
0.63x
0.67x
0.21x
0.48x
0.26x
Get a 2-3x
slowdown
time (s), orig time (s), Fun
1.77
4.78
0.63
2.08
79
189
11
23
n/a
764
2.7x
3.3x
2.4x
2.1x
n/a
Table 1. Lines-of-code and running time comparisons of synthetic and scientific models.
Write a quarter as much code
with Filzbach itself. We also compare the running
times of the original implementations
18
versus the Fun versions for MCMC-based inference using Filzbach, not including data
NPP Model
19
Related Work
•
•
•
•
•
•
Naive prototype (interpreter) reported at POPL’13.
Builds on work by Bhat et al., POPL’12.
•
We are less complete (admit fewer joint densities)
We have a soundness proof
We have a simpler algorithm (and fewer judgments)
We implement our algorithm, and study real models
We use a more expressive language:
integer operations, fail, general if and match,
deterministic let
20
Conclusion
•
We compile probabilistic programs
to their density functions
•
•
We validate the approach
by compiling existing ecology models
•
•
The algorithm is sound.
The implementation is reasonably efficient
Future work:
optimisation, improve completeness, clean up match
more complex real-life models or variations
different ways of treating hidden variables
•
•
•
21