A Monadic Model for Computations that Leak Secrets
Tom Schrijvers
January 7, 2015
1
The Probability Monad
1.1
Specification
We specify monads that model probabilistic computations in much the same way as Gibbons and Hinze [].
First we assume a type Prob to represent probabilities. In this paper, we use the rationals in the unit interval
for this purpose.
type Prob = Rational
Next we define a probabilities monads as monads that are equipped with a choose operation for probabilistically choosing between two alternatives.
class Monad m ⇒ MonadProb m where
choose :: Prob → a → a → m a
The idea is that choose p x y returns x with probability p and y with probability 1 − p. Following Gibbons
and Hinze, who are inspired by Hoare, we write choice in infix notation, ‘x J p I y’. Note that Gibbons and
Hinze define MonadProb in terms of a slightly different operation choice, which can be defined in terms of
choose:1
choice :: MonadProb m ⇒ Prob → m a → m a → m a
choice p mx my = join (mx J p I my)
which we denote in infix as ‘mx / p . my’. We prefer choose as the primitive operation because it can be
conveniently lifted in the monad transformer framework, which comes in handy later.
The uniform function from Gibbons and Hinze generates a uniform distribution from a given list.
uniform :: MonadProb m ⇒ [a] → m a
uniform [x]
= return x
uniform l@(x : xs) = return x / 1 ÷ length l . uniform xs
Another useful derived operation is fair binary choice:
(⊕) :: MonadProb m ⇒ a → a → m a
x ⊕ y = uniform [x, y]
Example 1.1. The MonadProb interface can be used to model unknown values of which only the probability
distribution is known. For instance, tuple1 denotes a tuple of that consists of any combination of boolean
values with equal likelihood.
1 and
vice versa, choose p x y = choice p (return x) (return y)
1
tuple1 :: MonadProb m ⇒ m (Bool, Bool)
tuple1 = uniform [(x, y) | x ← [True, False], y ← [True, False]]
Similarly, tuple2 consists with equal likelihood of any tuple of boolean values except for (False, False).
tuple2 :: MonadProb m ⇒ m (Bool, Bool)
tuple2 = uniform [(True, False), (False, True), (True, True)]
Example 1.2. The following pure function flips the boolean values in a tuple.
flip :: (Bool, Bool) → (Bool, Bool)
flip (x, y) = (not x, not y)
We can apply this program to one of the probabilistic tuples above in order to obtain the probability
distribution of the result.
test1, test2 :: MonadProb m ⇒ m (Bool, Bool)
test1 = tuple1 >
>= return ◦ flip
test2 = tuple2 >
>= return ◦ flip
Example 1.3. We can also use MonadProb to model computations that make probabilistic choices themselves. For instance, prog1 takes a tuple of boolean values and returns with equal probability the original
tuple or the flipped tuple.
prog1 :: MonadProb m ⇒ (Bool, Bool) → m (Bool, Bool)
prog1 p = p ⊕ flip p
We can apply the program to either input as follows:
test3, test4 :: MonadProb m ⇒ m (Bool, Bool)
test3 = tuple1 >
>= prog1
test4 = tuple2 >
>= prog1
1.2
Implementation
A Haskell implementation of a probability monad was proposed by Erwig and Kollmansberger [1]. We show
here how this implementation can be derived by starting from an ideal mathematical representation. In the
next section we repeat this derivation process to obtain a more complex monad.
Ideal Representation In theory, discrete probability distributions, represented by probability mass functions, are instances of MonadProb. A probability mass function fX : A → [0, 1] yields the probability of each
value in A. The probabilities add up to 1:
X
fX (x) = 1
x∈A
We restrict ourselves to finite distributions, i.e., those where fX (x) is non-zero for only a finite number of
values x.
Hypothetically, we could define the following operations for probability mass functions.
1 ,x = y
η(x) = λy.
P 0 , x 6= y
µ(p) = λx. mx p(mx) × mx(x)
mx / p . my = λz.p × mx(z) + (1 − p) × my(z)
2
Unfortunately, as we can glean from the above definitions, for two reasons probability mass functions
are unsuitable as monads in Haskell. Firstly, they require computable equality for the values. In contrast,
Haskell monads must be defined for all Haskell values, even those that do not support computable equality
(e.g., function values). Secondly, it not feasible for all Haskell values to finitely enumerate the finite set of
values whose probability is non-zero.
Finitely Enumerable Non-Zero Probability Values In order to be able to finitely enumerate all values
with non-zero probability, we can change our representation to (finite) sets of tuples (p, a) where a is a value
and p its non-zero probability. Hence, representations have the form fX : P((0, 1] × A) where any value
occurs at most once. Let us define the (finite) domain of such a reprensentation as the set of values:
dom(fX ) = {a | ∃p : (p, a) ∈ fX }
Any value that does not occur in the set has probability 0.
p , (p, a) ∈ fX
Pr fX (a) =
0 , a 6∈ dom(fX )
This representation forms a probability monad as follows:
η(x) = {(1,
Px)}
S
µ(mmx) = {( mx∈dom(mmx) Pr mx (a), a) | a ∈ mx∈dom(mmx) dom(mx)}
mx / p . my = {(p × Pr mx (a) + (1 − p) × Pr my (a), a) | a ∈ dom(mx) ∪ dom(my)}
In comparison with the previous representation, all enumerations are now bounded by the finite domain.
Unfortunately, there are still implicit equality checks hidden in the uses of the Pr function as well as in the
use of a set datastructure.
Equality-Free Representation In the last step we relax the representation from a set of tuples to a
list of tuples to relax one use of equality: fX : [[0, 1] × A]. Moreover, we can allow a finite number of
zero-probability values without problem.
In Haskell, this presentation is defined as follows, using a newtype wrapper.
newtype Dist a = D {runD :: [(Rational, a)]}
We also relax the condition that any value may occur at most once. Instead, we define the probability
of a value as the sum of the probabilities of its occurrences.
pr :: Eq a ⇒ Dist a → a → Prob
pr mx a = sum [p | (p, x) ← runD mx, x ≡ a]
While this interpretation of the representation requires computable equality, the MonadProb operations do
not:
instance Monad Dist where
return x = D [(1, x)]
m>
>= f = D [(p ∗ q, y) | (p, x) ← runD m, (q, y) ← runD (f x)]
instance MonadProb Dist where
choose p x y = D [(p, x), (1 − p, y)]
Hence, we can perform probabilistic computations with any Haskell values as long as we only observe or
interprete the computations at values with computable equality.
A convenient observation function is runD, which returns a canonical representation of a probabilistic
computation.
3
runD :: Ord a ⇒ Dist a → [(Rational, a)]
runD d = norm (runD d)
where
norm :: Ord a ⇒ [(Rational, a)] → [(Rational, a)]
norm l = map (λl → (sum (map fst l), snd (head l))) (groupWith snd l)
This function returns a sorted list of values and there probabilities. It is used in the pretty printing code
provided in Appendix A.
Example 1.4. We can pretty print the example programs with the above representation.
*Main> pretty
00
1/4
01
1/4
10
1/4
11
1/4
*Main> pretty
00
1/4
01
1/4
10
1/4
11
1/4
*Main> pretty
00
1/3
01
1/3
10
1/3
*Main> pretty
00
1/6
01
1/3
10
1/3
11
1/6
2
(tuple1 >>= return . flip :: Dist (Bool, Bool))
(tuple1 >>= prog1 :: Dist (Bool, Bool))
(tuple2 >>= return . flip :: Dist (Bool, Bool))
(tuple2 >>= prog1 :: Dist (Bool, Bool))
Leaking
Now we extend probabilistic computations with an additional effect: leaking information.
The idea is that an external (typically adversarial) observer does not know the exact input of the computation, nor its output for that input. He only knows the probability distribution of the input, and, from
that, can derive the probability distribution of the output. However, the secrecy of the computation is not
perfect; it leaks some information to the observer that potentially allows him to improve his guess on the
output of the computation.
2.1
Specification
The type class MonadLeak refines probability monads with the ability to leak information of type d:
class MonadProb m ⇒ MonadLeak m d | m → d where
leak :: d → m ()
The operation leak denotes the action of leaking a single piece of information. Note that the functional
dependency m → d requires that the type of leaked data must be derivable from the type m. This is not
essential, but convenient for the automatic type inference it engenders.
With MonadLeak we can model a number of variations on our example program.
4
Example 2.1. This program leaks the first component of the original tuple, and returns the tuple:
prog2 :: MonadLeak m Bool ⇒ (Bool, Bool) → m (Bool, Bool)
prog2 p =
do leak (fst p)
return p
Example 2.2. This program leaks with equal probability one of the two components of the tuple, and
returns the tuple:
prog3 :: MonadLeak m Bool ⇒ (Bool, Bool) → m (Bool, Bool)
prog3 p =
do b ← fst p ⊕ snd p
leak b
return p
Example 2.3. This program leaks with equal probability one of the two components of the original tuple,
and returns with equal probability the original tuple or the flipped tuple:
prog4 :: MonadLeak m Bool ⇒ (Bool, Bool) → m (Bool, Bool)
prog4 p =
do b ← fst p ⊕ snd p
leak b
p ⊕ flip p
2.2
Implementation
Like in the previous section we derive a practical Haskell implementation by starting from an ideal mathematical representation.
Ideal Mathematical Representation Mathematically, we are interested in a representation of the form
D → ([0, 1] × Dist A). In words, with every possibly leaked piece of data we associate:
1. a probability that it is observed, as well as
2. a (conditional) probability distribution for the resulting value given that observation.
(Note that in case the probability of the leaked data is 0, the particular result distribution is irrelevant.)
This representation suffers from similar practical problems as the ideal representation of the previous
section.
(1, m) , d = lift(m) = λd.
(0, ⊥) , d 6= η(x) = lift(η(x))
x J p I y = lift(xP
J p I y)
P
µ(mmx) = λd.( d1 π1 (mmx(d1 )) × d2 :d1 d2 =d Pr π2 (mmx(d1 )) (d2 ))
(1, η(())) , d = e
leak(e) = λd.
(0, ⊥)
, d 6= e
5
Practical Haskell Implementation Fortunately, we do not have to define a new leaking probability
monad from scratch. Instead, the well-known writer monad transformer WriterT [d]adds leaking to any
probability monad.
Firstly, any transformed probability monad is still a probability monad:
instance (MonadProb m, Monoid w) ⇒ MonadProb (WriterT w m) where
choose p x y = lift (choose p x y)
We see that now taking choose as the primitive operation comes in handy because it is easily lifted.
Secondly, the leak operation corresponds to the tell operation of the writer monad transformer.
instance MonadProb m ⇒ MonadLeak (WriterT [d] m) d where
leak x = tell [x]
Note that we instantiated the writer monad transformer here to the list monoid. This means that multiple
sequentially leaked pieces of data are sequentially collected in the list. Other choices of monoids would lead
to different ways of collecting leaked data (e.g., summing or taking the maximum).
Finally, we can compose the probability distribution monad from the previous section with the writer
monad transformer to obtain a leaking probability monad.
type Dist2 = WriterT [Bool] Dist
Example 2.4. Now we can pretty print all the examples using this representation.
*Main>
0
00
01
10
11
*Main>
0
00
01
10
11
*Main>
0
00
01
10
11
*Main>
0
00
01
10
11
*Main>
0
00
01
10
pretty
: 1/2
1/2
1/2
0
0
pretty
: 1/2
1/2
1/4
1/4
0
pretty
: 1/2
1/4
1/4
1/4
1/4
pretty
: 1/3
0
1
0
0
pretty
: 1/3
0
1/2
1/2
(tuple1 >>=
1 : 1/2
0
0
1/2
1/2
(tuple2 >>=
1 : 1/2
0
1/4
1/4
1/2
(tuple2 >>=
1 : 1/2
1/4
1/4
1/4
1/4
(tuple1 >>=
1 : 2/3
0
0
1/2
1/2
(tuple1 >>=
1 : 2/3
0
1/4
1/4
prog2 :: Dist2 (Bool, Bool))
prog2 :: Dist2 (Bool, Bool))
prog3 :: Dist2 (Bool, Bool))
prog3 :: Dist2 (Bool, Bool))
prog4 :: Dist2 (Bool, Bool))
6
11
*Main>
1
00
01
10
11
3
0
1/2
pretty (tuple2 >>= prog4 :: Dist2 (Bool, Bool))
: 2/3
0 : 1/3
1/4
0
1/4
1/2
1/4
1/2
1/4
0
Leaking Impact
Sometimes we are not interested in immediately exploiting the leaked information, but simply want to
know whether the leaked information can be exploited to improve upon the probability distribution of the
computation’s result.
reveal :: (Ord a, Ord b) ⇒ (a → Dist b) → (Dist a → Dist (Dist a))
reveal f d = structure (destruct (fmap (λx → (x, f x)) d))
where
destruct dadb = [(p ∗ q, a, b) | (p, (a, db)) ← runD dadb, (q, b) ← runD db]
structure trips = normalize (D (map outer (groupWith (λ( , , b) → b) trips)))
outer l
= let q
= sum (map (λ(p, , ) → p) l)
inners = map (λ(p, a, ) → (p / q, a)) l
in (q, normalize (D inners))
update :: (a → Dist b) → (Dist a → Dist (Dist b))
update f d = return (d >
>= f)
oneBit :: (Bool, Bool) → Dist Bool
oneBit p = fst p ⊕ snd p
model1 :: Dist (Bool, Bool) → Dist (Dist (Bool, Bool))
model1 = reveal oneBit
model2 :: Dist (Bool, Bool) → Dist (Dist (Bool, Bool))
model2 d =
do d0 ← reveal oneBit d
update prog1 d0
model3 :: Int → Dist (Bool, Bool) → Dist (Dist (Bool, Bool))
model3 n = repeatM n (reveal oneBit)
repeatM :: Monad m ⇒ Int → (a → m a) → a → m a
repeatM 0 f x = return x
repeatM n f x = f x >
>= repeatM (n − 1) f
normalize :: (Ord a) ⇒ Dist a → Dist a
normalize d = D (runD d)
instance Ord a ⇒ Eq (Dist a) where
d1 ≡ d2 = runD d1 ≡ runD d2
instance Ord a ⇒ Ord (Dist a) where
compare d1 d2 = compare (runD d1) (runD d2)
7
References
[1] Martin Erwig and Steve Kollmansberger. Functional pearls: Probabilistic functional programming in
haskell. J. Funct. Program., 16(1):21–34, January 2006.
A
Pretty Printing
instance (Enum a, Bounded a, Enum b, Bounded b) ⇒ Enum (a, b) where
fromEnum (x, y) = fromEnum x ∗ m + fromEnum y
where
m = fromEnum (maxBound :: b) − fromEnum (minBound :: b) + 1
toEnum i = (toEnum (i ‘div‘ m), toEnum (i ‘mod‘ m))
where
m = fromEnum (maxBound :: b) − fromEnum (minBound :: b) + 1
class Pretty t where
prettyDoc :: t → Box
pretty t = putStr (render (prettyDoc t))
instance Pretty Bool where
prettyDoc False = char ’0’
prettyDoc True = char ’1’
instance Pretty Rational where
prettyDoc r =
let n = numerator r
d = denominator r
in if n ≡ 0
then char ’0’
else if d ≡ 1
then text (show n)
else text (show n) char ’/’ text (show d)
instance (Pretty a, Pretty b) ⇒ Pretty (a, b) where
prettyDoc (x, y) = prettyDoc x prettyDoc y
instance (Pretty a) ⇒ Pretty [a] where
prettyDoc l = foldr (λx xs → prettyDoc x xs) nullBox l
instance (Ord a, Pretty a) ⇒ Pretty (Dist a) where
prettyDoc d =
vcat right els text "
" vcat left sps
where l = runD d
ps
= map (prettyDoc ◦ fst) l
els
= map (prettyDoc ◦ snd) l
sps
= zipWith stretch (map rows els) ps
stretch r ps = ps // emptyBox (r − 1) 0
instance (Enum a, Bounded a, Pretty a, Pretty d, Eq a, Eq d, Ord d, Ord a) ⇒ Pretty (WriterT [d] Dist a) where
prettyDoc d = bigBox
where
l = runD (runWriterT d)
aVals
= [minBound . . maxBound]
dVals
= nub [b | ( , ( , b)) ← l]
8
cols
= [[lkup b a l | a ← aVals] | b ← dVals]
colProbs = map sum cols
content
= zipWith (λcol prob → map (prettyDoc ◦ (/ prob)) col) cols colProbs
topHeader = zipWith (λd p → prettyDoc d text " : " prettyDoc p) dVals colProbs
sideHeader = map prettyDoc aVals
bigBox = hsep 3 bottom (map (vcat center1) (sideHeader : zipWith (:) topHeader content))
lkup x y [ ]
=0
lkup x y ((p, (a, b)) : xs) | a ≡ y && b ≡ x
=p
| otherwise
= lkup x y xs
9