Adversary Gain vs. Defender Loss
in Quantifying Information Flow
Piotr (Peter) Mardziel,† Mário S. Alvim,+ Michael Hicks,†
† University
of Maryland, College Park,
Federal de Minas Gerais
+ Universidade
UM
1 / 26
Quantified Information Flow [QIF]
I
Secrets leak to bad guys.
I
Quantify leakage of the secret.
information
secret
defender
channel
adversary
2 / 26
Why Quantified Information Flow?
I
Evaluate risks.
I
Evaluate relative merits of protection mechanisms.
I
Design incentives to keep adversaries from participating.
information
secret
defender
channel
adversary
2 / 26
Examples
I
Password authentication
I
Location-based services
I
Address space randomization
information
secret
defender
channel
adversary
2 / 26
Quantified Information Flow: The Approach
I
def
Flow = increase in adversary’s expected success
I
I
I
Model channel.
Model adversary behavior, exploitation.
Quantify expected success of optimal adversary.
before
observation
after
observation
3 / 26
Quantified Information Flow: The Approach
I
def
Flow = increase in adversary’s expected success
I
I
I
Model channel.
Model adversary behavior, exploitation.
Quantify expected success of optimal adversary.
before
observation
information
after
observation
3 / 26
Quantified Information Flow: The Approach
I
def
Flow = increase in adversary’s expected success
I
I
I
Model channel.
Model adversary behavior, exploitation.
Quantify expected success of optimal adversary.
before
observation
information
after
observation
3 / 26
Quantified Information Flow: The Approach
I
def
Flow = increase in adversary’s expected success
I
I
I
Model channel.
Model adversary behavior, exploitation.
Quantify expected success of optimal adversary.
before
observation
information
after
observation
3 / 26
Quantified Information Flow: The Approach
I
def
Flow = increase in adversary’s expected success
I
I
I
Model channel.
Model adversary behavior, exploitation.
Quantify expected success of optimal adversary.
before
observation
“flow”
information
after
observation
3 / 26
Quantified Gain: The Approach
I
∆ adversary’s gain
I
I
I
Model channel.
Model adversary behavior, exploitation.
Quantify (optimal adversary) gain.
before
observation
∆ gain
information
after
observation
3 / 26
Quantified Gain: The Approach
I
?
∆ adversary’s gain = ∆ defender’s loss
I
I
I
Model channel.
Model adversary behavior, exploitation.
Quantify (optimal adversary) gain.
before
observation
∆ gain
information
after
observation
3 / 26
Examples
I
Password authentication
I
Location-based services
I
Address space randomization
information
secret
defender
channel
adversary
4 / 26
Examples
I
Password authentication
I
I
Loss of bank contents = gain of bank contents
Loss of private info ≶ gain (theft) of identity
I
Location-based services
I
Address space randomization
information
secret
defender
channel
adversary
4 / 26
Examples
I
Password authentication
I
I
I
Location-based services
I
I
I
Loss of bank contents = gain of bank contents
Loss of private info ≶ gain (theft) of identity
Loss of privacy ≶ Gain in targeted advertising
Loss of house contents > Gain of stolen items
Address space randomization
information
secret
defender
channel
adversary
4 / 26
Examples
I
Password authentication
I
I
I
Location-based services
I
I
I
Loss of bank contents = gain of bank contents
Loss of private info ≶ gain (theft) of identity
Loss of privacy ≶ Gain in targeted advertising
Loss of house contents > Gain of stolen items
Address space randomization
I
I
(Small) loss of service < Gain of spam machine
Loss of critical service >> Gain of spam machine
information
secret
defender
channel
adversary
4 / 26
Examples
I
Password authentication
I
I
I
Location-based services
I
I
I
Loss of privacy ≶ Gain in targeted advertising
Loss of house contents > Gain of stolen items
Address space randomization
I
I
I
Loss of bank contents = gain of bank contents
Loss of private info ≶ gain (theft) of identity
(Small) loss of service < Gain of spam machine
Loss of critical service >> Gain of spam machine
... depends
information
secret
defender
channel
adversary
4 / 26
Examples
I
Password authentication
I
I
I
Location-based services
I
I
I
I
I
Loss of privacy ≶ Gain in targeted advertising
Loss of house contents > Gain of stolen items
Address space randomization
I
I
Loss of bank contents = gain of bank contents
Loss of private info ≶ gain (theft) of identity
(Small) loss of service < Gain of spam machine
Loss of critical service >> Gain of spam machine
... depends
loss 6= gain
information
secret
defender
channel
adversary
4 / 26
This work: Defender loss 6= Adversary gain
<
=
>
I
Defined model for information release with distinct defender
loss and adversary gain.
I
Both gain and loss are necessary to accurately quantify
defender loss.
I
Consequences about approximation:
I
I
I
Over-approximating adversary gain can be unsound
Over-approximating the channel (via partition refinement) can
be unsound
Worst-case (or best-case) metric to quantify the effect of
catastrophic (or fortunate) defender behavior.
5 / 26
Outline
<
=
>
I
Defined model for information release with distinct defender
loss and adversary gain.
I
Both gain and loss are necessary to accurately quantify
defender loss.
I
Consequences about approximation:
I
I
I
Over-approximating adversary gain can be unsound
Over-approximating the channel (via partition refinement) can
be unsound
Worst-case (or best-case) metric to quantify the effect of
catastrophic (or fortunate) defender behavior.
5 / 26
Caveat
<
=
>
I
Determining gain / loss is in the real world is hard.
I
We assume the “instantaneous” gain and loss are given.
I
I
Gain and loss functions, Secrets × Exploits → R
This work: analyze gain and loss dynamics as the adversary
learns about the secret through some channel.
5 / 26
Example: Pirate Treasure
I
Defender’s reasoning about the adversary stealing his treasure
and how to prevent it.
I
Approximately password authentication.
100% chance of loss
defender
6 / 26
Example: Pirate Treasure
defender
1/8
1/8
1/8
1/8
1/8
1/8
1/8
1/8
0
1
2
3
4
5
6
7
7 / 26
Secret Prior
0?
1?
2?
3?
4?
5?
6?
7?
adversary
8 / 26
Secret Prior = Defender Belief
1/8
1/8
1/8
1/8
1/8
1/8
1/8
1/8
0?
1?
2?
3?
4?
5?
6?
7?
adversary
I
Assume adversary knows defender behavior.
8 / 26
Exploitation
1/8
1/8
1/8
1/8
1/8
1/8
1/8
1/8
2?
3?
4?
5?
6?
7?
e
0?
1?
adversary
I
Adversary “raids” an island e for the treasure. If e = h he
succeeds.
8 / 26
Exploitation
1/8
1/8
1/8
1/8
1/8
1/8
1/8
1/8
2?
3?
4?
5?
6?
7?
e
0?
1?
adversary
I
Smith (FoSSaCS ’09): (prior) Vulnerability: expected
probability of optimal adversary with one guess being correct.
8 / 26
Exploitation: Measures of Success
1/8
1/8
1/8
1/8
1/8
1/8
1/8
1/8
2?
3?
4?
5?
6?
7?
e
0?
1?
adversary
Optimal adversary behavior:
I Guessing Entropy: Minimal number of guesses to find secret.
I Alvim et al. (CSF ’12): g -Vulnerability Gain/payoff
according to function g (secret, exploit).
8 / 26
Exploitation: Vulnerability
1/8
1/8
1/8
1/8
1/8
1/8
1/8
1/8
2?
3?
4?
5?
6?
7?
e
0?
1?
adversary
I
I
I
Connect probability of success to economic quantities.
If the treasure is worth w doubloons, the expected gain to
adversary and loss to the defender is w × V doubloons. Here,
w/8.
Will stick with expected probability of success using the term
“gain” in the remainder of this talk.
8 / 26
Observation
I
Adversary can “stake out” an island to check whether the
treasure is there.
1/8
1/8
1/8
0
1
2
yes
1/8
1/8
1/8
1/8
1/8
3
4
5
6
7
?
?
?
?
?
no
?
?
channel
?
adversary
9 / 26
Observation
I
Adversary can “stake out” an island to check whether the
treasure is there.
1/7
0
1/7
1
2
1/7
1/7
1/7
1/7
1/7
3
4
5
6
7
no
channel
adversary
9 / 26
Increased knowledge
I
I
Observation leads to increase in knowledge.
Which leads to increased odds of exploitation.
1/7
0
1
1/7
1/7
1/7
1/7
1/7
1/7
2
3
4
5
6
7
adversary
10 / 26
Increased knowledge ⇒ increased gain
I
(posterior) Gain: expected probability of optimal adversary
succeeding in one guess given observation(s).
I
I
Optimal island to stake out.
Optimal island to raid.
1/7
0
1
1/7
1/7
1/7
1/7
1/7
1/7
2
3
4
5
6
7
adversary
10 / 26
Increased knowledge ⇒ increased gain
I
(posterior) Gain: expected probability of optimal adversary
succeeding in one guess given observation(s).
I
I
I
Optimal island to stake out.
Optimal island to raid.
Here: 1/7 .
1/7
0
1
1/7
1/7
1/7
1/7
1/7
1/7
2
3
4
5
6
7
adversary
10 / 26
Observations over time
I
I
Adversary continues observations (stake outs) but only has
one exploitation chance (raid).
How does their expected gain grow when they have more time
to make observations?
11 / 26
Observations over time
I
I
Adversary continues observations (stake outs) but only has
one exploitation chance (raid).
How does their expected gain grow when they have more time
to make observations?
expected gain
2-0.0
2-0.5
2-1.0
2-1.5
2-2.0
2-2.5
2-3.0
0
1
2 3 4 5 6 7 8 9 10 11 12 13 14
T = (max) number of observations before raid
11 / 26
Observations over time
I
I
I
Adversary continues observations (stake outs) but only has
one exploitation chance (raid).
How does their expected gain grow when they have more time
to make observations?
Eventually the treasure will be lost.
expected gain
2-0.0
2-0.5
2-1.0
2-1.5
2-2.0
2-2.5
2-3.0
0
1
2 3 4 5 6 7 8 9 10 11 12 13 14
T = (max) number of observations before raid
11 / 26
Moving the treasure
I
Defender moves the treasure every once in a while.
1/8
1/8
1/8
1/8
1/8
1/8
1/8
1/8
12 / 26
Moving the treasure
I
Defender moves the treasure every once in a while.
I
Assume adversary knows the process with which the defender
does this.
1/8
1/8
1/8
1/8
1/8
1/8
1/8
1/8
12 / 26
Gain with moving treasure
I
Defender moves his treasure every 3 time steps.
expected gain
2-0.0
2-0.5
2-1.0
2-1.5
2-2.0
2-2.5
static treasure
moving treasure
2-3.0
0
1
2 3 4 5 6 7 8 9 10 11 12 13 14
T = (max) number of observations before raid
13 / 26
Gain with moving treasure
I
Defender moves his treasure every 3 time steps.
I
Eventually the treasure is lost.
expected gain
2-0.0
2-0.5
2-1.0
2-1.5
2-2.0
2-2.5
static treasure
moving treasure
2-3.0
0
1
2
3 4 5 6 7 8 9 10 11 12 13 14 ...
T = (max) number of observations before raid
∞
13 / 26
Costly Observation
I
I
Defender makes it harder for the adversary to stake out for
the treasure.
It costs 0.10 [treasure] to stake out an island.
0
1
2
yes
?
3
4
5
6
7
no
?
?
?
?
?
?
stake-out, −
?
wait
channel
adversary
14 / 26
Costly Observation
I
I
I
Defender makes it harder for the adversary to stake out for
the treasure.
It costs 0.10 [treasure] to stake out an island.
Gain = (1.0 if treasure raided) - (0.1 * num. of observations)
0
1
2
yes
?
3
4
5
6
7
no
?
?
?
?
?
?
stake-out, −
?
wait
channel
adversary
14 / 26
Costly Observation
I
I
I
Defender makes it harder for the adversary to stake out for
the treasure.
It costs 0.10 [treasure] to stake out an island.
Gain = (1.0 if treasure raided) - (0.1 * num. of observations)
I
Gain can no longer be interpreted as chances of
adversary successfully capturing the treasure.
0
1
2
yes
?
3
4
5
6
7
no
?
?
?
?
?
?
stake-out, −
?
wait
channel
adversary
14 / 26
Gain with costly stake outs
I
Defender still moves his treasure every 3 time steps.
expected gain
2-0.0
2-0.5
2-1.0
2-1.5
2-2.0
static treasure
moving treasure
moving treasure, observation cost
2-2.5
2-3.0
0
1
2 3 4 5 6 7 8 9 10 11 12 13 14
T = (max) number of observations before raid
15 / 26
Gain with costly stake outs
I
Defender still moves his treasure every 3 time steps.
I
Adversary gain bounded even in the limit.
expected gain
2-0.0
2-0.5
2-1.0
2-1.5
2-2.0
static treasure
moving treasure
moving treasure, observation cost
2-2.5
2-3.0
0
1
2
3 4 5 6 7 8 9 10 11 12 13 14 ...
T = (max) number of observations before raid
∞
15 / 26
Gain with costly stake outs
I
Defender wants ≤ 50% chance of losing his treasure.
I
Should he be satisfied with this result?
expected gain
2-0.0
2-0.5
2-1.0
2-1.5
2-2.0
2-2.5
2-3.0
adversary gain
0
1
2
3 4 5 6 7 8 9 10 11 12 13 14 ...
T = (max) number of observations before raid
∞
16 / 26
Gain with costly stake outs
I
Defender wants ≤ 50% chance of losing his treasure.
I
Should he be satisfied with this result?
I
Adversary gain does not measure defender loss.
expected gain
2-0.0
2-0.5
2-1.0
2-1.5
2-2.0
2-2.5
2-3.0
adversary gain
0
1
2
3 4 5 6 7 8 9 10 11 12 13 14 ...
T = (max) number of observations before raid
∞
16 / 26
I
Defender wants ≤ 50% chance of losing his treasure.
I
Should he be satisfied with this result?
I
Adversary gain does not measure defender loss.
2-0.0
2-0.0
2
-0.5
2-0.5
2
-1.0
2-1.0
2-1.5
2-1.5
2
-2.0
2-2.0
2
-2.5
2-3.0
adversary gain
defender loss
2-2.5
2-3.0
expected loss [probability]
expected gain [treasure]
Gain with costly stake outs
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 ... ∞
T = (max) number of observations before raid
16 / 26
I
Defender wants ≤ 50% chance of losing his treasure.
I
Should he be satisfied with this result?
I
Adversary gain does not measure defender loss.
I
Optimal adversary will keep on staking out indefinitely.
2-0.0
2-0.0
2
-0.5
2-0.5
2
-1.0
2-1.0
2-1.5
2-1.5
2
-2.0
2-2.0
2
-2.5
2-3.0
adversary gain
defender loss
2-2.5
2-3.0
expected loss [probability]
expected gain [treasure]
Gain with costly stake outs
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 ... ∞
T = (max) number of observations before raid
16 / 26
Dimensions of adversary strategy
I
Each adversary strategy induces both a gain and a resulting
defender loss.
I
Strategies: functions that determine adversary’s action based
on their past actions and observations.
gain
adversary
strategies
loss
17 / 26
Dimensions of adversary strategy
I
Each adversary strategy induces both a gain and a resulting
defender loss.
I
I
Strategies: functions that determine adversary’s action based
on their past actions and observations.
Our metric: expected defender loss assuming adversary
optimizes gain.
gain
αopt
g
adversary
strategies
s
loss
17 / 26
Gain + Loss
I
Cannot analyze a scenario in just one dimension.
gain
αopt
g
s
loss
18 / 26
Gain + Loss
I
Cannot analyze a scenario in just one dimension.
I
Gain only: not what a defender is interested in.
gain
αopt
g
g0
αopt
s
loss
18 / 26
Gain + Loss
I
Cannot analyze a scenario in just one dimension.
I
Loss only: miss out on disincentives.
gain
αopt
g
s
max loss
loss
18 / 26
Gain + Loss
I
I
Cannot analyze a scenario in just one dimension.
Loss only: miss out on disincentives.
I
In example: stake out cost must be ≥ 1/7 treasure units.
gain
0
αopt
0
g
g
αopt
s0
s
max loss
loss
18 / 26
Gain + Loss
I
I
Cannot analyze a scenario in just one dimension.
Loss only: miss out on disincentives.
I
Or miss bad incentives.
gain
00
0
αopt
0
g g
g
αopt
αopt
s0
s
s 00max loss
loss
18 / 26
Time
Model Overview
defender
secrets
h0
adversary influence
h1
`2
o1
`3
o2
h2
`1
channel
adversary
o3
observation
exploitation e
s
defender’s loss
loss / gain
g
adversary’s gain
19 / 26
Model Overview
Prior: initial secret distribution and distribution over
(non-deterministic) functions describing secret evolution.
Time
I
defender
secrets
h0
adversary influence
h1
`2
o1
`3
o2
h2
`1
channel
adversary
o3
observation
exploitation e
s
defender’s loss
loss / gain
g
adversary’s gain
19 / 26
Model Overview
Adversary optimization: optimize low inputs (channel
influence) and exploitation for maximal gain.
Time
I
defender
secrets
h0
adversary influence
h1
`2
o1
`3
o2
h2
`1
channel
adversary
o3
observation
exploitation e
s
defender’s loss
loss / gain
g
adversary’s gain
19 / 26
Model Overview
Measurement: measure the resulting defender loss.
Time
I
defender
secrets
h0
adversary influence
h1
`2
o1
`3
o2
h2
`1
channel
adversary
o3
observation
exploitation e
s
defender’s loss
loss / gain
g
adversary’s gain
19 / 26
Prototype, Prototyping Implementation
I
Describe models as probabilistic programs in monadic-style
OCaml.
I
Optimize adversary behavior via backward induction.
I
Compute the resulting defender loss.
I
Analyze a series of scenarios (including this talk’s examples)
I
Freely available online.
20 / 26
Approximations
I
Previously: both loss and gain are necessary.
I
Previously: loss and gain functions might be hard to ascertain
in the real world.
I
Also: channel might be uncertain or too hard to analyze.
I
Solution: over-approximations ?
21 / 26
Soundness Approximations
Does over-approximating gain or channel lead to
over-approximation of loss?
Time
I
defender
secrets
h0
adversary influence
h1
`2
o1
`3
o2
h2
`1
channel
C
adversary
o3
observation
exploitation e
s
defender’s loss
loss / gain
g
adversary’s gain
22 / 26
Soundness Approximations
Does over-approximating gain or channel lead to
over-approximation of loss?
Time
I
defender
secrets
h0
adversary influence
h1
`2
o1
`3
o2
h2
`1
channel
C
adversary
o3
observation
exploitation e
s
defender’s loss
loss / gain
g0 ≥ g
adversary’s gain
22 / 26
Soundness Approximations
Does over-approximating gain or channel lead to
over-approximation of loss?
Time
I
defender
secrets
h0
adversary influence
h1
`2
o1
`3
o2
h2
`1
channel
C0 w C
adversary
o3
observation
exploitation e
s
defender’s loss
loss / gain
g0 ≥ g
adversary’s gain
22 / 26
Soundness Approximations
Does over-approximating gain or channel lead to
over-approximation of loss?
Time
I
defender
secrets
h0
adversary influence
h1
`2
o1
`3
o2
h2
`1
channel
C0 w C
adversary
o3
observation
exploitation e
s 0 ≥ s?
defender’s loss
loss / gain
g0 ≥ g
adversary’s gain
22 / 26
Approximating Gain
Adversary gain (and defender loss): treasure is spread out
around a central island; secret = 4 is shown below.
expected gain
I
8
7
6
5
4
3
2
1
0
g
0
1
2
3
4
5
6
7
adversary action
23 / 26
Approximating Gain
expected gain
I
Very bad over-approximation:
g 0 (secret, exploit) ≥ g (secret, exploit)
8
7
6
5
4
3
2
1
0
g
g’
0
1
2
3
4
5
6
7
adversary action
23 / 26
Approximating Gain
Very bad over-approximation:
g 0 (secret, exploit) ≥ g (secret, exploit)
I
Not sound for loss.
expected gain
I
8
7
6
5
4
3
2
1
0
g
g’
0
1
2
3
4
5
6
7
adversary action
23 / 26
Approximating Gain
expected gain
I
Preference preserving approximation: g (secret, exploit1 ) ≥
g (secret, exploit2 ) ⇔ g 0 (secret, exploit1 ) ≥ g 0 (secret, exploit2 )
8
7
6
5
4
3
2
1
0
g
g’
0
1
2
3
4
5
6
7
adversary action
23 / 26
Approximating Gain
Preference preserving approximation: g (secret, exploit1 ) ≥
g (secret, exploit2 ) ⇔ g 0 (secret, exploit1 ) ≥ g 0 (secret, exploit2 )
I
Not sound for loss.
expected gain
I
8
7
6
5
4
3
2
1
0
g
g’
0
1
2
3
4
5
6
7
adversary action
23 / 26
Approximating Gain
expected gain
I
Linear scaling: g 0 (secret, exploit) = r ∗ g (secret, exploit) with
r > 0.
8
7
6
5
4
3
2
1
0
g
g’
0
1
2
3
4
5
6
7
adversary action
23 / 26
Approximating Gain
Linear scaling: g 0 (secret, exploit) = r ∗ g (secret, exploit) with
r > 0.
I
Sound approximation, but (arguably) not useful.
expected gain
I
8
7
6
5
4
3
2
1
0
g
g’
0
1
2
3
4
5
6
7
adversary action
23 / 26
Approximating Gain
Linear scaling: g 0 (secret, exploit) = r ∗ g (secret, exploit) with
r > 0.
I
(the only) Sound approximation, but (arguably) not useful.
expected gain
I
8
7
6
5
4
3
2
1
0
g
g’
0
1
2
3
4
5
6
7
adversary action
23 / 26
Approximating Channel
I
Approximate channel C with C 0 w C .
I
Example: C(secret) = “nothing” and C’(secret) = secret
24 / 26
Approximating Channel
I
Approximate channel C with C 0 w C .
I
I
Example: C(secret) = “nothing” and C’(secret) = secret
Not sound for loss.
24 / 26
Approximating Channel
I
Approximate channel C with C 0 w C .
I
I
Not sound for loss.
Example: Assume inverse relationship between gain and loss.
expected gain/loss
I
Example: C(secret) = “nothing” and C’(secret) = secret
4
loss
gain
3
2
1
0
0
1
2
3
4
5
6
7
adversary action
24 / 26
Approximating Channel
I
Approximate channel C with C 0 w C .
I
I
I
Example: C(secret) = “nothing” and C’(secret) = secret
Not sound for loss.
Example: Assume inverse relationship between gain and loss.
expected gain/loss
I
The more the adversary knows, the less loss is incurred by
defender.
4
loss
gain
3
2
1
0
0
1
2
3
4
5
6
7
adversary action
24 / 26
Soundness of Approximations
I
No “useful” approximations of gain are sound for loss.
I
Conjecture: no approximation of channel is sound for loss.
25 / 26
Conclusions
I
Model for information flow distinct adversary gain and
defender loss.
26 / 26
Conclusions
I
I
Model for information flow distinct adversary gain and
defender loss.
Both gain and loss are necessary for accurate measurement of
loss.
26 / 26
Conclusions
I
I
I
Model for information flow distinct adversary gain and
defender loss.
Both gain and loss are necessary for accurate measurement of
loss.
Unsound consequences for loss when over-approximating gain
or channel.
26 / 26
Conclusions
I
I
I
I
Model for information flow distinct adversary gain and
defender loss.
Both gain and loss are necessary for accurate measurement of
loss.
Unsound consequences for loss when over-approximating gain
or channel.
Implementation and Experiments
26 / 26
Adversary Gain vs. Defender Loss in Quantified Information Flow
I
I
I
I
I
Model for information flow distinct adversary gain and
defender loss.
Both gain and loss are necessary for accurate measurement of
loss.
Unsound consequences for loss when over-approximating gain
or channel.
Implementation and Experiments
http://ter.ps/fcs14
I
This paper, Oakland’14 paper, TR, Implementation,
Experiments
26 / 26