Combinatorial Agency
Michal Feldman
(Hebrew University)
Joint with:
Moshe Babaioff (UC Berkeley)
Noam Nisan (Hebrew University)
Hidden Actions


Algorithmic Mechanism Design: computational
mechanisms to handle Private Information.
(Classical) Mechanism Design



Private Information
Hidden Actions
We study hidden actions in multi-agents
computational settings
Example

Quality of Service (QoS) Routing [FCSS’05]:


We have some value from message delivery.
Each agent controls an edge:




succeeds with low probability by default.
succeeds with high probability if exerts costly effort
Message delivered if there is a successful sourcesink path.
Effort is not observable, only the final outcome.
source
sink
Modeling: Principal-Agent Model
exerts effort
cost: c >0
Principal
Agent
Does not
exert effort
cost: 0
Project succeeds with
high probability
Project succeeds
with low probability
Motivating rational agents to exert costly effort toward the welfare of the principal,
when she cannot contract on the effort level, only on the final outcome
“Success Contingent” contract. The agent
gets a high payment if project succeeds,
gets a low payment if project fails
Our focus is on multi-agents technologies
Our Model


n agents
Each agent has two actions (binary-action):





Project succeeds agent i receives pi (otherwise he gets 0)
Players’ utilities, under action profile a=(a1,…,an) and value v:



project succeeds, principal gets value v
project fails, principal gets value 0
Monotone technology function t: maps an action profile to a
success probability:
n
 t: {0,1}  [0,1]
t(a1,…,an)=success probability given (a1,…,an)
 i t(1, a-i) > t(0,a-i)
(monotonic)
Principal designs a contract for each agent


effort (ai=1), with cost c>0 (ci(1)=c)
no effort (ai=0), with cost 0 (ci(0)=0)
There are two possible outcomes (binary outcome):


The Principal’s
“input” parameter.
Agent i: ui(a) = t(a)·pi – ci(ai)
Principal: u(a,v) = t(a)·(v –Σipi)
Agents are in a game, reach Nash equilibrium.
The Principal’s
design parameter:
Used to induce the
desired equilibrium
Example: Read-Once Networks

A graph with a given source and sink

Each agent controls an edge, independently succeeds
or fails in his individual task (delivering on his edge)



Succeeds with probability ɣ<½ with no effort
Succeeds with probability 1-ɣ (>½>ɣ) with effort
The project succeeds if the successful edges form a
source-sink path.
a2=1
source
a1=1
Pr {x1=1}=1- ɣ
Pr {x2=1}=1- ɣ
a3=0
Pr {x3=1}=ɣ
sink
example: t(1, 1, 0) = Pr { x1  (x2  x3) =1 | a=(1,1,0) }
= (1- ɣ) (1- ɣ(1-ɣ))
Nash Equilibrium
Agent i’s utility
exerts effort
Does not exert effort
ui( 1,a-i ) = pi· t( 1,a-i ) – c
ui (1, ai )  ui (0, ai )  pi 
ui( 0,a-i ) = pi· t(0,a-i )
c
t (1, ai )  t (0, ai )
Di(a-i)

Principal’s best contract to induce eq. a=(a1,…,an):

pi= c / Di(a-i) for agent i with ai=1
 pi= 0 for agent i with ai=0
e.g., (1,0)
(1,1)

p1 
c
t (1,0)  t (0,0)
P2=0
p1 
c
t (1,1)  t (0,1)
p2 
c
t (1,1)  t (1,0)
Optimal Contract

the principal chooses a profile a*(v) that
maximizes her optimal equilibrium utility


c

u (a, v)  t (a ) v  

t
(
1
,
a
)

t
(
0
,
a
)
i
i 
 i:ai 1
Probability
of success
Total
payments
Research Questions

How does the technology affect the structure of the
optimal contracts?



What is the damage to the society due to the inability
to monitor individual actions?




Several examples (AND, OR, Majority …)
General technologies
“price of unaccountability”
What is the complexity of computing the optimal
contract?
Can the principal gain utility from mixed strategies?
Can the principal gain utility from a-priory removing
edges from the graph?
Optimal Contracts: simple AND technology
2 agents, g = ¼, c=1
s
x1
x2
t


c

u (a, v)  t (a ) v  

t
(
1
,
a
)

t
(
0
,
a
)
i
:
a

1

i

i
i


t(0,0) = g2 = (¼)2=1/16
 t(1,0) =t(0,1)= g(1g) = 3/16;
D0 =t(1,0)-t(0,0)=3/16 - 1/16 = 1/8
 t(1,1) = (1g)2 = 9/16
Principal’s Utility

1 agent exerts effort:
u((1,0),v) = t(1,0)·(v-c/D0) =
=3/16(v-1/(1/8))=(3/16)v-3/2

2 agents exert effort:
u((1,1),v) = t(1,1)·(v-2c/D1) = 9v/16-3

4
U(v)
0 agents exert effort:
u((0,0),v) = t(0,0)·v = v/16

At8 value of 6
there is a
6
“jump”
from 0
to 2 agents
2
0
0
5
10
-2
-4
v
15
Optimal Contract Transitions in AND and OR
OR
AND
x1
x1
x2
s
t
ɣ=1/4 optimal
to contract with 0
agents up to 6,
then with 2 agent
v
v
s
t
x2
2
g
g
Optimal Contract Transitions in AND and OR

Theorem: For any AND technology, there is only
one transition, from 0 to n agents.

Theorem: For any OR technology, there are
always n transition (any number of agents is
optimal for some value).
• We characterize all technologies with 1 transition
and with n transitions.
Proofs Idea-AND’s single transition


Observation (monotonicity): number of contracted
agents monotonically non-decreasing in v.
Proof for AND’s single transition:


At the indifference value between 0 and n agents,
contracting with 0<k<n agent has lower utility.
By the above observation, a single transition.
The 0 and n
indifference
value
8
6
U(v)
4
2
0
0
5
10
-2
-4
v
15
Transitions in AND and OR

Proof (AND):
k: number of contracted agents


k c

u (k )  t (k )   v 
t (k )  t (k  1) 



k c

 g n  k (1  g ) k   v  n  k
k 1
 g (1  g ) (1  2g ) 
this function has a single minimum point, thus maximized at
one of the edges 0 or n
Proofs Idea – OR’s n transitions



Let vk be the indifference point between k
and k+1 agents ( u(k,vk) = u(k+1,vk) )
We show that for OR: vk+1> vk
This ensures that k is optimal from vk-1 to vk
v1: The 1 ,2
indifference
value.
48
U(v)
v0: The 0 ,1
indifference
value.
58
38
0
28
1
18
2
8
-2
-12
v1>v0
0
20
40
v
60
80
Transitions in AND and OR

k: number of contracted agents
solve for v: u(k) = u(k+1), and let v(k) be the solution

we have to show: v(k+1) > v(k)  d
E.g., n=3
v(2)
v(0)
v(1)
d
General Technologies


In general we need to know which agents exert
effort in the optimal contract
Examples:
A1
A1
B1
t
s
A2

s
B2
(a) OR-of-ANDs technology

B1
A2
B2
(b) AND-of-ORs technology
In potential, any subset of agents (out of 2n
subsets) that exert effort could be optimal for
some v.
Which subsets can we get as an optimal
contract?
And-of-Ors (AOO) Technology

Example: 2x2 AOO technology
A1
B1
s
t
A2


{A1,B1}
{A1,B1,A2,B2}
v
B2
Theorem: The optimal contract in any AOO network (with
identical OR components) has the same number of agents in
each OR-component
f = h g

Proof: by induction based on following lemmas:


T
Decomposition lemma: if S=TUR is optimal on
f=hg on some v, then T is optimal for h on v·tg(R) and R is
optimal for g on v·th(T)
Component monotonicity lemma: the function vth(T) is
monotone non-decreasing (same for vtg(R) )
R
Decomposition Lemma
f = h g
T
if S=TUR is optimal on
f=hg on some v, then T is optimal for h on
v·tg(R) and R is optimal for g on v·th(T)

Proof:


ci



f
(
S
)
v

U ( S , v)

f

iS D i ( S \ i ) 



c
c

 h(T ) g ( R ) v   f i
 f i
iT D i ( S \ i )
iR D i ( S \ i ) 



g ( R )ci
ci

 h(T ) g ( R )v  

h
(
T
)
g
(
R
)

h
g

iT g ( R ) D i (T \ i ) 
iR h(T ) D i ( R \ i )



c
c
  g ( R ) g i
 h(T ) g ( R )v   h i
iT D i (T \ i ) 
iR D i ( R \ i )

i  T , D fi ( S \ i)  h(1, T \ i)  g ( R)  h(0, T \ i)  g ( R)  g ( R)Dhi (T \ i)
Similarly , i  R, D fi ( S \ i)  h(T )Dgi ( R \ i)
R
Component Monotonicity Lemma
The function vth(T) is monotone nondecreasing (same for vtg(R) )

Proof:








f:
S1 = T1 U R1 optimal on v1
h T1 R2 R1
S2 = T2 U R2 optimal on v2<v1
T2
By monotonicity lemma: f(S1) ≥ f(S2)
Since f=g·h, f(S1)=h(T1)·g(R1) ≥ h(T2)·g(R2) = f(S2)
Assume in contradiction that h(T1) < h(T2).
Since h(T1)·g(R1) ≥ h(T2)·g(R2) , we get g(R1) > g(R2).
By decomposition lemma, T1 is optimal for h on v1·g(R1), and T2 is
optimal for h on v2·g(R2)
As v1 > v2, and g(R1) > g(R2), T1 is optimal for h on a larger value than
T2 .
Thus, by monotonicity lemma, h(T1) ≥ h(T2)
g
And-of-Ors
x11

nc
s
t
xnl1

x1
xnl
nc
Theorem: The optimal contract in any AOO network,
composed of nc OR-components (of size nl) contracts with the
same number of agents in each OR-component.
Thus, |orbit(AOO)| ≤ nl+1
Proof: by induction on nc
 Base: nc=2
assume (k1,k2) is optimal on some v, assume by
contradiction k1>k2 (wlog), thus h(k1)>h(k2).
By decomposition lemma:
k1 optimal for h on v·h(k2)
k2 optimal for h on v·h(k1)>v·h(k2)
but if k2 optimal for a larger value, k2≥k1. in contradiction.
And-of-Ors
h2
k1 = k3
h

=
g
k3 = k2
k2
k2
h
h
h
assume (induction) that claim holds for any number of OR
components < nc






Assume 1st component has k1 contracted agents
Let g be the conjunction of the other (nc-1) comp.
By decomposition lemma, contract on g is optimal at v·h(k1), thus
by induction hypothesis has same number of agents, k2, on each
OR component.
Let h2 be conjunction of first two comp.
By decomp. Lemma, contract on h2 is optimal for some value and
by induction hypothesis has same number of agents, k3
We get k1=k3 (in first comp. k1 agents contracted), and k2=k3 (in
second comp. k2 agents contracted), thus k1=k2
The Collection of Optimal Contracts

Given t we wish to understand how the optimal
contract changes with v (the “orbit”).
Is there a structure on the collection of optimal contracts of t?

Monotonicity Lemma: The optimal contract
success probability t(a*(v)) is monotonic nondecreasing with v


So is the utility of the principal, and the total
payment
Thus, there are at most 2n-1 changes to the
optimal contracts (|Orbit(t)| ≤ 2n)
The Collection of Optimal Contracts

Observation 1: in the observable-actions case, only one set of
size k can be optimal (set with highest probability of success)

Observation 2: not all 2n subsets can be obtained
 Only a single set of size 1 can be optimal (set with highest
probability of success)
Can a technology have exponentially many different optimal contracts?


 2n 


n n 
Thm: There exists a tech. with
optimal
contracts
Open question 1: is there a read-once network
with exponential number of optimal contracts?
Exponential number of optimal contracts (1)


 2n 


n n 
Thm: There exists a tech. with
optimal contracts
Proof sketch:
 Lemma 1: all k-size sets in any k-admissible collection
can be obtained as optimal contracts of some t
S1
S3
S4
S2

Lemma 2: For any k, there exists a k-admissible
collection of k-size sets of size  1   n  


Collection of sets of
size k, in which every
two sets in it differ by at
least two elements
Based on error correcting code
 n  k 
Lemma 3: for k=n/2 we get
a k-admissible collection of
n
 2 
k-size sets of size
, as required.

n n 
Proof of Lemma 1
n
t(S)= ½ - eS
k
k-1
t(S’)= ½ - eS’
S’
S
S\i S’\i
S\i
t(S\i)= ½ - 2eS
1
S’\i
t(S’\i)= ½ - 2eS’
• marginal contribution of i  S is:
t(S) – t(S\i) = eS
Define t to ensure that the
marginal contribution of at least
one agent is very small
Claim: at vs=(ck) / 2eS2, the set S is
optimal:
• S better than any other set in
col. (by derivative of u(S,v))
• S better then any other set not
in col. (too high payments)


c

u ( S , v)  t ( S ) v  
iS t ( S )  t ( S \ i ) 

u ( S , v )
 0  eS 
e S
Let vs be v s.t. e S
ck
 ck
2v
2v
Exponential number of optimal contracts (2)

Lemma: For any n ≥ k, there exists an admissible collection of
k-size sets of size  1   n  
 n  k 
  

Proof: take error correcting code that corrects 1 error.
 Hamming distance ≥ 3  admissible
 Known:  codes with (2n/n) code words.
 Construct a code with sufficient # of k-weight words
 XOR every code word with a random word r. weight k w/
prob  n  n
  / 2
k 


Expected number of k-weight code words 1n   k 
 
There exists r such that the expectation is achieved or
exceeded
n
Research Questions


How does the technology affect the structure of the
optimal contracts?
What is the damage to the society / principal due to
the inability to monitor individual actions?




“price of unaccountability”
What is the complexity of computing the optimal
contract?
Can the principal gain utility from mixed strategies?
Can the principal gain utility from a-priory removing
edges from the graph?
Observable-Actions Benchmark (first best)





Actions are observable
Payment: an agent that exerts effort is paid
his cost (c)
Principal’s utility: u(a,v) = v·t(a) – Si|ai=1 c
Principal’s utility = social welfare sw(a,v).
The principal chooses a*OA, the profile with
maximum social welfare.
Social Price of Unaccountability

Definition: The Social Price Of Unaccountability
(POUS) of a technology is the worst ratio (over v)
between the social welfare in the observable-action
case, and the social welfare in the hidden-action
*
case:
sw(aOA
, v)
POU S  sup v 0
sw(a * , v)


a* - optimal contract for v in the hidden-action case
a*OA - optimal contract for v in the observable-action case

Example: AND of 2 agents:
s
t
v
Hidden actions
Observable actions
0
0
2
2
Principal’s Price of Unaccountability

Definition: The Principal’s Price Of Unaccountability
(POUP) of a technology is the worst ratio (over v)
between the principal’s utility in the observableaction case, and the principal’s utility in the hiddenaction case:
*
u p (aOA
, v)
POU P  sup v 0
u p ( a * , v)

a* - optimal contract for v in the hidden-action case
a*OA - optimal contract for v in the observable-action case

Price of Unaccountability - Results

Theorem: The POU of AND technology is
1 
POU    1
g




n 1

g
 1 
 1 g



unbounded for any fixed n≥2, when g0
unbounded for any fixed g<½ when n
Theorem: The POU of OR technology is
bounded by 2.5 for any n
Research Questions


How does the technology affect the structure of the
optimal contracts?
What is the damage to the society due to the inability
to monitor individual actions?




“price of unaccountability”
What is the complexity of computing the optimal
contract?
Can the principal gain utility from mixed strategies?
Can the principal gain utility from a-priory removing
edges from the graph?
Complexity of Finding the Optimal Contract
Input:
value v, description of t
Output: optimal contract: (a*,p)



Theorem: There exists a polynomial time algorithm to
compute (a*,p), if t is given by a table (exponential input).
Theorem: If t is given by a black box, exponentially many
queries may be required to find (a*,p).
sets of size n
Proof:
 for value v = c(k+ ½),
S’ is optimal
 Any algorithm must query
all sets of size k=n/2
to find S’ in the worst case
t(S)=1
S’
00100
0
sets of size
n/2
t(S)=0
sets of size 1
Complexity of Finding the Optimal Contract
Input:
value v, description of t
Output: optimal contract: (a*,p)

Theorem: For read-once networks, the optimal
contract problem is #p-hard



Proof: reduction from network reliability problem
Open problem 3: is it polynomial for seriesparallel networks?
Open problem 4: does it have a good
approximation?
Best Contract Computation
in Read-Once Networks

Proof (sketch): an algorithm for this problem can be used to
compute t(E) (probability of success)
G’
s


G
t
t
gx ½
Player x will enter the contract only for very large value of v
(only after all other agents are contracted), call this value vc
At vc, principal is indifferent between E and EU{x}




c
c
c



t ( E )  g x   v  

t
(
E
)

(
1

g
)

v



x
t
t


 iE g x  D i ( E \ i ) 
 iE (1  g x )  D i ( E \ i ) t ( E )(1  2g x ) 

t(E) 
(1  g x )  c
(1  2g x ) 2  v
Research Questions


How does the technology affect the structure of the
optimal contracts?
What is the damage to the society due to the inability
to monitor individual actions?




“price of unaccountability”
What is the complexity of computing the optimal
contract?
Can the principal gain utility from mixed strategies?
Can the principal gain utility from a-priory removing
edges from the graph?
Mixed Strategies
Can mixed-strategies help the principal ?
What is the price of purity ?



In the non-strategic case: NO (convex combination)
What about the agency case?
Extended game:
 qi : probability that agent i exerts effort
 t( qi,q-i ) = qi·t(1,q-i )+ (1-qi )·t(0,q-i )

Marginal contribution: Di(q-I ) = t(1,q-i ) - t(0,q-i ) ≥ 0
Nash Equilibrium in Mixed Strategies

Claim: agent i’s best-response is to mix with probability
q  (0,1) only if she is indifferent between 0 and 1
Agent i’s utility
High effort
Low effort
ui( 1,q-i ) = pi· t( 1,q-i ) – ci
ui( 0,q-i ) = pi· t(0,q-i )
ci
ui (1, qi )  ui (0, qi )  pi 
t (1, qi )  t (0, qi )


 t (q)



u
(
q
)

c


q
Agent i’s utility: i
i 
i
 D i (qi )

Principal’s utility: u (q, v)  t (q)   v 


ci 



i | qi  0 D i ( q  i ) 
g=0.25
Example:
OR with two agents

s
t
g=0.25
Optimal contract for v=110
 Pure strategies: both agents contracted: u = 88.12...
 Mixed strategies: q1=q2=0.96..: u=88.24...

Two observations:
 q1=q2 in optimal contract
 Principal’s utility is improved, but only slightly

How general are these observations?
Optimal Contract in OR Technology


Lemma: For any anonymous OR (any g,n,c,v), k{0,1,…,n}
agents exert effort with equal probabilities q1=…=qk  (0,1],
and n-k agents shirk. i.e. optimal profile: (0n-k, qk)
Proof (skecth): suppose by contradiction that (qi,qj,q-ij) s.t.
qi,qj (0,1) and qi > qj is optimal
y
(qi-ε,qj+yε,q-ij)
(qi,qj,q-ij)
qj
qi
t / qi 1  g  (2g  1)qi

t / q j 1  g  (2g  1)q j
For a sufficiently small ε , success
probability increases, and total payments
decrease. In contradiction to optimality
Optimal Contract in OR Technology
Example: OR with 2 agents:
Price of Purity (POP)

Definition: POP is the ratio between principal’s utility in mixed
strategies and in pure strategies
Optimal mixed contract


c
i

t (q * (v)) v  
 i|q* ( v ) 0 D i (q*i (v)) 
i


POP (t )  Supv 0

ci 

t ( S * (v)) v  

D
(
a
)
 iS *(v ) i i 
Optimal pure contract
Price of Purity

Definition: technology t exhibits

increasing returns to scale (IRS) if for any i and any b ≥ a
t(bi,b-i)-t(ai,b-i) ≥ t(bi,a-i)-t(ai,a-i)

decreasing returns to scale (DRS) if for any i and any b ≥ a
t(bi,b-i)-t(ai,b-i) ≥ t(bi,a-i)-t(ai,a-i)

Observations: AND exhibits IRS, OR exhibits DRS

Theorem: for any technology that exhibits IRS, optimal
contract is obtained in pure strategies

e.g., AND
Price of Purity





For any anonymous DRS technology, POP ≤ n
For anonymous OR with n agents, POP ≤ 1.154..
For any anonymous technology with 2 agents, POP ≤ 1.5
For any technology (not necessarily anonymous, but with
identical costs) with 2 agents, POP ≤ 2
Observation: the payment to each agent in a mixed
profile is greater than the min payment in a pure profile
and smaller than the max payment in a pure profile
Research Questions


How does the technology affect the structure of the
optimal contracts?
What is the damage to the society due to the inability
to monitor individual actions?




“price of unaccountability”
What is the complexity of computing the optimal
contract?
Can the principal gain utility from mixed strategies?
Can the principal gain utility from a-priory removing
edges from the graph?
Free-Labor


So far, technology was exogenously given
Now, suppose the principal has control over the technology in
that he can ex-ante remove some agents from the graph


as
before

Example: OR with 2 agents
s
t
s
Action set of agent i: ai  {1,0,}
 1: exert effort – succeed with probability d. cost=c
 0: do not exert effort - succeed with probability g< d. cost=0
 : do not participate – succeed with probability 0. cost=0
Action  “wastes free-labor” since action “0” increases the
success probability with no additional cost
t
Free-Labor
Are there scenarios in which the principal gains
utility from “wasting free-labor”?


The answer is: YES
Example: OR technology, n=2, g=0.2
g=0.2
s
0
1
2
v
t
g=0.2
1 removed

Theorem: for technologies with increasing marginal contribution
(e.g., AND), utilizing all free-labor is always optimal
Analysis of OR
g=0.01

g=0.25
g=0.49
Lemma: for any OR with n agents and g which is small
enough, there exists a value for which in the optimal contract
one agent exerts effort and no other agent participates
Version of the Braess’s Paradox


A project is composed of 2 essential components: A and B
And-of-Ors (AOO): allow interaction between teams
A1
B1
s
t
A2

B2
Or-of-Ands (OOA): don’t allow interaction between teams
A1
B1
t
s
A2

project succeeds if at least one of the
following pairs succeed:
(A1,B1) ; (A1,B2) ; (A2,B1) ; (A2,B2)
B2
project succeeds if at least one of the
following pairs succeed:
(A1,B1) ; (A2,B2)
Obviously, AOO is superior in terms of success probability
Version of the Braess’s Paradox
Example: g=0.2, v=110
A1
gi =1
s
A2
remove
middle edge
Or-ofAnds
A1
B1
B2
don’t remove
middle edge
A1
B1
B2
u(2,2) = 75.59..
B1
s
tt
ss
A2
t
t
A2
>
Andof-Ors
B2
u(1,1) = 74.17..
Or-of-Ands “wastes free-labor”.
Could the principal gain utility from removing middle edge?
Conclusion: it may be beneficial for the principal to isolate the teams
Summary

“Combinatorial Agency”: hidden actions in combinatorial
settings

Computing the optimal contract in general is hard

Natural research directions:



technologies whose contract can be computed in
polynomial time
Approximation algorithms
Many open questions remain
Thank You
[email protected]
Related Literature

[Winter2004] Incentives and discrimination

The effect of technology on optimal contract (full implementation)

[Winter2005] Optimal incentives with information about peers

[Ronen2005][Smorodinsky and Tennenholtz2004,2005]


Multi-party computation with costly information
[Holmstrom82] Moral hazard in teams

Budget-balanced sharing rules