1
Game Theory/
Mechanism Design/Auctions
in Multi-agent Systems
Outline
•
•
•
•
•
Examples
Motivation (why game theory?)
Basic Terms
Single-item Auctions
Combinatorial Auctions, VCG, & Falsename Bids
• Automated Mechanism Design
Makoto Yokoo
Kyushu University, Japan
[email protected]
http://agent.inf.kyushu-u.ac.jp/~yokoo/
Example: Standard (First-price)
Sealed-bid Auction
2
3
Second-price Sealed-bid Auction
(Vickrey Auction)
4
Mechanism : The highest bidder wins, but pays the value of
the second highest bid.
Mechanism: Each bidder (agent) privately bids
the price it is willing to pay, and the highest
bidder wins; pays the value of its bid.
Characteristic:
Demerit: Spying other agents’ bids is profitable.
–Bidding its true evaluation value (the maximal value
where it does not want to pay any more) is the optimal
strategy (incentive compatibility).
–Spying other agents’ bids is meaningless.
Pay the second
highest bid ($7000)
$8000
$7000
$6000
Spying is Useless Because...
Assume its own evaluation value is $8000, and found
that others’ highest bid is:
(I) less than $8000: its payment is the same to the case
when it submits the true evaluation value (without
spying).
(II) more than $8000: the agent cannot obtain positive
utility anyway.
Its profit cannot be more than the case when it truthfully
submits the true evaluation value.
$8000
$6000
$7000
5
6
Quiz: The larger also serves
for the smaller?
• If the highest bidder wins, and pays the
value equals to the third highest bid, is
honesty still the best policy?
$100
Pay the second
highest bid ($7000)
$150
$50
$8000
$7000
$6000
1
7
8
Answer
Outline
•
•
•
•
•
Examples
Motivation (why game theory?)
Basic Terms
Single-item Auctions
Combinatorial Auctions, VCG, & Falsename Bids
• Automated Mechanism Design
• Honesty is no longer the best policy.
– For the first and second bidder, if he wins,
he is going to pay only $50.
– Can increase his bid to infinity to beat the
opponent.
• If two identical units are sold (i.e., both
are winners), then honesty is the best
policy.
9
10
Why Game-theory/Economics
(contd.)?
Why Game-theory/Economics?
• Game-theory/Economics provide analytical tools
for the design/analysis of multi-agent systems
(MAS).
– Game-theory/economics are specially useful
when the MAS
• are not centrally designed.
• do not have a notion of global utility.
• will not necessarily act “benevolently”.
• Underlying assumptions:
– Each player (agent) has consistent
preference/utility.
– Each player is rational, i.e., tries to
maximize his/her utility.
– Each player chooses a strategy to play.
These assumptions only approximate human
behavior, but are fully applicable to MAS.
11
Why Game-theory/Economics
(contd.)?
• Increasing intersectional
topics:
– Internet auctions, electronic
commerce
John von Neumann
– resource allocation for
automated agents
From the theory for describing
human behavior to the theory
for designing systems!
Why Game-theory/Economics
(contd.)?
12
• MAS research community is divided into several
different groups (like clubs).
– e.g., logic, search/constraint satisfaction,
game-theory
• Each club has its own tradition.
• The members of each club speak their own
language.
• You need to join a club to publish your paper.
• To join a club, you need to speak their language
and understand their tradition.
• Game-theory/mechanism design is one of
mainstream clubs in MAS.
2
13
14
Characteristics of Player
Outline
•
•
•
•
•
Examples
Motivation (why game theory?)
Basic Terms
Single-item Auctions
Combinatorial Auctions, VCG, & Falsename Bids
• Automated Mechanism Design
risk neutral/averse
• risk neutral： only cares the expected utility
– e.g., indifferent between two lotteries:
1) he/she obtains 0 for the head and $100 for
the tail,
2) he/she obtains $50 for sure.
• risk averse： prefers that is more certain （even
with less expected utility）
– e.g., prefers getting $45 for sure to 1.
15
Assumption for Simplicity
16
The St. Petersburg Paradox
Quasi-linear utility: an agent’s utility is
defined as the difference between
its evaluation value of the allocated good
and its payment.
• If an agent wins a good whose evaluation
value is $100 by paying $90, its utility is
$100-$90=$10.
• If it does not win, its utility is $0.
• If it does win, paying $100, its utility is 0.
Assume the following lottery...
• Flip a coin, if it comes up tail, you get $2.
• If head, flip a coin again, if it comes up tail, you
get $4.
• If head, flip a coin again, ....
• If it comes up tail the first time at n-th trial, you
get $2n.
How much are you willing to pay to
participate this lottery?
17
Strategy & Equilibrium
Strategy: the way for choosing actions, i.e., how to
decide the bidding price
Dominant Strategy: the best strategy regardless of the
actions of other agents
– Paper-rock-scissors: no dominant strategy
– If only paper and rock are allowed, paper is the
dominant strategy
Dominant Strategy equilibrium: assume each agent
has a dominant strategy. Then, the combination of
dominant strategies is called dominant strategy
equilibrium.
Nash equilibrium: a weaker concept than a dominantstrategy equilibrium. A combination of strategy is a
Nash equilibrium if each strategy is a best response
to other strategies.
18
Desirable Properties of a Group
Decision Making Mechanism
• For each agent, there exists a
dominant strategy.
• The mechanism is robust against
various frauds (e.g., spying).
• A Pareto efficient outcome can be
achieved.
3
19
20
Pareto Efficiency
Social Surplus
Definition:
A state is said Pareto efficient, iff there
exists no other state that is
• Assuming agents’ utilities are quasi-linear,
if a state is Pareto efficient, the social
surplus (sum of all agents’ utilities) must
be maximized.
– better for one agent, and
– no worse for all the rest
×
×
Movie
Shopping
Zoo
Home
2
2
2
1
2
2
3
1
×
2
5
1
1
×
×
Movie 6
Shopping
9
6
Zoo
Home 3
2
2
2
1
2
2
2 →3 5 →4
3
1
1
1
21
Private/Common/Correlated Values
Pareto Efficiency in Auctions
Private Value: each agent knows its value
with certainty, which is independent from
other agents’ evaluation values (e.g.,
antiques which are not resold).
Common value： the evaluation values for all
agents are the same, but agents do not
know the exact value and have different
estimated values (e.g., US Treasury bills,
mining right of oil fields).
Correlated Value: Something between above
two extremes.
• The social surplus must be maximized;
the good must be allocated to the agent
who has the highest evaluation value.
• The agent whose evaluation value is $8000
wins and pays $7000.
• Utility of this agent: $8000 - $7000=$1000
• Utility of the seller: $7000
• Social Surplus: $8000
$8000
$7000
22
$6000
23
Mechanism Design
• Designing a mechanism is
determining the rules of a game.
• The designer cannot control the
actions of each agent.
–No way to force an agent to be
honest or to refrain from doing
frauds.
24
Mechanism Design
How can a designer achieve a certain desirable
property (e.g., Pareto efficiency)?
• Design rules so that:
– For each agent, there exists a dominant
strategy.
– In the dominant strategy equilibrium, the
desirable property is achieved.
4
25
Outline
26
First-price Sealed-bid
•
•
•
•
•
Examples
Motivation (why game theory?)
Basic Terms
Single-item Auctions
Combinatorial Auctions, VCG, & Falsename Bids
• Automated Mechanism Design
Mechanism：Each agent submits its
bid without knowing other agents’
bids. The agent with the highest bid
wins and pays its own price.
Dominant strategy： does not exist in
general.
27
Example: First-price
Example: First-price
• Assume you attend an auction on behalf of
your uncle.
• Your uncle specifies the maximal price you can
bid for the good ($100).
• The generous uncle will give you the difference
if you can buy it less than $100.
• If you failed to buy a good, you receive
nothing for the good.
• Assume there is only one opponent.
• Your opponent is also a proxy.
• You don’t know how much your
opponent will bid, but you know his
maximal price is uniformly distributed
among [0, 200].
29
Strategy for Bidding
• Clearly, bidding more than $100 is
meaningless (you must pay the
difference).
• If you bid $90 and win, your profit is $10.
• If you bid $1 and (by any chance) win,
your profit is $99.
• How much should you bid?
28
30
Answer
• We cannot tell the expected utility unless knowing
the strategy of the opponents.
• Let us assume the bidding price of the opponents
is uniformly distributed between [0, 100]
• If you bid low, the probability of winning is low,
but the utility when wins is large.
⇒high-risk, high-return
２５
• If you bid high, the probability utility
of winning is high, but the utility
when wins is low.
⇒low-risk, low-return
５０
１００
bidding price
• The best point is in between,
bidding the half (50), where the expected utility is
25.
5
31
Answer (contd.)
32
Dutch (descending)
• If the opponent uses the same strategy,
his bid is uniformly distributed between
[0, 100], since his maximal price is
uniformly distributed between [0, 200].
• This pair of strategies is (Bayesian) Nash
equilibrium, i.e., the best response with
each other.
• Even if you are not a proxy, i.e., the
maximal price is your own evaluation
value, you should use the same strategy.
Mechanism: The seller announces a
very high price, then continuously
lowers the price until some agent
says “stop”, then the agent wins the
good and pays the current price.
Dominant strategy： does not exist in
general.
33
Dutch (descending)
34
English (open cry)
• Strategically equivalent to
the first-price sealed-bid
auction
6900
7400
7900
8400
8900
– there is one-to-one
STOP!
mapping between the
strategy sets in two
auctions.
• Example:
– Dutch flower market
$8000 $7000 $6000
– Ontario tobacco auction
– bargain sale
Mechanism：Each agent is free to revise
its bid upwards. When nobody wishes
to revise its bid further, the highest
bidder wins the good and pays its own
price.
(Almost) Dominant strategy (in private
value): keep bidding some small
amount more than the previous
highest bid until the price reaches its
evaluation value, then quit.
35
Vickrey (Second-price Sealed-bid)
English (open cry)
• In everybody uses the
minimum increment
8000
strategy, the agent
with the highest
evaluation value wins
and pays the second
highest evaluation
value +ε.
• The obtained allocation $8000
is Pareto efficient.
36
7000
6000
Mechanism：Each agent submits its bid without
knowing other agents’ bids. The agent with
the highest bid wins and pays the value of the
second highest bid.
Dominant strategy (in Private value)： Bidding its
true evaluation value is a dominant strategy
• The obtained allocation in a dominant strategy
equilibrium is Pareto efficient.
• The obtained result is identical to English in
the dominant-strategy equilibrium.
$7000 $6000
6
37
38
Characteristics of mechanisms
(in Private Value Auctions)
Difficulties for Using Vickrey
Auction
• Dutch＝First-price Sealed-bid
• English＝Vickrey
• Under several assumptions, the expected
revenue of the seller is the same in all 4
auction mechanisms (revenue
equivalence theorem, Vickrey 1961）．
• Hard to understand!
• Do not know/aware of the true
evaluation value (even in private
value auctions).
• Cannot trust the seller.
• Do not want to reveal the
private/sensitive information.
– If there exists a Bayesian Nash equilibrium,
the expected revenue would be the same at
the equilibrium．
Sponsored Search
Is mechanism design/auction theory
useless?
39
40
Sponsored Search
• An advertiser bids for a search keyword.
• When a keyword is searched by a user,
links to advertisers are shown in the
decreasing order of bids.
• It enables an advertisement targeted
for a particular type of users.
• An advertiser pays only when a user
actually clicks the link （pay-per-click）.
Question: how to set the payment?
Payment Scheme
• In early systems, an advertiser pays the amount of her
bid (first-price).
– Setting an appropriate amount of a bid is difficult.
– An advertiser can send a dummy search request and
adjust her bid.
• The payment scheme is changed so that an advertiser
who obtains k-th slot pays the amount of k+1-st bid.
Lessons Learned:
• Second-price/Vickrey auction, was never widely used, now
became the most frequently used auction mechanism in
the world.
• For a mechanism, which is frequently used on the Internet
by users including automated agents, having theoretical
robustness is crucial.
41
42
Outline
•
•
•
•
•
Examples
Motivation (why game theory?)
Basic Terms
Single-item Auctions
Combinatorial Auctions, VCG, & Falsename Bids
• Automated Mechanism Design
7
43
44
Research Issues in
Combinatorial Auctions
Combinatorial Auction
• Finding the best combination of bids is a
complicated combinatorial optimization
problem
• Multiple different goods with correlated
values are auctioned simultaneously.
– Complementary: PC and memory
– Substitutable: Dell or Lenovo
• By allowing bids on any combinations of
goods, the obtained social surplus/revenue
of the seller can increase.
• e.g., FCC spectrum right auctions
– winner determination problem, one instance
of a set packing problem
– NP-complete
– Various search techniques are introduced
• How to describe the preference of an
agent is also a research issue --- 2m
subsets for m goods
45
Vickrey-Clarke-Groves Mechanism
(VCG) aka Generalized Vickrey
46
An Example of the VCG
Setting: three agents (agent 1, 2, 3) are
bidding for two goods.
coffee cake both
Agent 1 $6
$0 $6
• Each agent declares its evaluation values
for subsets of goods.
• The goods are allocated so that the social
surplus is maximized.
• The payment of agent 1 is equal to the
decrease of the social surplus except agent
1, caused by the participation of agent 1.
• Satisfies incentive compatibility and Pareto
efficiency.
Agent 2 $0
Agent 3
Result:
•Agent
•Agent
•Agent
$0
$0
$8
$5
$5
1 gets the coffee, 3 gets the cake.
1 pays $8－$5＝$3.
3 pays $8－$6＝$2.
47
Incentive Compatibility of VCG
• Goods are allocated so that social surplus is maximized.
• An agent can maximize its utility when the social surplus
is maximized.
social surplus when 1 does not participate ($8)
1‘s utility ($3)
1’s payment
($3)
1‘s evaluation value
($6)
social surplus except 1
when 1 does participate ($5)
Social Surplus ($11)
Limitations of VCG
• The VCG can be used for mechanism design in
general. It has theoretically well-founded.
– satisfies (dominant-strategy) incentive compatibility,
achieves Pareto efficient allocation in a dominant
strategy equilibrium.
• However, it has several limitations.
– Computationally expensive (NP-hard), the outcome
is not in the core, vulnerable against the collusion
of losers, vulnerable against false-name bids.
– It cannot guarantee budget balance/non-negative in
more general settings (apart from auctions).
We still need to design new mechanism for each
application domain.
48
8
49
False-name Bids
(Yokoo, et al. GEB-2004)
False-name bids
bids
50
A Case where the VCG is Vulnerable
Setting: two agents (Agent 1, 2)
An agent submits
several bids under
fictitious names.
• Detecting falsename bids is
virtually impossible,
since identifying
each participant on
the Internet is very
difficult.
coffee cake both
Agent 1 $6
$5 $11
Agent 2 $0
$0
$8
When telling the truth:
•Agent 1 gets both goods.
•payment: $8 －$0 ＝$8
coffee cake both
Agent 1 $6
$0 $6
Agent 2 $0
$0
$8
Agent 3 $0
$5
$5
When agent 1 uses a falsename 3 and splits its bid:
•Agent 1 gets both goods.
•payment: $3+$2=$5
52
51
Mechanism Design
Outline
• A mechanism M can be
Input q
considered as a function, which
takes a type profile q as an
(valuations of bidders)
input, and returns the result of
social choice (e.g., an allocation
o and payment p)
• We want the function to
Mechanism M
optimize some objective (e.g.,
（Auction Mechanism）
social surplus, revenue), while
it satisfy some constraints
(e.g., incentive compatibility,
Social Choice
individual rationality).
（allocation o
• Traditionally, such a function is
hand-crafted.
and payment p）
•
•
•
•
•
Examples
Motivation (why game theory?)
Basic Terms
Single-item Auctions
Combinatorial Auctions, VCG, & Falsename Bids
• Automated Mechanism Design
Automated Mechanism Design
(Conitzer & Sandholm, UAI-2002)
•
•
•
•
• Formalize the mechanism design problem as an
optimization problem (mixed integer programming
problem, MIP).
– Introduce (many) decision variables, each of which
represents a possible combination between an input
and an output.
– Determine the values of these variables, so that the
objective (e.g., social surplus) is maximized under
given constraints (e.g., incentive compatibility).
• Once a problem is formalized as a MIP, an off-the-shelf
optimization package can solve the problem (e.g.,
CPLEX by ILOG/IBM)
•
•
•
•
53
Example: Auction with 2
bidders and 1 good
Bidder 1 and 2 bid for one good.
Possible valuation： $100 or $50
Possible assignment for each bidder： win or lose
Possible type profile：
(q1, q2) =
(100, 100), (100, 50), (50, 100) or (50, 50)
Possible allocation:
(o1, o2) = (win, lose), (lose, win) or (lose, lose)
Payment （non-negative real value）
For each input, a mechanism determines the
allocation and payment:
– (q1, q2) → (o1, o2) , (p1, p2)
54
9
Constraints in MIP Formalization
（Incentive Compatibility）
Variables in MIP Formalization
• A bidder’s utility does not increase by mis-reporting.
• Expected utility for bidder 1 (whose valuation is 100),
and reported types are (100, 50):
– 100*prob_(100, 50)_(win, lose)
+ 0*prob_(100, 50)_(lose, win)
+ 0*prob_(100, 50)_(lose, lose)
– p1_(100, 50)
• Expected utility when bidder 1 mis-reports her valuation
as 50:
– 100*prob_(50, 50)_(win, lose)
+ 0*prob_(50, 50)_(lose, win)
+ 0*prob_(50, 50)_(lose, lose)
– p1_(50, 50)
Variables for allocations：
Example：prob_(100, 50)_(win, lose)： represents the
probability that for an input/bid (100, 50), the
allocation will be (win, lose).
• If a mechanism is deterministic, its value is 0/1.
• In short, we enumerate all possible combinations
of an input and an output, and assign 0/1 values.
Variables for payments：
Example：p1_(100, 50): represents the payment of
bidder 1 for an input/bid (100, 50).
• Its value is a non-negative real value.
≧
Result of Automated
Mechanism Design (AMD)
Advantages of AMD
• It is very flexible:
– We can choose any constraints: dominant-strategy
incentive compatibility, Bayesian Nash IC, budget
balance/non-negative, individual rationality（ex post,
interim, ex ante）, ...
– We can choose any objective: social surplus, revenue,
competitive ratio, ...
– We can choose the type of a mechanism:
deterministic/non-deterministic, ...
• It obtains a custom-made mechanism, which is specialized
to the possible range of inputs.
Can we design a mechanism automatically? No need for
human mechanism designers?
• The objective (e.g., maximizing social
surplus/revenue) is represented as a
linear function.
• Variable values are determined so that
they optimize the objective, while
satisfying given constraints.
Social Surplus
Revenue
(100, 100) (win, lose), (100, 0) (win, lose), (100, 0)
(100, 50)
(win, lose), (50, 0)
(win, lose), (100, 0)
(50, 50)
(win, lose), (50, 0)
(lose, lose), (0, 0)
Vickrey?
Reserve price=100?
Limitations of AMD
How can We utilize AMD?
• Types must be discretized for combinatorial optimization
（e.g., a real value between 0 and 100 → an integer
between 0 and 100）.
• The problem size of MIP formalization will explode soon.
• Assume # of participants is n, # of possible types is t, then
# of possible type profiles is tn.
• Assume # of goods is m, then, # of possible allocations is
(n+1)m.
• # of variables for allocations becomes tn ・ (n+1)m.
• When n=4, m=3, t=9, it is more than 820 thousand.
• CPLEX cannot handle such a large-scale MIP instance.
We cannot find a good custom-made mechanism for a realistic
size → Is automated mechanism design totally useless?
• We can use AMD for obtaining some hints/intuition for
designing a general mechanism.
• Select representative input values (e.g., high, middle, low)
• Run AMD for a small number of participants.
• Examine the result/table, and extract some characteristic
parts (e.g., different from existing mechanisms).
• Extract a generalized rule for allocation/payment by
human intuition.
• Examine the extracted rule.
• If it does not work, change input values and run another
round.
Although we don’t mention about AMD in our paper, we
secretly use AMD (other computer scientists also?).
59
58
60
10
62
Concluding Remarks
Further Readings (books)
• Game theory becomes a prerequisite
subject for most AI and MAS
researchers.
• Designing a mechanism is designing a
function, but there exist constraints
between not only the current input and
output, but between possible outputs of
different inputs.
• If you like designing an algorithm, you
would like designing a mechanism also.
• Textbook on MAS including game theory:
– Yoav Shoham, Kevin Leyton-Brown, Multiagent
Systems: Algorithmic, Game-Theoretic, and Logical
Foundations, 2008
• Textbook on Auction:
– Vijay Krishna, Auction Theory, Academic Press, 2002.
• Textbook on Combinatorial Auctions
– Combinatorial Auctions, Peter Crampton, Yoav
Shoham, Richard Steinberg, eds., MIT Press, 2006.
• Mid-level textbooks on economics in general:
– Andreu Mas-Colell, Michael D. Whinston and Jerry R.
Green, Microeconomic Theory, Oxford University
Press, 1995.
11