COLOR TEST
COLOR TEST
COLOR TEST
COLOR TEST
Dueling Algorithms
NICOLE IMMORLICA, NORTHWESTERN UNIVERSITY
WITH A. TAUMAN KALAI, B. LUCIER, A. MOITRA,
A. POSTLEWAITE, AND M. TENNENHOLTZ
Social Contexts
Normal-form games:
Players choose strategies to maximize
expected von Neumann-Morgenstern utility.
Social context games [AKT’08]:
Players choose strategies to achieve particular
social status among peers.
Social Contexts
Ranking games [BFHS’08]:
Players choose strategies to achieve particular
payoff rank among peers.
Two-Player Ranking Games
Bob
Alice and Bob play game:
Alice
G
1 Alice beats Bob in G
RG payoff of Alice:
½ Alice ties Bob in G
0 Alice loses to Bob in G
Implicit Representations
Succinct games [FIKU’08]:
Payoff matrix represented by boolean circuit.
NE hard to solve or approximate.
Blotto games [B’21, GW’50, R’06, H’08]:
Distribute armies to battlefields.
Implicit Representations
Optimization duels [this work]:
Underlying game is optimization problem.
Goal is to optimize better than opponent.
Ranking Duel
A search engine is an algorithm that inputs
• set Ω = {1, 2, …, n} of items
• probabilities p1 + … + pn = 1 of each
and outputs a permutation π of Ω.
Monopolist objective: minimize Ei~p[π(i)].
Ranking Duel
Competitive objective: Let the expected score of
a ranking π versus a ranking π’ be
Pri~p[ π(i) < π’(i) ] + (½) Pri~p[ π(i) = π’(i) ].
Then objective is to output a π that maximizes
expected score given algorithm of opponent.
Optimizing a Search Engine
User searches for object
drawn according to
known probability dist.
0.19
0.16
0.27
0.07
Search:
0.22
pretty shape
1.
(27%)
2.
(22%)
3.
(19%)
4.
(16%)
5.
(09%)
6.
(07%)
0.09
Choosing a Search Engine
1. Search for “pretty shape”.
2. See which search engine ranks
my favorite shape higher.
3. Thereafter, use that one.
0.19
0.16
Search:
0.27
pretty shape
0.07
Search:
0.22
0.09
pretty shape
1.
(22%)
1.
(27%)
2.
(19%)
2.
(22%)
3.
(16%)
3.
(19%)
4.
(09%)
4.
(16%)
5.
(07%)
5.
(09%)
6.
(27%)
6.
(07%)
Questions
Can we efficiently compute an equilibrium of a
ranking duel?
How poorly does greedy perform in a
competitive setting?
What consequences does the duel have for
the searcher?
Optimization Problems as Duels
Ranking
Binary Search
Routing
Finish
?
?
?
?
Start
Hiring
Compression
?
?
Parking
?
Duel Framework
Finite feasible set X of strategies.
Prob. distribution p over states of nature Ω.
Objective cost c: Ω × X
R.
Monopolist: choose x to minimize Eω~p[cω(x)].
Duel Framework
1 if cω(x) < cω(x’)
v(x,x’) = Eω~p
0 if cω(x) > cω(x’)
½ if cω(x) = cω(x’)
1. Players select strategies x, x’ from X.
2. Nature selects state ω from Ω according to p.
3. Payoffs v(x,x’), (1-v(x,x’)) are realized.
Results: Computation
An LP-based technique to compute exact
equilibria,
A low-regret learning technique to compute
approximate equilibria,
… and a demonstration of these
techniques in our sample settings
Computing Exact Equilibria
Formulate game as bilinear duel:
1.
2.
3.
4.
Efficiently map strategies to points X in Rn.
Define constraints describing K=convex-hull(X).
Define payoff matrix M that computes values.
Maps points in K back to strategies in original
setting.
Bilinear Duels
If feasible strategies X are points in Rn, and
payoff v(x, x’) is xtMx’ for some M in Rnxn, then
maxv,x v s.t.
xtMx’ ≥ v for all x’ in X
x is in K (=convex-hull(X))
Exponential, but equivalent poly-sized LP.
Ranking Duel
Formulate game as bilinear duel:
1. Efficiently map strategies to points X in Rn.
X = set of permutation matrices
(entry xij indicates item i placed in position j)
2. Define constraints describing K=convex-hull(X).
K = set of doubly stochastic matrices
(entry yij = prob. item i placed in position j)
Ranking Duel
Formulate game as bilinear duel:
4. Design “rounding alg.” that maps points in K back
to strategies in original setting.
Birkhoff–von Neumann Theorem: Can efficiently
construct permutation basis for doubly stochastic
matrix (e.g., via matching).
Ranking Duel
Formulate game as bilinear duel:
3. Define payoff matrix M that computes values.
Ep,y,y’[v(x,x’)]
= ∑i p(i) ( ½ Pry,y’ [x i = x’i ] + Pry,y’ [x i > x’i ])
= ∑i p(i) (∑i yij ( ½ y’ij + ∑k>j y’ik ))
which is bilinear in y,y’ and so can be written ytMy’.
Ranking Duel
Result: Can reduce computation time to poly(n)
versus poly(n!) with standard LP approach.
Technique also applies to hiring duel and
binary search duel.
Compression Duel
data
(each with prob. p(.))
Goal: smaller compression (i.e., lower depth in tree).
Classical Algorithm
:
Repeatedly pair nodes with lowest
probability.
Compression Duel
Formulate game as bilinear duel:
1. Efficiently map strategies to points X in Rn.
X = subset of zero-one matrices*
(entry xij indicates item i placed at depth j)
2. Define constraints describing K=convex-hull(X).
K = subset of row-stochastic matrices*
(entry yij = prob. item i placed at depth j)
* Must correspond to depth profile of some binary tree!
Compression Duel
Formulate game as bilinear duel:
3. Define payoff matrix M that computes values.
Ep,y,y’[v(x,x’)] = ∑i p(i) (∑i yij ( ½ y’ij + ∑k>j y’ik ))
which is bilinear in y,y’ and so can be written ytMy’.
Compression Duel
Bilinear Form:
maxv,x v s.t.
xtMx’ ≥ v for all x’ in X
x is in K (=convex-hull(X))
Problems:
1. How to round points in K back to a random
binary tree with right depth profile?
2. How to succinctly express constraints
describing K?
Approximate Minimax
Defn. For any ε > 0, an approximate minimax
strategy guarantees payoff not worse than
best possible value minus ε.
Defn. For any ε > 0, an approximate best
response has payoff not worse than payoff of
best response minus ε.
Best-Response Oracle
Idea. Use approximate best-response oracle to
get approximate minimax strategies.
1. Low-regret learning: if x1,…,xT and x’1,…,x’T
have low regret, then ave. is approx minimax.
2. Follow expected leader: on round t+1, play
best-response to x1,…,xt to get low-regret.
Compression Best-Response
Multiple-choice Knapsack:
Given lists of items with values
and weights, pick one from
each list with max total value
and total weight at most one.
Compression Best-Response
Depth:
1
2
3
4
Compression Best-Response
(each with prob. p(.))
x’ in K
For j from 1..n, list of depth j:
v( ) = Pr[win at depth j | x’ ]
w( ) = 2-j
… Kraft inequality
Other Duels
1. Hiring duel: constraints defining Euclidean
subspace correspond to hiring probabilities.
2. Binary search duel: similar to hiring duel, but
constraints defining Euclidean subspace more
complex (must correspond to search trees).
3. Racing duel: seems computationally hard,
even though single-player problem easy.
Conclusion
• Every optimization problem has a duel.
• Classic solutions (and all deterministic
algorithms) can usually be badly beaten.
• Duel can be easier or harder to solve, and can
lead to inefficiencies.
OPEN QUESTION: effect of duel on the solution
to the optimization problem?