1 The Multiplicative Weights Update Method [1]
1.1 Setting
• At each time step t = 1, . . . , T :
– We must decide for one of m experts
– Nature causes event xt to occur
– Event x incurs penalty M(e, x) to expert e. We suffer same penalty as chosen expert
• Goal: Difference between sum of our penalties and that of best expert should vanish
1.2 Multiplicative Weights Update Algorithm
• Input: Penalties M(e, x), number of rounds T , error parameter ε ∈ (0, 21 ]
i) Initialize w1,e := 1 for all e
ii) For t := 1, . . . , T :
a) Randomly pick an expert according to distribution Dt
b) Observe nature choosing event xt
c) Multiplicative weights update: wt+1,e := wt,e · (1 − εM(e, xt )) for all e
Table 1: Symbols and Implicit Definitions
Symbol / Definition
Meaning
m
M(e, x)
we,t
Pm
Wt := e=1 wt,e
w
dt,e := Wt,et
Dt := (dt,1 , . . . , dt,m )
Number of experts
Penalty for expert e when event x occurs
Weight of expert e at time step t
Total weight of players at time step t
Probability of player e being picked at time step t
Probability distribution for picking expert at time step t
1.3 Guarantees
• Theorem: Define M(Dt , xt ) := Ee∼Dt [M(e, xt )]. Suppose all penalties M(e, x) ∈ [−1, 1].
Then, for any expert e:
T
X
t=1
M(Dt , xt ) ≤ (1 + ε)
X
M(e, xt ) + (1 − ε)
≥0
X
M(e, xt ) +
<0
ln m
ε
Subscripts ≥ 0 and < 0 refer to rounds t with M(e, xt ) ≥ 0 and < 0 respectively. Remark:
Also holds when M(e, x) contains rewards instead of penalties.
• Note first:
WT +1 =
m
X
wT,e · (1 − εM(e, xT )) = WT · (1 − εM(DT , xT )) = · · · = n ·
e=1
T
Y
t=1
1
(1 − εM(Dt , xt ))
• We show inequality after multiplying with (−ε) and applying exp(·):
T
T
X
Y
exp −ε
M(Dt , xt ) ≥
(1 − εM(Dt , xt ))
t=1
[exp(x) ≥ 1 + x]
t=1
=
wT +1,e
1Y
WT +1
(1 − εM(e, xt ))
≥
=
n
n
n t
P
P
1
· (1 − ε) ≥0 M(e,xt ) · (1 + ε)− <0 M(e,xt )
n
1 − εx ≥ (1 − ε)x for x ∈ [0, 1]
1 − εx ≥ (1 + ε)−x for x ∈ [−1, 0]
X
M(e, xt )
= exp(− ln n) · exp ln(1 − ε)
≥
≥0
X
· exp − ln(1 + ε)
M(e, xt )
<0
h
X
X
ln n i
≥ exp −ε (1 + ε)
M(e, xt ) + (1 − ε)
M(e, xt ) +
ε
<0
≥0
ln(1 + x) ≥ x(1 − x) for x ≥ −0.68 . . .
• Assume from now on that all M(e, x) ∈ [−`, %] where 0 ≤ ` ≤ %.
• Corollary: The previous bound remains valid (simply scale all M(e, x) down by %) except
that the ln-term gets an additional factor %.
• Note that
(1 − ε)
X
M(e, xt ) ≤ (1 + ε)
X
<0
M(e, xt ) + 2ε`T
<0
and
P
(1 + ε)
t M(e, xt )
T
P
≤
t M(e, xt )
T
+ ε%
δ
• Corollary: Let δ > 0. With ε = 4%
(w.l.o.g., ε ≤ 21 ), we have after T =
every expert e:
P
P
M(e, xt )
% ln n
t M(Dt , xt )
≤ t
+
+ 2ε` + ε%
T
T
| εT {z
}
16%2 ln n
δ2
rounds, for
=δ/4+(δ`)/(2%)+δ/4≤δ
When ` = 0, sufficient to choose ε =
δ
2%
and T =
4%2 ln n
.
δ2
1.4 Examples
• Kalai and Vempala [3]: Event xt = cost(·, t) ∈ [0, 1]m and penalty M(e, xt ) = cost(e, t). Note:
Equivalent guarantees as “follow the perturbed leader”.
• Weighted-Majority Algorithm [4] is a deterministic variant: Event xt ∈ {0, 1} and M(e, xt ) =
1 − xt . Derandomized in that the majority gets all probability.
2
2 Applications
2.1 The Plotkin-Shmoys-Tardos Framework [5]
• Multiplicative weights method to determine approximate feasibility of linear program
Ax ≥ (1 . . . 1)T ,
x∈P
where A ∈ Rm×n and P ⊆ Rn is a convex set. Note: Optimization via binary search.
• Input: Error parameter δ > 0, A, P , parameters 0 ≤ ` ≤ %, a subroutine
• Output: x ∈ P with Ae x ≥ 1 − δ for all rows Ae ∈ R1×n , or prove system infeasible
• Intuition: Expert = constraint, nature = subroutine, event = vector in P
• The subroutine (“oracle”):
– Input: Probability distribution D = (d1 , . . . , dm )
P
– Output: Vector x ∈ P with m
e=1 de Ae x ≥ 1, or (correctly) say “infeasible”
– For all points x possibly returned assume for all constraints e that Ae x ∈ [−`, %]
• Adapt multiplicative weights update method as follows:
– M(e, x) := Ae x. Rationale: Reduce weight of well-satisfied constraints → similar in
spirit to Langrangian relaxation
– If subroutine ever returns
P “infeasible” return that system is infeasible: If x with Ax ≥
(1 . . . 1)T existed, then m
e=1 de Ae x ≥ 1 in particular.
1 P
– Otherwise, return x := T t xt . With ε, T as in corollary, we have for all constraints e0 :
P Pm
( e=1 dt,e Ae xt − 1)
0≤ t
[all xt were feasible]
T
P
P
M(Dt , xt )
Ae0 xt
= t
−1≤ t
+ δ − 1 = Ae0 x − (1 − δ)
[Theorem]
T
T
2.2 Multicommodity Flow
• Maximum multicommodity flow (let P be union of all paths for every commodity):
X
max
xp
p∈P
s.t.
1 X
xp ≤ 1 ∀e ∈ E
ce p3e
• Want to find flow x of volume X (if feasible)
d
• Define modified edge costs at time step t by c0t,e := ct,e
e
P
• Convex set P = {x ∈ RP
Expert = constraint/edge. Subroutine: At time
≥0 |
p∈P xp = X}.
P
P
P
P
step t, finds flow (= event) x ∈ P so that e dt,e c1e p3e xp = p xp e∈p c0t,e ≤ 1
→ minimized by putting all flow on shortest path w.r.t. edge costs c0t,e
P
• For x ∈ P , we can only guarantee Ae x = c1e p3e xp ≤ minXe ce =: %. Hence, number of
rounds T only pseudo-polynomial.
3
2.3 The Garg-Könemann Algorithm for Multicommodity Flow [2]
• Expert = edge, event = shortest path pt w.r.t. edge costs c0t,e
• Denote by c∗t the smallest capacity on path pt . Rewards M(e, pt ) :=
c∗t
ce
if e ∈ pt and 0 else
• Algorithm: In every round t, algorithm adds flow c∗t on path pt . Suppose fe contains the
current total amount of flow added to edge e. Then, the algorithm terminates after the round
in which fcee ≥ lnε2m for some edge e. Return flow (maxe { fcee })−1 · f .
• Let F = |f | total flow, g some maximum flow, G = |g|. Immediately before return:
P
X c∗ X
c∗t e∈pt c0t,e
F
t
P
P
=
≥
[pt is shortest path w.r.t. c0t ]
0
G
G
g
c
q
t,e
q∈P
e∈q
t
Pt
hP
i
X c∗t e∈p c0t,e
X X
gq
P
Pt gq ≥
=
c0t,e
c∗t
q3e ce ≤ 1, for g is feasible
e dt,e
q3e ce
e∈p
t
t
t
[D is probability distribution]
=
XX
t
dt,e M(e, pt )
e∈E
nX
o ln m
M(e, pt ) −
M(Dt , pt ) ≥ (1 − ε) max
e
ε
t
t
n f o ln m
nf o
e
e
= (1 − ε) max
−
≥ (1 − 2ε) max
e
e
ce
ε
ce
=
X
[Theorem with signs reversed]
[since
ln m
ε
≤ ε maxe
n o
fe
ce ]
• Running time: Each edge can be minimum capacity edge at most d lnε2m e times, therefore at
most m · d lnε2m e iterations.
References
[1] Sanjeev Arora, Elad Hazan, and Satyen Kale. The multiplicative weights update method: A meta
algorithm and its applications. Survey Paper, 2008. URL http://www.cs.princeton.edu/~satyen/
papers/MW_SDP.pdf.
[2] Naveen Garg and Jochen Könemann. Faster and simpler algorithms for multicommodity flow and
other fractional packing problems. SIAM Journal on Computing, 37(2):630–652, 2007. DOI: 10.1137/
S0097539704446232.
[3] Adam Kalai and Santosh Vempala. Efficient algorithms for online decision problems. Journal of Computer
and System Sciences, 71(3):291–307, 2005. DOI: 10.1016/j.jcss.2004.10.016.
[4] Nick Littlestone and Manfred K. Warmuth. The weighted majority algorithm. Information and Computation, 108(2):212–261, 1994. DOI: 10.1006/inco.1994.1009.
[5] Serge A. Plotkin, David B. Shmoys, and Éva Tardos. Fast approximation algorithms for fractional
packing and covering problems. Mathematics of Operations Research, 20(2):257–301, 1995. DOI: 10.
1287/moor.20.2.257.
4