How the Experts Algorithm Can
Help Solve LPs Online
Marco Molinaro
TU Delft
Anupam Gupta
Carnegie Mellon University
Primal-dual algo for online random order problems
using
black-box online learning to compute duals
Applications: (optimal) gen load-balancing, packing/covering LPs
GENERALIZED LOAD-BALANCING
•
•
•
•
𝑚 machines
Job: 𝑚 × 𝑘 matrix 𝐴1, each column is a processing option
Algorithm chooses 𝑝1 in simplex: fractional choice of processing
Load vector 𝐴1𝑝1…
• Goal: Minimize makespan
.9 .1 .2 .5 .3 .2
.3 .7 .1 .4 .4 0
.8
.4 .2 .1 .1 .7 0
.8 .4 .6 .3 .2 0
+
𝐴2
𝑡 𝑡
𝑡𝐴 𝑝 ∞
𝑝2
+…
𝑚
∞
1
𝑖 𝑝𝑖
= 1 , 𝑝1 ≥ 0
GENERALIZED LOAD-BALANCING
Random permutation model
• Collection of matrices, unknown, adversarial
• Job: matrix 𝑨1 sampled without replacement
• Algorithm: chooses 𝒑𝟏 in simplex (random)…
• Goal: minimize makespan
𝑨𝟏
𝒑𝟏
+
𝑨𝟐
Entries of 𝑨𝒕
in [0,1]
𝑡 𝑡
𝑡𝑨 𝒑 ∞
𝒑𝟐
+…
∞
GENERALIZED LOAD-BALANCING
Offline optimum: OPT = min
𝑡
𝒑 ∈Δ
𝑡 𝑡
𝑡𝑨 𝒑 ∞
Want: (1 + 𝜖)-competitive ratio (with high probability)
Alg ≤ 1 + 𝜖 OPT
In iid model: as long as OPT ≥ Ω
log 𝑚
𝜖2
[Devanur et al. 11]
– Primal-dual, exponential updates of dual
– Asked if techniques worked for random permutation model. How to
handle dependencies?
GENERALIZED LOAD-BALANCING
Thm: In the random permutation model, we get with high prob
log 𝑚
(1 + 𝜖)-approximation as long as OPT ≥ Ω
2
𝜖
• Primal-dual, using black-box online linear optimization for dual
• Abstracts exponential update of Devanur et al., explains why works
• Abstraction allow us handle dependencies in random permutation
ALGORITHM
• Idea: Capture ||. ||∞ in a linear way
Obj func:
𝑡 𝑡
𝑡𝑨 𝒑 ∞
= max 𝒘.
𝒘∈Δ
𝒕 𝒕
𝑡𝑨 𝒑
ALGORITHM
• Idea: Capture ||. ||∞ in a linear way
Obj func:
𝑡 𝑡
𝑡𝑨 𝒑 ∞
= max
𝒘∈Δ
𝒕 𝒕
𝑡 𝒘 (𝑨 𝒑 )
• Compute both 𝒑𝒕’s and “right” 𝒘 in online way
• Algorithm: at time t
– (primal step) chooses 𝒑𝑡 ∈ Δ to minimize (𝒘𝑡 𝑨𝑡 ). 𝒑𝑡
– (dual step) compute 𝒘𝑡+1 trying to maximize 𝒘𝑡+1 (𝑨𝑡+1 𝒑𝑡+1 )
How? 𝑨𝑡+1 and 𝒑𝑡+1 unknown
Online linear optimization
ONLINE LINEAR OPTIMIZATION
• Setup:
– First, algo chooses vector 𝑤 𝑡 in Δ
– Then, adversary chooses vector 𝑜 𝑡 in 0,1
– Reward: 𝑤 𝑡 𝑜𝑡
• Goal: maximize reward
𝑡𝑤
𝑚
𝑡 𝑜𝑡
• Algorithms with good regret bound [Arora et al. 12]
≥ 1 − 𝜖 max
𝑤∈Δ
𝑡
𝑤. 𝑜 𝑡
=
𝑡
𝑤 𝑡 𝑜𝑡
||
𝑡 ||
𝑜
∞
𝑡
log 𝑚
−
𝜖
ALGORITHM
• Idea: Capture ||. ||∞ in a linear way
Obj func:
𝑡 𝑡
𝑡𝑨 𝒑 ∞
= max
𝒘∈Δ
𝒕 𝒕
𝑡 𝒘 (𝑨 𝒑 )
• Compute both 𝒑𝒕’s and “right” 𝒘 in online way
• Algorithm: at time t
– (primal step) chooses 𝒑𝑡 ∈ Δ to minimize (𝒘𝑡 𝑨𝑡 ). 𝒑𝑡
– (dual step) compute 𝒘𝑡+1 trying to maximize 𝒘𝑡+1 (𝑨𝑡+1 𝒑𝑡+1 )
ALGORITHM
• Idea: Capture ||. ||∞ in a linear way
Obj func:
𝑡 𝑡
𝑡𝑨 𝒑 ∞
= max
𝒘∈Δ
𝒕 𝒕
𝑡 𝒘 (𝑨 𝒑 )
• Compute both 𝒑𝒕’s and “right” 𝒘 in online way
• Algorithm: at time t
– (primal step) chooses 𝒑𝑡 ∈ Δ to minimize (𝒘𝑡 𝑨𝑡 ). 𝒑𝑡
– (dual step) compute 𝒘𝑡+1 trying to maximize 𝒘𝑡+1 (𝑨𝑡+1 𝒑𝑡+1 ) via
online linear optimization
ALGORITHM
• Idea: Capture ||. ||∞ in a linear way
Obj func:
𝑡 𝑡
𝑡𝑨 𝒑 ∞
= max
𝒘∈Δ
𝒕 𝒕
𝑡 𝒘 (𝑨 𝒑 )
• Compute both 𝒑𝒕’s and “right” 𝒘 in online way
• Algorithm: For t=1..n/2
– (primal step) chooses 𝒑𝑡 ∈ Δ to minimize (𝒘𝑡 𝑨𝑡 ). 𝒑𝑡
– (dual step) compute 𝒘𝑡+1 trying to maximize 𝒘𝑡+1 (𝑨𝑡+1 𝒑𝑡+1 ) via
online linear optimization
At time n/2+1 start fresh, running from t=n/2+1..n
ANALYSIS (1/3)
• Algorithm: at time t
– (primal step) chooses 𝒑𝑡 ∈ Δ to minimize (𝒘𝑡 𝑨𝑡 ). 𝒑𝑡
– (dual step) compute 𝒘𝑡+1 via online lin optimization with adv. vectors {𝑨𝑗 𝒑𝑗 }𝑗≤𝑡
Want:
OBS:
𝑡 𝑡
𝑡𝑨
𝒑
∞
𝑡 𝑡
𝑡=1..𝑛/2 𝑨
Suffices:
≤ (1 + 𝜖) OPT whp,
if OPT ≥ Ω
𝒑 has same distribution as
𝑡 𝒑𝑡
𝑨
𝑡=1..𝑛/2
∞
𝑛
2
𝑡= +1..𝑛
log 𝑚
𝜖2
𝑡 𝑡
𝑨𝒑
≤ (1 + 𝜖) OPT/2 whp
Let 𝒑𝑡 be the optimal solution. Then
||
𝑡≤𝑛/2
𝑨𝑡 𝒑𝑡 ||
∞
<
~
𝑡≤𝑛.2
(dual) guarantee of
online lin optimization
𝒘𝑡 𝑨𝑡 𝒑𝑡
≤
𝑡≤𝑛/2
𝒘𝑡 𝑨𝑡 𝒑𝑡
?
< OPT/2
~
(primal) greedy wrt
duals
ANALYSIS (2/3): IN EXPECTATION
• Show in expectation: E[
𝑡 𝑡
𝒘
𝒐 ] ≤ 1 + 𝜖 OPT/2
𝑡≤𝑛/2
𝒐𝑡 = 𝑨𝑡 𝒑𝑡
• Ok if 𝒘𝑡 and 𝒐𝒕 are independent
• Issue: are not independent
𝒐𝒕
𝑨𝟏 , … , 𝑨𝑡−1
𝒘𝑡
in iid
Lemma: (low dependence) With high probability, we have for all 𝑡 ≤
𝑛/2
𝟏+𝟖𝟎 𝝐 OPT
𝐄 𝒐𝑡𝒕 |𝑨𝑡<𝒕 𝑡 ≤
.𝟏
n 𝑡 . 𝐄[𝑨𝑡 𝒑𝑡 ]
𝐄𝒘 𝑨 𝒑 ≈𝐄𝒘
• Uses a maximal Bernstein inequality to take care of all time steps
ANALYSIS (2/3): IN EXPECTATION
• Maximal Bernstein
ANALYSIS (3/3): HIGH PROB.
• Now with high probability:
𝑡 𝑡
𝑡≤𝑛/2 𝒘 𝒐
≤ 1 + 𝜖 OPT/2
~
• Issue: terms 𝒘𝑡 𝒐𝑡 are not independent… martingale concentration
• The sequence
is a martingale difference
• Freedman’s inequality
• Use bound 𝐄 𝒐𝒕|𝑨<𝒕 ≤
||
𝑡 𝑡
𝑡≤𝑛/2 𝑨 𝒑 ||∞ <
~
𝟏+𝟖𝟎 𝝐 OPT
n
𝑡 𝑡 𝑡
𝑡≤𝑛.2 𝒘 𝑨 𝒑
to control centering term, pred. var.
≤
𝑡 𝑡 𝑡
𝑡≤𝑛/2 𝒘 𝑨 𝒑
< OPT/2
~
ONLINE PACKING/COVERING LP
•
•
•
•
Packing/covering: non-negative data
Number of columns, right-hand-side known upfront
Columns (+coef in objective) come one by one, in random order
Goal: (1 − 𝜖) feasible solution, maximize total reward
max 𝑐𝑥
𝐴
𝑥
≤
𝑏
≥
𝑥 ∈ [0,1]𝑛
• Packing-only is well-studied [DH 09, Feldman et al. 10, MR 12, Kesselheim et al.
14, Agrawal et al. 14]
• No general results for packing/covering
ONLINE PACKING/COVERING LP
Thm: We get (1 − 𝜖)-approximation as long as
𝑏𝑖 ≥
log 𝑚
max 𝑎𝑖𝑗
𝜖2
𝑗
• Optimal guarantee for packing (indep Kesselheim et al. 14, Devanur-Agrawal 15)
• First general result for packing/covering (but requires technical assumption)
ONLINE PACKING/COVERING LP
• Idea: reduce online LP to gen load-balancing
• Elements
– Handle slightly negative loads in gen load balancing (well-bounded
instances)
– Simple reduction to gen load balancing assuming knows OPT
– Estimate OPT: pick out very valuable items, sampling + chernoff on
rest
• Cannot “scale down” solution to get feasibility
– Crucially used in Kesselheim et al. 14, Devanur-Agrawal 15…
CONCLUSION
Solving random order problems using duals from black-box online
linear optimization
Clean abstraction, allows to handle dependencies in random perm.
– Separates “optimization” and “probability” parts
Applications
– Generalized load-balancing
– (optimal) guarantees for packing/covering LPs
Open questions
1.
2.
3.
Seems very flexible. Apply techniques to other problems?
More general, realistic models
Remove technical assumption in packing/covering, or prove LB (minimax?)
THANK YOU!