Introduction
Multi-agent Network flows
Nash Equilibrium maximizing the flow
Conclusion
Nash Equilibria for multi-agent network flow
with controllable capacities
Nadia CHAABANE(1)
Cyril Briand (1)
(1) LAAS-CNRS,
Marie-José Huguet(1)
Université de Toulouse
ROADEF’15 - 25-27 February 2015
.
.
.
.
.
.
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
.
.
.
.
.
.
.
.
.
1 / 27
.
Introduction
Multi-agent Network flows
Nash Equilibrium maximizing the flow
Conclusion
Outline
1
Introduction
2
Multi-agent Network flows
Problem statement
Example
3
Nash Equilibrium maximizing the flow
Characterization of Nash equilibria
MILP formulation
First experiments
4
Conclusion
.
.
.
.
.
.
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
.
.
.
.
.
.
.
.
.
2 / 27
.
Introduction
Multi-agent Network flows
Nash Equilibrium maximizing the flow
Conclusion
Outline
1
Introduction
2
Multi-agent Network flows
Problem statement
Example
3
Nash Equilibrium maximizing the flow
Characterization of Nash equilibria
MILP formulation
First experiments
4
Conclusion
.
.
.
.
.
.
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
.
.
.
.
.
.
.
.
.
3 / 27
.
Introduction
Multi-agent Network flows
Nash Equilibrium maximizing the flow
Conclusion
Introduction
Optimization in a multi-agent context
A set of agents is involved in a common decision-making
process.
Every agent controls its own decision variables (strategy)
and is interested in maximizing its profit.
The agents’ profit can depend on the strategies of the
other agents.
⇒ Which strategy to adopt in order to fulfill the social goal,
provided agents’ objectives are satisfied ?
.
.
.
.
.
.
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
.
.
.
.
.
.
.
.
.
4 / 27
.
Introduction
Multi-agent Network flows
Nash Equilibrium maximizing the flow
Conclusion
Strategy requirement
Efficiency
Only non-dominated strategies are of interest because every agent
wants to maximize its profit.
Multi-objective optimization : Pareto optima
Stability
Given a strategy, no agent should be able to increase its profit by itself,
while decreasing the profit of others.
Game theory : Nash equilibria.
.
.
.
.
.
.
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
.
.
.
.
.
.
.
.
.
5 / 27
.
Introduction
Multi-agent Network flows
Nash Equilibrium maximizing the flow
Conclusion
Problem statement
Example
Outline
1
Introduction
2
Multi-agent Network flows
Problem statement
Example
3
Nash Equilibrium maximizing the flow
Characterization of Nash equilibria
MILP formulation
First experiments
4
Conclusion
.
.
.
.
.
.
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
.
.
.
.
.
.
.
.
.
6 / 27
.
Introduction
Multi-agent Network flows
Nash Equilibrium maximizing the flow
Conclusion
Problem statement
Example
Multi-agent Network with Controllable Capacities
Definition
Multi-agent Network < G, A, Q, Q, C, π, W > :
G = (V , E) is a Network :
each arc having its controllable capacity and receiving a
flow.
A = {A1 , . . . , Au , . . . , Am } a set of m agents ;
Each arc (i, j) belongs to exactly one agent ;
Eu set of arcs belonging to agent Au ;
Q = {q i,j }(i,j)∈E et Q = {q i,j }(i,j)∈E referred as the vectors
of normal and maximum arc capacities.
qi,j ∈ [q i,j , q i,j ] is the capacity of arc (i, j).
.
.
.
.
.
.
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
.
.
.
.
.
.
.
.
.
7 / 27
.
Introduction
Multi-agent Network flows
Nash Equilibrium maximizing the flow
Conclusion
Problem statement
Example
Multi-agent Network with Controllable Capacities
Definition (2)
C = {ci,j } is the vector of unitary costs.
The total cost∑incurred by agent Au for increasing its arc
capacities is (i,j)∈Eu ci,j (qi,j − q i,j )
π the reward given by a final customer per unit of flow
circulating in the Network.
W = {wu } sharing policy of rewards among the agents.
The total reward for agent Au is wu × π × (F − F )
F circulating flow in the Network.
fi,j flow circulating on arc (i, j) holds fi,j ≤ qi,j where
qi,j ∈ [q i,j , q i,j ]
.
.
.
.
.
.
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
.
.
.
.
.
.
.
.
.
8 / 27
.
Introduction
Multi-agent Network flows
Nash Equilibrium maximizing the flow
Conclusion
Problem statement
Example
Multi-agent Network with Controllable Capacities
Social goal
The customer wants to maximize the flow in the Network and give a
reward to agents.
Strategies of the agents
Individual strategy : capacities chosen by each agent for its arcs to
maximize his profit.
Zu (S) = wu × π × (F − F ) −
∑
ci,j (qi,j − q i,j )
(i,j)∈Eu
Type of the game
Non-cooperative game between the agents where each of them aims to
maximize its profit.
.
.
.
.
.
.
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
.
.
.
.
.
.
.
.
.
9 / 27
.
Introduction
Multi-agent Network flows
Nash Equilibrium maximizing the flow
Conclusion
Problem statement
Example
Example
Example of MA-MCF Network
Network G(V , U) with controllable capacities
Two agents : Blue (AB ) and Green (AG )
Reward and sharing policy : π = 120 and wB = wG =
.
.
.
.
.
1
2
. . . .
. . . .
F IGURE: Example of MA-MCF Network with two agents
.
. . . .
. . . .
. . . .
. . . .
.
.
.
.
.
.
.
.
.
10 / 27
.
Introduction
Multi-agent Network flows
Nash Equilibrium maximizing the flow
Conclusion
Problem statement
Example
Example (2)
Increasing the flow
Find an increasing path P in the residual graph such that costu (P) < wu × π
for all Au , with :
∑
∑
costu (P) = (i,j)∈P + ∩Eu ci,j − (i,j)∈P − ∩Eu ci,j
F IGURE: Example of MA-MCF Network with two agents
.
.
.
.
.
.
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
.
.
.
.
.
.
.
.
.
11 / 27
.
Introduction
Multi-agent Network flows
Nash Equilibrium maximizing the flow
Conclusion
Problem statement
Example
Example (3)
Decreasing the flow
Find a decreasing path P in the residual graph such that savu (P) > wu × π
for all Au , with :
∑
∑
savu (P) = (i,j)∈P − ∩E ci,j − (i,j)∈P + ∩Eu ci,j
u
F IGURE: Example of MA-MCF Network with two agents
.
.
.
.
.
.
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
.
.
.
.
.
.
.
.
.
11 / 27
.
Introduction
Multi-agent Network flows
Nash Equilibrium maximizing the flow
Conclusion
Problem statement
Example
Example (4)
Efficiency Vs. Stability
Strategy
S1
Flow
2
Reward
240
costG
80
costB
80
ZG
40
ZB
40
F IGURE: Strategy S1
.
.
.
.
.
.
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
.
.
.
.
.
.
.
.
.
12 / 27
.
Introduction
Multi-agent Network flows
Nash Equilibrium maximizing the flow
Conclusion
Problem statement
Example
Example (5)
Efficiency Vs. Stability
Strategy
S1
Flow
2
Efficiency Vs. Stability
Reward
240
costG
80
costB
80
ZG
40
ZB
40
Strategy S1
S1 maximizes the flow.
Is S1 Nash-stable ?
.
.
.
.
.
.
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
.
.
.
.
.
.
.
.
.
12 / 27
.
Introduction
Multi-agent Network flows
Nash Equilibrium maximizing the flow
Conclusion
Problem statement
Example
Example (6)
Efficiency Vs. Stability
Strategy
S1
Flow
2
Reward
240
costG
80
costB
80
ZG
40
ZB
40
F IGURE: Residual graph corresponding to the strategy S1
.
.
.
.
.
.
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
.
.
.
.
.
.
.
.
.
13 / 27
.
Introduction
Multi-agent Network flows
Nash Equilibrium maximizing the flow
Conclusion
Problem statement
Example
Example (7)
Efficiency Vs. Stability
Strategy
S1
Flow
2
Reward
240
costG
80
costB
80
ZG
40
ZB
40
F IGURE: Profitable decreasing path for agent AG
.
.
.
.
.
.
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
.
.
.
.
.
.
.
.
.
13 / 27
.
Introduction
Multi-agent Network flows
Nash Equilibrium maximizing the flow
Conclusion
Problem statement
Example
Example (8)
Agent AG can improve its own profit by modifying unilaterally its strategy.
Strategy
S1′
Flow
1
Reward
120
costG
10
costB
80
.
ZG
50
.
.
.
.
.
. . . .
. . . .
ZB
-20
. . . .
. . . .
. . . .
. . . .
.
.
.
.
.
.
.
.
.
14 / 27
.
Introduction
Multi-agent Network flows
Nash Equilibrium maximizing the flow
Conclusion
Characterization of Nash equilibria
MILP formulation
First experiments
Outline
1
Introduction
2
Multi-agent Network flows
Problem statement
Example
3
Nash Equilibrium maximizing the flow
Characterization of Nash equilibria
MILP formulation
First experiments
4
Conclusion
.
.
.
.
.
.
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
.
.
.
.
.
.
.
.
.
15 / 27
.
Introduction
Multi-agent Network flows
Nash Equilibrium maximizing the flow
Conclusion
Characterization of Nash equilibria
MILP formulation
First experiments
Characterization of Nash equilibrium
Nash equilibrium
A non-poor strategy S is a Nash equilibrium if and only if
there is no profitable path P such that :
I
∀Au ∈ A, ∀P ∈ P such that (i, j) ∈ Eu
costu (P) > wu × π
I
∀Au ∈ A, ∀P ∈ P
savu (P) < wu × π
.
.
.
.
.
.
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
.
.
.
.
.
.
.
.
.
16 / 27
.
Introduction
Multi-agent Network flows
Nash Equilibrium maximizing the flow
Conclusion
Characterization of Nash equilibria
MILP formulation
First experiments
Mathematical formulation
Finding a NE maximizing the flow
Max
s.t.
(i)
F
∑
(i,j)∈E fi,j
−
∑
(j,i)∈Er fj,i

 0 ∀i ̸= s, t
F ,i=s
=
, ∀i ∈ V

−F , i = t
(ii)
fi,j ≤ qi,j , ∀(i, j) ∈ E
(iii)
q i,j ≤ qi,j ≤ q i,j , ∀(i, j) ∈ E
(iv )
savu (P) < wu × π, ∀P ∈ Gf
Stability constraints
fi,j ≥ 0, ∀(i, j) ∈ E
.
.
.
.
.
.
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
.
.
.
.
.
.
.
.
.
17 / 27
.
Introduction
Multi-agent Network flows
Nash Equilibrium maximizing the flow
Conclusion
Characterization of Nash equilibria
MILP formulation
First experiments
Mathematical formulation
How to formulate the stability constraints ?
Linearize the Nash stability constraints as constraints of MILP.
(iv ) savu (P) < wu × π, ∀P ∈ Gf
Stability constraints
All path have to be computed ?
.
.
.
.
.
.
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
.
.
.
.
.
.
.
.
.
18 / 27
.
Introduction
Multi-agent Network flows
Nash Equilibrium maximizing the flow
Conclusion
Characterization of Nash equilibria
MILP formulation
First experiments
Mathematical formulation
How to formulate the stability constraints ?
Linearize the Nash stability constraints as constraints of MILP.
(iv ) savu (P) < wu × π, ∀P ∈ Gf
Stability constraints
All path have to be computed ?
Constraint reformulation
Identification of path P ∈ P(S) having maximum savu (P).
savu (P) < wu ×π, ∀P ∈ P(S) ⇒ max∀P∈P savu (P) < wu ×π
Computing the longest path for a given strategy using
primal/dual constraints.
.
.
.
.
.
.
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
.
.
.
.
.
.
.
.
.
18 / 27
.
Introduction
Multi-agent Network flows
Nash Equilibrium maximizing the flow
Conclusion
Characterization of Nash equilibria
MILP formulation
First experiments
Primal/dual constraints
Primal : longest path
Min
s.t.
(i)
(ii)
u
tn+1
tju − tiu ≥ δFi,j,u , ∀(i, j) ∈ E, ∀Au ∈ A
tiu − tju ≥ δBj,i,u , ∀(i, j) ∈ E, ∀Au ∈ A
tiu ≥ 0, ∀i ∈ V
Dual
Max
s.t.
(i)
∑
(i,j)∈E
∑
φui,j × δFi,j,u +
(i,j)∈Er
φui,j × δBj,i,u

 0 ∀i ̸= s, t
−1 , i = s , ∀i ∈ V , ∀Au ∈ A
−
=

1 , i = t. . . . . . . . . . . . . . . .
. .
. . . . . . . . . . . . . .
.
∈ {0, 1} ∀(i, j) ∈ E, ∀Au ∈ A
u
(i,j)∈E∪Er φi,j
φui,j
∑
∑
u
(i,j)∈E∪Er φj,i
.
.
.
.
.
.
19 / 27
.
Introduction
Multi-agent Network flows
Nash Equilibrium maximizing the flow
Conclusion
Characterization of Nash equilibria
MILP formulation
First experiments
Primal/dual constraints
At optimality, the value of a path in the residual graph is equal to the profit of
unitary decreasing the network flow (Complementary Slackness Theorem)
∑ u
∑ u
u
tn+1
≥
φi,j × δFi,j,u +
φi,j × δBj,i,u , ∀Au ∈ A
(i,j)∈E
(i,j)∈Er
u
The profit of the best decreasing path (represented by tn+1
) should be non
profitable for agent Au (stability) :
u
< wu × π, ∀Au ∈ A
tn+1
.
.
.
.
.
.
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
.
.
.
.
.
.
.
.
.
20 / 27
.
Introduction
Multi-agent Network flows
Nash Equilibrium maximizing the flow
Conclusion
Characterization of Nash equilibria
MILP formulation
First experiments
MILP formulation
Linearization of of the CST constraints
The right-hand-side of the CST constraints contains multiplication of two
variables :
φui,j binary variables and δFi,j,u integer variables
The multiplication can be linearized by introducing variables
ψ1i,j,u = φui,j × δFi,j,u
The set of inequalities (i) − (iv ) will ensure the linearization :
(i)
(ii)
(iii)
(iv )
ψ1i,j,u
ψ1i,j,u
ψ1i,j,u
ψ1i,j,u
≥ 0, ∀(i, j) ∈ E, ∀Au ∈ A
≤ φui,j × M, ∀(i, j) ∈ E, ∀Au ∈ A
≤ −δFi,j,u , ∀(i, j) ∈ E, ∀Au ∈ A
≥ M(φui,j − 1) − δFi,j,u , ∀(i, j) ∈ E, ∀Au ∈ A
.
.
.
.
.
.
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
.
.
.
.
.
.
.
.
.
21 / 27
.
Introduction
Multi-agent Network flows
Nash Equilibrium maximizing the flow
Conclusion
Characterization of Nash equilibria
MILP formulation
First experiments
First experiments
The algorithm performance was evaluated on a PC with Linux Ubuntu
server 12.04, 8 Go of RAM.
The MILP model was implemented using the C++ API for GUROBI
Optimization 6.0.0.
No standard benchmark instances exist for our problem
Benchmark instances
The instances were built up using the RanGen1 generator.
Consider 6 sets of benchmark instances with n ∈ {50, 100} and
m ∈ {2, 3, 5}.
For each problem size, 100 instances were generated.
.
.
.
.
.
.
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
.
.
.
.
.
.
.
.
.
22 / 27
.
Introduction
Multi-agent Network flows
Nash Equilibrium maximizing the flow
Conclusion
Characterization of Nash equilibria
MILP formulation
First experiments
First experiments
The reward π equals α × m × LPath, where LPath is the
value of the longest path from s to t in terms of costs and α
is a ratio.
Equal-sharing policy is considered.
To compare the solution and to analyze the influence of the
number of agents and the value of α, we varied the value
of α ∈ {0.1, 0.2, . . . , 0.9}.
.
.
.
.
.
.
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
.
.
.
.
.
.
.
.
.
23 / 27
.
Introduction
Multi-agent Network flows
Nash Equilibrium maximizing the flow
Conclusion
Characterization of Nash equilibria
MILP formulation
First experiments
First experiments
Some preliminary result
The solving phase remains fast since we solve instances
up to 100 nodes (almost 800 edges) in less than 571s
when m = 2.
The average relative flow increasing with respect to the
F
) in function of the percentage of
maximum flow (i.e. Fmax
the reward α.
.
.
.
.
.
.
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
.
.
.
.
.
.
.
.
.
24 / 27
.
Introduction
Multi-agent Network flows
Nash Equilibrium maximizing the flow
Conclusion
Characterization of Nash equilibria
MILP formulation
First experiments
First experiments
F IGURE: (a) n = 50
(b) n = 100
.
.
.
.
.
.
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
.
.
.
.
.
.
.
.
.
24 / 27
.
Introduction
Multi-agent Network flows
Nash Equilibrium maximizing the flow
Conclusion
Outline
1
Introduction
2
Multi-agent Network flows
Problem statement
Example
3
Nash Equilibrium maximizing the flow
Characterization of Nash equilibria
MILP formulation
First experiments
4
Conclusion
.
.
.
.
.
.
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
.
.
.
.
.
.
.
.
.
25 / 27
.
Introduction
Multi-agent Network flows
Nash Equilibrium maximizing the flow
Conclusion
Conclusion
Multi-agent Network flows
Multi-agent network flow problem with controllable capacities.
• Optimization problem = Find a Nash equilibrium that maximizes the
network flow.
Efficient MILP approach to solve it.
.
.
.
.
.
.
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
.
.
.
.
.
.
.
.
.
26 / 27
.
Introduction
Multi-agent Network flows
Nash Equilibrium maximizing the flow
Conclusion
Conclusion
Multi-agent Network flows
Multi-agent network flow problem with controllable capacities.
• Optimization problem = Find a Nash equilibrium that maximizes the
network flow.
Efficient MILP approach to solve it.
Perspectives
• Work in progress
Extend our approach to find an optimal sharing policy
• Future work
Extend the approach to model more general multi-agent
network expansion problem.
.
.
.
.
.
.
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
.
.
.
.
.
.
.
.
.
26 / 27
.
Introduction
Multi-agent Network flows
Nash Equilibrium maximizing the flow
Conclusion
Thank you for your attention !
.
.
.
.
.
.
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
.
.
.
.
.
.
.
.
.
27 / 27
.
Introduction
Multi-agent Network flows
Nash Equilibrium maximizing the flow
Conclusion
MILP formulation
Max
s.c.
(1)
F
∑
(i,j)∈E fi,j
−
∑
(j,i)∈Er fj,i

 0 ∀i ̸= s, t
F ,i=s
=
, ∀i ∈ V

−F , i = t
(2)
(3)
fi,j ≤ qi,j , ∀(i, j) ∈ E
q i,j ≤ qi,j ≤ q i,j , ∀(i, j) ∈ E
(4)
(5)
xi,j ≤ (q i,j − qi,j ) ≤ xi,j (q i,j − q i,j ), ∀(i, j) ∈ E
yi,j ≤ (qi,j − q i,j ) ≤ yi,j (q i,j − q i,j ), ∀(i, j) ∈ E
(6)
(7)
δFi,j,u = −ci,j + (ci,j − M)(1 − xi,j ), ∀(i, j) ∈ E, ∀Au ∈ A, (i, j) ∈ Eu
δFi,j,u = −M, ∀(i, j) ∈ E, ∀Au ∈ A, (i, j) ∈
/ Eu
.
.
.
.
.
.
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
.
.
.
.
.
.
.
.
.
27 / 27
.
Introduction
Multi-agent Network flows
Nash Equilibrium maximizing the flow
Conclusion
MILP formulation (2)
(8)
(9)
δBj,i,u = ci,j − (ci,j + M)(1 − yi,j ), ∀(i, j) ∈ E, ∀Au ∈ A, (i, j) ∈ Eu
δBj,i,u = −M(1 − yi,j ), ∀(i, j) ∈ E, ∀Au ∈ A, (i, j) ∈
/ Eu
(10)
(11)
tju − tiu ≥ δFi,j,u , ∀(i, j) ∈ E, ∀Au ∈ A
tiu − tju ≥ δBj,i,u , ∀(i, j) ∈ Er , ∀Au ∈ A
(12)
∑
(i,j)∈E∪Er
φui,j −
∑
∑
(i,j)∈E∪Er
u
ψi,j
−
∑
(13)
u
tn+1
≤−
(14)
u
< wu × π, ∀Au ∈ A
tn+1
(i,j)∈E
φuj,i
(i,j)∈Er

 0 ∀i ̸= s, t
−1 , i = s , ∀i ∈ V , ∀Au ∈ A
=

1,i=t
u
ψ2,i,j
, ∀Au ∈ A
.
.
.
.
.
.
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
.
.
.
.
.
.
.
.
.
27 / 27
.
Introduction
Multi-agent Network flows
Nash Equilibrium maximizing the flow
Conclusion
MILP formulation (3)
≥ 0, ∀(i, j) ∈ E, ∀Au ∈ A
≤ φui,j × M, ∀(i, j) ∈ E, ∀Au ∈ A
≤ −δFi,j,u , ∀(i, j) ∈ E, ∀Au ∈ A
≥ M(φui,j − 1) − δFi,j,u , ∀(i, j) ∈ E, ∀Au ∈ A
(15)
(16)
(17)
(18)
u
ψi,j
u
ψi,j
u
ψi,j
u
ψi,j
(19)
(20)
(21)
(22)
u
ψ2,i,j
u
ψ2,i,j
u
ψ2,i,j
u
ψ2,i,j
≥ −ci,j × φui,j , ∀(i, j) ∈ Er , ∀Au ∈ A
≤ φui,j × M, ∀(i, j) ∈ Er , ∀Au ∈ A
≤ −δBj,i,u , ∀(i, j) ∈ Er , ∀Au ∈ A
≥ M(φui,j − 1) − δBj,i,u , ∀(i, j) ∈ Er , ∀Au ∈ A
where
fi,j ≥ 0, ∀(i, j) ∈ E
qi,j ∈ N , ∀(i, j) ∈ E
fi,j , qi,j ∈ N , ∀(i, j) ∈ E
xi,j , yi,j ∈ {0, 1}, ∀(i, j) ∈ E
φui,j ∈ {0, 1}, ∀(i, j) ∈ E, ∀Au ∈ A
.
. . . . . .
.
. . . . . .
u
u
δi,j
, δj,i
∈ ℜ(∈ [−M, ci,j ]), ∀(i, j) ∈ E,. ∀A
u ∈ A
. . . .
. . .
. . . .
. . . .
.
.
.
.
.
.
.
.
.
27 / 27
.