Cooperative multi-agent systems
• In cooperative MAS agents strive to reach a common goal
and increase the combined utility of their actions
• Limitations on computational power and bandwidth
prohibit the global view of the task structure
• Exchange of complete local views between agents is
prohibitive
• Need to find a trade-off between the amount of information
exchanged and the negotiation’s contribution to increasing
combined utility
1
Contribution
The proposed negotiation protocol:
– Empirically shown to approach the utility
of distributed agents’ actions very close
to that of their aggregation
– Multi-parameter
– Uses marginal utility gain and cost to
drive the negotiation
2
Assumptions
• Agents possess local views of the task
structures that define their functionality
• Agents have access to a scheduler that takes
a task structure and utility function as input
and produces a set of schedules that
increase the utility function locally
3
Motivating example: task
allocation
The agent needs a certain non-local task to
be performed by some other agent.
The negotiation’s goal is to increase the
combined utility of actions of both agents
by choosing a certain quality and time range
of the non-local task.
4
Vocabulary 1
• Contractor – agent that has a non-local task
NL that needs to be assigned to another
agent
• Contractee – agent that performs this task
for the contractor
• Utility – weighted sum of quality, cost, and
duration of a schedule
5
Vocabulary 2
• Marginal utility gain (MUG) – local utility
increment for the contractor with NL
performed
• Marginal utility cost (MUC) – local utility
decrement for the contractee with NL
performed
6
Motivating example: task
structures
TCR
TCE
sum
sum
Task1
Task2
sum
M2
M3
M4
q:10 q:10 q:10
c:10 c:10 c:10
d:9 d:9 d:9
q:15
c:0
d:0
Task2
sum
sum
enables
M1
Task1
sum
enables
M5
q:10
c:10
d:9
deadline:50
enables
B1
B2
q:10 q:10
c:10 c:10
d:9 d:9
deadline:11 deadline:21
B3
q:10
c:10
d:9
B4
q:10
c:10
d:9
deadline:47
7
Contractor’s FSM
rcvMsgCounterProposal
always
evalCounterProposal
buildProposal
sndMsgProposal
Start
Wait
always
rcvMsgAccept&
HaveEnough
sndMsgFinish
EvalCounterProposal
Accept&HaveEnough
sndMsgFinish
Reject
sndMsgReject
rcvMsgAccept&
GetNext
Finish
generateNewProposal
sndMsgProposal
Accept&GetNext
NewProposal
8
Contractee’s FSM
rcvMsgReject
Wait
rcvMsgProposal
evalProposal
EvalProposal
Start
Reject
rcvMsgFinish
buildCounterProposal
sndMsgCounterProposal
Accept
Finish
sndMsgAccept
9
Binary search approach
Each next proposal is the midpoint between the contractor’s
current proposal and contractee’s current counter proposal.
Disadvantages:
• Finish time of first counter proposal may not be the actual
reasonable finish time
• Negotiation is about multiple issues (time range, quality)
while the binary search focuses only on finish time
• Binary search requires greater certainty about NL’s time
range
• Both agents are likely to cling to their own proposals,
greater the likelihood that some solutions are missed.
10
Proposal/counterproposal
information
• EST - Earliest start time of the non-local
task NL
• FT - Finish time
• minq - Requested quality
• MUC - Marginal utility cost
• MUG - Marginal utility gain
11
Protocol functions overview
• buildProposal:
– Creates the initial contractor’s proposal
including the time range and quality of NL and
MUG
– Insists on highest local utility by executing NL
according to the contractor’s best local schedule
12
Protocol functions overview
• generateCounterProposal:
– Creates the contractee’s counter proposals
including the time range and quality of NL and
MUC
– Initially insists on highest local utility by
executing NL according to the contractee’s best
local schedule
13
Protocol functions overview
• generateCounterProposal:
– Tries to increase combined utility by lowering
MUC
– Successive counter proposals relax the time
range and quality of NL until a schedule is
found s.t. MUG>MUC
14
Protocol functions overview
• generateNewProposal:
– Creates the next contractor’s proposal based on
the preceding proposal or counter proposal
15
Protocol functions overview
• generateNewProposal:
– Tries to increase combined utility by doing
multi-parameter heuristic-based search in
quality and time range space
16
Protocol functions: buildProposal
• buildProposal:
– produces the initially requested earliest start
time and finish time of the non-local task
according to the contractor’s best schedule
– has limited information about the NL’s quality,
cost, and duration
– the greater the time range of NL’s execution is
the closer to the high bound the requested
quality is
17
Protocol functions:
generateCounterProposal
• If no previous Counter Proposal exists:
– produces a “what-if” local contractee’s schedule
ignoring the time range and quality of NL in the current
proposal, but enforcing NL to be in the schedule, thus
delivering the minimum marginal utility cost for the NL
• If there is a previous Counter Proposal :
– Alternatively relaxes the NL’s time range and lowers
the requested quality from the current proposal until an
acceptable schedule is found (MUG>MUC)
18
Protocol functions:
generateNewProposal 1
• If Previous Proposal is acceptable for the
contractee:
– If the NL is out of initial time range and its quality
below average then increase the quality request and
make later the deadline for the NL
– If the NL is in initial time range and its quality is above
average then reduce quality request in the same time
range thus trying to lower MUC
– Otherwise shift time range later and reduce quality
request thus trying to lower MUC
19
Protocol functions:
generateNewProposal 2
• If Previous Proposal is not acceptable for
the contractee:
– If this is first counter proposal:
• If the initial time range is too short then the deadline
is moved later and lower quality is requested
• Otherwise make the initial range a bit longer than
the current estimate of NL’s execution time and
request higher than average quality
20
Protocol functions:
generateNewProposal 3
– If this is not first counter proposal
• Assume a previous counter proposal from the
contractee as current proposal
• If requested quality above average then shift the
time range later
• If requested quality below average then request
higher quality and shift the NL’s deadline later
21
Modified task structure
New_TCE
TCR
sum
sum
TCE
Task1
Task2
sum
M2
M3
M4
q:10 q:10 q:10
c:10 c:10 c:10
d:9 d:9 d:9
q:15
c:0
d:0
Task1
enables
Task2
sum
deadline:50
M41
M42
M43
sum
M5
q:10
c:10
d:9
exactly_one
sum
sum
enables
M1
M4
enables
B1
B2
q:10 q:10
c:10 c:10
d:9 d:9
deadline:11 deadline:21
B3
q:15
c:0
d:0
B4
q:10
c:10
d:9
deadline:47
22
Protocol variants
• Single step: the contractor sends a proposal PC to
the contractee which accepts it if
MUG(PC)>MUC(PC) or rejects it otherwise
• Multi-step-Multiple-Try: the agents perform the
negotiation process until a predefined number of
acceptable solutions are found or iteration limits
are exceeded
• Multi-step-Limited-Effort: the agents perform the
negotiation process until iteration limits are
exceeded
23
Experiment task structures
TCR
TCE
sum
sum
Task1
Task2
sum
enables
M1
M2
enables
M4
q:10 q:10 q:10
c:10 c:10 c:10
d:9 d:9 d:9
q:15
c:0
d:0
enables
M5
q:10
c:10
d:9
deadline:50
Task2
sum
sum
M3
est:10 est:24
Task1
enables
enables
B1
sum
B2
q:10 q:10
c:10 c:10
d:9 d:9
deadline:11 deadline:21
enables
B3
q:15
c:0
d:0
B4
q:10
c:10
d:9
deadline:47
24
Utility function
• S – schedule
Utility(S) = quality_gain(S)*quality_weight +
cost_gain(S)*cost_weight +
duration_gain(S)*duration_weight
25
Collected data
• Negotiation outcome: success if the combined utility is increased
• Utility gain: MUG - MUC
• Gain percentage: percentage of utility gain relative to the combined
utilities without the task allocation
• Solution quality: utility percentage relative to the utility of the
combined tasks
• Complexity of task structures: function of the number of
interrelationships and temporal constraints in the task structures
• Number of negotiation steps (proposal/counterproposal messages).
26
Protocol comparison
Success Average Gain
Percentage
Single-step
ANNS
Gain per
Step
Solution
Quality
2850
7.63
1.0
7.63
94.43
Multi-stepOne-Try
4088
10.17
1.48
6.87
96.69
Multi-stepTwo-Try
4088
11.9
4.69
2.55
98.23
Multi-stepThree-Try
4088
13.4
6.42
2.09
99.51
Multi-stepLimited Effort 4088
13.9
8.15
1.7
99.93
27
Complexity
•
•
•
•
ir1 – number of interrelationships in contractor’s task structure
tr1 – number of temporal constraints in the contractor’s task structure
ir2 – number of interrelationships in contractee’s task structure
tr2 – number of temporal constraints in the contractee’s task structure
Complexity of task structure = ir1 + tc1 + ir2 + tc2 + (ir1*tc1 + ir2*tc2 +
ir1*ir2 + tc1*tc2 + ir1*tc2 + ir2*tc1)/6
28
Dependence of SQ on complexity
102
100
solution quality
98
96
94
92
MultiStepLimitedSearch
MultiStepThreeTry
MultiStepTwoTry
MultiStepOneTry
SingleStep
90
88
0
5
10
15
complexity of task structures
20
29
Net negotiation gain
Assume that one negotiation step costs 0.5%
of the overall utility of the negotiating
agent’s schedules
Net negotiation utility gain = negotiation
utility gain – negotiation effort
30
Dependence of net gain on
complexity
102
net negotiation gain
100
98
96
94
92
MultiStepLimitedSearch
MultiStepThreeTry
MultiStepTwoTry
MultiStepOneTry
SingleStep
90
88
0
5
10
15
complexity of task structures
20
31
Observations
• The Multi-step-One-Try protocol is much better
than Single step in almost all situations(except for
very simple ones)
• The Multi-step-Two-Try and Multi-step-ThreeTry protocols are useful in moderately constrained
situations
• The number of constraints can be used to choose a
protocol that balances negotiation gain and effort
for a certain situation
32