USING AGENT COALITIONS FOR
IMPROVING PLAN QUALITY
Dang Thanh-Tung1 ,Baltazar Frankovic1 ,Ivana Budinska1 ,
Con Sheahan2 and Dang Bao-Lam3
1 Institute
of Informatics,
Slovak Academy of Sciences, Dubravska cesta 9, 84507, Bratislava, Slovakia
{utrrtung,utrrfran,utrrbudi}@savba.sk
2 Department
of Manufacturing and Operations Engineering,
University of Limerick, Limerick, Ireland
[email protected]
3 Department
of Design of Machines and Robots,
Hanoi University of Technology, 1 Dai Co Viet str. Hanoi, Vietnam
[email protected]
Abstract — Task execution in multi-agents systems might require cooperation
among agents. An agent coalition is a special kind of cooperation, whose purpose
is to improve the collective performance. Creating coalitions usually requires complicated negotiations among participating agents. In order to resolve this mentioned
problem, we try to suggest a negotiation protocol, which is based on FIPA standards.
In addition, some default strategies are proposed with the purpose to help agents in
decision making, while choose coalitions. These strategies could also be combined
one with others to adapt to various situations.
1
Introduction
Cooperation among autonomous agents occurs in many different application fields, e.g.
manufacturing, transporting, supply-chain systems, etc [1], [2], [3]. The main purpose
of agent cooperation is to fulfil as well the planned aim as possible. Let us consider an
example in which a number of agents (factories) cooperate one with another, each of them
owns a number of equivalent machines and has to satisfy any order defined by customers.
Of course, as an autonomous agent (factory is considered as one autonomous agent), the
assigned task could be accomplished without interventions or helps from other agents
(factories). But the usage of machines might not be as effective as each agent expects.
Some agents might have too busy plan, on the other hand, some others might have idle
machines. In such case, joining a common coalition can improve the performance of each
agent. If agents agree to share their machines and a part of the assigned tasks, then there
DANG THANH-TUNG, ET AL.
is possible to find a better manner (faster, cheaper, with more profits, etc.) to satisfy the
intended goals. This paper focuses on solving the problem how agents can negotiate and
decide to create such coalitions, with the purpose to improve the collective performance.
The agent performance could be measured by monetary units or any vector consisted of a
number of variables that characterize the production process.
In real-world domain, agents are usually self-interested and they tend to reach the personal benefit as much as possible. For that reason, the information that agents provide one
for another is usually incomplete. Moreover, agents can apply various manners within a
negotiation process to form coalitions. As a result, creating stable coalitions in the realworld domain is very complicated and time consuming. In order to make the problem to
be resolvable, some simplifications will be considered in this work, for example, the time
available for negotiation is limited; agents should agree with any common protocol for
negotiation; and some stop conditions should be clearly predefined in order to close the
negotiation process. The next simplifications are, namely: agents evaluate coalitions by a
reward function, which expresses monetary amount that the agent will receive within the
framework of the selected coalition. Each agent could be a member in one and only one
coalition at a time. No agent tends to cheat or damage another agent performance; another
assumption is that agents might be self-interested or not self-interested, but improving the
collective performance will have the highest priority for all participating agents. However, behaviour of self-interested and non self-interested agents might be different and it
will be analyzed later in Section 3 and 5.
2
Problem description
The problem solving is generally described as follows. Given a number of agents (producers), each of them owns a set of resources (the common label for machines, vehicles,
production lines, etc.), which represent an agent’s capability. Given a set of orders specified by customers, each of them consists of a number of tasks to be performed. Beside
that, customers might specify some extra conditions, e.g., deadline, penalty if the order is
lately accomplished, etc. These constraints will be taken into consideration within negotiation. Parameters associated with task performance, e.g. duration, production cost, etc,
are known for all agents, but the rewards that each agent can receive after accomplishing
orders are private information that only this agent knows. To simplify let us assume that,
although each task could be migrated from one agent to another, the result received after
execution is the same, regardless where that task is executed.
Assume that all agents can participate in a negotiation process to form coalitions. The
final coalition configuration might consist of a number of small disjoint coalitions, which
bring each agent better or at least as the same benefit as when it works alone. The main
task is to propose an approach of performing negotiation, which is able to guarantee
a stable final solution, i.e., the discussion among agents will converge to any concrete
coalition configuration, due to the time limitation, moreover, these created coalitions will
be stable on the basis of any predefined agreements or rules.
To understand better the problem solving, let us show an example - supply chain management problem, in which creating coalitions can increase the profits for each partner.
Example: four transport companies - called agents - need to deliver various products
to different customers - 6 retail dealers as shown in Figure 1. Each transporter delivers
USING AGENT COALITIONS FOR IMPROVING PLAN QUALITY
certain kinds of products, e.g. transporter T1 delivers milk, T2 sugar, T3 mineral water and
T4 toys, respectively. The time when customers wish to get desired products are specified
and known in advance for transporters. In addition, the payments that each customer
pays for packages of ordered products and their additional associated parameters, e.g.
size, weight, etc. are given. The main objective of the transport companies is to maximize
their individual profits.
One of the manners how to increase the profit is to reduce the delivering path or respectively the fuel consummation by sharing the use of vehicles. Transporters might agree to
join a coalition and share their transport capacity, for example, T1 and T2 join one coalition; T1 delivers milk and sugar to customers C1 , C2 , C3 ; T2 delivers the same products
milk and sugar to a group of customers C4 , C5 , C6 . In this case, the length of the path
that T1 and T2 have to carry is reduced to half, because only one transporter, not two
transporters like in the original situation, visits each customer. Similarly, T3 and T4 can
join a coalition and each of them is responsible for a disjoint group of customers, or T3
might take some orders from T4 and helps him to carry out, on the assumption that a part
of payoff that T4 receives, will be moved to T3 .
C2
C3
C1
T1
T2
C5
C4
T3
T4
C6
Figure 1: creating coalitions in supply chain system
However, transporters might join various coalitions and exploit many different manners
to share their vehicles usage. Consequently, there are an enormous number of possible
coalition configurations that agents - transporters can form. The objective of this paper is
to propose a method of performing negotiation in order to achieve such a solution, which
satisfies all participators requirements as best as possible.
3
Methods for solving - a case with non self-interested agents
The proposed approach consists of the following parts: definition of negotiation protocol,
specification of the stop condition for ending negotiation and default strategies for making
negotiation.
3.1
Specification of negotiation protocols
Agents need a common protocol for exchanging proposals. Using a common protocol has
many advantages towards a case when each of them uses different protocol, e.g. agents
DANG THANH-TUNG, ET AL.
save a lot of time needed for encoding and decoding messages. The proposed protocol
is based on the FIPA-Contract-Net that is commonly used in agent community (e.g. see:
http://sharon.cselt.it/projects/jade/ ). This interaction protocol allows the initiator agent
to send a Call for Proposal (CFP) to a set of specific agents (responders), evaluate their
proposals and then accept (or reject) the preferred one. The interaction protocol is in detail
described in the FIPA specification (see http://www.fipa.org/ ). In the following part, the
brief description of the usage of this interaction protocol is presented.
Beside the agents, which participate in the process of forming coalitions, there is another agent, whose functionality is to coordinate negotiation, to receive proposals, and
to audit agents behaviour. That agent is called mediator. Let us assume that agents are
fair (do not cheat) within negotiation. All agents are compulsory to provide the mediator
the truth information about their requirements (the reward function, production cost,...).
On the other hand, the mediator must not forward this information to other agents. We
consider a case when agents are not self-interested, so on the basis of the provided information, the mediator is able to declare the result of negotiation, i.e., the coalition is
successfully formed or not.
Negotiation is realized in rounds. In each round, only one agent can post a CFP message
that specifies the action to be performed and conditions associated with its execution. The
action is a set of tasks that the agent asks the other ones to help. The condition included
in the CFP message is a reward that the agent is willing to pay for these tasks execution.
Setting the reward in case with non self-interested agents will be discussed in Section
3.4. Conditions might involve the deadline when these tasks have to be accomplished
and delivered back to the initiator agent. The responder agents can reply by sending to
the mediator a propose message including the preconditions (i.e., the desired payoff, the
time of accomplishing these tasks) that they require for performing the received tasks. A
responder agent, however, can send a refuse message, if it is not able to help. The mediator
can then evaluate all the received proposals and consequently choose appropriate agents
to form coalitions, in order to complete these tasks. When a coalition is created, the
mediator informs all its members by sending them an accept-proposal message. The rest
of agents will receive a reject-proposal reason. Members of a coalition are compulsory to
keep all conditions that they declare in proposals.
In order to avoid endless negotiation, the mediator is else responsible for checking a
timeout parameter, i.e. a time that each agent has to answer. Only messages received
before than the timeout expires are considered for evaluation.
3.2
Specification of stop conditions
From practical point of view, discussion among agents should converge to any final solution. Agents should agree with some concrete coalitions, in which they can achieve better
benefits; or they should stop negotiation if there is not chance to improve the performance.
The first situation which leads to the end of negotiation is no agent is interested in
continuing negotiation, for example, calling or answering new proposals, because their
production capacity is fully exploited or production plans could not be improved. In the
example shown in Section 2, if all vehicle capacities of transporters are entirely used, then
the agents might not have interest in changing their transportation plans. In this case, the
mediator announces the end of negotiation.
USING AGENT COALITIONS FOR IMPROVING PLAN QUALITY
The next situation that can force to stop negotiation is when the timeout expires. In
order to apply this condition, at the beginning, the mediator announces the time available
for performing negotiation and the timeout after that each agent has to answer. Each
agent should be aware of that condition and has to keep it. If after the timeout expires, the
agent that is in turn does not post a CFP message, the mediator requires the next agent
in the order to act. If no agent posts a new CFP message, the mediator announces the
negotiation process is closed.
There might be such agents which do not have an interest in participation in forming
coalitions. Those agents can inform the mediator, and they then will be removed from the
list of candidates. If only one agent remains in the list of candidates, a negotiation process
will be stopped.
3.3
Strategies for choosing coalitions
When a new CFP message appears, all agents start a negotiation process and try to form
coalitions with the purpose to execute tasks included in this proposal. Agents calculate
and modify their production plan, so that the entire tasks included in this proposal will be
completed in time, accordingly the requirements specified by the initiator agent. Agents
should take into consideration all parameters like delays caused by transporting among
agents, limitations of production capacities, transport cost, etc., while try to reschedule
their production plans. Let us assume that within the framework of a coalition, agents
share their production plans, each agent has knowledge about the others plan and therefore
it is capable of rescheduling its production plan.
After rescheduling, if all requirements specified by the initiator agent are satisfied, each
agent sets its commission; that is a monetary amount the agent wants to obtain for tasks
execution. The commission should cover all the expenses related to tasks performance,
i.e. the cost of these tasks execution, the transport cost, etc. In case if these requirements
are not satisfied, agents can try to join another coalition, until the timeout expires. The
problem how agents specify their commission will be discussed in next section.
In general, the number of coalitions that each agent can join is very large. If there are
n agents, except for the initiator agent, each agent of the rest can join 2n − 2 different
coalitions. As a result, at the beginning each agent can receive 2n − 2 offers to join
coalitions. Due to the fact that forming coalitions of small size is more comfortable and
more practical than forming the large size ones. For that reason, the following approach
is suggested for forming coalitions.
Negotiation scheme for creating coalitions - small size is preferred
– Step 1: Agents try to form coalitions consisted of two members. If any coalition is
successfully created, all members of this coalition will not participate in next round
of negotiation.
– Step 2: Remaining agents try to create coalitions consisted of three members.
– ..
– etc.
In each next round, the size of examined coalitions increases by 1, and this process continues until the timeout expires. Besides preferring small size coalitions, agents might
choose coalitions by applying a different approach as following.
DANG THANH-TUNG, ET AL.
Choosing coalitions by using reference list: each agent has a list of potential candidates
for forming coalitions. At the beginning, that list includes all agents with the same priority. Sequentially, on the basis of the knowledge about other agents production plans,
each agent can reduce that list by removing the agents that do not have empty production
capacities, or can change the priority of these agents in its reference list.
For a new CFP, an agent requests the first candidate in its list, sequentially to the last
one, to form common coalitions.
The next method to choose coalitions is random choice. Agents are randomly assigned
to coalitions; afterwards, members of unsuccessful coalitions are merged to one group.
These remaining agents are then selected randomly to coalitions, etc. until the timeout
expires or all agents are members of any coalition.
In order to reduce redundant work, each agent indicates a configuration of successful
and unsuccessful coalitions. In next round of negotiation, each agent starts by requesting
members of the previous successful coalitions, because these configurations might have
higher chance to be successful than the one of the unsuccessful coalitions. Agents could
also attempt to form new coalitions, which have not been examined before. These new
coalitions might be examined before the previously unsuccessful ones; but it depends on
each agent decision. This approach is called choosing coalitions with adaptation, since
agents do not start negotiation from the beginning, but from some known configurations,
which have a high percentage of the success.
After stopping calculation, all formed coalitions send their answers to the mediator and
the coalition which requires the lowest reward will be considered as the winner. However,
the reward that the winner coalition requires has to be in the range of values where the
initiator agent is willing to pay; otherwise, the mediator announces the current round of
negotiation as unsuccessful.
3.4
Setting reward values and profit distribution
When the initiator agent posts a CFP message, it has to inform the mediator a value of
the payment that it is willing to pay for the tasks execution. The initiator agent is aware
of how much it will gain if the tasks included in the proposal will be completed and
delivered back in time. By comparing the current profit and the future one if the proposal
is accepted by any coalition, the initiator agent calculates the increase in benefit. From
practical point of view, a value of the payment should be top-bounded by that value.
For candidate coalitions, the rewards that they want to receive have to cover at least
the necessary expenses i.e. production and transporting cost. In addition, each agent
certainly wants to earn something; and therefore agents define a minimal reward that they
desire to get. The total sum of all minimal rewards defined by all members and the cost
(production, transporting) is the down-bound of the reward that the candidate coalitions
desire to get. If a value of the down limit is higher than the top bound of the initiator
agents payment, the proposal will not be accepted.
Since agents are assumed as non self-interested, and prefer to increase the collective
profit, so the initiator should set the payment by the maximal increase that it can gain.
In contrast, all candidate agents should set their commission by the minimal expense that
they need for task execution. The surplus between the payment that the initiator is willing
to pay and the requirement that a candidate coalition asks for task execution, will be
USING AGENT COALITIONS FOR IMPROVING PLAN QUALITY
equally distributed to all participators, i.e. initiator and members of the winner coalition.
The method presented above for setting rewards seems to be most objective. It eliminates cases when there is not a coalition to satisfy the proposal due to the rewards requirements, although they are practically able to do that. For example, the initiator can gain
10$ more, but it sets payment by only 7$. The best candidate coalition can execute tasks
included in the proposal for 5$, but each member sets its reward more 1$. Let the candidate coalition have three members, consequently, the final reward that the entire coalition
requires will be 8$. In this case negotiation will not be successful.
In general case, the top bound of the initiator agents payment and the down limit of the
candidate coalition rewards are included in the messages sent to the mediator and they
are not public for any agent. Besides these values, all coalitions and the initiator agent
can define for themselves an interval of the reward and payment, respectively, which is an
objective for negotiation, if the desired reward is higher than the willing payment.
4
Experimental simulation
To demonstrate the above presented theory, a number of experiments were made, in which
agents employ the above presented protocol and methods to form coalitions. In these
experiments, the number of agents is set by 20. The number of agents could be set larger,
but this parameter does not influence as much in the negotiation process as strategies for
making decision or a process of rescheduling.
Each agent receives an order that consists of a number of different products. Each
product consists of a specific number of basic operations, whose mutual relationships
are defined in advance. There are given several possible deadlines for accomplishing the
order, but the earlier the order is accomplished, the more an agent will gain.
After having a complete plan, the mediator asks one of agents to post a CFP. In case
when the execution plan of the selected agent cannot be completed before the earliest
deadline, it can take a part of tasks and asks the rest of agents to help to execute. Agents
can start a process of forming coalitions. In order to compare the proposed strategies, the
results achieved by using these strategies are compared with the optimal one, when agents
have unlimited time to investigate all possible configurations. In each experiment, agents
apply the same strategy to negotiate and to choose coalitions.
In these experiments agents are assumed as non self-interested. Agents set their rewards
and distribute among themselves profit according to the method presented in Section 3.4.
4.1
Proposals with small sets of tasks
In the first group of experiments, the initiator agent selects only a small number of tasks
to post. In “Figure 2”, there are results of comparison between the proposed strategies.
Based on the achieved results, some conclusions could be deduced as following.
The first strategy that prefers small size coalitions could faster achieve agreement than
all others. Coalitions are formed very quickly, mainly in the first round, where only coalitions of size 2 are examined; afterwards, agents from successful coalitions are removed
from the list of candidates for next rounds of negotiation. As a result, in most cases the
winner coalition consists of 2 agents only. On one side, agents can save a lot of time, but
on the other hand, only a small number of coalition configurations are examined when a
DANG THANH-TUNG, ET AL.
Achieved results/Optimal results
0.5
0.4
0.3
0.2
0.1
0
smal coalitions
reference list
random choice
choice with adaptation
Figure 2: Comparison among strategies - small packages of tasks
new CFP message appears. In addition, small coalitions are usually not able to resolve
a large set of tasks and therefore this approach is practical only when the set of tasks
included in the proposal is not too large. Methods for choosing coalitions based on the
reference lists and choice with adaptation achieve averagely the same quality results, and
they are the best among all the proposed ones. However, first rounds of negotiation usually lead to a failure. The explanation could be as follows. After a certain number of
rounds of negotiation, each agent has enough knowledge about other agents production
plans. Consequently, the reference list of each agent can indicate clearly which of the rest
agents is better candidate for creating coalitions (has the most free production capacity)
and which one is worse. The choice with adaptation strategy memorizes all successful,
but also unsuccessful coalition configurations. As a result, each agent has an outlook over
the potential candidates for forming coalitions. After many rounds of negotiation, each
agent probably knows precisely which one is preferred to request to join coalitions and
which one is not. But, on the other hand, both the strategies requires performing many
attempts to accurate the reference list or to have a precise survey over candidates. Since
only a small set of tasks is selected to post for rescheduling, negotiation is repeated for
many times and therefore agents have many opportunities to improve their information
base. That is why these both strategies achieve the best results in these experiments.
Random selecting is not as successful as the rest methods, due to the fact that many
coalitions are repeatedly examined, although they have been unsuccessful in previous
rounds. This strategy is really useful when the time limitation is short, because it guarantees a solution at any-time.
4.2
Proposals with large sets of tasks
In next series of experiments, each agent chooses such package of tasks together with
appropriate deadlines, so that if any coalition of agents agrees to execute the selected
tasks, then initiator’s benefit will be maximal.
The comparison results are shown in “Figure 3”. Similarly, the achieved results are
compared with the optimal ones when agents have unlimited time to experiment.
These results point out that, the quality achieved by applying the strategy that prefers
small coalitions decreases a lot while the quality of the rest methods increases. The
main reason is that many attempts in the first round lead to failure, because of the large
dimension of the task package; consequently agents do not have a time to examine other
coalition configurations.
Choosing based on the reference list achieves the best results, since after several rounds
agents could discover which partner is the best for forming coalitions (there are 20 agents,
USING AGENT COALITIONS FOR IMPROVING PLAN QUALITY
Achieved results/Optimal results
0.6
0.5
0.4
0.3
0.2
0.1
0
smal coalitions
reference list
random choice
choice with adaptation
Figure 3: Comparison among strategies - large packages of tasks
after maximally 19 rounds of negotiation with different partners, each agent is able to get
enough precise outlook over their plans of occupation). By applying reference lists, agents
reduce a lot of useless work, because coalitions which have low chances of acceptance
(whose members are in the last places of the reference lists) will be ignored.
Choosing with adaptation is not as successful as in the first series of experiments, since
the package of tasks is large, as a result, rescheduling might considerably change production plans. Some coalitions that are successful in the current round might not be accepted
in next rounds, due to these significant changes as well.
The improvement of the quality could be indicated in the random choice strategy. Random choice allows agents to experiment with unknown configurations, so there might be
a opportunity to get better solutions than in the first strategy (small size coalitions), where
agents often meet a failure while examine small configurations.
4.3
Proposals with random sets of tasks
In this series, each agent selects a random set of tasks to post. An agent starts by taking a
small set of tasks and tries step-by-step to increase it, if no agent asks to be removed from
the list of candidates. If some agents announce that they are not interested in receiving
further tasks, that means they do not have unused production capacity. In that case, the
initiator agent reduces a set of tasks included in its proposal.
Simulation results shown in “Figure 4” confirm again that using reference lists achieves
the best results. Reference lists help agents to choose a suitable partner for negotiation and
reduce a lot of useless efforts because of changes of production plans and the dimension
of the proposed set of tasks. On the other hand, concerning a variable dimension of task
sets included in CFP, agents apply not only one fixed strategy to perform negotiation
or reschedule. They use different strategies and could switch from one to another, with
respect to the content of proposals. Mixing strategies creates a number of the new ones,
which lead to a number of different results. In spite of numerous mixing strategies being
tested, there is not any significant evidence, which points out which one of them is best
for general situations.
4.4
Implementation and its combination with expert systems
The first version was completely implemented in C++ and PVM. The Internet is used as
medium for communication exchanges. The content of messages (proposals, answers,
etc.) is formulated by using a self-defined protocol, which guarantees the comprehensibility among agents.
JADE provides a set of predefined protocols, including such protocols for performing
DANG THANH-TUNG, ET AL.
Achieved results/Optimal results
0.6
0.4
0.2
0
smal coalitions
reference list
random choice
choice with adaptation
Figure 4: Comparison among strategies - different packages of tasks
negotiation as presented in section 3.1. For that reason, the current version is implemented
in Java and JADE. The difference between two versions is not significant, although Java
provides many possibilities to create a more user-friendly product.
In order to approximate practical situations, we have asked some managers to give comments and advices. Clearly, no person is willing to explain his/her tactics or his/her tactic is usually too complicated for implementation. Despite these difficulties, some their
comments are considered into implementation for case with non self-interested agents.
First, a coalition is usually formed for a long term purpose, so agents are not willing to
change their party again and again. From this point of view, small-size coalitions are more
practical and stable than the large ones. The first recommended approach is therefore to
combine two strategies - using reference list and - preferring small coalitions. Second,
special relationship might exist among some agents; agents from a so-called ”high interested” group are usually preferred over the rest ones. The mentioned problem could be
resolved by setting different priorities to the agents in the reference list at the beginning of
the solving process. Third, in order to maximize the collective profit, agents need exploring more coalitions and coalition configurations. When the proposal is too complicated,
cooperation of several coalitions could be needed to manage it. Of course, there are a
number of other practical problems that should be considered into implementation. Some
of them will be considered and implemented in near future.
5
Self-interested agents
In this paper, agents represent human being (e.g. manufacturers), so there is understandable that agents might be self-interested and they always aim to maximize their profit
as much as possible. In general, negotiation among self-interested agents often leads to
deadlock, if nobody is willing to make compromises. On the other hand, if agents stay
working alone, there is not chance to increase their profit, although each agent might
work with an optimal plan. Self-interested agents usually have a tendency to join such
a coalition, in which it can gain the most. Agents explore many different coalitions in
order to get the survey of possible rewards. At one time an agent might take part in forming several coalitions simultaneously. As a result, an agent could be involved in several
candidate coalitions. That fact, nevertheless, does not violate the assumption mentioned
in the introduction that says each agent could be a member of one coalition, because the
mediator selects only one winner coalition for each proposal.
One of reasons leading negotiation among self-interested agents to failure is the difference in choosing favourite coalitions for joining by each participator. There might be a
USING AGENT COALITIONS FOR IMPROVING PLAN QUALITY
situation when a group of agents with the same choice does not exist, as a result, coalitions cannot be created. However, this situation occurs only rarely, mainly when agents
do not have enough time to experiment various coalitions.
In Section 3, negotiation between non self-interested agents was dealt. In this case, the
main reason leading to failure of negotiation is the difference in reward values specified by
the initiator and candidate coalition. Agents are assumed to be not self-interested and the
entire agents aim to maximize the collective profit, so the initiator agent posts the maximal
payment that it can pay and coalitions post the minimal reward that they need to cover all
necessary expenses. Unlike this case, when agents are self-interested, they certainly want
to add some additional cost to their desired reward. Similarly, the initiator might leave
out something from its maximal payment. For that reason, negotiation usually leads to
unsuccess. To resolve this dilemma we assume that if negotiation fails, i.e. there is not
any coalition that accepts the proposal, the mediator does not finish the current round of
negotiation, but asks all agents to repeat their negotiation. Concerning the assumption
presented in the introduction that says improving the collective performance is always the
highest priority for the entire agents. Members of candidate coalitions probably decrease
their demands in order to meet the initiator’s specification. Such compromises could be
expected from many agents, but not all. There is a new issue, how much each agent should
decrease its demand, so that its coalition would be the winner one.
From the practical point of view, both the initiator and agents of candidate coalition
could withdraw something to achieve some agreement, but reductions never fall under the
ceiling that they can afford. For the initiator, the top limit is the maximal profit that it can
gain, and for members of candidate coalitions, the down limit of reduction is the minimal
cost for task execution.
6
Modification for case with self-interested agents
This section discusses some features added to the system for simulation of self-interested
agent’s behaviour.
The first added feature is that. Agents are allowed to participate in forming many coalitions at a time and could be member of many candidate coalitions for each proposal. Each
agent can ask and accept several requests simultaneously to form coalition. This point is
only a technical implementation problem.
Next functionality is added to solve cases, when the proposal is not accepted by any
candidate coalition. This functionality works as follows.
When there is not any coalition that accepts the proposal, the mediator asks all agents
to recalculate their plans and repost their new responds.
If all coalitions have a higher monetary requirement than the initiator agent offers, the
mediator asks both sides to negotiate. Besides the currently desired rewards, each agent
sends the mediator the true value of its ceiling. Only the mediator knows the true values
of rewards and therefore mediator is able to remove immediately irrelevant cases which
do not have chance to achieve agreement, e.g. when minimal expense coalition needs
is higher than maximal limit that the initiator can gain. In each round, both sides - the
initiator and candidate coalitions - have to modify their offer, and their change is at least
δ, that is constant defined by the mediator. In each round, each member of a candidate
coalition has to reduce at least δ/s from its current reward, where s is coalition’s size. If
DANG THANH-TUNG, ET AL.
any agent cannot reduce more than this value, it informs the rest of members of coalition.
The concerned coalition will not take part in next round of negotiation and sends to the
mediator an inform-message. Each agent sets its reduction by selecting a random value
from interval [δ/s, MaxAllowance] where
MaxAllowance = CurrentReward - MinReward and CurrentReward is an agent’s present
demand, and MinReward denotes minimal cost for task execution.
However, agents do not choose a random value with the same probability from interval
[δ/s, MaxAllowance]. The reason is as follows. When an agent fails repeatedly many
times in competition to win the proposal, it has a tendency to increase the reduction. In
contrast, when agent gets the proposal, it might not want to change the method for setting
reduction. Based on this assumption, a function for setting reduction is defined for each
agent as follows.
Denote T= MaxAllowance + δ/s and L= MaxAllowance - δ/s. Agent chooses a random
value x from interval [δ/s, MaxAllowance] with probability
P (x) = [k(x − T /2) + 1]/L,
where k is a constant and k ∈ [−2/L, 2/L]. It is easy to verify that P (x) ≥ 0, ∀x ∈
[δ/s, MaxAllowance] and,
Z
M axAllowance
P (x) dx = 1
δ/s
When an agent fails in negotiation, it increases parameter k, in contrast, it decreases k or
can leave k as the current value.
There are a restricted number of times that coalitions and the initiator can change their
offers and resend to the mediator. After a certain number of reconstructing offers, if
there is not any solution, then the mediator announces the current round of negotiation as
unsuccessful. Next agent in order will be asked for posting a new CFP.
These above presented functionalities are added to the original system and they are used
to simulate the behaviour of self-interested agents in a process of searching for better
production plans.
To compare the behaviour of self-interested and non self-interested agents, the same
experiments presented in Section 4 were made, but with self-interested agents. Evaluation of experiment results leads to various hypothesises. Achieved results pointed out
that negotiation among self-interested agents often fails at the beginning, due to high demand of each partner. However, this trend decreases in next rounds of negotiation, since
unsuccessful agents are willing to reduce more their demands in order to win the proposal.
In comparison with case when agents are non self-interested, negotiation among selfinterested agents is longer and the final effect, i.e. the collective performance is less
improved. Many winner coalitions do not execute tasks included in the proposal for minimal cost, but they reduce their rewards with the purpose to win the proposal. The overall
effectiveness is clearly less than when these tasks are executed for minimal cost and the
surplus is distributed equally to all participators.
On the other hand, self-interested agents are closer to the reality. The assumption that
improving collective performance is the highest priority for all agents makes the problem
solving much simpler. In future more realistic features will be considered to analyse.
USING AGENT COALITIONS FOR IMPROVING PLAN QUALITY
7
Comparison with some related works
Negotiation is usually associated with the game theory. Many papers that deal with this
topic try to propose different strategies based on the game theory, e.g., [4], [5]. The
essential difference between this paper and other ones is that agents are interested in
improving personal benefits, but not at the cost of damaging others. Negotiation therefore
has a different character from the approach based on the game theory.
Paper [6] could be considered as similar to this work, however in that work, agents
try to form coalitions with the purpose to resolve one task in each round of auction.
Forming coalitions is like auction, where each task is the subject of the auction, and the
winner is the coalition that offers the least production cost. The manager of the auction
will be responsible for distributing rewards among members of the winner coalition. In
comparison, our model allows agents to deal with different sets of tasks. Such cases are
closer to practice. The next essential difference is using reference list to guide choosing
coalitions. In the real-world domain, an agent might have different opinion about each
partner, so all agents are not the same partner for cooperation. In addition, in our model
small size coalitions are recommended in order to guarantee the stability. Creating large
coalitions might have better effectiveness; however, they are not as stable as the small
ones.
Paper [7] deals with a similar problem, but agents apply the fuzzy technique to evaluate coalitions. Fuzzy is an excellent technique for making decision, mainly when agents
have many alternatives to choose. Fuzzy could also be used to resolve many complicated
relationships among agents. However, complicated relationships as in the real-world domain are still not considered in our model. Agents have to make a deterministic decision
of joining coalitions or not; and therefore there are not too many alternatives to reason.
The technique used in our model is adequately comfortable for solving negotiation among
agents. But applying fuzzy technique might be the subject of the future work, since more
realistic situations will be considered to implement, in order to make product to be closer
to the reality.
8
Conclusion
This paper presents a new way to improve a quality of solving the planning and scheduling
problems. The benefits of each producer (considered as agent in this paper) could be
improved, if they join a coalition. By joining coalitions, agents can share their production
capacity. They can exploit their capacities more effectively; in addition, they can help one
another to improve their benefits.
In this paper, several practical strategies for evaluating and choosing coalitions are presented. All these strategies are applicable for making negotiation among agents. Negotiation is based on the FIPA-Contract-Net protocol that is supported by the FIPA standard.
The proposed strategies are deduced from practical situations and experience; moreover,
on the assumption that agents do not have complete information or knowledge about the
others. To verify and compare these proposed strategies, a number of experiments were
made. The achieved results point out many useful conclusions, which could help agents
to achieve better results in solving real-world situations.
On the basis of the analysis of the achieved results, this work could continue with the
DANG THANH-TUNG, ET AL.
following subjects.
– Applying fuzzy technique to resolve more complicated situations, where not everything could be expressed by differential or arithmetical equations.
– Integrating competition among agents to the negotiation, because this topic is very
up-to-date in daily business.
– Integrating learning capabilities, in order to allow agents to learn from historical
experience and to deduce new knowledge.
Acknowledgments
This paper is partially supported by APVT and VEGA grant agencies under grants No APVT 51
011602, APVT-51-024604 and VEGA 2/1101/21, the Slovak national project NAZOU (SPVV
1025/2004), and K-Wf Grid project (EU RTD FP6-511385).
References
[1] Frankovic B. and Budinska I. Single and multi-machine scheduling of jobs in production systems.
Advances in Manufacturing. Decision, Control, and Information Technology, Tzafestas, S.G. (Editor),
Springer Verlag, 1998.
[2] Zlotkin G. and Rosenschein J. Coalition, cryptography and stability - mechanisms for coalition formation in task oriented domains. In In Proceedings of the Twelfth National Conference on Artificial
Intelligence (AAAI-94), pages 432–437, 1994.
[3] Frankovic B. and Budinska I. Advantages and disadvanteges of heuristic and multi-agent approaches
to the solution of scheduling problem. In In: Proc. of the IFAC Conference on Control System Design,
Bratislava, pages 372–378, 2000.
[4] Caillou P., Aknine S., and Pinson S. Multi-agent models for searching pareto optimal solutions to the
problem of forming and dynamic restructuring of coalitions. In In Proceedings of the 15.th European
Conference on Artificial Intelligence, Lyon, France, pages 13–17, 2002.
[5] Kahan J. P. and Rapoport A. Theories of coalition formation. Lawrence Erlbaum Associates, Hillsdale
NJ, 1984.
[6] S. Kraus, O. Shehory, and G. Tasse. Coalition formation with uncertain heterogeneous information. In
In the proceeding of AAMAS-2003, pages 1–8, 2003.
[7] Mares M. Fuzzy coalition structures. Fuzzy Sets and Systems, 114, pages 23–33, 2000.
About the Authors
Dang Thanh-Tung received his MSc and PhD degree in 1996 from Slovak University of Technology in Bratislava, Slovakia and in 2003 from Institute of Informatics, SAS, respectively. His
research focuses on multi-agent systems, scheduling and discrete optimization, knowledge management and intelligent controlling.
Baltazar Frankovic is a member of IFAC/IFIP TC (technical committee), scientific society ECCAI, SSKI, J.von Neumann Society Budapest. He is a member of the Central European Academy
of Sciences and Arts. His current research interests in modeling and simulation of FMS, adaptive
and learning control in DEDS, MAS.
Ivana Budinska graduated from Slovak Technical University, Faculty of Electrical Engineering
in 1987, and received the PhD degree from Institute of Informatics, SAS, in 2003. Her research
interests in DEDS, control theory and MAS.
USING AGENT COALITIONS FOR IMPROVING PLAN QUALITY
Con Sheahan received his PhD degree from the University of Limerick in 1996. His research
interests on the domain of manufacturing resource optimisation, including operations research,
discrete mathematics and complexity theory.
Dang Bao-Lam received the MSc degree on Automation in 1998 at Slovak University of Technology, Bratislava, Slovakia and MSc degree on Mechanics of Machines in 2004 at Hanoi University
of Technology, Viet Nam. His research interests on kinematics, dynamics, control and simulation
of robotic systems.