COOPERATING AGENTS FOR PLANNING AND SCHEDULING
Frankovic, B., Dang, T. T.
Institute of Control Theory and Robotics
Slovak Academy of Sciences
84237 Bratislava, Dubravska c. 9, e-mail {utrrfran, utrrtung} @savba.sk
Slovak Republic
Abstract: In this paper we will discuss about a problematical using multi-agent system (MAS) for resolving the
problem scheduling production. The basic idea for achieving global optimal solution is a coordinating all agents to
resolve (in our case each producer is considered as one autonomous agent). Each agent possesses own goal but it
can improve own goal by coordinating with another or help another one to achieve its goal. We will analyze what
may bring the coordination between agents and propose several mechanisms for coordination. Also we will
discuss about a methods of evaluating bid that the agents order between themselves.
Keywords: Planning, Scheduling, Agent, Cooperating and Decision.
1.
INTRODUCTION
In the industrial world we often can meet a situation
when many manufactories cooperative as one team.
In a practice the centralization approach of control
manufacturing in many cases is ineffective, has
high charges or slowly react to new appeared
situation. From such reasons therefore a new
approach favoring decentralization of decision is
required. Let us assume in a town there are n
producers (for example: n branches Mc Donald’s);
each possesses own production lines that is able to
cover its requirements to a certain extent. It is
certainly each producer wants to the most profit and
the least loose by a way, that always tries to
equilibrate amounts produced and sold. But is
possible to happen such situation that one producer
has broken production lines or receives more
requirements than as it assumes, in such case it is
necessary to ask a help and cooperative with
another producers. All producers after such event
must change own plan to help this one with
unexpected situation. The process of reconstructing
plans can stop when each producer may fell
satisfied with any plan, which (the plan) but must
meet all given it’s requirements and concurrently
may improve its profit. Such approach coordination
between agents in practice can enable more
effectively using production capacities of each
producer, can help whole system react faster and
more optimal to a environment’s changes as in case
introduced above. It also can minimize a time needed
to terminating all orders and a result is more profit
for each one.
With similar problematical deals a [Khmelnitsky, et
al., 2000], but in this paper the authors propose a
method for continuous-time scheduling problem in
flexible manufacturing systems with all same
production constraints as discrete system. The
proposed approach to the search for optimal solution
is based on the maximum principle analysis and
time-decomposition methodology. In [Ni and
Goscinsky, 1993] the authors also discuss about
problem management facility resources and trading
services. The paper deals how the users share
resources and services, which are not available in
their local system. In [Wellman and Walsh, 2000] the
authors deal a problem negotiation between agents in
asynchronous system as auction, market. Each agent
(seller or buyer) has own tactic to choose the best bid
for itself. In [Durfee and Lesser, 1997] is presented a
problem nearly as in this paper but here all solvers
together search optimal solution for one common
problem - production scheduling. At first each solver
gets a part of whole plan (tasks) to execute, it has
only a local view of whole problem; therefore its
execution may be ineffective. Every can transmit to
another and ask another one to produce a part of its
tasks.
In such system as mentioned above, each producer
can be considered as one agent with own
intelligence and abilities. Each agent can work as
independent agent (autonomous) or cooperate with
another (cooperative). Our aim in this paper is
create a site of cooperating agents able being
configured in resolving these problems what can
appear in practice.
2. GENERAL DESCRIPTIONS
In general, it is possible to present one producer
(one Agent) as to be consisted of sub-components
(workshop, machine, products, tasks…), in this part
we will in detail present some features of an agent.
At first each agent has a set of orders (O) what it
must to do, each order may be consisted of several
products (P). The products may be presented as a
set of tasks (Task), execution methods (Me) for
each task. Each product may be noticed as:
Pi={{T1,Me11,…, Me1n},…,{Tk,Mek1,…, Mekn}}.
Let be noticed a set of machines as:
M={M1,…,Ml}. In the system the agent has a
function to schedule these products’ tasks to queue
such as well as it considers for itself. Let notice Pl
as a set of all possible plans for one concrete order,
when and where the agent has to send tasks to
execute. The agent-producer is defined as
following:
Definition1: The agent-producer is a structure:
{M, O, P, Pl, Cost, Utilization, Time…}, where
Cost, Utilization, Time… are a function joined with
each plan from δ∈Pl.
Definition 2: The multi-agent system (MAS) is a
structure:
{A, M, O, P, Pl, Cost, Utilization, Time…}, where
A={1,2…n} is a set of agents. Cost, Utilization,
Time… is a set of functions joined with each plan
from δ∈Pl.
For simplicity we can assume that all agents
possess an equal production lines, it means a set M
is same for all agents. Another assumption is that
each task may be executed in same machine in
various agents for same time. Each agent may
produce whole order in its own production lines, or
may send a part of its order to another agent
(require for help), which has a less work or is in
idle state in same time. The agents afterward may
negotiate between themselves to find such plan
with which each is contented. There are three main
policies necessarily to analyze:
1-The information policy
agents or all one. It is very difficult to estimate how
much a work agent can send to another agent to
produce when it cannot have all in detail information
about this agent’s plan. The request that agent sends
away may be also refused, such situation may happen
if the agent, which has received this request is very
busy or it does not agree with this request. Another
point in the first policy is each agent can send a
request a periodic without knowledge about it that
the neighboring agents can or cannot agree, or only
when some another agent states that it is able to
produce more. In our case all agents are coordinate,
therefore these requests that agent sends to
neighboring ones may be distributed arbitrary. Each
agent on a basis of own knowledge divides these
requests to smaller parts as well as it can and another
one can accept it.
A2
Requ
Respon
Requ
A1
Respon
A3
Fig. 1: An example agent’s coordination
In Fig. 1 is introduced an example, when agent A2
want to require both neighboring agents A1 and A3
to help. Each request directed to neighboring agent is
a part of its plan that it cannot terminate in time or
the agent A2 may consider as better if both agents
A1 and A3 can produce it.
The policy 2 generally discusses about principle how
the agents exchange these requests and the policy 3
about who the agent can send these requests, but in
this paper we won’t analyze them in detail.
In following part are defined some notions that we
will use in later sections.
Definition 3: -The termination time any product (Te)
is time when its last task is terminated.
- The plan (δ) is a sequence of all tasks that agent
must execute to fulfill the order.
- Cost plan, utilization plan (let notice as C(δ), U(δ))
are criterion functions characterize each plan.
- Optimal plan is such plan δ* that does not exist any
another plan with smaller cost and utilization and
concurrently can satisfy all constraints.
δ* is optimal plan ⇔ not ∃ δ∈Pl | C(δ)<C(δ*) ∧
U(δ)<U(δ*).
3. COORDINATIONS BETWEEN AGENTS
2-The transfer policy
3-The placement policy
The policy 1 exactly discusses about amount of
information each agent wants to send and receive to
(respectively from) one another agent, one group of
It is assumed that each agent in searching and
proposing own plan observes what another agents
do. However all agents may have any common aim
(for example all producers belong to one big
concern) but each of them also has a private goal.
To achieving this private goal each agent tries to
manipulate with own tasks that it must to do and
tries to coordinate it with neighboring agents,
although it is able to produce whole order. As
mentioned in previous section a volume of
information that the agent wants to exchange
depends on how it knows about another agents’
plan. In this section are presented some
mechanisms for exchanging information between
the agents (for illustrate example we assume a
case where there are 2 agents A1, A2 with all
properties as introduced above).
1.
The agents exchange all information about
own plan: This approach is the best for
agents’ planning but claims a large set of
date to exchange.
2.
Each agent exchanges only a part of own
plan when it is able to receive a next order:
This approach in practice is more effectively
than previous one because not all date are
useful for planning.
3.
The agent A2 makes own plan after it knows
about plan agent A1: it is special case of
mechanism 1. Agent A2 can improve own
plan but agent A1 is disadvantaged to A2.
4.
The agent A2 on a basis of historic plans
estimates what agent A1 will do, on a basis
of these estimations makes own plan and a
part what A2 wants to exchange with A1.
5.
Each agent can send another one a request
when it terminated planning
6.
Each agent can send a request even if
another are searching own optimal plan.
The mechanism 5 and 6 may also combine with
mechanism 4, here agent must know what to
exchange without knowledge about another agents’
plan.
It is also one difficult question stays here that how
agents choose a part of own plan to exchange. In
the real life each agent may possess different
algorithms for choosing, each one also may have a
different criteria and each one may observe all
neighboring agents by various ways. It is similar to
a case that all agents are playing cards; each of
them may propose one variant, which is considered
as the best for it. In such game, but all agents must
cooperate together to find a better solution and
subject to each mustn’t damage another one. The
negotiation process will continue until each agent
found any plan, with which it is satisfied. The
agents may choose according to the game theory,
theory of Markov decision process or use fuzzy
theory in decision-making… On a basis of these
theories and a set of historic date, then, each agent
may propose what it wants to exchange. In this
paper we won’t focus about this problematical.
In next Section are presented some assumptions for
agents’ negotiation.
4. A MECHANISM OF NEGOTIATION
At first, it is dealt a simple case when only two
agents take part in negotiation process, in next part,
then, is extended for a case with more agents
participated in negotiation process. Our idea is to
construct such model to enable agents to avoid
conflicts when they take part in negotiation process
or to avoid such instances where the negotiation
process can lead to infinity.
4.1 A case with two agents.
For simplicity is assumed that exist two agents: A1,
A2. The agent A2 requires agent A1 to produce a
part (∆) of its plan. Let notice a plan for the agent A1
and A2 as: δ1 and δ2. The part ∆⊂δ2 that the agent A2
wants to exchange with A1 may be a set of semiproducts (tasks) or complete products. It is assumed
to use a rule yes/no for negotiation, it means each
agent may agree or refuse an order what other
suggests it. A further assumption is each agent,
which wants to require another one to help, in a case
agreement must pay this agent not only all cost for
producing this request but also any payment (let us
notice it as a function Payoff(A2-A1)). On a basis of
these assumptions it is obtained a cost equation:
C(δ2*) = C(δ2) + Payoff(A2-A1) +
C(Transport).
(1)
where δ2* is a new plan for agent A2 after that the
agent A1 agrees to help. C(Transport) is a cost
function for transferring these tasks from A2 to A1
and again. The value of this function depends on
mutual locations between agents and a method of
transport. The value of a function Payoff(A2-A1)
depends on content of part ∆ that the agent A2 sends
to agent A1.
For the agent A1 is valid:
C(δ1*) = C(δ1) – Payoff(A2-A1).
(2)
where δ1* is a new plan for the agent A1 after
agreeing to help the agent A2.
For time conditions after exchanging plans are valid
these inequalities:
Time_end(δ1*) ≥ Time_end (δ1).
(3)
where Time_end (δ1) = {Te1,…,Ten} a set of
termination time of all products belonging to order of
the agent A1 if is used a plan δ1, similarly for
Time_end(δ1*). The inequality means all products
belonging to order of the agent A1 will be terminated
later or same as according to initial plan.
Time_end(δ2*) < Time_end(δ2).
(4)
This inequality is deduced from practice, because the
agent A2 certainly won’t ask the agent A1 to help if
it must pay more and its order is made later. In this
case Time_end(δ2*) includes still a time spending to
transferring tasks between the agents A1 and A2.
The assumptions that must to be satisfied to enable
this exchange are:
* Payoff(A2-A1) ∈ [min1,max1] where the agent
A1 can be satisfied with and
can afford to pay.
∈ [min2,max2] where the agent A2
These assumptions depicted above may be also
applied in a case where are more agents joined
negotiation process. The agents may negotiate one
with another in own group, each group also
negotiates with another group, etc… In each group
after a negotiation is obtained a global optimal plan,
with which all agents in this group are contented.
The final global plan must satisfy requirements of all
agent groups connected to negotiation process.
** Both new plans δ1* and δ2* must satisfy all given
time constrains, that means:
Time_end(δ1*) ≤ T01,
Time_end(δ2*) ≤ T02. where T01 and T02 are a set of
given deadlines for each product belonging to each
order.
From the assumption (**) is easy to deduce that for
the agent A2, the exchange is useful if the
Time_tran(∆) (spending time for transferring tasks
belonging to ∆ between agents) does not affect a
delay to initial plan as if these tasks are executed by
the agent A2.
The negotiation process begins by any initial bid of
the agent A2. The content of this bid is a part (∆) of
own plan what it wants to exchange and a payment
it orders the agent A1. This content is depended on
the agent A2 decision that it considers as well for
its production. The method of choosing bid is
discussed in previous section. The agent A1 after
receiving the bid from A2 can agree or refuse it if
this bid satisfies all its requirements or not. In a
case the agent A1 refuses, the agent A2 may choose
another bid and such recursive process may be
stopped in any state if both are satisfied with
proposed bid.
In general case both agents can order and receive
bid at once and each of them can behave similarly
as described above. The message that one agent
sends another, includes also a respond to the order
that it has received in previous step. The optimal
state for both agents and feasible algorithm for
negotiation may be defined as following:
Definition 4: The optimal state for both agents is
such state that any one of them does not want to
change own plan until another does not want too.
Definition 5: The stable negotiation algorithm is
such that it can lead the system to optimal state for
finite steps.
Theory 1: The negotiation process can be stopped
after finite steps if in each turn one of them retreats
by decreasing own demands.
Prof: (outline) By decreasing demands all agents
must agree with any plan because these demands
must always >0.
4.2 A case with a lot of agents.
A
A
A
A
A
A
Fig. 2: The example when are more agents connected
to negotiation.
Our purpose is to develop such model for agent
negotiations, when are more autonomous agents
participated in negotiation process to achieve one
common goal. To develop a complex model is
necessary to establish some another assumptions.
The first of them is: the agents in this model not only
execute one type of operations at time, but various
types, which may be divided into three categories as
follows:
• Communicative: which presents interactions
between agents,
• Observation: to gather information from another
agents that the agent is interested in,
• Executive: there are own operations that each agent
has to execute.
Another assumptions are:
• Communication is assumed to be fool proof,
• Durations of each task, methods for its execution,
all another constraints (order in which is executed,
amount, deadline) are fixed to all agents.
• Agents are assumed to be able to predict or learn in
manners based of own knowledge.
Our model is based in hierarchical approach (more
details introduced in [Frankovic-Dang, 01]), where a
negotiation is made successively in each level. It is
assumed that, the negotiation only exists between the
agents in same level and each organizing agent in
higher level has a capability to decide which agent in
lower level can be able to participate in negotiation
process. On a basis of this information, the tasks to
be performed, the order in which the tasks are to be
performed, and the deadline, the organizing agent
can project team of the agents in lower level, project
schedule or decides further to decompose these tasks
to individual parts and carries them forward to the
agents in lower level.
Obtained results by negotiation among agents, then,
are brought back to the organizing agent in higher
level. Information in feedback from the agents in
lower level reflects results of the organizing agent’s
decisions (may be consider it as a critic), and it can
make the organizing agent to choose other goal or
to change current its plan.
In each level, agent’s decision in a negotiation
process with other ones depends on these items:
• Information that it receives from its organizing
agent (request from a supervisor). Let denote it as
req. It is invalid for the agent in the highest level.
• Information that it receives from the agents in
lower level (critics from slaves). Let denote it as
cri. It is invalid for the agents in the lowest level.
• Information that it oneself obtains by observation
other agents in same level. An amount of this
information is depended on each agent’s
capability. Let denote it as obse.
any demand of the agent A1 or A2. The agent A1
agrees with payment in interval [1,6] and the agent
A2 is agreeable to pay from interval [0,5]. For both
agents is valid a condition: if each product is
terminated before a time 15 units, the agent gets 15,
if after this time gets 5 (deadline for all products is
15 time units).
In Fig. 4 is shown that, the agent A2 will terminate
one product after deadline (at time 16) therefore it
wants to ask the agent A1 to help. If both agents
exchange whole own plan as mechanism 1 in section
3, they may agree with such plans as shown in
following figure.
• Information that it obtains by exchange with other
agents in negotiation process. Let denote it as ex.
Request from higher level
Observation
Agent
Exchange
Feedbacks from lower level
Fig. 3: Input information for agent decision.
A agent’s decision, then is a function as:
Decision = f(req, cri, obse, ex)
(5)
A more about hierarchical approach applied in
decision and negotiation process is presented in our
work [Frankovic-Dang, 01].
5. EXAMPLES
For illustrate example is chosen a case with two
agents A1 and A2. Initially each agent has an own
plan as autonomous agent. The plan is optimal for
agent’s execution in this moment. In the Fig. 4 are
shown these plans, where slots with same pattern
are tasks of one product and these must be executed
one after another. Each agent possesses two various
machines.
Fig. 5: A new plan obtained by using mechanism 1.
After exchanging information the agent A2 asks A1
to produce its tasks T1, a payment that A2 wants to
pay A1 is 2. Because the agent A1 can add the task
T1 to own plan without violation a deadline
condition and the payoff the agent A2 orders is
feasible for it, then both agents can agree with new
plan. In the new plan a machine 1 (Ma11) of agent A1
can execute the task T1 at time 1 because of a time
of transport between them. Similarly the machine 2
(Ma22) of agent A2 can continue to execute this
product till at time 8 however task T1 is terminated
at time 7 in the agent A1. The agent A2 gets 10 more
because one its product before negotiation is
terminated after deadline and in a new plan it is
terminated earlier than this deadline (task T2 in new
plan is terminated at time 12 units). A cost that the
agent A2 must pay for transport (2 times) and the
agent A1 is 4. Overall profit agent A2 is increased to
6 (=10-4) and agent A1 is increased to 2 (a payoff
that it receives from agent A2). By negotiation both
agents really achieve a better results than they work
independently.
If both agents use a mechanism 5 from section 3, the
agent A2 can ask the agent A1 to produce task T2
because it is terminated after deadline however the
agent A1 can refuse it. With same data as above both
agents may agree with such plan:
Fig. 4: Initial plans agent A1 and A2.
Next assumptions are: C(transport) between A1
and A2 is 1. Time_tran(∆) is 1 unit time where ∆ is
Fig. 6: A new plan obtained by using mechanism 5.
In this case the agent A2 terminates two products in
a bounds of deadline because the task T2 must be
delivered to the agent A2 after 1 unit time. The
profit for both agents is same as previous case. The
mechanism 6 from section 3 may bring an
improvement such that the agent A1 considers these
tasks received from the agent A2 as own tasks. In
searching optimal plan the agent A1 may schedule
these tasks before own one and it can help the agent
A2 to terminate soon.
enables to react dynamically and effectively to each
change in the environment.
References
1.
Benjamin N. Grosof: Conflict Resolution in
Advice Taking and Instruction for Learning
Agents. http:// www.watson.ibm.com
2.
Diana F. Gordon: APT Agents (2000): Agents
That Are Adaptive, Predictable and Timely. In
the Proceeding of the first Goddard Workshop
on Formal Approaches to Agent-Based
Systems.
3.
E. Khmelnitsky, K. Kogan and O. Maimon
(2000): A Time Decomposition Method for
Sequence-Dependent Setup Scheduling under
Pressing
Demand
Conditions.
IEEE
Transaction on Automatic Control, Vol. 45,
No. 4.
4.
Y. Ni and A. Goscinsky (1993): Resource and
Service Trading in a Heterogeneous Large
Distributed System. In Proceeding of the
IEEE Workshop on Advances in Parallel and
Distributed Systems. Princeton, New Jersey,
October.
5.
M. Wellman and W. Walsh (2000):
Distributed
Quiescence
Detection
in
Multiagent Negotiation. In Proceeding of the
fourth International Conference Multiagent
Systems. July.
6.
M. Wellman and J. Hu (1998): Multiagent
Reinforcement
Learning:
Theoretical
Framework and an Algorithm. In Proceeding
of the fifteenth International Conference on
Machines Learning (ICML-98). Madison, WI.
7.
E. H. Durfee and V. L. Lesser (1997): Using
Partition Plans to Coordinate Distributed
Problem Solvers. In Proceeding of the tenth
International Joint Conference on AI, August.
Another related problem is that we want to continue
to develop a model for agent negotiation in
hierarchical approach such, which can be applied in
every general situation. Another is to improve
general theory for agent negotiation to enable to
achieve better solution in appropriate time.
8.
S. Haffner, A. Monticenlli, A. Garcia, J.
Mantovani and R. Romero (2000): Branch and
bound algorithm for transmission system
expansion planning using a transportation
model. IEEE Proc.-Gener. Transmission
Distribution, Vol. 147, No. 3, May.
In this paper we have tried to apply theory MAS to
resolve a big problem as coordinating producers to
achieve better results for all concerned members.
We have proposed several mechanisms for
coordination these producers to achieve a feasible
plan for all one. In section 5 the obtained results
show our propositions really bring an improvement,
the producers by coordination can increase own
profit, also they can use free capacities to help
another one to achieve its goal. The MAS approach
is near with real life than centralization and it
9.
B. Frankovic, T. Dang, I. Budinska (2001):
Agent based process for production
scheduling. In Proceeding of the fifth IEEE
International
Conference
on
Intelligent Engineering Systems (INES -01).
Helsinki, Finland
There are many results that we obtained by using
these mechanisms described in section 3 or their
combinations but in this short paper we cannot
show all them.
6. CONCLUSION
In the practice it is very difficult to choose optimal
number of agents at any instant to negotiate because
someone may be satisfied with own plan that does
not want to change, another also is in a bound of
own capacities. Another problem is each agent may
possess different rule for valuation bid and may
happen a situation that the rule of any agent
conflicts with the rule of another one and
negotiation process may continue to infinity. For
example one agent prefers a minimizing cost and
agrees with longer production time, another one
prefers against: minimizing production time
however cost may be larger. In such case they may
never achieve any plan if anyone of them does not
concede. The papers [Grosof, 2000] discuss about
such problem where one agent may meet two
requests to execute from two another agents but
these requests are mutually conflicted.
A mechanism 4 from section 3 requires the agents
to possess such knowledge, which enables the
agents to predict other plan. They may use Qlearning algorithm, about this problem is discussed
in many papers, or enters to coalition with another
to resolve this problem together. In future work we
will deal about this problem.