Integrating a Distributed
Agent-Based Simulation into
an HLA Federation
Gary Kratkiewicz
Amelia Fedyk
Daniel Cerys
Purpose of this Session
• Understand how to:
– Use distributed multi-agent systems, with a
specific emphasis on the Cougaar architecture
– Integrate a distributed multi-agent simulation
into an HLA federation
– Design a full-scale Cougaar society for
integration with an HLA federation
Outline
•
•
•
•
•
•
Motivation and Goals
Distributed Agent Computing and Cougaar
Integration Approach
Lessons Learned
Design considerations for Cougaar societies
Conclusion
Motivation
• Military combat simulations lack
sophisticated logistics components
• Logistics support is critical to combat
operations
• Cougaar-based logistics simulations provide
capabilities not available in current military
simulation systems
• Roadblock: lack of interoperability
Purpose
• Demonstrate the ability of Cougaar societies
to interact with other simulations via HLA
• This would allow the following:
– Credible proposals of simulation systems based
on Cougaar-HLA linkages
– The addition of realistic simulation of logistics
to combat simulations
Technical Goals (1)
• Demonstrate that a Cougaar-based society of
agents can act as an HLA federate
• Show that Cougaar time mechanisms can
integrate with HLA time synchronization
• Confirm that a Cougaar society can work with
a variety of RTIs:
– HLA 1.3 and HLA 1516
– Fully asynchronous and partially asynchronous
– Java and non-Java based
Technical Goals (2)
• Develop techniques for creating or
modifying Cougaar societies to be HLA
federates
• Identify
– Further areas of research
– Additional functionality required in a full-scale
Cougaar HLA federate
Outline
•
•
•
•
•
•
Motivation and Goals
Distributed Agent Computing and Cougaar
Integration Approach
Lessons Learned
Design considerations for Cougaar societies
Conclusion
Agents
• Agents are independent software entities that
react to events and initiate actions by
themselves
• Different agent-based architectures yield
agents with varying capability levels
• Agents can have or define roles, tasks, beliefs,
desires, or intentions
• Agents can be static or mobile
• Some agents or agent systems are considered
intelligent
Distributed Agent Computing
• A distributed system is a collection of
separate processes or information systems
that can act together as a single system
• Distributed agent computing involves
agent-based systems that operate in a
distributed manner
• Purpose:
– Implement complex behavior
– Model or simulate complex systems
Cougaar
• Cognitive Agent Architecture
• Java-based architecture
• Rich design can model diverse objects and
interactions
• Agents are Cougaar components with a
defined functionality and local memory
store (Blackboard)
• The behavior of the agent emerges from the
composite of plugins
Cougaar
• Blackboard(BB) implements a publish/
subscribe API
• Plugins view objects on the BB by creating
subscriptions
• Logic Providers are lightweight agent
components responsible for messaging and
BB modifications
• A collection of agents make up a cougar
society
Cougaar
PLATFORM OS
JAVA VM
J AVA VM
COUGAAR NODE
COUGAAR NODE
Agent Binder
Agent Binder
PlugIn
AGENT
AGENT .
FRAMEWORK
SVCS.
Agent Binder
AGENT .
PlugIn
Plugin
AGENT
Binder
Agent Binder
FRAMEWORK
SVCS.
Plugin
Binder
PlugIn
Plugin
Binder
Plugin
Binder
COUGAARNODE SERVICES
Plugin
Binder
Plugin
PlugIn
PlugIn
Binder
Agent
Agent Binder
Binder
Plugin
Binder
AGENT
.
AGENT
PlugIn
Plugin
Binder
Plugin
Binder
Plugin
Binder
Binder
Plugin
Binder
FRAMEWORK
SVCS. Agent Binder
Plugin
PlugIn
PlugIn
COUGAAR NODE SERVICES
P LATFORM S ERVICES
Plugin, Agent, and Node Structure of the Cougaar Architecture (K. Kleinmann et al, 2003)
UltraLog
• 4 year project sponsored by DARPA
• Layer built on top of Cougaar Architecture
• Developed to model military logistics
within a distributed multi-agent system
• Added Security, Robustness and Adaptivity
to the cougar infrastructure.
• Test society models interaction between
large set of military organizations.
UltraLog
• Society runs in strictly planning
mode(predictive solution) and execution
mode(simulated solution)
• Solution includes detailed transportation
and supply chain plan
• Society capable of garbage collection of
completed/stale BB objects
FCS Supportability
• Future Combat Systems (FCS) Supportability
extends Ultralog Functionality
• Agents designed to reduce the logistics
footprint
• Additional business logic added to plugins to
model highly mobile, self-sufficient units.
• Plugins enhanced to accept 3rd party
simulation data
Outline
•
•
•
•
•
•
Motivation and Goals
Distributed Agent Computing and Cougaar
Integration Approach
Lessons Learned
Design considerations for Cougaar societies
Conclusion
HLA Overview
Simulations
Simulations
Simulations
Simulations
Data
Data
Data
Collectors
Data
Collectors
Collectors
Collectors
Interfaces to
Interfaces to
Live
Interfaces
Players to
Live
Interfaces
Players to
Live Players
Live Players
Interface
Runtime Infrastructure (RTI)
Federation and Society Design
• Created small UltraLog logistics society
– Pared down to model only bulk fuel consumption
– 2 fuel-consuming organizations (agents)
– 18 other organizations in supply chain and chain
of command
• Two society federates based on above:
– High-fidelity logistics society for general
simulation
– Demand generation society to model fuel
demand
Prototype Society
NCA
HNS
USEUCOM
USAEUR
5-CORPS
1-AD
2-BDE-1-AD
1-35-ARBN
1-6-INFBN
47-FSB
21-TSC-HQ
3-SUPCOM-HQ
37-TRANSGP
DISCOM-1-AD
7-CSG
6-TCBN
123-MSB
316-POL-SUPPLYBN
110-POL-SUPPLYCO
900-POL-SUPPLYCO
Non-Cougaar Federate
• Performed verification with Java-based,
non-agent, non-Cougaar federate
• Demand generation society to model fuel
demand
RTI Selection
• Tested prototypes with following HLA RTIs:
– Pitch 1516 LE
– Pitch 1.3 LE
– DMSO NG 1.3
• Demonstrated the feasibility of using Cougaar
with:
– HLA 1.3 and IEEE 1516 RTIs
– Fully asynchronous (Pitch) and partially
asynchronous (DMSO) RTIs
– Java-based (Pitch) and non-Java (DMSO) RTIs
Simulation Models
• Cougaar time:
– Continuously advances with real (wall clock) time
– Can be advanced ahead to some future time
• Mapping to simulation model:
– Advance time with real time or ahead in equal steps 
time-stepped simulation model
– Advance time in unequal steps  event-driven model
• Most Cougaar simulations run for hours and
simulate operations over months
– Equal steps  scaled real-time
– Unequal steps  non-real-time
Time Management
• Cougaar time
–
–
–
–
Initialized to physical time
Advances at same rate as wall clock
Can be stepped to future point in time
Society should be in quiescence before stepping
• Cougaar and HLA time synchronization
– None: not useful for simulation, OK for planning
– Conservative: appropriate
– Optimistic: not appropriate; Cougaar cannot roll back
time
• OK since different federates can use different
synchronization methods
Iterative Development Approach
1. Build a federation without HLA time
management
•
•
Two federates
Allowed us to get running quickly
2. Build a federation with HLA time management
•
•
Two federates
Added RTI time synchronization
3. Build a federation with a multi-node Cougaar
society
•
•
Three federates total
Two federates map to one society
Phase 2: HLA Time Management
High - Fidelity Logistics
Society
Cougaar Node
Demand Generation Society
Cougaar Node
Advance
Advance
NCA
Time
NCA
Time
1 - 35 - ARBN
1 - 6 - INFBN
1 - 35 - ARBN
1 - 6 - INFBN
Ambassador
Ambassador
Service
Service
Federate 1
Federate 2
Time Management
HLA RTI
Supply Task Interaction
Phase 3: Multi-Node Society
High - Fidelity Logistics
Cougaar Node
Advance
Society
Cougaar Node
Demand Generation Society
Cougaar Node
Advance
NCA
Time
NCA
1 - 35 - ARBN
1 - 6 - INFBN
Ambassador
Service
Federate 1
1 - 35 - ARBN
1 - 6 - INFBN
Ambassador
Ambassador
Service
Federate 2
Time Management
HLA RTI
Time
Supply Task Interaction
Service
Federate 3
Results of Integration Experiments
• Demonstrated interoperability between Cougaar and
HLA RTIs in various combinations
• Demonstrated various operations:
– Subscribing a Cougaar federate and plugins to specific HLA
interactions
– Interfacing a Cougaar society to the RTI via a single
ambassador class
– Interfacing a Cougaar society to the RTI via a Cougaar service
– Synchronizing society time via the HLA mechanism using
conservative synchronization
– Detecting tasks in one society, transferring the task info via
HLA interactions, reconstituting the tasks in a second society
• Learned techniques and identified design considerations
Outline
•
•
•
•
•
•
Motivation and Goals
Distributed Agent Computing and Cougaar
Integration Approach
Lessons Learned
Design considerations for Cougaar societies
Conclusion
Interfacing Between Society and RTI
• Singleton ambassador class
– Separate ambassador class per RTI
– Performs following actions:
•
•
•
•
Instantiates the RTI’s ambassador class
Joins the federation execution (creating it first if necessary)
Connects to interactions and objects
Handles the interactions and object attributes from the
federation
• Distributes them to the appropriate agent in the society
• Later moved ambassador class to Cougaar service
• Separate class instantiated per Cougaar node
Time Synchronization: Phase 1
• Used a servlet-based, user-directed time
advancement mechanism
– Allowed us to operate with a variety of
simulation models
– Used for all phases
• Initially implemented without HLA time
management
– For development purposes only
– Advanced time in high-fidelity society
– Time advance passed to demand generation
society via interactions
• Small society – always ready
Time Synchronization: Phase 2
• Conservative time synchronization
– Standard HLA mechanism
– Modified version of the user-directed Cougaar
time advancement mechanism.
• Cougaar mechanism modified
– Obtains permission from the RTI before
advancing time in the society
– Required ambassador class, adding plugins to
handle time, and modifying the time advance
servlet
Time Management Details
1. User manually advances time via servlet
– Must manually determine quiescence
2. Servlet places time change request object on agent’s
blackboard
3. Time plugin subscribes to object on blackboard
– Reads it when it appears and calls ambassador service
4. Service checks for pending request; if not, sends
request to RTI
5. RTI grants advance when appropriate
6. Service calls callback method in plugin
– Changes society time
– Stores new federation time
7. RTI delivers appropriate messages to society
Mapping Agents, Actions, and Objects
• Cougaar societies contains many objects
– Map these objects to HLA actions and objects carefully
to ensure acceptable performance
– Data can be transferred in HLA via interactions (events)
and/or object attributes
• Mapping is heavily dependent on the society
design
– Cougaar agents or asset objects map to HLA objects
– Cougaar events map to HLA interactions
– Cougaar objects such as UltraLog tasks could be
mapped either way
• Depends on how long they can live and whether their contents
change
Outline
•
•
•
•
•
•
Motivation and Goals
Distributed Agent Computing and Cougaar
Integration Approach
Lessons Learned
Design considerations for Cougaar societies
Conclusion
Federate Granularity
• Federate granularity: the level of mapping of
federates to portions of a Cougaar society
– Consider size and organization of the society, and
design of other (non-Cougaar) federates
• In our demonstration prototypes, each Cougaar
node mapped to a federate.
• In a large complex society, a logical group of
agents (covering multiple nodes) might constitute
a single federate
– Transportation organization, combat organization, etc.
• Single agents could map to individual federates
– Make sure this makes sense
Society Design
• Selection of agents when federation
contains multiple societies
• Functional split
– Same agents, different functions
• Organizational split
– Different agents in logical groupings
Time Synchronization
• Need automated time advance mechanism
– Trigger time advance requests automatically
• Based on schedule or events
– Check for quiescence before making request
• Society time is always advancing
– Time step needs to be large enough that society
time advances an insignificant amount between
steps
Adaptive Society Design
• Prototype HLA ambassador class hardcodes HLA
interactions, objects, attributes
– Need to modify class when plugins change
• Better way: hardcode nothing in the ambassador class
– Each plugin would
• Register with the ambassador class
• Tell it which interactions and objects it was interested in
– The ambassador class would then
• Dynamically subscribe to the requested interactions
• Read the FOM file to get the attributes for each interaction
• Dynamically get the attribute handles
• Leverage Cougaar object discovery to match up
objects in federates with those in the society
Operation Independent of Federation
• For more flexibility and survivability, design your
society to operate
– Whether or not the appropriate federates are present
– If not available, use default plugins
• Or, use the Cougaar Message Transport Service
(MTS) and create an HLA-specific link protocol
– Appropriate where a society
• May or may not be a member of a federation
• Has a number of other communication channels at its disposal
– MTS would select communication method based on
availability and prioritization of channels
– Allows interoperation with simulations that exist outside of
both the Cougaar society and the HLA federation
Outline
•
•
•
•
•
•
Motivation and Goals
Distributed Agent Computing and Cougaar
Integration Approach
Lessons Learned
Design considerations for Cougaar societies
Conclusion
Conclusion (1)
• Successfully Integrated a Cougaar-based agent
society as a federate in V1.3 and 1516 HLA
federations
• Integration leverages the benefits of both
architectures
– Distributed agent-based architecture
– Standard simulation interoperability architecture
• Established how to
– Interface the society and the HLA RTI
– Synchronize society time with HLA RTI time
– Map agents, objects, and actions in the society to HLA
objects and interactions
Conclusion (2)
• Examined integrating a full-size Cougaar
society with an HLA federation
• Lessons learned can be applied to largescale efforts
– Such as integrating large Cougaar-based
logistics simulations with combat simulations
– Such integrated simulations would provide
greater effectiveness than combat-only
simulations
Integrating a Distributed
Agent-Based Simulation into
an HLA Federation
Gary Kratkiewicz
Amelia Fedyk
Daniel Cerys