Using a Goal/Action Architecture
to Integrate Modularity and LongTerm Memory into AI Behaviors
Marc Atkin
John Abercrombie
Why Do We Want Better AI?
• Bad AI is always noticed
• AI often inconsistent between
games – why can’t we build on
previous work?
• The AI is a big part of what makes a
game world immersive and fun
What’s Stopping Us?
• What makes AI complex?
– AI has many possible reactions to many situations
– Many behaviors are operating on the AI at any given
time
– Hard to account for all the possible interactions
The Basic Idea: Use Modularity
• It’s not just for coding anymore!
• Produces AI that is easier to
understand and debug
• Produces AI that it reusable within
and between projects
• The key point: Design the AI
architecture so that it facilitates
modularity
Part 1: Tyrion
Tyrion
• An architecture for specifying and
executing AI behavior
• Implements a language in which to
write behavior
• Encourages hierarchical, modular
design
Resources, Goals, and Actions
• Goal: A description of a
desired world state
• Action (or behavior):
A method for achieving a goal
• Resource:
An entity required to
perform an action
Resource Hierarchy
Team
Squads
Vehicles
Driver/Gunners
Characters
Legs/Arms/Head
Resource hierarchy
Team
Strategic level
Squads
Tactical level
Vehicles
Driver/Gunners
Characters
Legs/Arms/Head
Individual
Units
Motor Control
Action/Goal Hierarchy
Attack Goal
Squad
Attack From All Sides
Character
Attack Goal
NormalAttack
Legs /
Arms
MoveTo Goal
MoveTo
Attack Goal
FlankingAttack
FireAt Goal
Fire Weapon
Action Structure
• Goal satisfied by action
• Selection heuristic
– Evaluates appropriateness of action
in a given situation
• Message callback functions for child
actions
• Message callback functions for sensors
• Body (Unreal script latent code)
– Typically executes over a number of game ticks
Action Idioms
• Simplify writing action bodies
• Encapsulate commonly performed
tasks
• Examples:
– WaitForAllGoals
– WaitForAnyGoal
Execution Model
• 10 times a second:
– Iterate over every resource:
•
•
•
•
Order unmatched goals by priority
Find actions that achieve them
Tick every running action
Tick every active periodic sensor;
send messages
• If an action completes, mark its goal as achieved/failed;
send messages
Part 2: Building Modular Actions
SWAT4
• What is SWAT4?
– First person tactical shooter
– Player is team leader
– Single player – player commands four
officer AI’s
– Object is to kill or subdue all
enemies, and rescue the hostages
What Makes Up the SWAT4 AI?
• Hierarchical resources for
organizing behaviors
Tactical
Squad
Individual
Character
Motor Control
Movement
Weapon
• Layer behaviors to form more
complex behaviors
Modularity in SWAT4
• Start from the ground up – Motor Control
– The Legs - Movement Resource
• Basic Movement – move to a position, move to a
particular object, follow another AI
• Lower body rotation
– The Arms – Weapon Resource
• Aiming – aim at a position, to a rotation, at
something
• Item usage – (i.e. guns, grenades, toolkit,
explosives)
Modularity in SWAT4
• The Character
– Controls arms and legs
– Plays special full body animations
• The Squad
– Controls individual characters, tells
them what to do so they can work
together
Reuse
• Tyrion framework encourages tasks to
be broken apart into sub-tasks
– Higher level behaviors use lower level
behaviors to accomplish complicated tasks
– Lower level behaviors can be used over and
over again by different higher level
behaviors
• i.e. Movement, aiming, weapon usage
Layering
• Each behavior on a resource is
responsible for controlling everything
below it
– i.e. Officers move in a formation while
aiming at points of interest in the world
(Fall in Command)
• Using multiple characters in a squad
allows group to work together to
accomplish more
Organizing Behaviors
Use of action idioms
– waitForGoal, waitForAllGoals,
waitForAnyGoals,
waitForResourcesAvailable
• Parent action posts child goals to
accomplish tasks
– on the same resource or one lower in
the hierarchy
•
Organizing Behaviors
In SWAT4 the Squad posts child
goals on characters (the officers)
• Character behaviors usually post
child goals on movement and/or
weapon resources
• Each action can wait for child goals
to complete using action idioms
•
Example: Move Squad to a Point
• Squad behavior (MoveSquadTo):
– Posts MoveTo Goal on each officer
– WaitForAllGoals(MoveToGoals)
• Character behavior (MoveTo):
– Posts Aim Around Goal on weapon
resource
– Posts MoveToLocation Goal on
movement resource
– WaitForGoal(MoveToLocation)
Stack Up (on a door)
• User points at door, gives command
– Squad behavior determines points for
Officers to stack up at
– Each officer locomotes to point given while looking
around or aiming
at the door if
they are able
1
1
2
2
How Stack Up Works
Squad Behavior (Squad Stack Up)
– Post Stack Up goal on each officer
• Pass Stack Up Point as
destination parameter
– WaitForAllGoals(StackUpGoals)
• Character Behavior (Stack Up)
– Post Aim At Door goal
– Post MoveTo(StackUpPoint) goal
– WaitForGoal(MoveToGoal)
•
•
Result: Movement & Weapon behaviors
act independently of each other
Move & Clear
Inherits from the Stack Up behavior
– Re-use functionality of getting AI’s stacked up
on the door
• Adds new behaviors:
– After finishing stacking up (but not trying the
door):
• Officer near the door opens it
• Another Officer told to start moving as
soon as the door is opened
– Team then moves through the door into next
room to points specified by a designer
•
Breach, Bang & Clear
Inherits from the Move & Clear behavior
– Open Door behavior replaced: first officer
blows door open using C2 explosives or a
breaching shotgun
– Second officer throws flashbang grenade
into room before entering
• Squad organizes characters
• Some basic character behaviors are reused
across many commands
– i.e. deploy grenade, deploy shotgun, or
deploy C2
•
Squad StackUp
Stack Up
Squad Move
& Clear
Move &
Clear
Squad Breach
Bang & Clear
Throw
Grenade
Aim Around
Use
Grenade
Move To
Move To
Move To
Move To
Move To
Deploy C2
Deploy
Shotgun
Aim At
Open
Door
Inherits From
Child Behavior
Summary and Results
Framework is key to organizing use of system
• Time to prototype and develop AI behaviors
reduced
• Interesting SWAT4 Stats:
– Number of Squad Behaviors: 33
– Number of Character Behaviors: 65
– Number of Weapon Behaviors: 10
– Number of Movement Behaviors: 11
– Number of Months Programming: 16
(May 2003 – September 2004)
•
Part 3: Long-term Memory
The Problem of Forgetful AI
• “Oh hello – have we
met before?”
• State machines don’t
remember past events
• What’s a natural way to
represent task-related
memory?
Two Types of Memory
• Task memory:
– What needs to be done
– The context of the current action
– Represented by the set of goals
• Cognitive memory:
– Sensor data that has accumulated during the AI’s life
time
– Represented inside sensors and actions
Tribes: Vengeance
• First-person shooter
• Emphasizes speed and freedom of
movement (jetpacks!)
• 3rd in the series
A Day in the Life of a Grunt
• A Grunt AI with three goals:
– PatrolGoal (priority 40)
– AttackGoal (priority 50, dormant)
– DodgeGoal (priority 90, dormant)
• AttackGoal will activate when an
enemy is sighted
• DodgeGoal will activate when a
visible projectile will hit the AI
Patrol Attack Dodge
Goal Goal Goal
Patrol is executing
legs
arms
40
Patrol Attack Dodge
Goal Goal Goal
Patrol is executing
Enemy spotted!
Sensor wakes up AttackGoal
Attack executes; posts subgoals
for legs & arms
legs
40
arms
legs
arms
X
40
50
50
Patrol Attack Dodge
Goal Goal Goal
Patrol is executing
Enemy spotted!
Sensor wakes up AttackGoal
Attack executes; posts subgoals
for legs & arms
legs
arms
legs
X
40
arms
Projectile spotted!
Sensor wakes up DodgeGoal
Dodge executes; needs legs;
arms continue executing Attack!
40
legs
arms
50
50
X 50
X
40
50
90
Patrol Attack Dodge
Goal Goal Goal
Patrol is executing
Enemy spotted!
Sensor wakes up AttackGoal
Attack executes; posts subgoals
for legs & arms
legs
arms
legs
Dodge finishes: legs resume
Attack subgoal
X
40
arms
Projectile spotted!
Sensor wakes up DodgeGoal
Dodge executes; needs legs;
arms continue executing Attack!
40
legs
X 50
X
40
50
arms
legs
arms
50
50
X
40
50
50
90
Seeing the Big Picture
• Task memory: An AI’s goal set
constitutes the “big picture” of tasks to
be accomplished
– Failed or interrupted goals hang around
• Cognitive memory: actions potentially
contain lots of stored data
– High level actions have long life span; their data sticks around
for that time, too
Seeing the Big Picture
• Explicit goal representation:
AI can reason about remaining tasks
• Behaviors can be chosen that
achieve multiple goals
• Separate rules decide when to
remove unachieved goals
Part 4: Collaboration
Why Collaborate?
• Increases productivity
• Both working on FPS’s starting at about the
same time using the same engine (Unreal)
• Development teams based in Boston
(SWAT4) and Canberra, Australia (Tribes)
• Teams shared code for rendering,
networking, and more
• Tyrion was an already tested system
How to Work Together?
•
•
•
•
Communication
Shared Codebase
Perforce
Programming (of course!)
Communication
• Initial technical design document was
written in Australia, then discussed via
e-mail between offices for weeks
• E-mail response time was usually a day
(due to time difference)
• More complicated work could be
accomplished over the phone
• An in-person visit helped SWAT4 make
greater use of Tyrion’s features
Communication: Lessons Learned
• We underestimated the amount of time
required for effective communication
– During design phase, both of us spent about
1/3rd of our time communicating – less later on
• We underestimated the amount of time
required for “tech support”
• Collaboration allowed both teams to
identify weaknesses in Tyrion that we
would discuss and address
Shared Codebase
• Shared code between offices to
speed development of both Tribes
& SWAT4
• One shared codebase, which both
projects could integrate code from
or to
Perforce
• Very useful program, allowed for
easy collaboration between offices
• We didn't have to worry about
sharing the code, we got to worry
about the code itself!
Programming
• Marc did most of the implementation
• John's prior experience in Unreal helped
optimize and streamline the code
• We both felt comfortable editing the code
in the shared codebase for our own needs
• Code today is clean and well commented,
although it does contain a few
#if IG_SWAT and #if IG_TRIBES3
preprocessor definitions!
Programming: Lessons Learned
• Would have been nice to have shared a
common AI base class
• Both games exposed weaknesses that
were addressed
– Early in SWAT, the idle goal needed to be
satisfied by two subclasses of the base Idle
Action class
• Change was made to system to allow subclasses of
actions to satisfy goals
– Claiming leaf (motor control) resources
issue
Programming: Lessons Learned
• Inheritance vs Building Blocks
– Building blocks: new behaviors are implemented
by combining simpler ones
– Inheritance: Change an existing behavior by subclassing  Tyrion didn’t intend for this
– Not everyone will use your engine the way you
intended
• Having multiple ways of accomplishing tasks allows
clients to create and innovate
• Engine’s strength determined by client’s effective use
Collaboration Results
• The shared development of using Tyrion
worked well
• We feel both games had challenging and
fun AI
• Its use has made prototyping AI
behaviors easy and quick!
• Marc got to wake up at 7 a.m. in the
morning for a fire alarm while staying at
John's house
The Future
• We move on to new projects, and will
continue to use Tyrion
• Still finding improvements and new
ways of using Tyrion
• Engine Improvements
– Better goal/sensor integration
– Make AI more “designer-friendly”
– Shared AI subclass
Questions?