Managing
Complexity
in the
Halo2 AI
Damián Isla
Bungie Studios
Managing
Scalability
in the
Halo2 AI
Damián Isla
Bungie Studios
Scalability?
No! Scalability!
Why Scalable AI?
More is BETTER
The brute force approach to common sense: the
more unique situations you can recognize and
react to, the more common-sensical you are.
(e.g. Cyc, OpenMind projects)
3 Dimensions of Scalability
heretic grunt
swarms
brute captain
Miranda
elite major
flood
hunters
challenge
hell-jumpers
Variety
sniper marine
DRAMA!
story
Variation
pacing
boarding
drive the warthog
Volume
Shoot the needler
Strafe target
fight
perch
melee attack
hide
The Ultimate Goal
Designer
I want X to do Y whenever Z!
Engineer
Okay. (To himself) Hmm … and I know exactly
where to put that …
Designer
I’m happy now. (Trundles back to design-bubble)
AI Design Requirements
• Transparency
– Facilitate the ongoing narrative in the player’s head
• Coherence
– Focused action, right priorities
• Directability
– For the designer
• Workability
– For the engineer
See [Butcher & Griesemer]
GDC 2002
Managing
Scalable
DecisionMaking
in the
Halo2 AI
Damián Isla
Bungie Studios
Level
scripting
Character
definition
Decision
making
Perception
Memory
Level
scripting
Character
definition
Decision
making
Perception
Memory
Behaviors
Behavior: a program that takes
temporary control of the actor
in order to get something done
Behaviors are action over time:
• Move to a point
• Orient
• Shoot
• Crouch
• Play dialogue
• Trigger “actions”
World
e.g.,
• Perception
Fight & Knowledge Model
• Vehicle entry
• Throw
Targetgrenade
Selection
• Find cover
• Cover friend
• Check body
• Wake up a sleeping grunt
• Etc. Behavior
Halo2: ~130 in total
Behaviors have notion of
relevancy and duration
Looking
Aiming /
Shooting
Moving
Talking
Animation, Physics, Sound
The Combat Cycle
Search
Idle
Combat
Selfpreservation
Flight
Decision-Making
Behavior Tree
Retreat
(or DAG)
Flee
Cover
Self-preservation
Guard
Grenade
Good
•
•
•
Intuitive
Modular
Scalable
Vehicle strafe
Charge
Root
Melee
Vehicle fight
Engage
Bad
•
•
•
Never really that simple
Debugging is hard
Memory is hard
Idle
Fight
Presearch
Search
Uncover
Guard
Investigate
Guard
Decision routines
• Parent custom
Child 1
Child 2
Parent
• Child-competitive
?
Child 3
Child 4
Decision routines
Child-competitive strategies
• Analog Relevancy
• Binary Relevancy
–
–
–
–
–
Prioritized-list (root, engage)
Sequential
Sequential-looping (search)
Randomized
One-off randomized (postcombat)
Child 1
Child 2
Parent
?
Design Principle #1 :
Given A, B or C, always
choose D (all of the above)
Child 3
Child 4
Impulses
Problem: What happens (with a prioritized list) when the
priority is not constant?
Charge
Engage
Fight
Vehicle entry
Unless the
player is in
vehicle, in
which case...
Charge
Engage
Vehicle entry
Fight
Impulses
Solution: Separate alternative
trigger conditions out into
separate impulse
Charge
Player vehicle
Vehicle entry
entry impulse
Engage
Fight
Two execution options
• In-place
• Redirect
Vehicle entry
Charge
Design Principle #2 :
Formalize complexity
(don’t hide it)
Self-preserve
Self-preserve on
damage impulse
Root
Engage
Fight
Vehicle entry
Impulses
The other purpose of impulses:
Charge
Player vehicle
Vehicle entry
entry impulse
HACKS
Engage
Fight
Vehicle entry
Or, “localized code with ad-hoc
Functionality”
Charge
Self-preserve
Self-preserve on
damage impulse
Root
Engage
e.g.
• crouch_on_danger impulse
• dive impulse
Fight
Vehicle entry
Behavior Metadata
Problem: In determining relevancy, we check the same conditions over and
over and over and over
– Actor’s vehicle status (infantry, driver, passenger)
– Actor’s alert status (idle, in combat, visible target)
Solution: Use metadata to describe execution conditions
– Force all behaviors to declare their execution conditions
– Halo2: “conditions” bitvector compared to “actual state” bitvector
Vehicle
Retreat
Flee
Grenade
Self-preservation
Cover
Self-preservation
Guard
Cover
Guard
Grenade
Grenade
Vehicle strafe
Charge
Root
Charge
Melee
Root
Root
Vehicle fight
Engage
Suppressing fire
Fight
Engage
Engage
Presearch
Suppressing fire
Fight
Grenade
Presearch
Grenade
Search
Uncover
Search
Uncover
Search
Guard
Investigate
Guard
Investigate
Guard
Shoot corpse
Postcombat
Check corpse
Idle
Guard
Idle
Guard
Idle
Guard
Acts as a behavior/impulse mask (modifying the basic structure of the tree itself)
Design Principle #4 :
Variation from a
stable base
Custom Behaviors
Problem: character-type specific behaviors
– Don’t want all characters evaluating behaviors that only one type can do
– E.g. grunt’s “retreat from scary enemy” impulse
Solution: attach custom behaviors to tree
Retreat
Flee
Retreat
Self-preservation
Self-preservation
Flee
Retreat on scary
enemy impulse
Retreat on danger
impulse
Root
Grenade
Root
Deploy turret
impulse
Charge
Charge
Vehicle fight
Engage
Generic
Grenade
Vehicle fight
Engage
Fight
Fight
Search
Search
Guard
Guard
Grunt
Stimulus Behaviors
Problem: rare event-driven behaviors tested for every tick
e.g., Grunts flee when their leader dies
actor
Vehicle
Retreat
Flee
Grenade
Self-preservation
“Actor Died”
The world
events
Root
Stimulus
System
(event
handlers)
1 sec.
Cover
Guard
Leader dead
retreat impulse
Grenade
Charge
Vehicle fight
Engage
Fight
Search
Solution: Stimulus Behaviors
Behaviors or impulses dynamically and
asynchronously placed into tree at
specified location
Guard
Joint Behaviors
Blackboard
“Hey, buddy
you wanna do
this?”
“Hey,
anybody
wanna do this?”
“Sure, why not?”
Remember
Design
Principle #1!
(always choose D,
all of the above)
Behavior Summary
•
•
•
•
•
•
•
Behavior DAG
Binary decision routines
Impulses
Metadata
Custom Behaviors
Stimulus Behaviors
Joint behaviors
The behavior tree is
fundamentally a
DYNAMIC structure!
Level
scripting
Character
definition
Decision
making
Perception
Memory
Memory and Memory
Problem: Persistent behavior state is impossible (too much data to keep
around!)
Solution: Only keep around state for behaviors that are running
Grenade
Self-preservation
Cover
Guard
Grenade
Vehicle strafe
Charge
Root
Melee
Vehicle fight
Engage
Suppressing fire
Fight
Presearch
Search
Uncover
Guard
Investigate
Charge state
Melee state
Grenade
Behavior
State
Root state
Engage state
unused
Problem: What about state we need to keep around regardless of whether
we’re running the behavior or not? (e.g. last grenade-throw time)
The Anatomy of Memory
…memory is a complicated thing.
Persistent per-behavior
Store memory as…
• Behavior state (short-term)
• Persistent per-behavior
• Persistent per-target
Per-target
Per-target
Behavior
Per-target
Props are also
• Perception caches
• Our knowledge model
“Props”
Behavior-state
(short-term)
The Anatomy of Memory
e.g. Search
• Aquire target 1
• Lose sight of target 1
• Search target 1
Persistent per-behavior
Per-target
– Last known location inspected
– New search point selected
• Aquire target 2
Per-target
Behavior
– Search deactivated, short-term
state discarded
• Target 2 killed
• Switch to target 1
• Search target 1
– Where do we search?
– Existing search point used
Per-target
Remember
Design
Principle #2
(formalize complexity,
don’t hide it)
Behavior-state
(short-term)
Level
scripting
Character
definition
Decision
making
Perception
Memory
Designer Direction
Variety
• Character definition
– Model, animation graph
– Behavior parameters
– Static control
Level
scripting
Character
definition
Decision
making
• Level-scripting
– Dynamic Control
Variation
Perception
Memory
Static Control: Variety
• Character types
– Elites
– Grunts
– Marines
• Variants
–
–
–
–
Gold elite
Red elite
Honor guard elite
Jetpack elite
• Each variant gets a .character
definition file
A LOT of parameters to
author and maintain!
64(variants) x
80 (behaviors/variant) x
3 (parameters/behavior) =
15,360
Static Control: Parameters
•
Character files divided into
parameter blocks
–
–
–
–
–
Vitality properties
Perception properties
Search properties
Weapon usage
Etc.
•
Blocks can be instantiated or not
•
Hierarchy: children characters
instantiate only significant
variations from parent
•
Root of tree is the “generic”
character, which provides
“reasonable” values for all
parameters
Generic
character
Marine
Helljumper
Grunt
Heretic
grunt
Elite
Grunt
major
Elite
major
Jetpack
elite
Elite
ultra
Remember Design
Principle #4!
(Variation from a
stable base)
Static Control: Styles
Styles are cool:
• Behavior mask
• Bias control parameters
• Superposable
(Another reason why Impulses
are great)
Dynamic Control: Organization
Halo1:
• “Encounters”
– Groups of actors
– Groups of areas
Halo2:
• “Squads”
– Groups of actors
• “Zones”
– Groups of areas
• “Orders”
Problem: moving actors
from one encounter to the
next was “architecturally
discouraged”
– A mapping between the
two
Dynamic Control: Orders
The Metaphor:
• “Take that hill!”
• “Hole up in that bunker!”
Or
“Occupy this space and behave this way!”
• Position direction
• Behavior direction
• Blah blah blah
Order
“Rout”
Order
“Forward”
Order
“Initial”
Orders and Behavior
Behavior direction
•
•
•
•
•
Vehicle behaviors enabled/disabled
Active camouflage enabled/disabled
Follow player enabled/diabled
Rules of engagement
Style
– Behavior mask
– Bias control parameter
“Alright, men, engage active camouflage and move
forward cautiously. And don’t open fire until the
Master Chief does!”
In summary
Area
Squad
Order
Design
Principle #5 :
Work with your
representations
Style
Style
Character
Definition
Recap
Design Principles:
1. Always choose D
(all of the above)
2. Formalize complexity
(don’t hide it)
3. Embrace the hackery
4. Variation from stable base
5. Work with your representations
Technical Takeaways
•
•
•
•
•
Dynamic behavior tree
Binary relevancy
Behavior masking
Principled approach to memory
Take smarts out of the
behavior and put it in the
knowledge model
Conclusions
• All systems need to “play ball”
with the core decision
mechanism
• Not looking for “smart”
architecture, looking for
expressive architecture
The Ultimate Goal
Designer
I want X to do Y whenever Z!
Questions?
Damián Isla
[email protected]