Artificial Intelligence in
Game Design
Goal-Oriented Action Planning
Planning
• Creating series of actions to meet some goal
• Planning actions to meet multiple needs
– Chosen so final state has highest utility
– Intermediate states after part of sequence should not be unacceptable
– Allows fast actions to be chosen in logical circumstances
• Planning actions with multiple steps before payoff
– Purchase ingredients
– Cook ingredients in oven
– Eat meal
– No effect on hunger until after last step
Planning Example
Action
Effect on Fun
Effect on Energy
Paint
-2
+1
Go to Concert
-9
+3
Nap in Chair
+1
-2
Sleep in Bed
+5
-10
Need
Current
level
After Paint
After Go to After Nap
Concert
in Chair
After Sleep
in Bed
Fun
8
62
02
92
132
Energy
6
72
92
42
02
100
85
81
107
169
Total effect
Best action if single action allowed
Will be very tired afterward!
Planning Example
• Better approach for two large needs:
– Take fast action to relieve one
– Then take another action to relieve other
• Example:
– Take nap before concert
Need
Current level
After Nap in Chair
Then After Go to Concert
Fun
8
92
02
Energy
6
42
72
100
107
49
Total effect
Better than taking
single action
Planning and Search Trees
• Must try all possible combinations of actions
– Compute total discontentment for each path
– Choose path with lowest total
root
Go to
Concert
Paint
Paint
Paint
Paint
Go to
Conc
ert
Nap
in
Chair
Sleep
in
Bed
Nap
in
Chair
Paint
Paint
Paint
Go to
Conc
ert
Paint
Nap in
Chair
Go to
Conc
ert
Go to
Conc
ert
Go to
Conc
ert
Nap
in
Chair
Go to
Conc
ert
Sleep
in
Bed
Nap
in
Chair
Go to
Conc
ert
Nap
in
Chair
Nap
in
Chair
Sleep in
Bed
Nap
in
Chair
Sleep
in
Bed
Sleep
in
Bed
Paint
Sleep
in
Bed
Go to
Conc
ert
Sleep
in
Bed
Nap
in
Chair
Sleep
in
Bed
Sleep
in
Bed
Planning and Search Trees
• Expanding search tree is costly
– Given a possible actions
– Given n possible levels
• an possibilities to test
– Will only be able to test small number of action combinations
(lookahead limit)
– Even two actions enough to seem intelligent
• Looks like character thinking ahead
Planning and Search Trees
• Algorithm: depth limited search
– Depth-first search to some fixed limit n
•
an leafs in tree
– At each leaf, compute discontentment
– Just keep track of best path found so far
• Storage cost: n
Best sequence of
actions found so far
Current path
being tested
Pruning and Search Trees
• Can cut off search down obviously bad branches
– Branches with intermediate state unacceptably bad
– Can possibly save search time (but no guarantee)
Character dead
Character dead
Discontentment
= 172
No path can be better
than best found so far,
so no further search
No path can be better
than best found so far,
so no further search
Planning and Preconditions
• Actions may have preconditions before they can be
taken
Preconditions
Action
Postconditions
• Multiple actions required to meet needs
Preconditions which
character meets
Action
Postconditions
meets
Preconditions
Action
Postconditions
meet need
Planning and Preconditions
• Example: Microwaving food
Uncooked food
in fridge
Get food
Have uncooked
food
meets
Have uncooked food
Have microwave
Cook food in
microwave
Have cooked
food
meets
Have cooked
food
Eat food
Hunger - 5
STRIPS Planning
• Simple planning for worlds with preconditions on actions
– “Stanford research intelligent planning system”
– Prohibitively expensive for complex problems (thousands of
possible actions, prerequisites, etc.)
Preconditions
Action
Postconditions
– Widely used in games where actions limited
(Quake, Age of Empires, etc.)
• Plan of attack for single NPC
• Series of actions to meet large-scale goal
(“increase number of citizens farming”)
STRIPS Planning
• Regressing searching:
Working backwards from goals to preconditions
goallist = {unmet needs}
while (unmet goals in goallist) {
choose goal G from goallist
select action A with G in postconditions
remove G from goallist
add unmet preconditions of A to goallist
}
STRIPS Planning Example
• Initial goal
• Choose “attack” action
which meets initial goal
• Precondition added to list
• Choose “load weapon” to
meet precondition
• Its precondition added
• That precondition already
true, so plan complete
Selecting Actions
• Different characters  Different set of possible actions
• Algorithm selects from that character’s actions which
meet prerequisites
Key in current
room
Get Key
Door unlocked
Door unlocked Open Door
Be in room
Smash Door
Be in room
Planning as Search
• Can take form of search tree
– Depth-limited search
• Must backtrack if current plan has unmeetable preconditions
– Adds appearance of intelligence
– Can be expensive if happens too often
No available
action has this as
postcondition
Backtrack and try
this action
Key in
current room
Get Key
Door unlocked
Door unlocked Open Door
Pick Lock
Door unlocked
Planning as A* Search
• Can use A* to choose next action to add to tree
• Cost of path found so far
– Actions assigned costs
– Cost of rest of plan from current state =
sum of actions from current state to goal
• Heuristic estimate of rest of path
– Number of unmet preconditions
– Possibly weighted by some “expected cost” measure
Precondition
Precondition
Precondition
Action
Action
Goal
Planning as A* Search
• Example:
–
–
–
–
“Open Door” action has cost 1
“Get Key” action has cost 2
“Pick Lock” action has cost 5
“Key in current room” precondition estimated cost = 1
Try first since
estimated path
cost = 4
Try next since
estimated path
cost = 6
Key in
current room
Get Key
Door unlocked
Door unlocked Open Door
Pick Lock
Door unlocked
Planning as Tree Building
• Actions with multiple preconditions represented as trees
– Actions with preconditions  Nodes with preconditions as
branches
– Actions with no preconditions  Leafs
Order troops to collect
1000 wood
Have 1000
wood
Build Cannon
Factory
Have 1000 iron
Order troops to
collect 100
wood
Build Iron
Mine
Order troops to mine
1000 iron
Mine iron
Produce Cannons in
Factory
Build Cannons
Planning and Scheduling
• Key question:
How to schedule the actions that result from plan
– Some branches may have to be accomplished before others
– Some branches can be done in parallel
Can be done in
parallel
Build Cannon
Factory
Order troops to collect
1000 wood
Order troops to
collect 100
wood
Build Iron
Mine
preceeds
Order troops to mine
1000 iron
preceeds
Mine iron
Produce Cannons in
Factory
Build
Cannons
Hierarchical Planning
• Overall plan created at different levels of abstraction
• Low-level subplans created to do higher level tasks
– High-level tasks have preconditions and postconditions
– High-level tasks have methods for expanding into subplans
– Steps of subplan accomplishes postconditions given preconditions
Task
Preconditions
Task
Postconditions
Task
Main plan
Preconditions
Action
Task
Action
Postconditions
Subplan
Action
Action
Hierarchical Planning
General plan to capture a city
Gather
weapons
Gather
wood
Have weapons
Move to
city
At city
Attack city
Build bows
and arrows
Move to
bridge
Cross
bridge
Move to
city
Subplan generated for this particular
city, level of technology, etc.
At city
Dynamic Replanning
• Conditions used to create plan may no longer apply in
course of executing plan
– Bridge NPC destroyed by player while gathering weapons
• Can create a different subplan to accomplish specific
task without replanning entire plan
– “Move to city” subplan accomplished by building boats
Have weapons
Move to river
Gather
wood
Move to
city
Build boats
At city
Cross river
in boats
Move to
city
At
city
Planning for Hierarchical Teams
• Goals of team should direct goals of individuals
• Individuals take actions to support overall actions of team
Planning for Hierarchical Teams
• Individual units meet goal based on individual abilities
– No micromanaging from overall system
Search all
rooms
Pick Lock
Open Door
Look in Room
with X-ray Vision
Overall
team goal
Smash
Door
Look in
Room
Look in
Room
Planning for Hierarchical Teams
• Individual units can balance team goals with individual
goals to give appearance of independent thought
• Each individual/subteam has relevance measure for
different goals
• Meets goal with highest current relevance
Goal
Relevance
Charge
TakeCover
Retreat
Patrol
0.6
0.8
0.3
0.0
Current action
Planning for Hierarchical Teams
• Individual goal modified by goals of team
– Example: Leader orders squad to charge, and not retreat
Goal
Charge
TakeCover
Retreat
Patrol
Relevance
0.6
0.8
0.3
0.0
Modifier
0.5
0.0
-0.5
0.0
Total
1.1
0.8
-0.2
0.0
Current action
Planning for Hierarchical Teams
• If individual goals strong, team goals may be overridden!
– Example: Leader orders squad to patrol
– This unit currently under ambush
Goal
Charge
TakeCover
Retreat
Patrol
Relevance
0.4
0.8
0.6
0.0
Modifier
0.0
0.0
0.0
0.5
Total
0.4
0.8
0.6
0.5
Current action