Multiagent Probabilistic
Smart Terrain
Dr. John R. Sullins
Youngstown State University
Multi-Agent Search in Games
• Player in some
room on this
level
• Multiple guards
searching for
player
• Some rooms
more likely
than others
John Sullins
Youngstown State University
Multiagent Probabilistic Smart Terrain
CGAMES 2011
2
Multi-Agent Search in Games
• Guards must
“divide up”
rooms in
plausible way
• Focus on most
likely rooms
• While making
sure all rooms
searched
John Sullins
Youngstown State University
Multiagent Probabilistic Smart Terrain
CGAMES 2011
3
Outline
• Goals of multi-agent probabilistic search
• Background: Probabilistic smart terrain
• Estimating global expected distances to targets
that meet goals of group
• Matching agents to targets
• Demonstration on examples
John Sullins
Youngstown State University
Multiagent Probabilistic Smart Terrain
CGAMES 2011
4
Multi-Agent Smart Terrain
• Assumptions:
– “Teams” of NPCs with same goal (such as “find player”)
– One NPC finds target that meets goal 
entire team succeeds
• Must be fast solution
– No time for complex negotiations among characters
– Plausible behavior from POV of player sufficient
John Sullins
Youngstown State University
Multiagent Probabilistic Smart Terrain
CGAMES 2011
5
Plausibility Benchmarks
• Cooperative behavior:
–
–
–
–
Room 1 closer to G1 and more probable than room 2
G1 should still move to room 2
G2 can cover room 1
Both rooms searched quickly
John Sullins
Youngstown State University
Multiagent Probabilistic Smart Terrain
CGAMES 2011
6
Plausibility Benchmarks
• Probability as a factor:
– Room 1 much more probable than room 2
– G1 should move directly to room 1
– Player overwhelmingly likely to be found there
• Main purpose: Find goal, not search all targets
John Sullins
Youngstown State University
Multiagent Probabilistic Smart Terrain
CGAMES 2011
7
Plausibility Benchmarks
• Divide and conquer:
– G1 closer to both rooms and could explore both
– G2 should still move to the rooms also even though
closer to neither
– If G1 moves to one room, G2 can quickly cover the other
John Sullins
Youngstown State University
Multiagent Probabilistic Smart Terrain
CGAMES 2011
8
Outline
• Goals of multi-agent probabilistic search
• Background: Probabilistic smart terrain
• Estimating global expected distances to targets
that meet goals of group
• Matching agents to targets
• Demonstration on examples
John Sullins
Youngstown State University
Multiagent Probabilistic Smart Terrain
CGAMES 2011
9
Smart Terrain
• Target meets goals  transmits “signal”
– Signal moves around objects, weakens with distance
• Character follows signal to target
John Sullins
Youngstown State University
Multiagent Probabilistic Smart Terrain
CGAMES 2011
10
Limits of Smart Terrain
• Normal smart terrain not
appropriate for all situations
• “Guard search” example:
– Player “transmits signal”
– Guards follow directly to player
– Obvious cheat!
• Guards should have to
search for player based on
probabilities
John Sullins
Youngstown State University
Multiagent Probabilistic Smart Terrain
CGAMES 2011
11
Probabilistic Smart Terrain
• Targets broadcast signal of form “I meet goal”
“I may meet goal with probability P ”
John Sullins
Youngstown State University
Multiagent Probabilistic Smart Terrain
CGAMES 2011
12
Expected Distances
• Expected number of tiles character must
travel from tile x to target that meets goal
dmax
Dist(x) = Σ
 (1 – pi )
d=0 di < d
Summed over all distances
up to some maximum dmax
Probability no target within
d tiles of x meets goal
(otherwise sum could be infinite)
(assumption of conditional
independence)
John Sullins
Youngstown State University
Multiagent Probabilistic Smart Terrain
CGAMES 2011
13
Expected Distances
• Compute Dist(x) for adjacent tiles x
• Move to adjacent tile with lowest Dist(x)
p = 0.7
distance = 6
p = 0.6
distance = 8
John Sullins
Youngstown State University
Multiagent Probabilistic Smart Terrain
CGAMES 2011
14
Outline
• Goals of multi-agent probabilistic search
• Background: Probabilistic smart terrain
• Estimating global expected distances to targets
that meet goals of group
• Matching agents to targets
• Demonstration on examples
John Sullins
Youngstown State University
Multiagent Probabilistic Smart Terrain
CGAMES 2011
15
Global Expected Distances
• Each agent Aj estimates moves until goal found
by some agent (not necessarily itself)
– Distances to
targets Aj is
moving towards
– Distances of
other agents to
targets Aj is
moving away from
John Sullins
Youngstown State University
Multiagent Probabilistic Smart Terrain
CGAMES 2011
16
Finding Closest Targets
• Step 1: Each agent Aj determines set of targets Tj
that it is closer to than any other agent
John Sullins
Youngstown State University
Multiagent Probabilistic Smart Terrain
CGAMES 2011
17
Partitioning by Direction
• Step 2: For each possible next tile for Aj,
determine which targets Tj direction  Tj
would be closer in that direction
•
•
•
•
Tj left
Tj right
Tj up
Tj down
John Sullins
Youngstown State University
Multiagent Probabilistic Smart Terrain
CGAMES 2011
18
Global Expected Distances
• Step 3: Compute global expected distance for each
possible next tile x based on targets ti  Tj
dmax
Dist(x) = Σ
 (1 – pi )
d=0 di < d
• ti  Tj direction  di = distance(x, ti ) + 1
• ti  Tj direction  di = min(distance(Ak, ti ))
j ≠k
(distance to closest other uncommitted agent)
John Sullins
Youngstown State University
Multiagent Probabilistic Smart Terrain
CGAMES 2011
19
Global Expected Distances
•
•
•
•
Tj left
Tj right
Tj up
Tj down
John Sullins
Youngstown State University
Multiagent Probabilistic Smart Terrain
CGAMES 2011
20
Global Expected Distances
• Example: Expected global distance for A1 moving left
– Neither room reached in < 4 moves
– A1 reaches room 2 (probability 0.4) in 4 moves
– A2 reaches room 1 (probability 0.5) in 5 moves
John Sullins
Youngstown State University
Multiagent Probabilistic Smart Terrain
CGAMES 2011
21
Moving Agents
• Each agent computes global expected distances for
surrounding tiles
A1 moves left
A1 moves right
• Each agent then moves to tile with the lowest global
expected distance
John Sullins
Youngstown State University
Multiagent Probabilistic Smart Terrain
CGAMES 2011
22
Global Expected Distances
• Probability of targets is also an important factor
A1 covers R2, A2 covers R1
John Sullins
Youngstown State University
A1 covers R1, A2 covers R2
Multiagent Probabilistic Smart Terrain
CGAMES 2011
23
Outline
• Goals of multi-agent probabilistic search
• Background: Probabilistic smart terrain
• Estimating global expected distances to targets
that meet goals of group
• Matching agents to targets
• Demonstration on examples
John Sullins
Youngstown State University
Multiagent Probabilistic Smart Terrain
CGAMES 2011
24
Assigning Agents to Targets
• Closest targets in direction of tile x with minimum Dist(x)
now assigned to that agent
• May not be considered by any other agent this move
R2 assigned to A1
John Sullins
Youngstown State University
R2 cannot be considered by A2
Multiagent Probabilistic Smart Terrain
CGAMES 2011
25
Committing Agents to Targets
• Agent now committed to those targets
• That agent will not be used by other agents to compute
their Dist(x)
A1 committed to R2 and will not
move to R1
Cannot be used by A2 to
determine global expected
distance to R1
John Sullins
Youngstown State University
Multiagent Probabilistic Smart Terrain
CGAMES 2011
26
Subsumed Agents
• Some agents may not be closest to any targets
A1 closer to both
A2 closest
to neither
• Agents “subsumed” by other agents
– Initially, no move chosen
– Are reconsidered after other agents choose directions
John Sullins
Youngstown State University
Multiagent Probabilistic Smart Terrain
CGAMES 2011
27
Subsumed Agents
• Targets agents moving away from are released to
subsumed agents
– ti Tj , ti Tjx where Dist(x) is minimum
– A1 chooses to move towards R2
– R1 released for consideration by other agents
– A2 now uses R1 to move left
John Sullins
Youngstown State University
Multiagent Probabilistic Smart Terrain
CGAMES 2011
28
Overall Algorithm Structure
for each move {
while (unassigned targets && uncommitted agents) {
– find closest uncommitted agent to each unassigned target
(agents with no targets are subsumed)
– for (each non-subsumed agent A) {
– A computes Dist(x) based only on other uncommitted agents
– A committed to move in direction with minimum Dist(x);
– A’s targets in that direction assigned to A;
– Targets not in that direction released for next cycle of loop
}
}
}
John Sullins
Youngstown State University
Multiagent Probabilistic Smart Terrain
CGAMES 2011
29
Subsumed Agents
• Agents may still be subsumed if
all targets assigned to other
agents
– Loop ends without all agents being
assigned targets
– A1 best move is down
– Both R1 and R2 closer in that
direction
– Both R1 and R2 assigned to A1
– No remaining targets for A2
John Sullins
Youngstown State University
Multiagent Probabilistic Smart Terrain
CGAMES 2011
30
Subsumed Agents
• Base subsumed agent
move on all targets
regardless of what
other agents are doing
– Use original single-agent
probabilistic smart
terrain formula
– Gives agent appearance
of doing something
John Sullins
Youngstown State University
Multiagent Probabilistic Smart Terrain
CGAMES 2011
31
Subsumed Agents
• Agents may eventually
not be subsumed
– Agent moves to area
with multiple targets
– Will move towards one
target and away from
others
– Those other targets now
available to subsumed
agents in area
John Sullins
Youngstown State University
Multiagent Probabilistic Smart Terrain
CGAMES 2011
32
Outline
• Goals of multi-agent probabilistic search
• Background: Probabilistic smart terrain
• Estimating global expected distances to targets
that meet goals of group
• Matching agents to targets
• Demonstration on examples
John Sullins
Youngstown State University
Multiagent Probabilistic Smart Terrain
CGAMES 2011
33
Performance on Initial Example
• Player may be in
one of 7 rooms, 3
with “treasure”
• 3 guards searching
for player
• Probability player in
a given “treasure”
room = 0.2
• Probability player in
a “non-treasure”
room = 0.1
John Sullins
Youngstown State University
Multiagent Probabilistic Smart Terrain
CGAMES 2011
34
Dynamic Targets
• “Target” tiles placed one step inside each room
– Gives guards appearance of “looking” in a room
• Probabilities change when guard reaches tile
– Player not in room  probability set to 0
– Guard now influenced by other rooms
John Sullins
Youngstown State University
Multiagent Probabilistic Smart Terrain
CGAMES 2011
35
Performance on Initial Example
John Sullins
Youngstown State University
Multiagent Probabilistic Smart Terrain
CGAMES 2011
36
Initial Example Modified
• “Gold” room
moved to upper
left
• Guard 2 moves
to jewel room
instead
• Guard 1 path
also altered as
result
John Sullins
Youngstown State University
Multiagent Probabilistic Smart Terrain
CGAMES 2011
37
Ongoing Work
• Testing with actual players
– Implement algorithm as game (Unreal engine)
– Goal: steal treasure while avoiding guards
– Player can see guard movement, guards use
algorithm to search for player
• Do NPC guard actions appear plausible to
players?
John Sullins
Youngstown State University
Multiagent Probabilistic Smart Terrain
CGAMES 2011
38
Multiagent Probabilistic
Smart Terrain
Dr. John R. Sullins
Youngstown State University
Player Found in Large Example
• Player tile
probability set
to 1
• Other target
probabilities
set to 0
• All guards will
now converge
on player
John Sullins
Youngstown State University
Multiagent Probabilistic Smart Terrain
CGAMES 2011
40