Game and Tree Searching
10/19/2004
TCSS435A Isabelle Bichindaritz
1
Learning Objectives
• Introduction to games
• Optimal decisions in games
• Alpha-Beta pruning
• Real-time decisions
10/19/2004
TCSS435A Isabelle Bichindaritz
2
Introduction to Games
• Games as Search Problems
– Frameworks: two-player, multi-player; zero-sum;
perfect information
– Minimax algorithm
• Perfect decisions
• Imperfect decisions (based upon static evaluation function)
– Issues
• Quiescence
• Horizon effect
– Need for pruning (alpha-beta pruning)
10/19/2004
TCSS435A Isabelle Bichindaritz
3
Introduction to Games
• Perfect Play
– General framework(s)
– What could agent do with perfect info?
• Resource Limits
– Search ply
– Static evaluation: from heuristic search to heuristic game tree search
– Examples
• Tic-tac-toe, connect four, checkers, connect-five
• Chess, go
• Games with Uncertainty
– Explicit: games of chance (e.g., backgammon, Monopoly, blackjack)
– Implicit
10/19/2004
TCSS435A Isabelle Bichindaritz
4
Games versus Search Problems
• Unpredictable Opponent
– Games are adversarial search problems
– Solution is strategy, contingency plan
– Time limits
• Unlikely to find goal
• Must approximate
• Plan of Attack
– Algorithm for perfect play (J. von Neumann, 1944)
– Finite horizon, approximate evaluation (C. Zuse, 1945; C.
Shannon, 1950, A. Samuel, 1952-1957)
– First chess program (Turing, 1951)
– Pruning to allow TCSS435A
deeper Isabelle
search
(J. McCarthy, 1956)
Bichindaritz
10/19/2004
5
Types of Games
• Information: Can Know (Observe)
– … outcomes of actions / moves?
– … moves committed by opponent?
• Uncertainty
– Deterministic vs. nondeterministic outcomes
– Thought exercise: sources of nondeterminism?
10/19/2004
TCSS435A Isabelle Bichindaritz
6
Minimax
• Games with two players MIN and MAX are a search problem
with:
– Initial state
– Successor function
– Terminal test / state
– Utility function (objective / payoff)
• Win / loss / draw (chess)
• +1 / -1 / 0
(chess)
• -192 … +192
(backgammon)
10/19/2004
TCSS435A Isabelle Bichindaritz
7
Minimax
10/19/2004
TCSS435A Isabelle Bichindaritz
8
Minimax
• Perfect play for deterministic, perfect-information games
• Choose move to position with highest minimax value
= best achievable payoff against best play
• Simple example: 2-ply game, a ply being a half-move
• Game ends after one move each by MAX and MIN (one move
deep)
10/19/2004
TCSS435A Isabelle Bichindaritz
9
Minimax
10/19/2004
TCSS435A Isabelle Bichindaritz
10
Minimax Algorithm:
Decision and Evaluation
 what’s this?
 what’s this?
10/19/2004
TCSS435A Isabelle Bichindaritz
11
Properties of Minimax
• Complete?
10/19/2004
TCSS435A Isabelle Bichindaritz
12
Properties of Minimax
• Complete?
– … yes, provided following are finite:
• Number of possible legal moves (generative breadth of tree)
• “Length of game” (depth of tree) – more specifically?
• A finite strategy can exist even in an infinite tree!
– Perfect vs. imperfect information?
• Q: What search is perfect minimax analogous to?
• A: Bottom-up breadth-first
10/19/2004
TCSS435A Isabelle Bichindaritz
13
Properties of Minimax
• Complete?
– … yes, provided the tree is finite:
• Number of possible legal moves (generative breadth of tree)
• “Length of game” (depth of tree) – more specifically?
• Optimal?
10/19/2004
TCSS435A Isabelle Bichindaritz
14
Properties of Minimax
• Complete?
– … yes, provided the tree is finite
• Optimal?
– … yes, provided perfect info (evaluation function) and
opponent is optimal!
– … otherwise, guaranteed if evaluation function is
correct
• Time Complexity?
10/19/2004
TCSS435A Isabelle Bichindaritz
15
Properties of Minimax
• Complete?
– … yes, provided the tree is finite:
• Optimal?
– … yes, provided perfect info (evaluation function) and
opponent is optimal!
• Time Complexity?
– Depth of tree: m
– Legal moves at each point: b
– O(bm) – NB, m  100, b  35 for chess!
• Space Complexity?
10/19/2004
TCSS435A Isabelle Bichindaritz
16
Properties of Minimax
• Complete?
– … yes, provided the tree is finite:
• Optimal?
– … yes, provided perfect info (evaluation function) and
opponent is optimal!
• Time Complexity?
– Depth of tree: m
– Legal moves at each point: b
– O(bm) – NB, m  100, b  35 for chess!
• Space Complexity? O(bm) – why?
10/19/2004
TCSS435A Isabelle Bichindaritz
17
Resource Limits
10/19/2004
TCSS435A Isabelle Bichindaritz
18
Static Evaluation Function
Example: Chess
10/19/2004
TCSS435A Isabelle Bichindaritz
19
Do Exact Values Matter?
10/19/2004
TCSS435A Isabelle Bichindaritz
20
Cutting Off Search
10/19/2004
TCSS435A Isabelle Bichindaritz
21
Cutting Off Search
• Issues
– Quiescence
• Play has “settled down”
• Evaluation function unlikely to exhibit wild swings in value in near
future
– Horizon effect
• “Stalling for time”
• Postpones inevitable win or damaging move by opponent
• See: Figure 6.9 R&N
• Solutions?
– Quiescence search: expand non-quiescent positions further
– No general solution to horizon problem at present
10/19/2004
TCSS435A Isabelle Bichindaritz
22
Why Prune?
10/19/2004
TCSS435A Isabelle Bichindaritz
23
•
Game
Tree
Search
Perfect Play (TBD from Perfect Information)
– Common framework: two-player (or multi-player), zero-sum
– Deterministic or known probability distribution function, totally
observable
– Minimax algorithm given ability to search for arbitrary ply
• Assumptions about
– … game?
– … opponent?
• Specifications and Ramifications
– Combinatorics: finite depth (m), breadth (b) of search
– Resource limits: need for static evaluation function
• Practical Issues
– Effects of depth cutoff
– Need for pruning
10/19/2004
TCSS435A Isabelle Bichindaritz
24
- Pruning
• Properties of - pruning
– Pruning does not affect final result
– Good move ordering improves effectiveness of pruning
– With “perfect ordering,” time complexity = O(bm/2)
• Depth of tree: m
• Legal moves at each point: b
• Effective branching factor becomes b1/2
• Can easily reach depth 8 and play good chess
– A simple example of the value of reasoning about which computations
are relevant ( a form of metareasoning)
10/19/2004
TCSS435A Isabelle Bichindaritz
25
- Pruning
•
 is the value of the best (i.e. highest value) choice we have found so far at
any choice point along the path for MAX
•  is the value of the best (i.e. lowest value) choice we have found so far at
any choice point along the path for MIN.
10/19/2004
TCSS435A Isabelle Bichindaritz
26
Alpha-Beta (-) Pruning:
Definitions
10/19/2004
TCSS435A Isabelle Bichindaritz
27
Alpha-Beta (-) Pruning Example
≥3
MAX
MIN
≤2
3
≤ 14 ≤5
2
MAX
3
10/19/2004
12
8
2
TCSS435A Isabelle Bichindaritz
14
5
2
28
Alpha-Beta (-) Pruning Example
What are ,  values here?
≥3
MAX
MIN
≤2
3
≤ 14 ≤5
2
MAX
3
10/19/2004
12
8
2
TCSS435A Isabelle Bichindaritz
14
5
2
29
Alpha-Beta (-) Pruning:
Modified Minimax Algorithm
10/19/2004
TCSS435A Isabelle Bichindaritz
30
Alpha-Beta (-) Pruning Properties
• Perfect Play (TBD from Perfect Information)
– Common framework: two-player (or multi-player), zero-sum
– Deterministic or known probability distribution function, totally observable
– Minimax algorithm given ability to search to arbitrary ply
• Assumptions about
– … game?
– … opponent?
• Specifications and Ramifications
– Combinatorics: finite depth (m), breadth (b) of search
– Resource limits: need for static evaluation function
• Practical Issues
– Effects of depth cutoff
– Need for pruning
10/19/2004
TCSS435A Isabelle Bichindaritz
31