Automatic Construction of
Static Evaluation Functions
for Computer Game Players
D1 Miwa Makoto
Chikayama & Taura Lab
Outline

Introduction

Computer Game Player
Static Evaluation Functions
 Automatic Construction of Static
Evaluation Functions

GLEM
 Static Evaluation Functions from domain
theory


Conclusion & Future Work
Computer Game Player

Game

2-player, zero-sum, deterministic, and perfect
information game
 ex.

chess, Shogi, Go, etc…
Computer Game Player
receives a position and returns the next move
 Important Features

 Evaluation
Functions
 Game Tree Search (& search strategy)
Static Evaluation Functions (SEFs)


Static ≒ without search
Evaluate positions and assign their heuristic
values


Probability to win
Distance to goal …
E(p) = number of lines open for Ｘ –
number of lines open for O
E(p) = 3 – 2 = 1
Evaluation Value
Static Evaluation Functions (SEFs)
Rank treating positions
Static
Evaluation
Function
Evaluation
value
+1
WIN
+2
+1
0
-1
-2
-1
LOSE
Game Tree Search
Tic Tac Toe with
horizon = 2
MAX
1
Static
-1
0
evaluation
function
Static evaluation fuctions decide
the computer game players’ behavior
1
0
1
1
-1
-1
1
0
-1
0
-2
1 MIN
2
Construction of SEFs

Two requirements
evaluation accuracy
 speed


trade-off
Representation of SEFs

Combinations of features
 feature

Number of pieces, Position, etc…
Construction of SEFs

To write SEFs by hand, developers need ...

Deep knowledge about the treating game


To find useful features and essential features
Much effort to tune


To avoid oversights
ex. self-play, play with other players
What if no experts exist?
What if the player is stronger than human experts?
Automatic construction of SEFs
Automatic Construction of SEFs

Construction from simple features

Simple features
 A set
of units to represent a position
ex. Black stone on A1 (Othello)

2 kinds of methods
Direct Method
 Hybrid Method

Automatic Construction of SEFs

Direct Method

High-level combination of simple features


Feature




Genetic Programming, Neural networks, etc…
High expressivity
Much resources
Difficult to analyze
Ex.

TD-Gammon, The Distributed Chess Project
Automatic Construction of SEFs

Hybrid Method


Linear combination of high-level features
from simple features
Feature (in comparison with direct method)




Less expressive
Less resources
Easy to analyze and optimize constucted SEFs
Fast SEFs
Ex.

ZENITH, GLEM
GLEM (M. Buro, 1998)

Generalized Linear Evaluation Model

Application : Othello

Logistello
 One
of the strongest computer Othello players
 won the human champion in 1997
Method
1. Extraction
of atomic features
2. Generation of configurations by
conjunction of features
3. Weighting configurations using linear
regression
Atomic features

Simple binary features


ex. Black stone on A1
Extracted by hand

ex. Othello
 2£64

features - places and their stones
Can not be expressed by a combination of
other features.
Configurations

Extracting all conjunctions of atomic
features is unreasonable


ex. Othello - 2128 conjunctions
Extract only frequent conjunction
(=Configurations)

frequent
 at
least N times in the training set
Pattern

Treating the whole
position at once costs
high
Extract a part of the
position by hand
 Pattern values are
pre-computed and
preserved in a hash
table
 Fast Evaluation

Patterns in LOGISTELLO
Pattern
Evaluation using hash tables
Patterns in LOGISTELLO
Application : Othello
13 game stages (every 4 discs)
 Sum of 51 pattern values


1.4 million positions/sec on Athlon 2000+
1.5 million weights
 17 million training positions
 10x speedup


compared with the old one which won the
human champion in 1997
SEFs from domain theory
(Kaneko et al. 2003)

Based on ZENITH (Fawcett 1993)

Application : Othello
SEFs from domain theory
1. Generation
of logical features from domain
theory
2. Extraction of evaluation features from
logical features and selection
3. Weighting evaluation features using linear
regression
Domain Theory
Rules and goal conditions written by a set
of Horn Clauses
 ex. Othello

Logical Features
Features generated from domain theory
 Generation Strategy

1. Translate
domain theory
2. Remove expensive features
3. Select features by backward elimination
method
Logical Features

Translations
Decomposition
split conjunction
remove negation
split arithmetic comparison
split arithmetic calculation
Abstraction
remove least constraining term
remove a variable
Goal Regression
regress the formula
Specialization
remove a feature’s disjunction
replace call to recursive predicate with base case
find invariant variable values
Evaluation Features

Evaluation Features
Boolean value
 Conjunction of facts
ex. blank(a1) Æ owns(x, a2) Æ owns(o, a3)
(=white can play on square a1)

 fact

a clause without body
ex. owns, blank
Extraction of Evaluation Features
Translate logical feature to patterns.
 logical feature
f(A) :- legal_move(A, o).
unfolding

unfolded features
legal_move(a1, o) :- blank(a1), owns(x, a2), owns(o, a3).
legal_move(a1, o) :- blank(a1), owns(x, b1), owns(o, c1). ….
extract conjunction of body

Evaluation Features
blank(a1) Æ owns(x, a2) Æ owns(o, a3)
blank(a1) Æ owns(x, b1) Æ owns(o, c1) ….
Selection of Evaluation Features

Frequency


Reject high- and extreme low-frequency
evaluation features
Approximated Forward Selection

Select high correlation pattern iteratively
Fast access to features

Hasse diagram

Kernel Extraction
Experimental Results
11,079 logical features
 8,592,664 evaluation features without
selection
 800,000 positions for training and 6,000
positions for testing
 Positions of 55 and 60 discs
 Range of evaluation values : [-64, 64]

Accuracy
number of evaluation features in evaluation functions
as well or better than GLEM in accuracy
Speed
Athlon MP 2100+
30,000 positions/sec (1.4 million in GLEM)
number of evaluation features in evaluation functions
Conclusion

Automatic Construction of Static
Evaluation Functions
GLEM
 SEFs from domain theory

Future Work
More sophisticated feature selection
 Automatic extraction of GLEM’s pattern
 More expressive combination of simple
features
