GP End-Chess
Evolution of Chess Endgame Players
Ami Hauptman & Moshe Sipper
1
Outline
 Introduction
 The Game of Chess – a solved problem?
 Important differences between human and artificial chess
players




Chess Endgames - features & building blocks
GP problem definition
Experiment and Results
Future work
2
The game of Chess
 First developed in India and Persia
 Considered THE complex game of strategy and
inventiveness
 Enormous search space
 Roughly 50 possible moves at mid-game
 A typical game consists of a few dozen moves
 Estimated at 1043 in 40-move game (Shannon, 1950)
Elephants don’t play Chess(?)
3
The game of Chess – AI history
 First chess AI at 1958 – novice level
 Machine strength increasing linearly
 1997 – defeat of former world champion, Garry
Kasparov, by IBM’s deep blue
 Last years – performance still increasing
 Mainly Hardware
 Also Software
 … The End ?
4
The game of Chess
 …NO!
 Deep(er) blue use extreme brute-force,
traversing several millions of boards ps
 Very little generalization
 Virtually no human resemblance
 Deemed theoretically uninteresting
 Chomsky: As interesting as a weight lifting competition
between machine and man
 Low A to I ratio; Low return
5
The game of Chess –
Basic concepts
 8x8 board
 Each player starts with 16 pieces, of 6
different types, and may only move 1 piece
per turn
 A piece can only move into an empty square or
into one containing an opponent’s piece (a
capture)
 Win by capturing the opponent’s king
6
The game of Chess – pieces
 Pawn: may only move forward (or capture diagonally)
 Bishop: diagonals
 Knight: L shaped moves. The only “unblockable” piece
 Rook: Ranks & files
 Queen: Bishop & Rook combined
 King: 1 square in any direction. May not move into
attacked square
Values :
1
3
3/3.5
5
9
∞
7
The game of Chess – example
 White has over
30 possible moves
 If black’s turn –
can capture pawn
at c3 and check
(also fork)
8
The game of Chess –
Check and Checkmate
 “Checking” is attacking
opponent’s king.
Opponent must respond
 “Mating” (Checkmate)
is when the opponent
can’t avoid losing the
king – and thus
forfeiting the game
9
Man vs Machine
10
Human & Artificial Players –
AI search
 AI uses search to
assign a score to a
board
 Traverse the move
tree from leaves - up
 Select the best child
using scores found
 Only partial tree
Computer is O (Max) opponent is X
11
Human & Artificial Players –
The Machine
12
Human & Artificial Players –
The Machine
 Millions of boards (nodes) per second
 Little time for each board – less knowledge
 Smart search algorithms –
 pruning
 Alpha-beta variants (negascout etc.)
 Still use heuristics at end – can’t see all tree
 Most research revolves around search
 Human resemblance minimal – humans use
little search
13
Human & Artificial Players –
Humans
 Humans use problem solving cognition
 Deeply knowledge based –
 Extensive “theory” exists
 Numerous books and institutions
 Massive use of pattern recognition
 Also use search but
 Less deep
 Only develop “good” positions
 More efficient – less nodes for “same” result
 Reminiscent of greedy search
 Not only in chess
14
Human & Artificial Players –
Grand Masters - Findings
 Play against several opponents at the same
level they play against a single opponent
 GMs and novices: same level of performance
when memorizing a random board; differ when
memorizing real game positions
 GM eye movements show they only scan
“correct” parts of board
 Strong Amateurs use the same meta-search as
GMs - equally deep, same nodes, same speed;
Differ in knowledge of domain (De Groot)
15
Endgames - example
 White’s turn: mate in 5,
with Qe6+
 Features include:
 #moves for black king
minimal
 Attacking, un-attacked
 Checking
 Officers same line\row
 Black’s turn: draw with:
Rc1+, then Qg5 – fork
& exchange
16
Endgames (2) - features
 Few pieces remain (typically: king, 0-3
officers and sometimes pawns)
 Fewer options, but more moves for each piece
 Trees still extremely large
17
Endgames - Building Blocks
 Main goals
 Reduce search by “smart” features of the board
 Use more game knowledge as humans do
 Allow more complex features to be built by
supplying basic ones (terminals) and building
methods (functions)
 Schemata evolution
18
Features Example - Fork
 My piece is:
 Attacking 2 or more pieces
 Protected or not attacked
 Opponent pieces:
 Unprotected
 OR protected but of greater
value
 Example: black must
exchange Q for R because
of fork
19
Fork: Traditional AI search
 Only 3 legal moves for
black
 Find that one of white’s
next moves (out of 23
possible) captures black
queen
 Check all following moves
for more piece exchanges
 Sometimes, still check
other moves (non
capturing)
 At end of search –
compare remaining pieces
 No fork “concept”
20
Features Example – Fork
Feature Search (GP)
 One of the features is
isMyFork function – Checks
all previously defined
conditions
 Also, use some smaller
building blocks:




Is Opponent piece Attacked?
Is attacking piece protected?
Is opponent in check?
Value of attacked piece
21
GP Problem Definition
 Terminals & Functions
 Numerous “chess terminals” and ERCs
 Non-chess funtions
 Fitness
 Tournament
 Run parameters
 Termination
 We will see each element in the following
experiments
22
Endgame experiments conducted
 KRKR – each player has 1 king and 1 rook
 KRRKR – King with 2 Rook against King and Rook
 KRRKRR
 KQKQ – Kings and Queens
 KQRKQR – Combined
23
Basic program architecture
 Generate all possible moves (depth=1)
 Evaluate each board with GP individual
 Select board with best score (or stochastically
decide between equal)
 Perform best move
 Repeat process with GP opponent until game ends
(or until only kings left)
24
KRKR Endgame
 Each player has 1 King, 1 Rook
 “Toy” problem for chess endgames
 Theoretical draw (experts never lose this)
 Some interesting positions exist
25
KRKR Endgame what needs to be learned (1)
 Avoid losing rook
 Don’t move to
attacked, unprotected
squares
 Vice versa - capture
opponent’s rook if
able
Black to move – white loses Rook
26
KRKR Endgame what need to be learned (2)
 Avoid getting king
stuck in edges
 Again, take
advantage if
opponent does this
Black to move – mate in 1
27
KRKR Endgames - Terminals
 Used in first runs:
 Is My Rook Attacked, Is Opp Rook
attacked
 Is {My, Opp} Rook Protected (two as
above)
 Is {My, Opp} Rook In Play
 Num Moves {My, Opp} king
 {My, Opp}-King’s proximity to edges
 Is Mate
 ERCs: ± {0.25, 0.5, 1} * MAX
 MAX = 1000 (empirically)
28
KRKR Endgames - Functions
 Boolean
 OR2, OR3, OR4
 AND2, AND3, AND4
 NOT
 Arithmetic - +, -, *
 Combined - <, =, >, IF
 STGP
 For now, no “chess” functions, only
terminals
29
KRKR Endgames - Fitness
 Competitive, Random-2-ways
 Each individual plays against k randomly selected
opponents
 Each game counts for both players
 For each encounter
 Several games (typically 4) are played
 Short games - ~5-8 moves per player
 Each game starts at a random legal position
 Safe start - no piece is attacked at the beginning
30
KRKR Endgames – Fitness (2)
 Scoring method:
 Victory: 1-2 points
 Piece count advantage (theoretical win) – ¾ point
 Draw: ½ point
 After advantage – 0 points
 Loss: 0 points
31
KRKR Endgames – Parameters
 Population size - 80
 #Generations - 150..250
 Operators:
 Reproduction 0.35
 Crossover 0.5
 Mutation 0.15 (including ERC mutation)
 Termination – ~10-25 hours
32
KRKR Endgames – Results
 Every 10 generations, best individual played
against:
 Best of generation 0
 An opponent performing random moves
 Longer games: ~10-12 moves per player
 50-150 games
 Games were doubled – each player staring
from both positions
33
KRKR Endgames – Results
 Bad results – no distinct improvement
 Several reasons:
 Arithmetic operations problematic – we get large
numbers
 Mate not distinct enough (traditionally terminates
the search)
 Boolean functions not clear enough
 Slow Runs due to large trees with repeating
functions
34
KRKR Endgames –Improvements
 Boolean functions
 Divided to good and bad
 Example: Is-My-King-In-Check changed to Is-MyKing-Not-In-Check
 Mate changed to 1000*Mate
 Added Not-My-Rook-Attacked-Unprotected and
Opp-Rook-Attacked-Unprotected
35
KRKR Endgames – Results Improvements
 Also consulted Chess
Experts – added more:
 Is-Opp-King-BehindRook
 Split to
 Opp-King-Prox-Rook
 Opp-King-Behind-Rook
 Is-Stalemate (only kings
left)
Black moves and White loses Rook
36
KRKR Endgames – Results Improvements
 Arithmetic functions canceled
 Although Still using floats for terminals
 Also divide to good and bad: NumNotMovesOppKing
 Theoretically justified – more “logical” search in
literature
 Empirically - need more logical rules, and not : (
> (+ (#moves-k #moves-opp-k) 5.5))
 Memoization – saves more than ½ the time
37
KRKR Endgames –
Final Results
 Improvement
 Above 75% of games against random end in
advantage or mate
 Still, too few mates, even when score for win is
increased – difficult to learn move sequence
 Same against best of generation 0 (almost random)
 The main thing that was learned was avoiding
getting the rook captured
38
KRRKR Endgames
 Example (right)
 Very good for
white
 Black king exposed
 2 rooks close
 Next move –
captures rook
 (mate in 5)
39
KRRKR Endgames - goals
 One player has 2 rooks, the other – 1
 Not theoretically drawn
 We want one generalized individual for all
endgames and positions (Not one for each
endgame):
 Each player needs to play both advantage, draw
(KRKR) and disadvantage
 Terminals need to be more general
40
KRRKR Endgames - changes
 Terminals - changed and added to cope with
changing state
 Material-Count (recall each rook = 5)
 Num-My-Pieces-Not-Attacked, since now there
are more than 1
 Is-My-King-Protecting-Piece and My-OfficersSame-Line to allow more complex considerations
 Functions
 If-Adv-Then-(left child)-Else-(right child)
 Eventually divided to 3 trees
41
KRRKR Endgames - changes
 Also added – comparing differences to parent
node
 Boolean Is-Material-Increase, which compares to
the parent node (board)
 Material decrease is not needed since considering
only my move
 Not-My-King-Moves-Decrease to further use
number of moves for king
42
KRRKR Endgames –
Opponents
 Random forsaken; Best-of-0 still used but less
 Added new opponent – MASTER
 a program we wrote based on consultation with
experts, highest being International Master Boris
Gutkin, ELO 2400 (only about 3000 of those…)
 Used ~50 general positions and rules derived from
them, together with scores for each
 Defined a strategy (“Expert”) accordingly
 Tested evolved programs against it
 Human competitive?
43
KRRKR Endgames –
Fitness
 Test were conducted by assigning each player
both roles for each position
 Fitness was refined – score effected by:
 Starting position (advantage or disadvantage)
 End result – win, loss or draw
 Adv position ending in draw receives a score of
near zero
 Dis-adv ending in a draw will receive better than
0.5
44
KRRKR Endgames –
Results
 Expert-defined performed extremely well against
Random and Best0
 Evolved programs performed generally as well as
expert defined, sometimes better
Percent of favorable results in game outcomes
45
Main Experiment – KQRKQR
 Most complex endgame
we worked with
 Still theoretical draw
 Highly position
dependant – “noisy”
 Larger trees
 2 officers
 Queens
 Easier to mate
46
KQRKQR Endgames - changes
 Added – more “heavy” terminals (and
components)
 Boolean Is-Not-Mate-in-one, most time consuming
but necessary
 Boolean Is-My-King-Not-Trapped
 Not all king’s moves lead closer to edges
 Important but vague – usually happens with complex
terminals
 My-Officers-Same-Line
47
Simple Terminals
Genome Summary
Complex Terminals
Function Set
NotMyKingInCheck
EvaluateMaterial
If3(B,F,F)
IsOppKingInCheck
IsMaterialIncrease
Or2(B,B)
MyKingDistEdges
IsMate
Or3(B,B,B)
OppKingProximityToEdges
IsMateInOne
And2(B,B)
NumMyPiecesNotAttacked
OppPieceCanBeCaptured
And3(B,B,B)
NumOppPiecesAttacked
MyPieceCannotBeCaptured
Smaller(B,B)
ValueMyPiecesAttacking
IsOppKingStuck
Not(B)
ValueOppPiecesAttacking
IsMyKingNotStuck
IsMyQueenNotAttacked
IsOppKingBehindPiece
IsOppQueenAttacked
IsMyKingNotBehindPiece
IsMyFork
IsOppPiecePinned
IsOppNotFork
IsMyPieceNotPinned
NumMovesMyKing
NumNotMovesOppKing
MyKingProxRook
OppKingDistRook
MyPiecesSameLine
OppPiecesNotSameLine
IsOppKingProtectingPiece
48
KQRKQR Endgames –
New Opponent
 CRAFTY, second in the 2004 Computer Chess
Championship (held at Bar-Ilan)
 Uses brute force methods; State-of-the-art
search algorithms
 Specializes in Blitz games (typically 3 minutes
per game)
 We limited to 5 secs per move, enough to
scan ~1.5 Million boards with pruning
49
KQRKQR Endgames –
Our parameters
 Used lookahead of depth 2
 Typically ~5 secs per move
 Simple Minimax search, but not Alpha-Beta
 Played 5-6 moves per game
 Never cancelled a game, even if it started
with mate-in-4 (which CRAFTY easily saw)
 Played each position 2 times, switching places
 ~100 games - reduce noises in starting positions
50
Results: Master
51
Results: CRAFTY
52
Multiple Endgames
 Aim for general-purpose strategies
 All endgames used during evolution
 Results:
%Wins
%Advs
%Draws
Master
6
2
68
CRAFTY
2
4
72
53
Sample GP-Endchess
Tree 0:
(If3 (Or2 (Not (Or2 (And2 OppPieceAttUnprotected NotMyKingInCheck) (Or2 NotMyPieceAttUnprotected 100*Increase)))
(And2 (Or3 (And2 OppKingStuck NotMyPieceAttUnprotected) (And2 OppPieceAttUnprotected OppKingStuck) (And3 1000*MateInOne OppKingInCheckPieceBehind NotMyKingStuck)) (Or2 (Not NotMyKingStuck) OppKingInCheck)))
NumMyPiecesUNATT (If3 (< (If3 (Or2 NotMyPieceAttUnprotected NotMyKingInCheck) (If3 NotMyPieceAttUnprotected
#NotMovesOppKing OppKingInCheckPieceBehind) (If3 OppKingStuck OppKingInCheckPieceBehind -1000*MateInOne))
(If3 (And2 100*Increase 1000*Mate?) (If3 (< NumMyPiecesUNATT (If3 NotMyPieceAttUnprotected -1000*MateInOne
OppKingProxEdges)) (If3 (< MyKingDistEdges #NotMovesOppKing) (If3 -1000*MateInOne -1000*MateInOne
NotMyPieceATT) (If3 100*Increase #MovesMyKing OppKingInCheckPieceBehind)) NumOppPiecesATT) (If3
NotMyKingStuck -100.0 OppKingProxEdges))) (If3 OppKingInCheck (If3 (Or2 NotMyPieceAttUnprotected
NotMyKingInCheck) (If3 (< MyKingDistEdges #NotMovesOppKing) (If3 -1000*MateInOne -1000*MateInOne
NotMyPieceATT) (If3 100*Increase #MovesMyKing OppKingInCheckPieceBehind)) NumOppPiecesATT) (If3 (And3 1000*MateInOne NotMyPieceAttUnprotected 100*Increase) (If3 (< NumMyPiecesUNATT (If3 NotMyPieceAttUnprotected
-1000*MateInOne OppKingProxEdges)) (If3 (< MyKingDistEdges #NotMovesOppKing) (If3 -1000*MateInOne 1000*MateInOne NotMyPieceATT) (If3 100*Increase #MovesMyKing OppKingInCheckPieceBehind))
NumOppPiecesATT) -1000*MateInOne)) (If3 (< (If3 100*Increase MyKingDistEdges 100*Increase) (If3 OppKingStuck
OppKingInCheckPieceBehind -1000*MateInOne)) -100.0 (If3 (And2 NotMyPieceAttUnprotected -1000*MateInOne) (If3 (<
NumMyPiecesUNATT (If3 NotMyPieceAttUnprotected -1000*MateInOne OppKingProxEdges)) (If3 (< MyKingDistEdges
#NotMovesOppKing) (If3 -1000*MateInOne -1000*MateInOne NotMyPieceATT) (If3 100*Increase #MovesMyKing
OppKingInCheckPieceBehind)) NumOppPiecesATT) (If3 OppPieceAttUnprotected NumMyPiecesUNATT MyFork)))))
Tree 1:
(If3 NotMyPieceAttUnprotected #NotMovesOppKing 1000*Mate?)
Tree 2:
(If3 1000*Mate? NumMyPiecesUNATT -1000*MateInOne)
54
Summary
 Draw and better against Master-defined
 Draw against a world class opponent
 On limited conditions (not a full game, time ,etc.)
 Shows deep search may have an alternative
 Fast, pattern-oriented search suggests more
human resemblance
 Search and lookahead are still important
55
Future Work
 Add more pieces
 Improve evolution speed
 Parallel nets
 Stronger board representations
 Develop more cognitive models using evolution
 Search scheme space as well as game space
 Tackle beyond endgames
 Openings and mid-game
 General game concept schemas (?)
56