PCG
RAPP
Simplified Boardgames
Generalized RAPP
The End
Evolving Chess-like Games
Using
Relative Algorithm Performance Profiles
Jakub Kowalski, Marek Szykuła
EvoApplications
31 March 2016
Jakub Kowalski, Marek Szykuła
Evolving Chess-like Games Using Relative Algorithm Performance Profiles
PCG
RAPP
Simplified Boardgames
Generalized RAPP
The End
Procedural Content Generation
Jakub Kowalski, Marek Szykuła
Evolving Chess-like Games Using Relative Algorithm Performance Profiles
PCG
RAPP
Simplified Boardgames
Generalized RAPP
The End
Procedural Content Generation of complete games
Task
Generate complete rules of high quality games
Applications
Games for humans
Games for AI (competitions)
Problems
Game rules are usually complex
How to measure quality of a game?
Jakub Kowalski, Marek Szykuła
Evolving Chess-like Games Using Relative Algorithm Performance Profiles
PCG
RAPP
Simplified Boardgames
Generalized RAPP
The End
Relative Algorithm
Performance Profiles
Jakub Kowalski, Marek Szykuła
Evolving Chess-like Games Using Relative Algorithm Performance Profiles
PCG
RAPP
Simplified Boardgames
Generalized RAPP
The End
RAPP (Nielsen et al. 2015)
Idea
In good games,
better algorithms should play better than worse algorithms.
Application
Evaluate game quality
by comparing performance of different algorithms.
Jakub Kowalski, Marek Szykuła
Evolving Chess-like Games Using Relative Algorithm Performance Profiles
PCG
RAPP
Simplified Boardgames
Generalized RAPP
The End
RAPP (Nielsen et al. 2015)
Idea
In good games,
better algorithms should play better than worse algorithms.
Application
Evaluate game quality
by comparing performance of different algorithms.
GVG-AI tests
Atari-like games (VGDL)
Simulation-based framework (Java)
Real-time responses.
One player games only (puzzles).
Jakub Kowalski, Marek Szykuła
Evolving Chess-like Games Using Relative Algorithm Performance Profiles
PCG
RAPP
Simplified Boardgames
Generalized RAPP
The End
Research (Nielsen et al., EvoGAMES 2015)
Performance comparison
Jakub Kowalski, Marek Szykuła
Evolving Chess-like Games Using Relative Algorithm Performance Profiles
PCG
RAPP
Simplified Boardgames
Generalized RAPP
The End
Research, cont. (Nielsen et al., IEEE CIG 2015)
Evolution
Based on two controllers: DeepSearch and DoNothing
Fitness function:
RD(score) + RD(wins) + win 50 + win lose
, where:
4
RD means relative difference
win 50 is -1 if win in fewer than 50 frames
win lose is 1 if the game can be both won and lost
Jakub Kowalski, Marek Szykuła
Evolving Chess-like Games Using Relative Algorithm Performance Profiles
PCG
RAPP
Simplified Boardgames
Generalized RAPP
The End
RAPP extension
Generalization of RAPP;
Formalization of the method;
Application for two player, zero-sum games;
Generation and evolution tests.
Jakub Kowalski, Marek Szykuła
Evolving Chess-like Games Using Relative Algorithm Performance Profiles
PCG
RAPP
Simplified Boardgames
Generalized RAPP
The End
Simplified
Boardgames
Jakub Kowalski, Marek Szykuła
Evolving Chess-like Games Using Relative Algorithm Performance Profiles
PCG
RAPP
Simplified Boardgames
Generalized RAPP
The End
Definition
Simplified Boardgames (Björnsson, ECAI 2012)
Turn based; two players; zero-sum games;
Rectangular board; fixed initial position; max one piece per square;
One piece movement per turn;
Capturing only at destination square;
Winning conditions:
reaching a goal square using a certain piece,
captured some number of opponent’s pieces;
Draw occurs when the preset maximum game length is reached.
Set of piece’s move rules
Regular language over an alphabet Σ containing triplets (∆x, ∆y , on):
∆x, ∆y are relative column/row distances;
on ∈ {e, p, w } describes the content of the destination square:
e – empty square,
p – square occupied by an opponent piece,
w – square occupied by an own piece.
Jakub Kowalski, Marek Szykuła
Evolving Chess-like Games Using Relative Algorithm Performance Profiles
PCG
RAPP
Simplified Boardgames
Generalized RAPP
The End
Examples
Move examples
rZkZ0Z0s
o0Zna0o0
6
0ZbZ0Z0o
5
ZpZnZpZ0
4
0ZPO0Z0Z
3
Z0ZQZNA0
2
0O0Z0OPO
1
S0Z0Z0J0
8
7
a
b
c
d
e
f
g
∗
h
Qd3-a3: (−1, 0, e)(−1, 0, e)(−1, 0, e)
Qd3-f5: (1, 1, e)(1, 1, p)
nd5-f6: (2, 1, e)
Qd3-f3: (1, 0, e)(1, 0, w )
∗ Deep
Blue vs Garry Kasparov, 1997, Game 6, Move 19 (last)
Jakub Kowalski, Marek Szykuła
Evolving Chess-like Games Using Relative Algorithm Performance Profiles
PCG
RAPP
Simplified Boardgames
Generalized RAPP
The End
Examples
Simplified Chess example
<--BOARD-->
8 8
rnbqkbnr
pppppppp
........
........
........
........
PPPPPPPP
RNBQKBNR
<--GOALS-->
200 &
@P 0 7, 1 7, 2 7, 3 7, 4 7, 5 7, 6 7, 7 7 &
@p 0 0, 1 0, 2 0, 3 0, 4 0, 5 0, 6 0, 7 0 &
#K 1 &
#k 1 &
<--PIECES-->
P (0,1,e) + (-1,1,p) + (1,1,p) +
(0,1,e)(0,5,e)(0,-4,e) + ... &
N (2,1,e) + (2,-1,e) + (-2,1,e) + (-2,-1,e) + (1,2,e)... &
R (0,1,e)^* + (0,1,e)^*(0,1,p) + (0,-1,e)^* + ... &
Jakub Kowalski, Marek Szykuła
Evolving Chess-like Games Using Relative Algorithm Performance Profiles
PCG
RAPP
Simplified Boardgames
Generalized RAPP
The End
Generalized RAPP
Jakub Kowalski, Marek Szykuła
Evolving Chess-like Games Using Relative Algorithm Performance Profiles
PCG
RAPP
Simplified Boardgames
Generalized RAPP
The End
Method
Model selection
1
Defining the set of example games
2
Defining the set of example algorithms (player profiles)
3
(Selecting model games)
4
Selecting evaluation algorithms
5
Generation and evolution
Jakub Kowalski, Marek Szykuła
Evolving Chess-like Games Using Relative Algorithm Performance Profiles
PCG
RAPP
Simplified Boardgames
Generalized RAPP
The End
Method
Example sets
Example games
Gardner,
Action Man’s Chess,
Petty Chess,
Half Chess,
Demi-chess,
Los Alamos Chess,
Cannons and Crabs,
Small-Deacon Chess,
Shatranj,
Chess.
Example algorithms (heuristic functions)
Constant/Weighted
+ Mobility
+ Control
+ Goal
Jakub Kowalski, Marek Szykuła
Evolving Chess-like Games Using Relative Algorithm Performance Profiles
PCG
RAPP
Simplified Boardgames
Generalized RAPP
The End
Method
Average score of algorithms
Jakub Kowalski, Marek Szykuła
Evolving Chess-like Games Using Relative Algorithm Performance Profiles
PCG
RAPP
Simplified Boardgames
Generalized RAPP
The End
Model
Extracting model
Model games
Division into two sets with minimal sum of distances
dist(G, H) =
Σni=1 Σnj=i+1 (PG [i, j] − PH [i, j])2
.
n(n − 1)/2
(1)
Result: Action Man’s Chess, Cannons And Crabs, Chess, Los Alamos,
Shatranj, Small-Deacon.
Model algorithms
Take k algorithms which maximize spread
spread(Pmodel ) = Σki=1 Σkj=i+1 |Pmodel [i, j] − 0.5|2 .
(2)
Result (k = 3): CG, WGC, WGCM
Result (k = 4): C, CG, WGM, WGC
Result (k = 5): C, CG, WGM, WGC, WGCM.
Jakub Kowalski, Marek Szykuła
Evolving Chess-like Games Using Relative Algorithm Performance Profiles
PCG
RAPP
Simplified Boardgames
Generalized RAPP
The End
Model
Representation graph of the relative performance models
Jakub Kowalski, Marek Szykuła
Evolving Chess-like Games Using Relative Algorithm Performance Profiles
PCG
RAPP
Simplified Boardgames
Generalized RAPP
The End
Evolution
Generation
Restrictions (chess-like games)
Symmetric initial position
Two rows of pieces per player.
Front row contains only pawns (or empty squares)
Back row contain one piece of the king type.
Win by capturing the enemy king.
Win by reaching the opponent’s back row using a pawn.
Parameters
Board width, height ∈ {6, 7, 8},
Number of non-winning figures ∈ {3, 4, 5}.
turnlimit = 3 × width × height + random({0, . . . , 19}).
Jakub Kowalski, Marek Szykuła
Evolving Chess-like Games Using Relative Algorithm Performance Profiles
PCG
RAPP
Simplified Boardgames
Generalized RAPP
The End
Evolution
Fitness function
Features
B=
|scorew −scoreb |
n
is a balance, where
n is the number of plays,
scorew be the percent of points scored by white player,
scoreb be the percent of points scored by black player.
Q=
s
n
is a game’s quickness, where
game is too short if it ends in 10 turns,
s plays were qualified as too short.
D is distance to the model using modified formula (1),
dist(G, Pmodel ) =
Σni=1 Σnj=i+1 (|PG [i, j] − PH [i, j]|)1
.
n(n − 1)/2
(1a)
Fitness value
f =
(1 − D)(1 − B)(1 − Q) if game is playable,
−1
otherwise.
Jakub Kowalski, Marek Szykuła
(3)
Evolving Chess-like Games Using Relative Algorithm Performance Profiles
PCG
RAPP
Simplified Boardgames
Generalized RAPP
The End
Evolution
Genetic operators
Crossover
Roulette wheel parents selection
Uniform crossover (except king squares)
Mutation
Piece mutation regenerates the rules of a random piece.
Position mutation changes the content of a random square.
Selection
Best n games from parents and children sets.
Jakub Kowalski, Marek Szykuła
Evolving Chess-like Games Using Relative Algorithm Performance Profiles
PCG
RAPP
Simplified Boardgames
Generalized RAPP
The End
Experiments
Experiment setup
Generation
4 parameter sets (for piece rules generation);
200 generated games (50 per set);
evaluated using 3 and 4 model algorithms.
Evolution
same 4 parameter sets;
20 generations;
3 and 4 model algorithms: 12 runs with population size 10;
4 algorithms: 12 runs, population size 16, increased mutation rate.
Jakub Kowalski, Marek Szykuła
Evolving Chess-like Games Using Relative Algorithm Performance Profiles
PCG
RAPP
Simplified Boardgames
Generalized RAPP
The End
Experiments
Evolution results
3 algs.; population size 12
Jakub Kowalski, Marek Szykuła
4 algs.; population size 16
Evolving Chess-like Games Using Relative Algorithm Performance Profiles
PCG
RAPP
Simplified Boardgames
Generalized RAPP
The End
Experiments
Results comparison
Variant
Generated
Evolved(12)
Evolved(16)
Example
Max.
0.907
0.971
3 algs.
Avg.
Promising
0.671
29%
0.911
100%
0.858
0.811
Jakub Kowalski, Marek Szykuła
90%
Max.
0.942
0.959
0.978
0.959
4 algs.
Avg.
Promising
0.704
34%
0.916
100%
0.922
100%
0.859
100%
Evolving Chess-like Games Using Relative Algorithm Performance Profiles
PCG
RAPP
Simplified Boardgames
Generalized RAPP
The End
Experiments
Example of evolved game (fitness 0.9538)
Jakub Kowalski, Marek Szykuła
Evolving Chess-like Games Using Relative Algorithm Performance Profiles
PCG
RAPP
Simplified Boardgames
Generalized RAPP
The End
Summary
Contribution
Generalization and formalization of RAPP approach for games
evaluation.
Application in Simplified Boardgames class.
Conclusions
Works best as a sieve with human intervention in the last stage.
Time consuming (requires many expensive simulations).
By using MCTS with different time limits can be made
knowledge-free.
Jakub Kowalski, Marek Szykuła
Evolving Chess-like Games Using Relative Algorithm Performance Profiles
PCG
RAPP
Simplified Boardgames
Generalized RAPP
The End
Thank you
Jakub Kowalski, Marek Szykuła
Evolving Chess-like Games Using Relative Algorithm Performance Profiles