poster.qxp
27/11/2006
14:32
Page 1
Exploration-exploitation
in Go: UCT for MonteCarlo Go
Go is an asian game with more than 4000 years of history.
Go is much more computer challenging than chess: the
branching factor is huge and there is no «good» evaluation function
by Sylvain Gelly1
and Yizao Wang1,2
[email protected]
[email protected]
1. TAO, INRIA Futurs/LRI, UMR 8623,
Paris-Sud University, France
2. CMAP Ecole Polytechnique, France
Tree based search
Multiarmed bandit approach of Go:
each position is a bandit
each move is an arm
play using UCB [1] algorithm
= UCT [2] algorithm
Some tried variations
Specific problems:
Go has a lot of arms;
non stationary;
huge treesize;
must be conservative.
==> don't explore all
arms at first
==> parametrized UCT
==> parsimonious tree [6]
Discussion
results very sensitive to the
parametrization;
one simulation per leaf is the best;
UCB is here often used with very little
information;
metaUCT to generate an opening
database.
MonteCarlo Evaluation function
Monte Carlo Go [3,6] for evaluating
position p:
consider r a random player;
from p, r plays against itself till the end;
the final score is alloted to p.
Parallelization of UCT
Parallelization with shared memory:
straightforward;
very efficient.
Parallelization on a cluster is difficult:
ratio evaluation/communication times;
most nodes are explored only a few times;
may unbalance the exploration in the tree;
==> no very positive results so far
Results
MoGo
MoGo
MoGo
MoGo
is first ranked since August 2006 among programs on Computer Go Server (9x9) [4];
has been the first program to reach 2000 ELO on the Computer Go Server (now is around 2200);
won the two October 2006 KGS [5] international tournaments (9x9 and 13x13);
won the two November 2006 KGS [5] international tournaments (9x9 and 13x13).
Acknowledgements: Rémi Munos (INRIA/Sequel), Olivier Teytaud (INRIA/TAO), Pierre Arnaud Coquelin (INRIA/Sequel), Rémi Coulom (INRIA/Sequel)
[1] Peter Auer, Nicolo CesaBianchi,and Paul Fischer. Finitetime analysis of
the multiarmed bandit problem. Machine Learning, 47(2/3):235256,2002.
[2] L. Kocsis and C. Szepesvari. Banditbased montecarlo planning. ECML 06, 2006.
[3] B. Bruegmann. Monte carlo go. 1993.
[4] Computer Go Server, http://cgos.boardspace.net/9x9.html
[5] Kiseido Go Server, http://www.gokgs.com/tournList.jsp
[6] Rémi Coulom. Efficient selectivity and backup operators in montecarlo tree search.
Proc. of the 5th International Conference on Computers and Games, 2006.
NIPS 06