Turn-based Game Engine and AI
Melvin Winstrøm-Møller s072435
Niels Thykier s072425
August 31, 2010
Contents
Table of Contents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
List of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1
4
1 Preface
5
2 Introduction
6
I
8
Assignment specification and analysis
3 Reasoning behind the project
3.1 Introduction to game engines and related considerations . . .
3.2 Types of games . . . . . . . . . . . . . . . . . . . . . . . . . .
3.3 Technical requirements of TN and RTP games . . . . . . . .
3.4 Competitiveness of a new TN game engine . . . . . . . . . . .
3.5 Summary and final assignment description . . . . . . . . . . .
3.6 Potential weaknesses and difficulties of the chosen assignment
.
.
.
.
.
.
.
.
.
.
.
.
9
9
10
11
13
14
15
4 Engine requirements
4.1 Game developer requirements
4.1.1 Graphics . . . . . . .
4.1.2 Game Logic . . . . . .
4.1.3 Multiplayer . . . . . .
4.1.4 AI . . . . . . . . . . .
4.1.5 Resource IO . . . . . .
4.2 Feature requirements . . . . .
5 AI
5.1
5.2
5.3
5.4
requirements
Aspects of AI in TN games
AI tasks and technologies in
AI tools . . . . . . . . . . .
AI feature requirements . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
17
17
19
19
20
21
22
22
. . . . . . .
TN games .
. . . . . . .
. . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
24
24
27
30
31
.
.
.
.
.
.
.
6 Technology
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
34
1
II
Engine
7 Design
7.1 The original implementation
7.2 Event-based system . . . . .
7.3 Graphics system . . . . . .
7.4 Game logic system . . . . .
36
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
37
37
38
39
39
8 Implementation
8.1 Graphics system . . . . . . .
8.2 Game logic system and AI . .
8.3 Multiplayer system . . . . . .
8.4 Resource IO and management
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
41
41
42
43
44
9 Discussion
45
III
47
AI
10 AI Design
10.1 TN-focused libraries . . . . .
10.1.1 Path finding . . . . . .
10.1.2 Information extraction
10.2 Machine learning . . . . . . .
. . . . . . . .
. . . . . . . .
and analysis
. . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
49
49
49
51
52
11 AI Implementation
11.1 Selection of AI methods . . . . . . . . . . .
11.1.1 Path finding . . . . . . . . . . . . . .
11.1.2 Information extraction and analysis
11.1.3 Machine learning . . . . . . . . . . .
11.2 Implementation of AI methods . . . . . . .
11.2.1 Path finding . . . . . . . . . . . . . .
11.2.2 Blocking . . . . . . . . . . . . . . . .
11.2.3 Influence mapping . . . . . . . . . .
11.2.4 Genetic Algorithm . . . . . . . . . .
11.2.5 Case-based Reasoning . . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
55
55
56
56
59
62
62
63
64
65
67
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
script engine
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
71
71
72
72
73
74
74
75
12 Script Engine
12.1 The idea and outcome . . . . . .
12.2 Script engine design . . . . . . .
12.2.1 Script engine components
12.2.2 Linking scripts . . . . . .
12.3 Lessons learned . . . . . . . . . .
12.3.1 Code analysis . . . . . . .
12.3.2 Debugging scripts and the
13 Discussion
76
2
IV
V
Conclusion
79
Appendix
82
14 Abbreviations and acronyms
83
Bibliography
85
3
List of Figures
11.1 Influence map . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
11.2 Blue team crushing the red team. . . . . . . . . . . . . . . . . . .
4
65
69
Chapter 1
Preface
This thesis is submitted as part of the requirements for obtaining a Bachelor’s
Degree at the Technical University of Denmark. The project, which corresponds
to 15 ECTS points per author, was carried out at the Department of Informatics and Mathematical Modelling under the supervision of Assistant Professor
Henrik Pilegaard in the period from April to August 2010.
The background for picking this project was the continuation of a hobby
project. Extending the project into a bachelor project both meant that we could
be allowed to work on our own project, as well as challenge ourselves in several interesting fields of computer science, including algorithms, multithreading,
language design and artificial intelligence. The project was on a scale neither
of the authors had experienced before, and the experience of working on such a
varied and interesting project have been worth the many hours put into it.
As part of making the project come true as well as helping us on our path,
we would like to thank our supervisor Henrik Pilegaard for his great humor and
our long and fruitful conversations with him, which brought many useful ideas
for the project. We would also like to thank the people behind Battle for
Wesnoth for putting their graphics under GNU General Public License version
2, which allowed us to reuse the graphics for the game implementation.
5
Chapter 2
Introduction
Abstract Game engines have become increasingly important in the development of games as computing resources have increased. However, competing in
this market is difficult, as there is an ever increasing number of game engines.
This thesis introduces a game engine focused on a game genre not previously
supported by engines: turn-based node-based (TN) games. This genre of games
cover a large variety of games, including many card and board games, as well as
more advanced computer games. Given the special properties of these games,
an engine taking advantage of these properties is designed and implemented.
As part of the development of the engine, candidate methods for an AI
library is tested and implemented. Developing AI is frequently a challenging
task, but AI support is frequently rare or lacking in game engines, partly due
to the variety of games. Focusing on a specific genre of games, AI methods are
chosen and tested in order to determine whether an AI library focused on TN
games can successfully support a TN engine.
Overview This report describes a bachelor thesis in the field of computer
science. The first goal of the thesis was to design and implement a flexible game
engine focused on the niche of games classified as both turn-based and nodebased. The second goal was the implementation of several AI libraries and tools
to help support the engine, and investigate the value of including AI libraries
and tools to the game engine.
Structure of the report The first part of the report, introduction, introduces the bachelor thesis and the structure of the report.
In part 1, assignment specification and analysis, the background for the as6
signment is described, and the assignment is analysed thoroughly, resulting in a
set of requirements for both the engine and the AI methods. Part 1 is concluded
by a technology chapter discussing the chosen technology in the project.
In part 2, engine, the engine design and implementation are described and
discussed. The focus is on the subsystems of the engine and their support of
TN games, as well as the overall design goals of separation of concerns. Part 2
is concluded with a discussion on the resulting design and implementation, and
presents the fulfilled requirements from part 1.
In part 3, AI, the requirements for the different methods are described in
the design. The implementation describes and discusses the selections of the
methods, and then describes and evaluates their implementation. The design
and implementation of the scripting engine is then described and discussed.
Part 3 is concluded with a discussion on the AI methods and their usability to
TN games and game engines.
The report then ends with the conclusion on the results of the thesis.
7
Part I
Assignment specification
and analysis
8
Chapter 3
Reasoning behind the
project
This section discusses several aspects of game engines, game types and other
issues relevant to the chosen assignment. In the end of the section, the conclusions from the aspects are summed up, and finally a section discusses the
potential weaknesses and difficulties of the chosen assignment, and how they
may be dealt with.
3.1
Introduction to game engines and related
considerations
In the early days of game development, the relatively low complexity and cost
of developing games combined with the few computer resources available made
it undesirable to create engines that enabled code reuse. Instead, it was cheaper
and more effective to simply start over for each game, especially as platforms and
technologies changed rapidly. For instance, hand-coded assembly code was used
to increase performance by taking advantage of each platforms capabilities, and
the games often had to be recoded somewhat when porting to new platforms. 1
1 The below links describes how assembly language was and still is used in game creation.
Describes how assembly code was used to port a game to the PDP-7.
http://www.courses.fas.harvard.edu/˜lib215/reference/history/spacetravel.html
Describes how some consoles required assembly.
http://www.eidolons-inn.net/tiki-index.php?page=SegaBase+Saturn
Discusses the issues with porting games. Includes comments about how assembly code and
other platform-specific code is typically minimised
http://www.bit-tech.net/gaming/2009/08/18/the-problem-with-porting-games/1
9
As computer resources (such as RAM, CPU, etc.) increased considerably,
and long-term technologies were developed2 , game engines became feasible. By
abstracting away the underlying details, and reusing many of the different software systems used in the games (such as graphics, logic, sound, network, etc.),
several advantages are achieved:
• Increased portability
• Decreased cost and development time
• Decreased complexity and skill requirements
These advantages can sometimes come at the cost of flexibility and speed. Often,
game engines have to be designed to prioritise between 2 factors: The more
specialised the engine, the more assumptions can be made, and the easier it is
to achieve high performance using the engine. However, this generally comes at
a cost in flexibility. This can help explain why it is not all games today that
are developed using game engines - by creating a code base using few or none
existing software components, a highly specialised and optimal code base can
be created for the game in question.
Another motivation is the lack of a game engine for a particular genre of
games. While game engines offer many advantages, they are also difficult software systems to develop. Many different subsystems have be developed and
interact in an efficient manner, and many design choices must be made.
Furthermore, adopting a game engine for future games is a considerable
technology investment, both in terms of knowledge of using the engine in the
development team, and in the technology of the engine itself. For the engine
to be adopted, there have to be considerable advantages attained by using the
engine - both in terms of an advantage over developing an engine from scratch,
and from using an existing engine. A new game engine needs an edge, such as
being specialised for a certain genre of games, for which there are no existing
engines that fits very well.
3.2
Types of games
One of the genres for games are turn-based, node-based (TN) games. This
genre of games are most generally known from board games. They cover traditional games like Chess, Checkers, Nine Men’s Morris, Ludo, etc., more
modern games like Space Hulk, Settlers of Catan, Descent, Monopoly, Axis
& Allies, etc., and computer games such as Advance Wars, Final Fantasy
2 Examples are the graphics libraries OpenGL and DirectX, which were developed in the
start 1990’s, and are still used today. They are often the back-bone in the graphical parts of
engines, and are quite portable. OpenGL supports a large number of operating systems and
even some consoles
10
Tactics, the Civilization series (and its clones), the Age of Wonders series,
Battle for Wesnoth, etc.. The board games can in general be turned into
computer games, and often have been.
A turn-based game is a game where the game flow is partitioned into welldefined and visible parts, called turns. Reflexes and motion control have little
or no importance in these games. Instead, the ability to create tactics and
strategies, or to figure out the puzzles the game presents, is generally more
important.
Whether the time allocated for each turn is unlimited or limited differs, but
the time allocated to each turn is generally enough to let the player consider
what to do.
The alternative to turn-based is real-time, in which the game continues,
independent on whether the player acts or not. While in practice all computer
games have logic partitioned into small time units, real-time games generally
have very small time units.
A node-based game is a game in which the board and spaces of the game can be
modeled as a graph with nodes. A chess board, a hexagonal board and other,
more special boards (such as with varying space sizes) can all be modeled as a
graph with nodes.
The main alternative to node-based games are position-based games. In these
games, spaces and dimensions are represented through positions. A game which
is still node-based, even though it may seem position-based, is the real-time
strategy (RTS) game Warcraft 2, which features a large grid. While some may
consider it position-based, it is considered node-based in this project since its
granularity is not small enough to be impractical to represent as nodes in a grid.
An example of a position-based game is the RTS game Command and Conquer:
Generals.
While games are often either turn-based and node-based, or real-time and
position-based, games that mix the types exist. In the rest of the report, turnbased, node-based games will be called TN, while the real-time, position-based
games will be called RTP.
3.3
Technical requirements of TN and RTP games
From a software engineering point of view, the two general types of games
described above have different technical requirements. RTP requires the state
11
to be updated constantly and needs constant communication between the game
and the player. This place constraints on how much time each component may
take to do their tasks. This is unlike TN games, in which several components
have a relatively long time to perform their tasks. For instance, AIs in some
Chess games may take several seconds or even minutes to compute their moves,
and this is accepted by the players.
Game logic One very important component is the game logic structure. By
game logic is first and foremost meant the code that modifies the game state.
It is in the game logic that the game rules are defined, through defining what is
possible and what is not. In RTP games, game logic is frequently mixed with
other code that handles issues like graphics, AI, etc. This gives fine-grained
control and integration between game rules and the rest of the game. In a
typical RTP game, a game loop is run, in which small scripts modifies the game
state in each step. Typically, many (20-80) steps are run per second. While this
fits RTP games well, it makes much less sense for turn-based games, which only
needs to modify the game state whenever a player takes an action.
AI Another, often important part of a TN game is the AI, which is somewhat
different from the AI in RTP games. The AI in RTP games frequently have to
meet constant deadlines, because of the need for quick decisions. To help the
AI meet these deadlines, the AI is typically run in the same loop as the game
logic (sometimes mixed with the game logic).
Another option is to execute all or some of the AI in a special AI loop,
typically with a lower frequency to give extra time for complex analysis. This
makes little sense for turn-based games, since the players makes the choices,
and a complete separation of game logic and AI not only helps game designers,
it also increases modularity and separation of concerns.
Networking Networking is also another important part of many games, RTP
as well as TN. RTP games have constant state changes. Keeping the traffic low
in RTP games is entirely possibly given the right approach, even in games with
a huge game state and frequent game changes 3 . However, these approaches can
complicate matters, and turn-based games have much looser constraints (much
more time to communicate much less game state, for instance in the form of
player actions), making it plausible that a single network subsystem may be
able to serve the needs of many simple TN games, or the least common subset
of their needs.
Summary Given these differences in technical requirements, it seems likely
that a properly designed TN game engine will have a considerable advantage
12
over a similar RTP game engine in regards to development of TN games.
3.4
Competitiveness of a new TN game engine
The vast majority of game engines supports real-time first and foremost, and we
have failed to find any turn-based engines, which were not focused on a specific
game. However, we have found multiple requests for turn-based engines.
A flexible TN engine would be clearly preferred over RTP engines for TN
games, since it would support TN games much better, and it would have little
competition from other TN engines, since no or few flexible TN engines exists.
A new flexible, TN game engine would therefore have a clear edge over other
game engines in regards to development of TN games.
AI libraries and tools Many TN games are tactical or strategic in their game
play, and it is frequently difficult to implement a proper single-player version
of a TN game without a good AI. However, developing a good AI is one of the
more time consuming and difficult tasks of developing a game. Supporting AI
development through the inclusion of specialised libraries and tools would give
the engine an edge.
Nevertheless, AI libraries are often difficult to implement, because a considerable part of the AIs written contain game rules, and are therefore specific
to a particular game. Still, a genre of games typically contain some of the
same AI tasks and solutions. For instance, first-person shooter (FPS) games
often employ state machines, and some engines implement direct support for
state machine programming 4 . In regards to TN games, a common feature is
the graph-based nature of these games, and most of these games would benefit
considerably from an AI subsystem handling common tasks on graphs, notably
search tasks.
AI is also a rapidly developing field, and experimenting with new AI techniques not used in games before could give an edge in the quality of the AIs.
It might also help pave the way for the use of these techniques in the game
industry.
3 http://www.gamasutra.com/view/feature/3094/1500˙archers˙on˙a˙288˙network˙.php
an article describing the networking used in Age of Empires 1, 2 and ”RTS3” (possibly Age
of Mythology). The networking only communicates player actions, and the game designers
went to great lengths to ensure that the same code would always run the same on multiple
computers - for instance, to get random numbers, pseudo-random number generators was
used, exchanging the root number between all clients at the start of the game. The process
was complicated and was a source of obscure bugs, but made networking feasible
4 UnrealScript, a game engine used for several projects:
http://www.unrealtechnology.com/
Manual describing the UnrealScript, including their support for state machine programming
13
3.5
Summary and final assignment description
Based on the observations and discussion in the previous sections, we believe
that the design and implementation of a TN game engine would make an interesting and valid project. While the game industry and the game engine industry
is fiercely competitive, there is a lack of competition in the niche of flexible TN
game engines. Combined with the important design differences in the structure
of TN and RTP games, it seems possible that such an engine could achieve
success, and maybe be the start of a new type of game engines.
Furthermore, investigating whether such a type of engine could benefit from
an AI library could help increase the advantage of such an engine over traditional
game engines when it comes to TN games. We have noticed that while many
different game types employ many different AI methods and technologies, TN
games share many common traits, most notably the graph-based structure.
Revealing which AI techniques are useful could not only be useful to give the
engine an edge, but could help guide future engine development. This help
includes RTP engines for those AI techniques tested which are not specific to
TN games.
Given these considerations, we have decided on the following assignment
description.
Assignment description:
1. Design and implement a flexible TN engine.
2. Investigate several AI techniques as candidate parts of a possible AI subsystem in the engine.
As part of the development, we have also decided to implement a game on top
of the engine, such as to help the development of the TN engine, as well as to
help test and evaluate the AI techniques.
To help guide the development, we also have some long-term goals. These
are not expected or guaranteed to be fulfilled in the project, or in the near
future, but they will be important in regards to guiding the requirements and
designs of the game engine.
Long-term goals:
1. Create a flexible, TN game engine, which will be used to create at least
directly in the language:
http://unreal.epicgames.com/UnrealScript.htm
14
one successful game.
2. Make the TN game engine facilitate easier development of turn-based
games than starting from scratch or using existing game engines.
3. Attract developers to help the chances of success of the engine.
3.6
Potential weaknesses and difficulties of the
chosen assignment
While we believe that the chosen project assignment is good, there are some
difficulties. Particularly, the disadvantages of a TN game engine against an
RTP game engine, and the question of why no one have made a successful TN
game engine before.
The main disadvantage of a TN game engine against an RTP game engine
is the types of games that each engine type can support. If games are split up
in regards to turn-based and real-time, and node-based and position-based, it
is seen that an RTP engine is considerable more flexible.
The type of games the engine types TN and RTP supports:
Turn-based
Real-time
Node-based
TN, RTP
RTP
Position-based
RTP
RTP
Examples of games that are node-based and real-time includes the old RTS
games Dune 2 and Warcraft 2, which feature large grids. Examples of games
that are position-based and turn-based include Darkwind: War on Wheels
and Brotherhood of Battle.
While research indicates that the bulk of games fall into the category of
either TN or RTP, successful games still occur in the categories of real-time,
node-based games, and turn-based, position-based games. Furthermore, games
that mix the genres, for instance having one part of the game being TN, and
another part being RTP, are only supported by RTP style engines.
We believe that this is somewhat of a problem, but not a critical one. There
are many successful, pure TN games, and not all players desires a mix of genres.
Specifically, there are a group of players that appreciates that fast reflexes are
not required to play and enjoy TN games. Mixing the genres might prevent this
group of players from enjoying a game.
15
The question of why no one have made a successful TN game engine before
might indicate that such an attempt is futile. Two counter-arguments against
this claim could be:
• There are more RTP games than TN games, and this single fact can
explain why RTP engines are the most common type. However, the game
market have grown enormously in the last decades, and we believe that the
TN game market has grown with it, making the niche market big enough
for specialised engines.
• Secondly, TN games require less computing resources than RTP games.
This might explain why there have been more development in RTP game
engines instead of TN game engines. It could also suggest that any TN
game engine needs to focus on a design that grant advantages over not
using the engine, such as providing AI subsystems useful to TN games,
and tailoring the engine design especially well to TN games.
16
Chapter 4
Engine requirements
The main source of requirements is the users of the game engine, the game
developers. There are some general requirements from these users. Defining and
investigating their requirements will be important to the success and usefulness
of the engine.
The following sections will first discuss the game developers requirements,
then the engine design will be discussed, and finally the requirements will be
fleshed out.
4.1
Game developer requirements
When designing a game engine for game developers, there are some general
characteristics that are important for game developers:
• Portability
• Cost and development time
• Complexity and skill requirements
From a game engine point of view, these are addressed by focusing on different
aspects:
• Providing generic subsystems in the engine to handle common tasks, such
as graphics, networks, etc. These subsystems should hide difficult and
time-consuming aspects of the aforementioned issues. For instance, mul-
17
tiple engines hide which graphics API is used.
1
• Providing (visual) rapid development tools, such as level editors, animation editors, scripting, all often combined in an integrated editor. This
both eases and speeds up development. While not necessarily part of the
engine, the engine can be specifically tailored to support these tools. 2
• Choosing the right technology. If the technology chosen is platformdependent, neither the engine or its games will be usable on other platforms. While this is done by some game developers, it usually means a
considerable decrease in the potential players of the game. See the technology part for more.
The second aspect is important, but not necessarily vital. Editors can always be
built upon the engine or added, and while they speed up the content creation,
it will not make the game run faster, have better graphics, become more stable,
etc., apart from the fact that the time saved by editors can be used to improve
these aspects. Several engines provide no or few development tools.
Considering that the project time is limited, the rapid development tools
may not be provided; while they are worth the time, they take a lot of time to
implement, and the engine itself has higher priority. Besides, if the engine has a
good design, it will be easier to add these tools later, including when the engine
is being used.
Amongst the subsystems, the pure basics are needed: a way to handle graphics, a way to write game logic and AI code and a User Interface (UI) system.
In this vein, the game logic and AI code is especially interesting, because it
seems likely that these subsystems can be structured more efficiently for TN
games than typical for RTP engines, given the differences in structure of TN
and RTP games. A subsystem that is not necessary, but also interesting, is the
multiplayer subsystem. Not only is multiplayer often important for TN games,
it might also be handled differently in a TN engine, possibly giving the engine
another edge over RTP engines.
A subsystem which is not strictly necessary, but is important nonetheless, is
the resource reading and writing. Such systems are useful when resources like
textures, sounds, levels, objects, save games, configuration, etc., needs to be
1 The jMonkeyEngine provides no visual editors, and hides whether JOGL or LWJGL is
used.
http://www.jmonkeyengine.com/wiki/doku.php/complete˙features˙list
The OGRE graphical engine provides no visual editors, and hides whether Direct3D or
OpenGL is used.
http://www.ogre3d.org/about/features
2 The Unity engine is designed such that it is possible to pause the game, step-wise go
through the game, and even edit resources such as textures, sounds and scripts on the fly.
http://unity3d.com/unity/features/
18
handled. A few games can be made that do not need such resources or contain
them directly in code. However, resource IO is integral to most games, including
those in the TN genre, and developing and testing without resource IO could
hide flaws in the engine design. The inclusion of some form of resource reading
and writing would therefore be desirable.
Below the different subsystems will be discussed. Several factors will be
considered when discussing them, such as project time constraints, usefulness,
how they maybe differ from their RTP equivalents, etc.
4.1.1
Graphics
The graphics subsystem is needed, because it is the main way to communicate
output to the players in the vast majority of games, and because it is a major
part of the aesthetics of most games. Most graphics systems are split into
whether they support 3D or not. While 3D can enhance the game experience a
lot, it is not vital, and it can take a lot of work and good design to make a good
3D engine. Besides, it may be possible to integrate the engine with existing 3D
engines, such as the jMonkeyEngine, which is based on Java and is primarily a
3D engine. As long as the graphics system is replaceable, implementing a basic
2D system should prove sufficient.
Some 2D TN games are not animated, but simply shows the graphics as they
are, and updates the graphics immediately once something happens. While this
is easy and efficient, it is not optimal for all TN games, and to help ensure that
the game engine can properly handle 2D graphics which includes animation, the
engine must implement it as well.
4.1.2
Game Logic
The game logic system can be written in many ways. In RTP engines, the
typical game logic system is that of a step system: perform a number of steps
each second, such as 30 times per second. This is a good system in a real-time
game, since it handles the need to change the game state continually.
In contrast, the game state in turn-based games does not need to change
continually, but only when the player takes an action. This means that a turnbased game needs to sort of “stand-by” in an RTP engine, and wait till the
player actually acts. In RTP games, where synchronisation is important, the
mixing of the game logic, AI, graphics and sound code is quite practical. But in
a TN game where such synchronisation is not necessary, such mixing indicates
that concerns are not separated.
A system where concerns are separated is desirable for multiple reasons. For
19
instance, if the competitiveness of an AI needs to be tested, the graphics and
sound would be easy to turn off, if the code is not mixed. However, whether
such separation can be achieved can not be answered at this point, since the
subsystems are interdependent. For instance, the graphics system needs to
access the game state, and needs to be able to react to state changes and player
actions.
A key difference between TN and RTP games is that RTP games have constantly changing game state, which a TN game does not. For RTP engines, the
assumption that the game state changes constantly is typically a part of how
game logic is structured and executed. Frequently, a game loop is run in the
engine, which executes code that can change the game state. While this fits
RTP games very well, it does not fit TN games well. Game state only needs to
be changed once the player acts, and changing it at any other time is generally
an error. Furthermore, code in the game loop may not take too long to execute,
or the game may end up lagging, since the game loop does not only handle
game logic, but also graphics and IO. This means that some events may have
to be split up or run outside the game loop, indicating a need for some sort of
synchronisation.
If a game logic system can be implemented, which does not expect the game
logic to change the game state constantly, but instead expects that the game
logic only changes the game state in reaction to a player action, it would give
considerable advantages over RTP engines when writing TN games. It seems
easy to implement a logic system which supports reacting to the player; wait
until the player acts, and then run the game logic code related to that action.
This would make it easier to write TN games, and should therefore be included
as a requirement. On the other hand, how to separate concerns is not immediately clear, and should therefore be an optional requirement, because it may
not be possible without too much work.
4.1.3
Multiplayer
Multiplayer is a great subsystem to have, because it enables playing between
multiple players and not just against an AI. Playing against an AI is rarely as
interesting and unpredictable as playing against human players, and some games
can skip the often difficult challenge of implementing a good AI by implementing
multiplayer. Furthermore, the replay value can be increased through the use
of multiplayer modes. 3 Nearly all board games are designed to be played
by multiple human players, which makes multiplayer an integral part of most
computer game translations of such games.
Some TN games support hotseat for multiplayer, such as Heroes of Might
and Magic 3. Hotseat means that the game is played on the same computer by
two or more players, and the “hot seat” refers to the changing of seats between
20
turns.
Either hotseat or network multiplayer should be implemented. This would
help ensure that the game engine systems supports multiple players at once.
Without such a system, a hidden constraint for multiplayer modes may be
discovered late in the process.
Since TN games do not need to constantly update the state, the loose constraints may allow the design and implementation of a general multiplayer system. If this system is effective enough for the vast majority of TN games,
it would make networking considerable easier for developers. Investigating a
system which will make multiplayer relatively easy for TN games in general is
desirable, but it may take a bit too much time to design and implement properly.
4.1.4
AI
The AI system is often important in TN games. There are several facets of an
AI system:
• Game logic data structures
• AI libraries
• AI tools
In regards to data structures, the main data structure is the graph, represented
by nodes connected to each other. Supporting this data structure directly in
the game engine makes a lot of sense, because all TN games needs to have
nodes. Furthermore, having the same general node structure means that general
algorithms can be developed for this data structure. Of course, different games
may require different nodes, and the node data structures/algorithms should
handle this.
AI libraries and tools can be both TN specific and general-purpose. While
many engines do not provide these, or provide them for a very specific task
(such as path finding in FPS-games), AI is frequently important for TN games
as mentioned in an earlier section. Since we have decided to implement several
AI techniques, the AI tools and libraries are discussed in their own section.
3 All MMO-games (massively multiplayer online) are meant to be played for years. The
RTS game Starcraft, released in 1998, have grown a large community worldwide, and even
a professional league, based on its multiplayer.
http://en.wikipedia.org/wiki/StarCraft˙professional˙competition
21
4.1.5
Resource IO
The resource IO system includes the reading and writing of different game resources. While it is not strictly necessary to have, games often contains many
different resources, and incorporating the handling of resources early will make
it easier to use and test the engine, such as in the form of test games.
4.2
Feature requirements
Given the discussion above, 2 lists of required features have been made. The
primary requirements must be met. The secondary requirements include items
which would be quite good to have, but may not be possible, for instance due
to the time constraints on the project.
Primary:
1. Basic 2D graphics system. The game engine must support 2D graphics,
including animation.
2. Basic TN game logic system. The game engine must support a game logic
system. The system must support TN games at the structure level.
3. Basic multiplayer. The sample game must support either hotseat or network multiplayer.
4. Basic TN AI system. The game logic and AI system must support a
flexible, general node data structure, which will make the basis for general
algorithms. “Flexible” and “general” means that the data structure and
possible algorithms should be customisable to any given game, such that
re-implementation of the data structure and general algorithms will not
be needed for the vast majority of TN games.
Secondary:
1. Implement a basic TN multiplayer system. The implementation of a general multiplayer system, that both serves the vast majority of TN games,
and makes multiplayer implementation in games using the engine easy.
2. Support playing over the internet. Support and testing for playing over
the internet. Complete support would include a lobby service, such that
finding other players would be very easy.
3. Completely separate game logic, graphics and AI code. The complete
separation of game logic, graphics and AI code, such that each part in a
22
game can be coded somewhat separately without concern for the others.
4. Basic resource IO system. The system must handle the reading and writing
of resources, and must make it easy to organise and find resources in the
game.
23
Chapter 5
AI requirements
The first part of this section will introduce and discuss AI aspects in the context
of TN games. It will then present common tasks and related technologies of TN
games. A discussion of the desirable aspects of candidate AI libraries and tools
in regards to a TN engine then follows. A special type of AI tool, a domainspecific language focused on supporting AI, is discussed as a candidate to the
TN engine. Finally, a list of requirements is presented.
5.1
Aspects of AI in TN games
It has been discussed in previous sections why AI in TN games play an important
role. This section will focus on different aspects which are important in regards
to AI. In the end of this section, the needs that AI libraries and tools need to
fulfill will be summarised.
An aspect to consider about AIs in games is that the AI code is often specific
to one or a few games. There are 2 reasons for this. As a consequence of
the different game rules the AI have to solve different goals and tasks. Games
sharing game logic concepts, such as graphs, may share some tasks. The second
reason is that the code may be optimised for performance. Different games have
different task dimensions. For instance, one game handling a few units path
finding through a complicated, weighted graph; versus another game handling
30.000+ units real-time path finding through a small and simple graph. Using
the code of a game in place for the code of another game may lead to performance
issues.
As mentioned earlier, this is less of a problem in TN games, where there is
24
generally plenty of time to compute AI tasks. Furthermore, the common data
structure, the graph, makes it likely that some tasks are common for many or
even the majority of TN games. The premier example is path finding. Of course,
the issue of intractability is frequently a problem in AI, even in TN games, and
making the general handling of the common tasks effective enough may become
an issue.
Another aspect to consider is that different requirements can be given to AIs for
different games. While some AIs are focused on being as competitive as possible,
such as is often the case in the game Chess, other AIs try to be entertaining by
not being as competitive as possible. In these games, a good AI can be given
faults and random behaviour to ensure that it is more entertaining.
As Jonny Ebbert, the lead designer on Dawn of War 2, said1 :
Add frailty but avoid stupidity. A.I.s need to make mistakes for
the player to exploit from time to time but they shouldn’t look dumb
doing it. A.I’s aren’t fun to play if they always trigger their abilities
perfectly when they have the chance . . . But they need to stay in the
range of competency when they do make ‘mistakes.’ It’s a fine line
to walk, but an important line.
Randomness Making parts of the AIs decisions random might mean that it
becomes more competitive. Some players utilise the fact that some game AIs
are fairly or completely predictable to lure the AI into traps. While this is
entertaining in some games, it might lead to boredom, since the same, simple
strategy/tactic may work every single time. Adding random behaviour can
break this tendency.
An element that can help create game longevity and re-playability is if the
AI can change its behaviour over time, for instance by reacting to different
strategies used by the player. While the behaviour may well not be optimal in
these cases, it may defeat the player if the player depends on using the same
strategy or tactic, i.e. trap, over and over again.
Cheating In this report, “cheating” is not defined as letting the AI and the
player have different premises, such as the AI having a bigger base in a special
mission. Instead it is the hidden changes in rules for the AI that work to its
advantage.
Cheating AIs are not only common in some games but also encouraged by
game developers. To quote the lead designer on Dawn of War 2, Jonny Ebbert2 :
1 http://kotaku.com/5271733/the-three-or-more-or-less-laws-of-gaming-ai
25
Cheat wherever you can. A.I.s are handicapped. They need to
cheat from time to time if they’re going to close the gap . . . Never
get caught cheating. Nothing ruins the illusion of a good A.I. like
seeing how they’re cheating.
An example comes from RTS games, in which it is common that the AI is
granted resources for free when it needs them, or seeing all of the game state in
a game with hidden information.
Another reason for cheating can be efficiency. As an example in first-person
shooters, letting the computer players ask the engine for the position of the
player is much more effective, than having them compute it themselves.
While many games accept cheating, some TN games are both challenging to
write AI for and is expected not to cheat. Examples are the games Chess and 9
Mens Morris, in which the game state is completely visible. However, cheating
can be used together with a well-developed AI to enhance the game experience.
A well-developed AI can still be weak, and adding a little cheating can uphold
the illusion of no cheating, while still giving a good challenge. Furthermore,
a well-developed AI can more easily afford doing deliberate faults, faults that
may help make the AI entertaining, if it also cheats. Thus, even TN games that
cheat can benefit from AI tools and libraries.
Changes to game rules Game AI often contains knowledge collected from
human players playing the game. Properly designed, simple game rules can give
rise to complex tactics and strategies. An example which can clearly illustrate
this is the simulation program Life3 , in which 4 simple rules and an initial
setup in a graph can give rise to surprising and complex results. However, the
creation of these interesting and complex rules can be difficult to attain for game
developers. Constant testing and tweaking may be required to get the desired
result. Not only does the game rules have to give rise to complex games, they
also have to be entertaining for human players.
Apart from the development process, game rules can also be changed once
the game is already deployed. This may be done for many reasons, including
balancing the difficulty level, leveling out the power level in multiplayer games,
or creating a new experience. This occurs frequently, especially in multiplayer
games, where a fair fight is expected by players. Another example is variations
of known games, such as Chess.
The frequent changing of game rules can affect the AI. Since the game AI
frequently contains game rules and human experience on how to play, rewriting
the AI may be required, and if the AI is dependent on human knowledge to work
2 http://kotaku.com/5271733/the-three-or-more-or-less-laws-of-gaming-ai
3
Also known as Conway’s Game of Life.
26
properly, the game must be tested somewhat before the AI can be updated. AI
libraries and tools that facilitate changing the AI quickly or even automatically
in response to new game rules will be useful to ensure that the AI is up to date
without much work.
Summary Given all these aspects, a summary of qualities that can be useful
for TN AI libraries and tools can be extracted:
• Support for common tasks occurring specifically in TN games, such as
path finding and map analysis.
• Support for unpredictable behaviour and faults, such as through randomness or changing approach over time.
• Support for rapidly reacting to game rule changes, either allowing quick
adoption through general libraries and tools, or even automatic adoption.
5.2
AI tasks and technologies in TN games
Path finding A main task of any game AI is to analyse the game state and
extract useful information. In any TN game, the graph is a major data structure.
Moving around pieces is very common, and this often requires path finding
of some sort. There exists many different search algorithms for graphs, and
implementing one or several of these is likely to benefit the vast majority of TN
games.
Of course, some TN games, like Chess, benefit less from this searching,
because the board can be represented as a grid instead of a graph, and the units
move in special ways according to this grid. But this is the exception rather
than the rule. Another use of graph searching is the search of more abstract
graphs, such as game trees in Chess, where the possible moves from each player
is considered.
Information extraction and analysis Apart from the task of searching and
finding paths, the main game graph can also be investigated in other ways. In
TN games, positioning often plays a crucial role. Extracting information based
on positioning, such as finding positions according to some criteria, can be very
useful in many TN games.
Examples include finding the nearest, desirable resource, the strongest enemy
that can be taken, a good position for blocking the opponents units, finding the
spot for the next base, etc. Even in games such as Nine Mens Morris, it could
be a task such as finding places suitable for getting 3 pieces in row without the
27
opponent interfering.
While these techniques are not specific to TN games, and also occur in realtime, node-based games, more time for complicated analysis is available in TN
games. This means that there is less of a need for algorithms optimised for
speed for specific games, and that general algorithms should be reusable for the
majority of TN games.
Decision making The previously mentioned methods are specific to TN
games. Many other information gathering and analysis technologies also exist, which are not specific to TN games.
Whether some of these are especially suited to a majority of TN games is
unknown to us, but investigating a selection of technologies may uncover their
usefulness as AI libraries for TN engines.
Once information has been extracted, the AI must decide what to do. While
the information extraction can be specific to TN games, the decision making is
not, except that the time constraints are less strict than for RTP games. Many
different technologies and methods can be used in this section, and there is no
general methods or technologies that fit TN games particularly well in general.
A general trait for the majority of TN games is that there is an opponent
of some kind. It is possible to have TN games that have no opponents, but
still use AI. Possible uses of AI in such games include using AI to automate
certain tasks for the player. However, such a use appears to be rare. In the
following sections, decision making will therefore assume that the tasks include
adversaries. Note that TN games do not need to be zero-sum; TN games include
examples in which actions are mutually beneficial between players.
A few TN games fit a certain area of AI technology related to adversarial search
(AS), and it is mentioned here since AS is specific to turn-based games4 . In
the note, several games are mentioned use adversarial search. These include
Checkers, Othello, Chess, backgammon, Go, etc., all of which are not just turnbased, but also node-based. For these games, the technology used is generally
very successful, and could be seen as a candidate library for a TN game engine.
However, the technology generally assumes that information is never hidden,
and this is a major constraint. Many TN games deal with incomplete information, and this means that the AI library would only be useful for a small amount
of TN games.
4 Page 161 and 162 Artificial Intelligence: A Modern Approach, Second edition, Stuart
Russell, Peter Norvig.
28
The typical playthrough of a TN game rarely consists of one or two actions.
Instead, several actions are taken over the course of a game. Usually, several
actions are taken in sequence to fulfill one or more goals. For instance, in Chess,
this could simply be to win, or in a more advanced game like Civilization, it
could be a sequence of actions trying to establish a new base. Actions would
include preparing units fit for the task, finding a suitable site for the base,
preparing roads and “terraforming” the landscape, preparing transportation
and defence for the units, etc.
However, the basic assumption in regards to the games of the TN engine
is that games have opponents. This means that any planning must be able to
handle opponents trying to prevent the plans. While this situation is common to
many types of games, not just TN games, investigating AI technologies handling
this could still yield good results, since planning is a difficult and complex
problem, especially when combined with adversaries.
Machine learning Another aspect of TN AI is utilising domain-specific knowledge (DSK). In typical TN games, DSK is programmed directly into the AI
using the AI programmers knowledge of how to play the game. While this can
help making the AI competitive, and avoid intractability issues through use of
heuristics, it means that changing the game rules causes a need for rewriting
the AI to some degree.
An AI discipline that may help solve this issue is machine learning. Machine
learning depends on the AI repeatedly playing games, learning as it plays, and
becoming better over time. Many algorithms and methods relate to this area,
such as the genetic algorithm, which can take a set of heuristic values and
seek a global optimal solution. While the use of heuristics may not seem clear
immediately, it is used in many games to help solve intractability. For instance,
in Chess, it is popular to use heuristics when evaluating a game board, instead
of having to calculate all possible results that can come from that board and
use that as its score, which is intractable for most Chess board situations. One
way to use heuristics would be to let the AI find the values for the heuristic
itself.
If the game rules changes, the AI may be able to adapt to this by learning
that the previous used tactics are no longer effective, and finding other tactics
that are effective now.
Another possible advantage of machine learning is that the AI may exhibit
surprising (and competitive) behaviour. This may help the re-playability, since
a changing AI gives different challenges over time. It may also help competitiveness, because an AI that can learn might become less predictable and more
competitive. It may also let the AI react properly towards traps laid by the
player, based on the players assumption that the AI always reacts the same.
29
A disadvantage of the use of machine learning is that it needs players to test
against, and using human players is frequently not practical. Furthermore, it
takes time to train the AI, and typical machine learning problems can occur.
In the example with the genetic algorithm, it may end up finding a “locally
optimal” solution, and end up never finding the globally optimal solution.
5.3
AI tools
A main purpose of game engines is to decrease the complexity and requirements
of development. While the use of libraries and existing components can save
time, the libraries frequently have to be used in a way specific to a particular
AI. This is especially true in regards to AI, since game rules generally affect the
development of AI in many ways.
In many game engines, a frequently developed game engine tool is the domainspecific language (DSL), which is often known as “game scripting language” in
the game development field. These languages provide several features and language constructs that make game development easier. Frequently, the purpose
of the language is to program game logic, which is frequently subject to change
both during and after development.
However, some script languages provides support for AI development, which
can be seen in the UnrealScript language. This language is developed for an
RTP engine focusing on first-person shooters, and the well-known use of state
machines for programming AI agent behaviour is supported directly in the language via special constructs. Part of the reasoning behind this was that state
machine programming in the language of the engine, C++, was unmaintainable,
difficult and error-prone. By providing a language with direct support, these
problems are handled much better5 .
Other AI tools include describing the game rules in a DSL. This effort is often
part of the project for general game playing, the goal for creating a single AI
which is capable of playing multiple games. The advantages in regards to game
rules changing are obvious; the AI does not need to be rewritten. Not only that,
once an AI is updated for one game, it may perform better in several games.
Effectively, it might cut away the whole cost and time of developing AI for any
single game.
Several projects have worked with this approach. One project is the Zillions
5 UnrealScript, a game engine used for several projects:
http://www.unrealtechnology.com/
Manual describing the UnrealScript, including their support for state machine programming
directly in the language:
http://unreal.epicgames.com/UnrealScript.htm
30
of Games project, which thousands of games have been written for using their
DSL6 . Another project is the General Game Playing project of the Stanford
Logic Group, which has featured competitions in which multiple different AIs
have to compete for which is the better in many different games (40+ games in
the 2007 competition)7 .
While such projects are very interesting and incredibly useful for the games
they support, the types of games they support is currently very limited. According to the authors of Zillions of Games, the project badly supports hidden
information, and simple arithmetic can not be used in the game rules. We
believe that at the current AI technology level, these kinds of constraints are
necessary in order to have general AIs that perform well. Since the large majority of TN games may be excluded by these constraints, the approach is not
practical here, even though the future developments in AI may make it practical.
Of the 2 approaches presented above, the first is the practically useful in this
project. However, finding common elements of game AIs is not always obvious.
A possible way to find these is to look for commonly performed tasks, that
may be error-prone, ineffective, etc., and try to abstract them into language
constructs.
Even without language constructs, an AI DSL would still be of some use.
Providing a sandbox, in which AI scripts can not crash the game engine, and
provide the ability to program the AI without having to recompile the engine,
maybe even programming the AI on the fly, are possible benefits. However,
DSLs are not without their disadvantages. Besides taking considerable time to
design and implement, they typically require more memory and/or processing
power for the same tasks, and they take extra time to learn for game developers.
5.4
AI feature requirements
Given the above discussion, 3 parts has been focused on:
• TN-focused libraries.
• Machine learning libraries.
• DSL for AI.
6 http://www.zillions-of-games.com/
7 http://en.wikipedia.org/wiki/General˙Game˙Playing
http://en.wikipedia.org/wiki/Zillions˙of˙Games
http://games.stanford.edu/
http://cadia.ru.is/wiki/public:cadiaplayer:main
31
We have chosen to work with TN focused libraries because we believe that it
is possible to implement TN-libraries that will be beneficial to the majority of
TN games. Furthermore, some libraries are obvious; such as path finding, with
a solid basis for general path finding problems.
The machine learning libraries were chosen for several reasons. The primary
reasons are their advantage in regards to game rules, DSK and replay value.
Furthermore, machine learning is useful to TN games in general. While not
every game requires them, even abstract TN games such as Chess can take
advantage of them. Finally, machine learning is a research field like most AI,
and machine learning is still a new theme in game AI. Applying it to TN games
may help uncover which techniques are useful to TN games, and which are not.
The DSL for AI have been chosen partly because a DSL supporting TN AI
is very useful to developers, and partly because designing and implementing a
DSL is a challenging and exciting goal.
Based on the discussion of the previous sections and the above, 2 lists of requirements have been made. The primary requirements that must be met. The
secondary requirements includes items which would be quite good to have, but
may not be possible, for instance due to the time constraints on the project.
Primary:
1. Graph-based search library. The game engine must provide and test at
least one method related to searching in graphs. The method should be
somewhat general, being usable in multiple TN games.
2. Machine learning library. The game engine must provide and test at least
one method related to machine learning.
3. Graph-based information extraction and analysis. It would be good to
utilise the graph-based structure of TN games in regards to information
extraction and analysis. At least one method of information extraction
and/or analysis should be provided and tested.
Secondary:
1. A TN AI domain-specific language should be supported. The language
should have at least one TN AI specific language construct. The language
should provide better support for AI than the engine language. While
the utility of such a language would be great, it is included as a secondary
32
feature due to several factors: implementing a language is a big task which
may not be possible given the time constraints; and extracting the specific
language constructs may not be possible in the given time.
We believe that the primary requirements are attainable within the given time
constraints, and that they are interesting and useful. All the libraries should be
able to support the majority of TN games, and implementation should not take
too long. The secondary requirements are not many, but a scripting language
is a big task. Furthermore, the primary requirements can always be extended
with more methods.
33
Chapter 6
Technology
The programming language chosen for the project is Java. Traditionally, most
engines use C or C++ in their engines 1 . Besides being widespread, C++ is a
popular choice for making games and game engines due to its speed, its support
of abstraction facilities, including object-oriented and generic programming, and
its ability to interface and take advantage of hardware directly. In this light,
Java may seem as being the suboptimal choice.
There are two main reasons for choosing Java: the first being that we are much
more familiar with Java than C++, and that using an unfamiliar technology
(which is not just the programming language itself, but libraries, coding solutions, development tools, building process, etc.) could have considerable negative impact on the project by having to invest a lot of time in the technology.
The second reason is that we believe that Java is well suited for making games.
Speed matters a lot in game development, and Java was considerably slower
than C++ in its early days.
However, the present speed of Java is today on par with C++, and Java
is widely supported. Furthermore, the speed at which graphics handling is
executed, a major competition point for many games, matters today less and
less on the speed of languages, and more on the graphics card, where the major
bulk of computation happens. While C++ can interface faster with the graphics
card, and interfacing to be done through native calls (to, for instance, C++),
1 http://en.wikipedia.org/wiki/List˙of˙game˙engines
the majority of listed open source engines uses C or C++; the well known commercial game
engines, Source (http://source.valvesoftware.com/programming.php) and Unreal Engine
(http://www.unrealtechnology.com/technology.php), are written in C++
34
the speed penalty has turned out not to be an obstacle for fast 3D graphics 2 .
Another indication of Java being suitable is the emergence of Java based
game engines 3 and several successful Java games for the PC, of which some
both run directly within the browser and include 3D graphics, only assuming
that a JVM is installed4 . Many C++ engines do not support the running of
games within a browser.
Regarding platform support, Java supports all of the major operating systems, and is widespread on many different phones. However, Java is not supported directly on any of the main gaming consoles (Playstation 3, Xbox 360,
and Nintendo Wii), and for game developers wishing to run their games on these
consoles, Java is not an option.
Finally, Java has little game technology compared to C++. While this may
change, given that Java has become a valid alternative for games, the current
situation is that C++ provides better support for game development.
Overall, Java is a decent technology choice for PC and phone game development.
The main issue with Java is the lack of support on the 3 major gaming platforms
(Xbox 360, Playstation 3, Wii), and the greater support that games based on
C++ enjoy.
If game developers wish to develop games for the console, they must avoid
Java, and game engines based on Java. We believe that the main market for TN
games is the PC, but if it becomes a priority to support the consoles, a possibly
expensive and time-consuming technology shift is needed.
In the case that another technology than Java is needed, we would recommend C++, considering its great support in regards to game technology, its
decent features considering game development, and the support for consoles.
However, the relative difficulty of learning and using C++ should be considered.
2 http://jogamp.org/jogl/www/
3 http://www.jmonkeyengine.com/home/
a popular java game engine
4 Three examples of Java games, featuring 3D graphics, running in the browser:
http://www.banghowdy.com/
http://www.tribaltrouble2.com/
http://www.runescape.com/ - (client side in Java)
35
Part II
Engine
36
Chapter 7
Design
In this chapter we describe some of the important choices in the engine design.
7.1
The original implementation
The project started using an existing code base, which had been developed
without a lot of consideration to design. This section will briefly describe some
of the issues with the original implementation.
Separation of concerns In the original implementation, there was little separation of concerns. For instance, the game logic code was intermixed with
the graphical code. This design was inspired by RTP engines, where the close
integration provides fine-grained control over several subsystems, such as graphics, sound and AI. As described in the analysis, such a close integration is not
needed in a TN game engine due to the lack of constant state changing.
Several difficulties were experienced with this system. For instance, game
logic developers had to understand the graphics system in order to code game
logic. Another issue was that replacing the graphics system would be very
difficult. This became necessary later in the lifetime of the game engine, and
while it might have been possible to write a wrapper around the new graphics
system, it would have been a suboptimal solution.
Another difficulty was that mixed game logic and graphics code meant that
it would be non-trivial to run the game logic code without any graphics being
executed. For instance, if a machine learning method was to be used, running
any graphics at the same time would be a waste of resources.
37
Internal and external communication All clients shared the same immutable game state. For each move a new game state would be generated,
which would be passed to the clients.
A design like this greatly complicated things like adding network multiplayer
support. The network module would have to either transmit the new state
entirely or generate a “state-difference” and transmit that to the client. Neither
of these two options are very optimal, specially when they had to be done once
per move and not per turn.
Another issue was the constant regeneration of the game state. While TN
games require less resources than RTP games, their game state can still become
large. Constant regeneration of the game state does not scale well in the size
of the game state, and this would hurt performance. Using another design for
internal communication which scales better is desirable.
7.2
Event-based system
In order to improve the engine, we replaced parts of the previous implementation
with an event-based system. With this design, the goal is to obtain several
aspects that were missing in the previous implementation:
• Subsystems that support TN games especially well.
• Better separation of concerns.
• Better communication.
Introduction The event-based system is based around looking at subsystems
as clients and the control subsystem as the server. When there is a change in
the game state, the engine will emit an event to all clients.
Events are first and foremost game logic events, which means that they intend
to change the game state. However, other subsystems such as the graphics
system can still take advantage of events. For instance, if it receives an event
about a unit moving to a new position, it can use this event to animate the
move in the GUI.
For clients, this changes a number of aspects compared to the old design.
First off, the clients now receives a small notification of what changed instead
of the entire state. Clients are no longer required to use the same representation
of the game state; in fact they now have to maintain their own private copy of
the game state if they need such a thing.
38
While this costs extra memory since some clients now have to maintain their
own game state, it added the flexibility for clients to examine the state both
before and after processing the given event.
The other major change is that all clients now receive an event when something changes. Previously they were only informed of the current state when
it was “their turn”. This infrastructure is also much better at handling clients
that only observe the game.
Changes to the subsystems In the following sections, the subsystems that
were affected a lot by the change to the event-based system is described below.
7.3
Graphics system
The graphics system is mainly used as an interface between the game and the
player. In this discussion, the graphics system will not just encompass the
drawing part of the system, but also related issues like a graphical user interface,
and the IO with the player. This makes the graphics system a considerably more
complicated system. A part which is separate from the graphics system, but is
commonly included in the graphics system, is the game logic system. The game
logic system is needed because the graphics system needs to know the state of
the game for several reasons, and using an existing system saves the work of
reimplementing a whole system to handle logic just for graphics.
An example can help illustrate the practical value of integrating the graphics
system with GUI, IO and game logic. Imagine the simple clicking of a unit
on the screen. The IO system communicates a click, and this is registered in
the graphics control. It is then computed what graphics were clicked, and the
corresponding unit in the game logic is retrieved. Now, the GUI is updated with
the game state of the unit, such as health left, and possible actions based on the
unit is shown. This simple example illustrates the need for a close integration
of the aforementioned systems. Of course, it should also be noted that the
game logic system does not need to know anything about the graphics, and
this important detail means that the same game logic system can serve a game
recorder, a graphics system, or an AI player equally well without problems.
7.4
Game logic system
The changes to the game logic system is profound. Instead of communicating
and sharing game state with all components, it is possible for each component
to have their own game state. While this separate concerns nicely, it can also
39
create a scalability problem. If the game state is large, and each (AI) player
have their own copy of the game state, the number of game states would be
linear in the number of players.
However, the game state has to become very large before this can become
a problem, and in the worst case, it should be possible to set up AI players
to use the same game state. In the case that several human players play a
game, the likely that they don’t use the same computer rises with the number
of players (hotseat playing is rarely used with more than 2 players), and thus
more memory will also be available.
Another change that the system can lead to is different game states for different
players, assuming that different events can be sent to different players. This is
especially relevant for hidden information games. In these games, the players
frequently have different information about the world, and therefore also different game states. In games such as Age of Empires 2, each game instance have
the complete game state, even though games in general are based on hidden information. The cause of this is due to the need for high-performance networking
- by making it possible to run the same simulation on all machines, much less
information needs to be transferred.
However, this means that a particularly clever player could hack the game
and extract this information, allowing the player to see what other players see,
counter their strategies immediately, never going into traps, etc., basically cheating. While this may be somewhat difficult, it is still possible, because each game
instance have the complete game state.
Such cheating is impossible if the game states on each computer really is
different. The only way to cheat would then be to watch the network traffic
from the server, or compromise the server which would know the whole game
state, but this can be seen as considerably more difficult.
40
Chapter 8
Implementation
8.1
Graphics system
Drawing systems The graphics system is based on 2D-drawing. In regards
to 2D-drawing, there are 2 general methods of drawing. The first is the spritebased drawing, in which images stored as pixels are drawn to the screen. Sprites
are a series of images which together forms an animation. The second method is
the vector-based drawing, where images stored as vector descriptions are drawn
to the screen using vector-drawing libraries.
The main differences between the two are that vector-based drawing generally takes up very little space and can be scaled without loss of precision,
while sprite-based drawing can contain much greater detail and draws faster,
especially with hardware acceleration. Most 2D-games use sprite-based drawing. This is a common choice, since speed is one of the foremost needs when
drawing. It is also due to the speed that sprite-based drawing will be used in
this project.
Time-fraction drawing When drawing in RTP games, the drawing is frequently handled in the main game loop together with the game logic. As discussed before, this mixing of different concerns is accepted in RTP games, since
the game state may change constantly, and precise timing is important.
This is not the case with a TN game engine. In fact, based on the events
that the graphics system receive, the drawing can be completely predicted until
the next event is consumed, since the game state only changes when events are
received.
Based on this, it was decided to try and implement a drawing system based
41
on time fractions. This can be used to implement a system in which the animation speed can change with animations playing smoothly. This is typically not
possible in RTP games. For instance, if the animation speed is set to 10 percent,
the game loop of the RTP game has to go ten times slower than normal. This
means that the frame-rate might drop from 60 frames-per-second to 6 fps. Of
course, animation speeds are frequently of little use in RTS games since RTP
games generally are designed to go at a very specific pace.
TN games do not have this expectation, since the player is really just observing the animation to see what have transpired, not because the player needs to
react to what happens. And in this case, if the player is not all that interested in
the events, the animation speed can just be turned up until something exciting
happens.
When tested in the game, the animation speed turned out to be quite useful for the game, and being able to quickly speed through battles until some
interesting situation happened helped the debugging of the game.
Hardware acceleration During the development of the game, we encountered issues with frame rate obtaining a decent frame rate. This was curious,
because only a few hundred images were drawn to the screen every 10 milliseconds. Investigating this, we found that the main culprit was the non-accelerated
nature of Java2D. This meant for instance that images containing transparency
were drawn much more slowly than images without transparency. Basically,
anyone using the engine would be forced to consider such an issue when implementing the drawing of their game.
With today’s graphic cards in PCs such issues should not occur. To solve
this, we decided to try and add support for hardware acceleration. This was
achieved by using JOGL, a binding to OpenGL for Java. With JOGL it was
possible to maintain a much more acceptable frame rate.
8.2
Game logic system and AI
The main data structure of the game logic system, the node, is designed to be
as flexible as possible. While many TN games have a “board” that is generated
from a lot of small equal-sized areas, such as a grid made out of squares, a couple
of games cannot easily be modeled in that manner. Examples include Risk and
Axis & Allies, which consist of a board with different sized subareas. Due
to the intend to support as many TN games as possible, it was decided not to
assume that the game fields would always be based equal-sized areas.
This also affected the kind of node structuring that could be used. Since
grids were not an option, the main data structures for consideration was the
42
adjacency list and adjacency matrix representations. However, this choice was
trivial. Game graphs in TN games are generally very sparse, and there are very
few, if any, TN games that have dense game graphs. Furthermore, adjacency
matrices have a high memory cost especially for spare graphs. Based on this,
the adjacency list was chosen as the method for storing neighbours.
Adding and removing nodes Early in the process it was decided to make
the node data structure more efficient through the assumption that no nodes
were ever removed or added. This allowed a couple of optimisations in the
implementation of the node structures. The reasoning behind the assumption
was that there are very few TN games that handles the addition or removal of
nodes. Also implementing support for modifying a live graph could have added
a considerable amount of needless complexity.
However, it turned out that adding and removing nodes were quite useful
in regards to game engine tools. Particularly, the map editor is built on top of
the game engine using an ad-hoc extension to inform the engine that the graph
should be made mutable. This saved a lot of time developing and keeping the
editor up to date, since it was a lot easier to reuse parts of the game specific
code, when the editor was built on top of the same infrastructure.
8.3
Multiplayer system
The event driven design had the positive side effect of greatly simplifying the
network implementation. It turned out to be enough just to create a serialization and a deserialization algorithm for the events that allowed them to be
transmitted via the network.
The event-based approach complicated an unforeseen area, namely waiting
until given request to be processed by the game engine; which was first discovered when working on the AI leading to an fragile ad-hoc solution for the
issue.
One major issue is that a given request can result in multiple events; a
possible solution to this could be to add an identifier to all requests and have
the engine emit an event using that identifier to signal that the request has been
processed or rejected.
Alternatively events could be bundled into a set of events related to the
request that triggered them. It would still require some identifier linking the
request and the events, but it would allow clients to trivially relate all events to
the request.
Neither of these solutions have been implemented; while either would be a
43
vastly more robust solution than the ad-hoc solution currently implemented,
time was invested improving other parts of the engine.
8.4
Resource IO and management
The resource management (RM) system was not planned to be a part of the
game engine. It was originally written as a part of the script engine to help it
locate binary and source files, which were stored in different locations.
While preparing the initial demonstration of the game engine, it was discovered that the game and parts of the engine had a lot of assumptions about
where to find the resources, such as sprites. To solve this the resource handling
system of the script engine was extended and slowly integrated into the rest of
the code.
The RM system handles the part of finding and reading resources. The method
is based on the one used in Neverwinter Nights (NWN), where the game has
a predefined list of locations to look for resources.
In both this game engine and in NWN if multiple locations have a resource
with the same name, the game will use the resource from the location with
highest priority.
Beyond file system locations, the RM system was extended to support reading resources from zip/jar files, since this was the format used in the demonstrations of the game during the development. Due to its extendible nature, it
is possible to add handlers for other containers, such as tar files.
There are at least two very likely candidates for improvements to the RM. One
would be caching; either of the resource location or the data itself. Currently
only the latter is possible and must be done by the resource using code itself.
The other improvement would be to add support for writing resources. Currently only the game needs to write resources when creating a new map in the
game editor. Nevertheless, if the scripting engine was completed, it might need
to compile source scripts into binaries.
44
Chapter 9
Discussion
Overall, the event-based system proved very successful for the TN game engine. Utilising this system is recommended to anyone implementing a TN game
engine.
The separation of concerns was achieved to a high degree. Though the
graphics system still uses parts of the game logic system internally. The high
degree of separation still gave several advantages. In regards to testing, the AI
machine learning methods could be run without any graphics at work due to
this separation.
The communication was improved a lot by the event-based system, and it
created a good alternative to the state-changing game loop employed in RTP
games. Since the events can be sent over the internet as well as internally,
using the event-based system to implement automatic multiplayer handling is a
prospect worth investigating.
Development of a flexible TN game engine The primary requirements
of designing and implementing game engine systems focused on TN games have
all been attained.
1. Basic 2D graphics system. The game engine supports 2D graphics, including animation, as well as a time fraction system and hardware acceleration.
2. Basic TN game logic system. The event-based system helped the game
logic system avoid the game loop of RTP games.
3. Basic multiplayer. The sample game supports both hotseat and network
multiplayer.
4. Basic TN AI system. The implementation of the node data structure is
45
flexible and can support the vast majority of TN games.
The secondary requirements of designing and implementing game engine
systems focused on TN games have partly been attained. Requirement number
1, 2 and 3 was only partly implemented.
1. Implement a basic TN multiplayer system. While the event-based system
may be utilised to provide such a system, it was not reached within the
time constraints in the project.
2. Support playing over the internet. Playing over the internet was only
supported partially due to time constraints. In order to play over the
internet, ip-address and port number of the host is needed, and no game
lobbys exist for the game/engine.
3. Completely separate game logic, graphics and AI code. A high degree
of separation was achieved, but it turned out to be impractical to make
graphics completely independent of the game logic in practice. However,
the game logic system was independent of any other system. Advantages
in regards to testing the AI was also achieved, due to the separation.
4. Basic resource IO system. A basic system to handle resources was implemented and used successfully.
46
Part III
AI
47
The following chapters will present the design and implementation of the AI
libraries and tools, as well as test and evaluate the methods used, and finally
discuss the methods.
The first chapter, design, will describe and analyse the different requirements
given to the different AI techniques. The second chapter, implementation, describes the chosen methods, the reasoning behind choosing them, and their
implementation and issues found during the implementation. The third chapter
will describe the scripting engine. The final chapter, discussion, will discuss the
techniques and their use, reflect on whether further investigation is needed or
recommended, and describe other potential techniques that the findings indicate
might be useful. It will also discuss the value of adding AI libraries and tools
to a TN game engine.
48
Chapter 10
AI Design
The purpose of the following sections is to describe and analyse the requirements
given to the different AI techniques. A major reason for including the AI libraries
and tools is to estimate the usefulness of adding such tools and libraries to a
TN game engine. If methods which are suboptimal at providing support to TN
games are chosen, it may negatively affect the estimation of the value of such
methods in a game engine. To avoid choosing such methods, several aspects and
requirements are discussed that must be considered when choosing methods.
This section will present this description and analysis.
The description and analysis is organised according to their general category:
• TN-focused libraries.
• Machine learning libraries.
• DSL for AI.
10.1
TN-focused libraries
10.1.1
Path finding
According to the primary requirements for the AI, at least one method for path
finding in graphs must be provided. Furthermore, the method should be as
general as possible, to ensure that it is useful for many TN games, and not just
a few.
Several path finding methods are available for graphs. While this gives a
great deal of options, care must be taken, since some of these have special
49
constraints. Special constraints might well mean that the path finding method
would only be usable in a few games. To ensure that the chosen path finding
method(s) will be widely useful for TN games, the following requirements and
aspects were considered when choosing path finding methods.
Graphs and edge weights In regards to TN games, the graph model varies
a bit. In a lot of TN games, such as Nine Men’s Morris and Axis & Allies,
the edge weight is always 1. Sometimes, the edge weight might differ a bit, for
instance in the game of Axis & Allies where “land” units cannot go into “sea”
nodes, and the reverse with sea units, but this can be handled without requiring
weights on the edges.
However, many other TN games gives weight to edges between nodes. These
include Battle for Wesnoth, Civilisation, Advance Wars, etc., in which not
only the path weights are different from 1, but they differ from unit to unit.
Supporting different path weights from unit to unit should be supported by any
path finding method chosen.
Changing edge weights An aspect to consider is that the path weights in a
TN game not only depends on the different game pieces, but frequently changes
over time. A common example is that game pieces which can move can block
for each other. This means that any path finding method should be able to
handle changing weights properly.
An example of a path find method that might not handle changing weights
properly is a method that requires that the shortest paths can be computed
once in order for it to be effective computing-wise. Since path weights change,
the shortest paths may change as well, and this invalidates the computation of
the method.
Hidden information Another important aspect is the issue of hidden information. Many TN games contain hidden information, and this is frequently
seen as covering parts of the graph. Furthermore, as game pieces are moved on
the graph, which part of the graph that is hidden may change.
This becomes an issue in several ways. First, the path finding should handle
the hidden parts of the graph when calculating a route. Second, when the units
move, more of the graph might be revealed, and the path finding method may
become ineffective if it has to start over its path finding each time a small part
of the graph is revealed.
When choosing path finding methods, the aspect of how a method handle
hidden and changing information should therefore be considered.
50
Summary In all, it was decided that the following aspects were the guiding
ones in regards to choosing which path finding method to implement.
1. Supporting edge weights in the graph.
2. Changing edge weights during the game.
3. Hidden and changing information.
10.1.2
Information extraction and analysis
Information extraction and analysis is the field of transforming knowledge about
the game state into a form which is easier for the AI to reason about and to
decide on. In order to determine which requirements should be given for the
methods, the common extraction and analysis tasks for TN games is investigated.
Graph-based methods For TN games, the position of the game pieces on
the nodes generally plays a big role.
A major representation and organisation of knowledge in TN games is the
graph. While it is possible that there is methods that does not directly work
with nodes but which are still very useful to TN games in general, it seems more
likely that methods directly working with graphs will be better suited to TN
games.
An observation supporting the previous assumption regarding graphs-related
methods are more promising is that positioning frequently plays a major role in
TN games. Positioning relates to the place at which a game piece is placed in
the graph. The importance is seen in the vast majority of TN games.
An obvious example is Chess, where the position of the game pieces plays
a major role, both at a strategic and tactical level. At the strategic level, protecting the king by position pieces is important, and getting the pawns forward
is important, since the late-game may give the victory to the side with pawns
closest to the opposing line (since a pawn reaching the final line can become a
queen). At the tactical level, the threat which pieces pose to each other, and
the king especially, can determine the game. In fact, the game is won when the
game pieces are positioned such that the enemy king is checkmate.
Given these considerations, only extraction and analysis methods that work
with graphs will be chosen. While this might risk preventing the discovery of a
non-graph method that somehow supports the vast majority of TN games very
well, choosing graph-based methods will give some level of guarantee that the
method is in fact useful to the majority of TN games.
51
General application Another consideration is the generality of the method.
Some methods may be very useful to a small subset of TN games, but if these
methods are not useful at all to the rest of the TN games, the methods will
not be worth anything to the majority of the game developers using the engine.
For instance, some methods assume both perfect information, full determinism,
two-player, etc., and these constraints limit the use of the methods.
While it can be difficult to determine whether a method is useful to many
or to few TN games, the assumptions the method make can help to easily
decide whether it supports many or few. For instance,s if a method assumes
that perfect information is available, a lot of complex and interesting TN games
using hidden information are excluded.
Based on the need for supporting many and not few, only analysis and
extraction methods that does not place severe constraints on the types of TN
games possible will be considered.
Summary In all, it was decided that the following aspects were the guiding
ones in regards to choosing which information extraction and analysis methods
to implement.
1. Methods that works directly with graphs.
2. No methods with constraints that severely limits the supportable TN
games.
10.2
Machine learning
The field of machine learning concerns the gathering of information knowledge
and using it to improve performance. Related to its use in games, machine
learning is often used to let agents learn how to compete better. Machine
learning features a vast collection of methods and technologies, and this makes
it difficult to choose. This makes it desirable to find requirements and aspects
to choose machine learning methods from.
The chosen methods of machine learning should be general. This requirement
is not trivial to satisfy. Different machine learning methods can be applied to
different parts of a game independent of the type of the game. This means that
it makes little sense to try to analyse which machine learning methods that are
especially useful for TN games.
We have looked at several papers that describe experiments regarding the use
of machine learning in games [3, 4, 5] as basis for our choice and own experiment.
52
Learning and feedback systems In regards to generality, some might claim
that the categorisation of type of feedback for machine learning can help determine which methods are useful for games. According to Russel and Norvig [1,
p 650] the 3 types of feedback systems are:
• Supervised learning
• Reinforcement learning
• Unsupervised learning
In a supervised learning task a number of problems along with their solution is
given. The task is to create a function that maps each problem to the correct
solution. An example is the task of determining which images in a set contain
buses. A human superviser has marked the bus images. Given this training
set, the learning task is to create a function that correctly finds the bus images.
The function should be able to correctly determine whether new images contain
buses.
Reinforcement learning means that the solution to the problem is not known,
but that the environment reacts favorably or unfavorably to actions. This may
be seen as common to games, in which doing a wrong move can have immediate
and strong penalties. For instance, trading a queen for a pawn in Chess usually
leads to a much worse position.
The final feedback type, unsupervised learning, is a special kind of learning
since it never knows whether what it did was correct or wrong. It constitute a
class of problems in which the goal is to find patterns in the given data.
Based on this, reinforcement learning may be seen as the general type of
learning problems for games, including TN games. The issue with this generalisation is that the other types of learning methods still occur in games, and may
still be useful. For instance, the supervised learning method is used in a white
paper on game AI 1 to try and improve an AI developed for the TN game Go.
Other categorisations may be used, but this categorisation is the most useful
when determining the nature of the learning problem that the developer faces[1].
Based on these considerations, the learning method will not be chosen based on
its type. Instead, it will be considered in regards to implementation and use.
Ease of use Another requirement is a less functional one, but still very practical. It is the issue of ease of use. Machine learning is an interesting and
advanced field, and it touches on several other fields, such as statistics. This
also means that it has its own issues and problems. Part of the idea of provid1 http://doc.utwente.nl/66613/
53
ing AI libraries is to lessen the implementation time and learning complexity,
but machine learning methods that pose problems that is exotic to the regular
game developer may cause the method not to be used, except by a minority
of game developers. Of course, by implementing such libraries in an engine,
and providing plenty of documentation resources, may help against such situations. But simpler machine learning methods are still much more likely to be
successful than complicated ones. If a complicated machine learning method is
implemented, the implementation should consider ease of use.
Summary The following list of aspects have been chosen to guide which machine learning methods to implement.
1. Machine learning methods which have been successfully used in games.
2. Ease of use.
54
Chapter 11
AI Implementation
The purpose of this chapter is to describe the chosen methods and the reasoning
behind choosing them, as well as describing and justifying their implementation. Furthermore, issues found during the implementation are described and
discussed.
11.1
Selection of AI methods
This section describes the selection of AI methods and reasons for their selection.
Apart from the aspects discussed in AI Design, another important consideration
was made regarding the selection of AI methods.
While the focus for the selection of methods was based on the design derived from the assignment specification analysis, the development method also
affected the design. Basically, the game and its AI players was developed alongside the engine and the AI libraries, and this meant that the AI methods was
partly chosen based on what the AI players needed in the game.
This have obvious disadvantages, such as the risk for choosing algorithms
that are especially suited to that game instead of a more general one, it still
carries the advantage of testing the algorithms in a game. AI libraries that
are tested using simple simulation tests may not show hidden problems and
constraints that occur once they are used in a game.
55
11.1.1
Path finding
In regards to the requirements of path finding found in design, the following 3
aspects were determined as being important:
1. Supporting edge weights in the graph.
2. Changing edge weights during the game.
3. Hidden and changing information.
A* is a simple, frequently used and well-known path finding algorithm in the
game development field, based on Dijkstra’s algorithm. It has a decent running
time, and its memory requirements are proportionally to the size of the graph
structure in the game, meaning that memory is not likely to be an issue, at least
when used for path finding in the game graph. It supports path weights, and
it does not support or require precomputation, meaning that it matters little
whether or not the edge weights of the graph are changed during the game.
While it does not support negative weights, encountering negative weights were
deemed to be a very rare case in TN games.
However, it does not handle hidden and changing information well. If new
information becomes available after it has computed the shortest path, A* must
be recompute the path from scratch.
The second algorithm considered, D*, is an algorithm used in robotics. The
special thing about D* is that it does not assume that everything is known;
instead, it computes the shortest paths assuming that unknown territory has
no obstacles. When an obstacle in previously unknown territory is revealed, it
recomputes the parts of the computation that was effected, and continues. This
algorithm is especially suited to robots exploring and the situation is similar to
scouting units in TN games with hidden information.
Due to time constraints and the simplicity of implementation, A* was chosen.
While it does not handle hidden or changing information well, it can still handle
it. D* is still an interesting candidate, and investigating its usefulness for TN
games is recommended, especially for games with hidden information.
11.1.2
Information extraction and analysis
In regards to the requirements of information extraction and analysis found in
design, the following 2 aspects were determined as being important:
1. Methods that works directly with graphs.
56
2. No methods with constraints that severely limits the supportable TN
games.
Blocking
Methods that work directly with graphs include the methods related to areas
such as labeling, routing (path finding), network flow, sub-graph finding, etc.
While it may be difficult to model common tasks in TN games to problems
which is handled in one of these areas (apart from routing), we managed to find
one task in which one of the areas above gave the solution.
During the development of the game, a need for blocking enemy units appeared. When blocking, a primary goal is to use as few units as possible to
prevent enemy units from entering some area. This usually means finding the
smallest gorges, or making a ring around the target. As an example, this tactic can be used in Battle for Wesnoth to protect a wounded unit or in older
Civilization games to keep others out of “your” territory.
Blocking as a max-flow min-cut problem This task can be modeled as
a problem in network flow, more specifically as a max-flow min-cut problem in
which all nodes have a flow capacity of 1. Nodes having flow instead of edges having flow can easily be modeled as edges having flow. In the problem, a source and
a sink have to be indicated. A typical modeling in TN games would compare to
having the enemy units as the source, and then the area(s)/building(s)/unit(s)
to be protected as the sink. Now, a min-cut algorithm will then find the nodes
which are the least required to block the enemy completely, since a flow in a
node will be equal to having one unit/building occupying that node, and the
smallest blocking is equal to a maximum flow in the flow capacity 1 graph.
Several algorithms exists for the solution of this graph problem. However,
since every single node have a flow capacity of 1, the Ford-Fulkerson (FF) algorithm was a good candidate.
The running time of FF is described as: O(E · max(f )), in which E is the
number of edges in the graph, and f is the maximum flow in the graph. Two
important observations are made. The graphs in TN games are generally sparse,
with the number of edges being proportional to the number of nodes. We have
no knowledge of a TN game in which the highest number of edges for any nodes
become larger than 10. This means that E can be replaced with V (the number
of vertices, or in this report, nodes).
Second, the maximum flow is, logically, maximum equal to the number of
nodes, since each node have flow capacity one. This means that the running
time of FF is given as O(V 2 ) in the worst case, which is a very good running
time for max-flow problems.
57
Summary Because of this, the FF was implemented and used as a method in
the AI library. While other max-flow algorithms may be more effective on graphs
with more flow, no tasks requiring more flow have been encountered, and the
FF was therefore both adequate and effective regarding the needs. If such tasks
is encountered, investigating the other max-flow algorithms is recommended.
Influence mapping
Another method, more commonly known in regards to games, is the method of
influence mapping. In this report, influence mapping is defined as a mapping
from each node to some value, typically a real value value. The mapping is
obtained by going through each relevant game piece, and propagating their
“influence” to their node or the area around them. Whenever influence from two
or more game pieces is collected in the same node, some operation is performed
on it (frequently it is summed up). Basically, influence mapping is a spatial
analysis that tries to show the influence of different aspects in the game.
Examples and possibilities There are many options for influence mapping.
A typical use in strategic and tactical games is to calculate the extent of one owns
units influence against the enemy’s influence. This is simply done by creating 2
influence maps: one for the allied units, and one for the enemy units. In order
to get a picture of who influences which area, the maps are simply subtracted,
and the difference map is then obtained. Other interesting influence maps can
be gained by performing various algebraic functions on influence maps. 1
Grids and graphs While some regard influence maps as being inherent to
grids 2 , this is not necessarily the case. In the game Risk 2, which is a TN
game with non-grid, irregular size nodes, influence maps are used as a help to
the player to analyse the game graph (whether the AI utilises influence mapping
is unknown).
If it is assumed that the graph is a grid, several methods can be combined
with influence mapping, such as cellular automata and several image analysis
methods. However, in order to be general and include non-grid TN games, these
are not considered. If extra support for grids is desired, these methods are worth
looking at, since they make the analysis of influence maps somewhat easier, for
instance being able to locate areas of influence.
1 http://gameschoolgems.blogspot.com/2009/12/influence-maps-i.html
http://gameschoolgems.blogspot.com/2010/03/influence-maps-ii-practical.html
http://www.slideshare.net/mobius.cn/influence-map
2 http://www.slideshare.net/mobius.cn/influence-map
58
Summary Given influence mappings flexibility, and its ability to map properties over to a spatial construct, it is very useful in tactical and strategic games
in which space is an important dimension, such as the majority of TN games.
However, it is not without its issues. Influence mapping basically generates
heuristics: the influence propagated by a game piece has no clear relationship
with it, and it can require testing and fine tuning to obtain a good propagation
method in different problems.
In all, influence mapping can be useful to the majority of TN games. It
is also easy to implement, and has been tested in the game development field.
Given that they work directly with graphs, it has been decided to include them.
11.1.3
Machine learning
In regards to the requirements of machine learning found in design, the following
2 aspects were determined as being important:
1. Machine learning methods which have been successfully used in games.
2. Ease of use.
Genetic algorithm
The general idea of a genetic algorithm (GA) is to evole a data set in a method
similar to the biological evolution process. Each data set is called a “chromosome”, and each data part is called a “gene”. In order to evolve, a population
of the chromosomes is needed. These chromosomes are frequently randomly
generated. Given the starting chromosomes, the following steps are performed:
1. Selection. Evaluate the fitness of each chromosome. Pick the fittest chromosomes and discard the rest.
2. Reproduction. The surviving chromosomes produce children by being
paired off, and new child chromosomes inherit data from each parent.
3. Mutation. For each of the new child chromosomes, mutate genes at random.
4. Kill off the parent chromosomes, and start over with selection.
GA, which is a reinforcement learning method, was considered due to its
tried and successful use in games by several others 3 [4].
3 http://www.springerlink.com/content/v9clljnrr4erd4qq/
59
In regards to games, a typical use of GAs is to optimise a set of heuristics. [4]
The evaluation function then consists of whether the generated heuristics in a
chromosome lead to winning or losing, or some degree in between. In a TN game
where the heuristics to optimise is for the AI player, the evaluation function
would consist of playing a game and see how well the AI player performed.
Issues regarding fitness evaluation A requirement of the GA is that the
evaluation function must return values that the GA can converge on. This can
be a problem in games, since games are usually either only a loss, a draw, or a
victory. For instance, if a population is evaluated, and the chromosomes all lost
their games, the evaluation function will make picking impossible if the only two
values returned are ”it lost” and ”it won”. Instead, if the fitness function returns
a broad spectrum of values, indicating that some chromosomes lost more than
others, the chromosomes that lost less can be picked. This broad spectrum of
values can be attained in two ways for games: The first is to implement an
estimate of how well the chromosome performed, such as a score, and modify
it heavily dependent on victory or loss. While this is effective, the estimate
may be wrongly chosen, and such may optimise tendencies that does not lead
to better performing chromosomes.
The second is to let the games be played repeatedly, and let the randomness
sort it out. For instance, if each chromosome is evaluted 50 times, the chromosomes that won the most victories can be regarded as the most fit. While
this approach helps ensure that the best performing heuristics are picked, it
also means that the evaluation time is much higher, since the games have to be
played many more times.
Another important issue regarding the fitness evaluation is the frequent requirement of an opponent. While this is not always an issue, it is generally an
issue if the heuristics is used to improve the heuristics of an AI player in a TN
game. While human opponents may be the most varied and the best players to
test against, the need for repeatedly and quickly test games makes it generally
impractical to use humans. This means that opponents are needed.
Ease of use An important advantage of the GA is that the algorithm itself
is completely separate from the evaluation function. This means that the algorithm can be used between several games and fine-tuned, and let the game developers develop the evaluation function. The evaluation function is frequently
simple, based on giving the game a score, or repeating the game and basing the
score off number of wins out of games played. This means that the algorithm
might be easy to implement in an AI library for a game. The only issue that
remains is the structure of the chromosomes, which may differ from game to
game, and which the algorithm should be able to handle.
60
Summary Based on the above considerations, it has been decided to use the
GA. The method is both easy to use and has been successfully used in games.
While it has some issues regarding its use in TN games that must be considered,
these issues can be dealt with.
Case-based reasoning
Case-based reasoning (CBR) is a learning method centered around solving current a problem, also called a case. The idea is that the machine will have or
through trial and error generate a database of problems, solutions attempts and
some measure of how successful the attempt was. When given a new problem, it should extract and adapt a solution based on the previous cases it has
encountered.
Humans frequently use learning style in their every day life. Consider driving
a car; if faced with a Mercedes, most licensed drivers are able to use it, even if
they never drove that model before.
CBR systems can be implemented in several ways, but the systems commonly
includes phases similar to the below:
• Match previous cases by finding similar cases to the current case.
• Adapt the solution of the most fitting case to the current task.
• Learn by recording a new case and store the success of the adapted solution.
CBR was considered due to its tried and successful use in research papers
concerning games, and its usability in complex environments [3, 5]. CBR is a
reinforcement learning method.
Decision making A main feature of CBR in games is that it can help make
choices. The “solution” of a case can be a move, a tactic or even a strategy.
Through its learning mechanisms, CBR learns which solution to choose in a
given case. An obvious use to this is the perceived advantage of machine learning, such as when rules are changed. In this situation, CBR can learn anew
which strategies, tactics and choices should be chosen in different cases.
This also makes it different from GA, since GA does not directly support
decision making, but first and foremost the optimisation of heuristics. Another
difference is that CBR can adapt to different situations much easier, which
improves its decision making. GA seeks a set of heuristics that is good in
general, and can not switch between sets of heuristic values depending on the
61
situation.
Issues with CBR Some of the main issues with CBR are that it can be
difficult to use. Many factors have to be considered: how to retrieve cases,
how to adapt solutions, how to learn from new cases, how to maintain the
case database, how to represent cases, how to structure solutions, etc. These
considerations make CBR a learning method which might be difficult to use
and implement properly. Furthermore, the structuring of cases and solutions
may be different from game to game, and that makes it harder to implement a
general CBR system.
Of course, these issues with CBR may make an argument for implementing
it in an engine. If it is possible to handle the majority of issues in the engine
code, the game programmers will have a considerably easier time utilising CBR.
Given that CBR is directly useful in decision making, this makes it an attractive
learning method.
Summary Given the above considerations, it has been decided to implement
CBR. While the method is non-trivial to implement well, implementing it in
an engine may save game developers a lot of the issues with it. Furthermore,
several sources have described successful use of CBR in games, including TN
games.
11.2
Implementation of AI methods
This section describes the implementation of AI methods. It briefly covers
each implementation and the reasoning behind it, and highlights the interesting
aspects of the implementation. The art of testing the method is also described,
and any test results are also presented.
11.2.1
Path finding
The implementation of A* was fairly trivial. The highlights of it was the extensions given to it, which naturally developed along with the game.
A simple extension was the setting of its edge weight function. This meant
that different units could use different weight functions when path finding. But
in the actual game, there was only one unit type. It turned out that weight functions were still useful, because it both allowed searching for paths which ignored
enemy units, allied units, or units in general. Furthermore, since technologies
were implemented in the game, some units could end up having a changed
62
weight function. Making it easy to set the weight function made these changes
considerably easier.
As part of extending it, it was decided to give it a ”stop”-function. This
stop-function was given the current node visited and the distance traveled, and
decided if the A* algorithm should terminate. With this stop-function, several
interesting uses was found. A simple use was to find the nearest nodes of a
certain type, such as resource nodes or nodes with enemy units.
Another extension to the A* algorithm was to allow it to have multiple sources
to start from. While this would not find the shortest paths from each source to
each node, it did allow some other useful operations. Consider for instance the
issue of finding the number of enemies in a certain search range from a group
of allied units. By using A* from the set of the nodes of the allied units, and
using a weight function that ignores allies, a stop function could be used to add
each enemy found and add it to a set, finally stopping once the search range is
reached.
In all, the A* algorithm proved effective and useful enough in itself, but its
extensibility made it much more flexible and useful. It covered all path finding
needs and more. This made A* a very useful algorithm, and it is believed that
it will also be useful in an AI library. However, it was not tested on a game with
hidden information, and investigating D* for those cases is still recommended.
But A* can definitely be recommended for path finding in TN games.
11.2.2
Blocking
The implementation of the Ford-Fulkerson algorithm was somewhat trivial. In
the first iteration of the game rules, in which “food” was of major importance,
the FF was vital in order to decide whether or not it would be possible to circle
in and protect more food store than the enemy, such that it would be possible
to starve the enemy to death.
The use of such a strategy required that areas could be easily defended. The
nodes of the games each had their own type, and the “mountain” type could
not be passed by any unit. The full blockage from these nodes helped create
the easily defendable areas. However, FF had to be run on some part of the
graph suspected of being easily defendable and worth defending.
In order to handle this requirement, it was decided to split the graph up
into smaller areas, based on loose connectivity. For each area, the amount of
food in it would be computed, and whether it contained enemies or not was
also computed. To find the areas, techniques inspired from image analysis was
used, including binary morphology. Since image analysis primarily uses data
organised as grids, the methods had to be converted to be usable in general
63
graphs.
After each area had been computed, FF could simply be run on it, and the
actual number of nodes which needed to be blocked could be found. Based on
the number of blocking nodes, the food, and whether enemies was contained in
the area, the AI could compute if it was possible to starve the enemy to death,
and if so, which areas to occupy with some number of troops.
However, once the game rules changed away from food, FF was not used.
This was because holding some area became much less important, while aggressive expansion was needed in order to win. Given the right circumstances, FF
could still be used, if it would be possible to cut off a majority of the map from
the opponent player, but this circumstance was mostly based on speculation.
Based on this, several points can be concluded. The first is that FF is useful
for a specific set of tasks, primarily blocking. The second conclusion is that
techniques from image analysis is a useful future source of investigation for TN
games. Image analysis contains many useful and computing-effective techniques
and methods for spatial data, especially for grids. While not all methods can
be converted to non-grid graphs, some can be converted as described in the use
of FF.
11.2.3
Influence mapping
The implementation of influence mapping was another trivial implementation.
The most interesting aspect of influence mapping was how it was used in the
AI.
From a general perspective, influence mapping can be used to assist human
players by providing a visual analysis of the map. Since influence mapping is a
mapping from nodes to values, each node can be highlighted based on the value
in the influence map. In the game, influence maps was used this way, partly to
help indicate the map, and partly to help debug the influence mapping.
In regards to its use in the game AI, influence mapping was used for several
purposes. One use was to analyse which part of the map was controlled by
which player. The unit influence map was used for this, in which each unit
propagated influence. The map was then calculated for each player, and one
players map was then subtracted from the other players map. Thus, positive
values would indicate which areas was controlled by the player, and negative
values would indicate which areas was controlled by the opponent.
Another use was to determine how many turns different interesting map
features was from each unit. These features could be enemy units, resources
and bases. Organising this information in an influence map proved practical,
and was used in the AI to weight the desirability of goals: if a feature was far
64
Figure 11.1: Screenshot from the game showing influence mapping. Red means
enemy, yellow means allied.
away, it was less desirable.
Overall, influence mapping proved to be a useful, intuitive and effective
method of providing spatial analysis. An issue with influence mapping turned
out to be that the values had to be weighted properly, and that goals based on
influence mapping gave less control and required more fine tuning. This fine
tuning took a considerable time and a large amount of trial-and-error, until the
task was given to the genetic algorithm. The implementation time for the GA
paid itself back a couple of times in time not wasted on trial-and-error, especially
as the AI and the game rules were changed. Based on this, influence mapping
is deemed to be a method useful to the majority of TN games, and some of its
disadvantages can be handled by machine learning methods such as GA.
11.2.4
Genetic Algorithm
In order to keep the GA implementation simple and effective, it was decided to
first and foremost support just 1 data type: floating points. While this requirement may seem very limiting, it was determined to satisfy the requirements of
the AI, since the GA was primarily used to determine the heuristic weights.
The implementation of the GA was somewhat simple. Of note is the evaluation, which was based on a point system. As discussed in the choosing of the GA,
the 2 options available for evaluation is the point system and the repeated-games
system. In order to avoid needless computation, the repeated-games system was
discarded. The point system was used, with a fairly simple point-giving system:
65
1. Massive bonus/penalty for winning/losing.
2. Small bonus/penalty for winning/losing x turns before the end.
3. Small bonus/penalty for having more/less units.
4. Small bonus/penalty for having more/less points.
This system proved to be sufficiently advanced to give rise to proper evaluation in the game.
The rest of the implementation was not very interesting. The selection part
chose a percentage of the best performing and discarded the rest, the reproduction part chose parents at random and chose randomly which gene to pick for
each place. Finally, the mutation part added or subtracted a random value to
genes at random.
The use of the GA was more interesting. The GA help alleviate the problem
of fine tuning the heuristics in the AI. Instead of having a game developer giving
values to each heuristic based on intuition, and then try to test values by playing
through games slowly, the GA made it possible to automatically get decent
values for the heuristics. Furthermore, it also helped reveal which heuristics
was the most important. In a typical run, the most important heuristics would
get sensible values after just a few generations. Instead, some heuristics did not
seem to matter much at all, randomly having positive or negative sign even in
late generations. Overall, the GA made the heuristics AI player go from losing
battles generally to winning them generally.
Another interesting effect on the AI was its adaptability against players. In
one case, the AI with heuristics went from losing to get some overwhelming
victories in just a few generations. When the authors decided to investigate,
it turned out that the GA had caused the AI player to utilise a previously
unknown strategy, and revealed a flaw in the other AI player: By playing really
aggressively and getting units near the enemy base, it became possible for the
heuristic AI player to sneak in a troop at an opponent base. This meant that the
unit production for the opponent decreased considerably. Since the opponent
AI was not programmed to defend its bases, this meant that it would lose based
simply on the difference in unit producing capacity.
However, this revealed an important issue with the GA. When training,
the risk of optimising the heuristic AI player for a single map against a single
opponent instead of improving in general became apparent. This was observed,
since training in 2 different maps lead to 2 different sets of heuristics. While
generally similar, the difference was still an issue. This phenomenon is known as
overtraining, which is known to happen for different learning algorithms. Game
developers have to be aware of this issue if they use GA. One possible way to
handle it is to test the heuristics on a “control set” of maps for each generation:
by observing whether the heuristics performance on this set decrease, the GA
66
can be stopped before overtraining begins hurting the performance on maps in
general.
Overall, GA proved to be a simple, useful and effective learning method for
the game implemented. It also turned out that a simple evaluation function was
enough to get good performance with the GA. However, care must be taken in
order to avoid overtraining.
11.2.5
Case-based Reasoning
The system A CBR system can be extended greatly. In order to handle the
time constraints, it was decided to implement a very simple CBR system. The
main part of the system was the case. In this CBR system, the case consisted
of the following:
< x1 , x2 , ..., xn , tactic, success, timestried >
The values from x1 to xn are the game state features, such as gold, enemy gold,
number of units, enemy units, numbers of turn left, etc. These are used to
match cases, and to determine if cases are similar enough. The tactic is the
solution of the case. The success gives an indication of how successful using this
tactic have been so far. The number of times tried indicates how often the case
have been matched and its solution used, and is included to help ensure that
the AI player is inclined to try out tactics it has not tried a lot.
The CBR system in the main AI player support 3 CBR modes: none, retrieval and modify. In the mode none, the CBR system is not used. This is
useful for testing and providing the main AI player with an opponent. In the
mode retrieval, cases are retrieved from the case base, but the cases are not
modified in regards to success and number of times used. In the mode modify,
cases are both retrieved and modified once the game has been evaluated.
In a typical game with the main AI player using modify against another
instance of the main AI player using none, the following process is observed:
The player first loads the case base. Then, every fourth turn, cases are retrieved
to get a solution for the current situation. It was decided to do this at every
fourth turn, since this means that each tactic have time to have some sort of
influence on the game. If the tactic was chosen every turn, determining whether
the tactic was successful would have been difficult. Furthermore, the success of
the game at that point is recorded. Once the game ends, the final success of
the game is computed, and each case chosen is modified with the success. The
success is based on the success 4 turns after it was chosen, as well as the success
of the whole game. The number of tries is also incremented.
The success determination is chosen based on two desires for success. The
67
success of the whole game should be considered, because solutions should help
lead to winning the game. Secondly, the success of the solution itself should be
considered, not the overall success, which can be influenced by other solutions.
In an advanced CBR system, case base maintenance is important to ensure
that the case base does not grow too large. Using analysis of the existing cases
means that cases which are too similar can be merged, thus allowing for more
cases in the case base. However, due to time constraints, it was decided not to
implement any form of case base maintenance, and instead simply set a limit
on the number of cases that may be added. Adding case base maintenance is a
useful extension to the CBR system.
Retrieval The retrieval of cases is based on the features, success and times
tried of a case. The features of the case is used to collect those cases which are
similar to the current case. To compare, the features are seen as points in a
N-dimensional space, and the similarity is based on the unweighted Euclidean
distance between the two points. In the case that one or more cases are considered, the chance for selecting the case is based partly on how many times it has
been tried before, and partly on the success. In general, cases that have been
tried a low number of times, less than 5, is tried first. If all cases have been
tried, the most successful cases are the most likely cases to be chosen. This
system encourages exploration.
In the case that no similar case is found, a number of new cases having the
same features but different solutions is saved in the case base. After considerable
testing, the solutions that work better will get a greater success score, and they
will be preferred over the less successful solutions.
Game playing and testing The two AI players that were available at the
time of testing the CBR was the StandardPlayer and the InfluencePlayer. InfluencePlayer differed from StandardPlayer by using GA and CBR, and it used
influence mapping to a considerably greater degree than StandardPlayer. In
practice, we observed that InfluencePlayer performed considerably better than
StandardPlayer.
However, an issue was encountered in regards to testing and balancing the
game. Making a map that was fair to both players turned out to be a difficult
challenge. Since players does not take their turn simultaneously, the player that
moves first will get resources first, and thereby get an edge.
When trying to balance this by adding more units to the other team, it turned
out that the power balance was easily overturned to the other side. Basically, no
maps were created that featured winning chances between 40-60 percent for the
same AI player playing against itself. This meant that the test had to assume
that the AI player either was in a winning position or a losing position. In order
68
Figure 11.2: Blue team crushing the red team.
to test its capabilities, the AI player to use the CBR was placed in the losing
position. Furthermore, the CBR using AI player was determined to start from
scratch. The test tried to determine if the player could learn to win using CBR.
The test table is shown below:
Games
40
40
40
40
80
Winning percentage
12.5
20
20
35
36.25
Case numbers
0..300
300..1040
1040
1040..1500
1500
Training mode
Modify
Modify
Retrieve
Modify
Retrieve
As is seen from the results, the training of the Influence Player resulted in a
rise from winning 12.5 percent of all games to win 36.25 of all games. Given the
above tests, we believe that the CBR system works as intended.
Issues There were several issues with the CBR method. One issue is that the
features chosen can affect the performance of CBR quite heavily. If features
are chosen that does not include information relevant in regards to choosing
one solution over the other, the CBR system will be unable to learn which
solution to choose in the given cases. Furthermore, considerable requirements
to features can be given. If features are too complex, the case base will take up
too much space, and case matching will become both more complex and require
more computing time. The question is then if simple features can be chosen for
69
games and still be effective.
Based on the above test results, the conclusion is that it is possible in at
least some games. While it may not be possible to grasp every important factor
when selecting features, enough relevant features can still be gathered such that
learning can happen. This is supported by the results of Aha, Molineaux and
Ponsen, which describes how an AI player learned to win given some fairly
simple features[3].
70
Chapter 12
Script Engine
We had two reasons for attempting to develop a script engine. We wanted to
attempt to write a language that would assisting game developers to write AIs
for their games. The other intended use was to provide an easy way to write
games for the engine and create “custom content” for such a game.
12.1
The idea and outcome
The idea was to create a language with built-in support for AI specific tasks to
ease the development, which was superior to a general purpose programming
language. Unfortunately late in the development we realised that due to the
nature of AI programming, general purpose languages are generally an excellent
choice. This is especially true now that more and more imperative programming
languages adopt some of the more functional paradigms such as closures.
In our experience TN game AI development benefits more from having a
strong base library to handle some of the common tasks. It is possible that
engines with tight integration between AI and the game itself might benefit
from language level support as is the case in the RTP game engine Unreal
Engine 1 .
In the end we decided to halt the development of the script engine due our
revelation and also become writing a script engine takes a lot of time, which
would be better spent else where.
The basic script engine had been completed and it could support a large
1 http://unreal.epicgames.com/UnrealScript.htm
71
subset of “usual” language features of a general purpose programming or scripting language2 . The big issue was that it had been a stand-alone component,
developed independently of the rest of the game engine.
To integrate it we would have to not only rewrite parts of the game engine
to use the script engine, but at least also write a core library for the scripts to
easy interface with the game engine. However, as mentioned other components
showed more promise and we decided to work on those instead. Nevertheless,
we have made some observations and experiences about writing a script engine,
which we have provided below.
12.2
Script engine design
This section will describe the setup and some of the choices involved in creating
a script engine.
12.2.1
Script engine components
A script engine consists of quite a few related components. The exact number of
distinct components vary from engine to engine. This particular engine is made
up of a compiler, an assembler and a virtual machine (VM). This modularisation
is by no means a requirement for a successful setup; especially it is easy to merge
the assembler into the compiler or even the VM.
In this setup, the compiler will parse the “high level” script language, analyse
it, apply a few optimisations on it and output some “script assembly code”.
The assembly code is basically an ordered list of instructions that needs to be
executed. The assembler translates these instructions into byte-code used by
the VM. Execution of the resulting code is done by the VM.
Assembler Including the assembler as a separate component had several advantages. The assembly language itself is very close to the actual byte-code and
gave a step-by-step recipe of what the VM was going to execute. Secondly, the
assembly code being plain text meant that the compiler could output annotations and hints about the assembly instructions. These hints proved quite useful
in debugging the compiler without having to support these annotations in the
byte-code itself.
Another major advantage was that it was possible to implement new bytecode instructions merely by updating the VM and the assembler. Adding support for new things in the compiler was often more complex and updating it
2
A notable exception to this is the lack of support for “global” or “static” variables.
72
all the time slowed down prototyping. In the end the compiler was updated to
support assembly code written directly in the script; this gave the best of both
worlds, since the prototypes could be written in the high level language to the
extend the compiler supported.
12.2.2
Linking scripts
It is common for programmers to write libraries or utility code that is used
throughout their project and script writers are no exception to this. The question was how to handle this re-usage and there are two well known ways to
handle this, which is also used in most operating systems.
When linking a program to a library you either link statically or dynamically.
With static linking, all the code used by the program is embedded into the
program. The other approach is to record the requirement for this library in
the program and then when the program is loaded, a runtime linker will also
load the library and finalize the linking of the program.
Static linking greatly simplifies a lot of things. The library is embedded in
the script; this can be exploited to generate simpler byte-code by handling all
library parts as if they had been written directly in the script itself. With static
linking there is also no need to write a runtime linker; the compile time linker can
be handled by having the compiler read the source of the static library, which
means the compiler do not need to be able to read the byte-code of libraries in
order to generate its output.
The advantages of using runtime linking can greatly reduce the size of the
binaries as well as the memory footprint. When two scripts use the same library,
the price for the library is only paid once with runtime linking. It also makes
it possible to cache a heavy used library such as the core script libraries. A
complication of using dynamic linking is that when your runtime linker detects
it needs a library, it has to have some way of finding said library.
For better and worse, when a library is updated the runtime linking will
automatically pick up this change. This allows a library to be patched without
having to recompile all scripts that use it. On the other hand, an incompatible
change will now break all scripts linked to it. Such a change is also likely cause
a recompilation to fail against the new library, thus static linked scripts have
no benefit here if there is a bug in the old version of the library.
While it is possible to mix the styles; that is linking statically against some
libraries and dynamically against others we decided to go with pure dynamical
linking. The primary argument for dynamic was that it would allow developers
to release their library without all the hassle of reminding people to recompile
or relink all their scripts to the new version.
73
Consequences To cope with some of the issues with dynamic linking we
adopted a Java-like requirement for naming the scripts. At least on Linuxbased systems, shared libraries can have a name (SONAME) different than its
file name. The system solves this by having a library cache that maps the
SONAME to the file name. Instead of having to create and update such a
cache, we decided to make a trivial algorithm for mapping a script name to file
name. This is enforced in the runtime linker, which refuses to complete linking
if the script it loaded is named differently than expected from its file name.
This restricted was inspired by and analogous to file name convention of java
and class files in Java.
The other issue with dynamic linking is how to handle global writable variables particularly if the VM is expected to allow multiple threads. With static
linking it is trivially solvable by embedding all the writable global variables directly into the script itself and have each thread allocate its own copy of these
variables. Unfortunately this is much less trivial when the global variables are
spread across several libraries. In our setup we never implemented global variables nor constants; the work on the script engine was halted before it became
relevant.
Nevertheless, were we to resume the development of the script engine, we
would need to solve this. The prime candidate solution considered so far, would
be to have each thread generate a table where it would keep its global variables.
To conserve memory read-only global variables can be stored in the part of
the script that is shared between threads; though this may require additional
complexity in the byte-code.
Please note that the complete list of advantages and disadvantages of either
linking style are far beyond the scope of this document; the things listed above
were the primary considerations for choosing runtime linking.
12.3
Lessons learned
In this section we will shortly describe some of the lessons we learned while
implementing the script engine.
12.3.1
Code analysis
When parsing a script, the compiler will need to check the semantics of the
input. Originally this was implemented as a part of the parser. Nevertheless,
the language was based on a context-free grammar; in some cases the compiler
lacked the context to decide if the script made sense or not. To solve this most
74
of the checking was deferred till after the parsing.
Originally the compiler used its own description of the script to analyse and
optimise the output. However, this tree form was inferior to work with and was
later replaced with a static single assignment (SSA) based form. SSA was quite
an asset when it came to doing simple optimisations.
There exist other representations, in this case the choice was heavily influence
by both the amount of documentation on SSA available and that GCC 3 adopted
it as an internal representation in 2004 4 . Depending on the language type SSA
may not be the right choice; particularly compilers for functional languages tend
to prefer “continuation-passing style”.
12.3.2
Debugging scripts and the script engine
A great issue with working with a script engine is that there are so many points
of failure. When the VM reports an error the bug could be in the script, the
VM itself, the byte-code generation, the assembler, the compiler or in any mix
of the above.
To assist in the bug hunting, we had the VM generate a stack trace of the
script execution. We had the compiler emit its entire representation of the
script as a comment into the assembler; particularly we had a snapshot of the
representation before and after its optimisation pass. This before and after view
greatly assisted in finding bugs, when the optimisation was too aggressive. Also
the compiler could write line numbers for the assembly instructions it wrote;
which compensated for the lack of support in the assembler and byte-code to
handle instruction to line number mapping in our main development branch.
3 “GNU
Compiler Collection” - http://gcc.gnu.org
4 http://gcc.gnu.org/projects/tree-ssa/
75
Chapter 13
Discussion
Several methods related to AI have been implemented and tested. This section
will discuss the implemented methods and the results from implementing them.
Path finding The A* algorithm turned out to be more extensible than the
authors believed. It is generally used in the game development field, and it
also seems like an excellent algorithm for TN games. Furthermore, it fulfilled
the requirements given to it in the design phase, except for hidden information.
Whether A* in practice will handle hidden information well or not will have to
be investigated. Another algorithm, D*, may prove useful here.
Blocking The implementation of Ford-Fulkerson turned out to be very effective. However, not all TN games have need of blocking, and this methods
utility may be decreased by this. Therefore, it is the authors recommendation
that such algorithms are not the first priority when implementing an AI libray
for a TN game engine.
During implementation and use of Ford-Fulkerson, we discovered that the
area of image analysis might be useful to TN games. Image analysis concerns
itself with spatial analysis of large data sets, frequently organised as grids. Some
TN games have graphs that can be modeled as grids, and these games might
be able to benefit from these graphs. Other TN games have graphs that cannot
be modeled as grids, but some image analysis algorithms can still be changed
to fit these graphs. Investigating this area for TN AI is recommended.
Influence mapping Influence mapping was used for several different tasks.
Furthermore, influence mapping turned out to be trivial to debug, since an in-
76
fluence can be visualised easily. Influence mapping is especially suited to TN
games, since nodes can be used for the mapping. Influence mapping is not limited to grid-based games, but some methods that can be used together with
influence maps, such as cellular automata, assumes a grid structure. Influence
mapping turned out to be useful, and supporting it in a TN AI library is recommended.
Genetic Algorithm The Genetic Algorithm proved very useful for automatic
setting of heuristics. While some issues occured, specifically with overtraining
against one opponent or on one map, these issues did not appear to be considerably noteworthy in the game. An AI player that trained on one map could
still perform well on another map. Furthermore, GA is easy to separate into a
library, requiring only an evaluation function in regards to use. Supporting GA
in any AI game library is recommended.
Case-based Reasoning Case-based reasoning is an advanced method, and
decent results were gained with it. However, the tuning and implementation of
CBR can make it difficult for game developers to use it properly. While further investigation is recommended, easy and general use of CBR may require
specialised tools to help handle the complexity, and that makes it a heavy investment for a TN AI library. The method is still very useful, since it can handle
decision making well in TN games. For game developers they should ensure that
CBR is a good choice before implementing it, and AI library developers should
investigate it further before deciding to support it.
Testing In regards to the testing methods, it turned out that the changing
game rules and having a game under development meant that several different
AI methods could be tested. Another advantage was the chance to test the
utility of machine learning in regards to changing game rules. It was observed
that as not only when the game rules changed, but the AI too, letting machine
learning handle the heuristics gave better and faster results than letting game
developers tune the systems by hand.
On the other hand, not having a game to go on from meant that testing also
became more difficult. For instance, the CBR system that was implemented
required different tactics. While these tactics were already implemented for
Aha, Molineaux and Ponsen[3], no such tactics existed for this game before the
CBR systems needed them. In fact, there was no guarantee that the rules of
the game allowed multiple tactics, instead of one general tactic which worked
all the time.
Another issue in regards to testing was that the game logic and game AI was
completely separate from multiplayer, graphics, GUI, etc. This resulted in that
77
the machine learning could compute without having to share resources with the
aforementioned systems, thus speeding it up.
Adding AI tools and libraries to a TN game engine The primary requirements of implementing and testing AI methods in a practical game has all
been attained.
1. Graph-based search library. The A* algorithm has been successfully implemented and tested, and results indicate that it can indeed be used in
the vast majority of TN games.
2. Machine learning library. Two methods related to machine learning have
been successfully implemented and tested. These are Genetic Algorithm
and Case-based Reasoning. Especially GA is useful in general.
3. Graph-based information extraction and analysis. Two methods related
to graph-based information extraction and analysis have been successfully
implemented and tested. These are blocking method and influence mapping. The results indicate that influence mapping will be useful in the
vast majority of TN games.
Script Engine The original idea of providing language support for AIs proved
to be very difficult for this particular game engine. While this has been done
with success in other engines such as the Unreal Engine, it is possible that this
language level support makes more sense for RTP based games or engines were
different components are integrated more tightly.
A possibility for further research is looking into well-established TN games
and investigate which problems occur. The reasoning for creating the UnrealScript was repeated issues in multiple games with maintainability and errors;
the language constructs were aimed at solving these issues.
78
Part IV
Conclusion
79
In the project, a game, game engine and AI library have been designed,
implemented and tested. Furthermore, a scripting engine have been partly
implemented.
Game engine The implemented game engine consists of several subsystems.
All of these subsystems are geared towards supporting TN game. The most
important aspects were the node-based data structure and the separation of
concerns.
The node-based data structure chosen can support the vast majority of TN
games. By designing a general data structure with few assumptions, it was
possible to support this wide variety games.
The separation of concerns was achieved by implementing an event-based
system. This system may not have been effective in a real-time engine, but
meant that the different subsystems achieved a degree of separation rarely seen
in game engines.
AI library The implemented AI library contains two groups of methods for
supporting AI: graph-based methods and machine learning methods. 3 of these
implemented methods stood out.
The A* path finding algorithm turned out to be effective and efficient. The
vast majority of TN games can utilise it given the graph structure. However,
the ability of A* to handle the important aspect of hidden information was not
tested. D* is an algorithm designed to handle hidden information, which were
not implemented due to time constraints. It is recommended that future work
evaluate D* for TN games, especially if they contain hidden information.
The influence mapping method turned out to be flexible and useful at combining spatial information with game specific properties. For instance, estimating which areas were controlled by a given player.
The Genetic Algorithm turned out to be both easy to use and effective at
optimising heuristics in the AI. An important aspect is that the algorithm is
independent of the game domain.
Script engine The implementation of an AI domain specific language was
not successful. During the development, no aspects of the AI players implementation gave rise to language constructs which would be better than a general
purpose language. AI languages have been successfully implemented in other
RTP engines, such as the Unreal Engine. Whether such language constructs can
be created for TN games is unknown. It is recommended that further research
is based on well-established games, since these may provide a better basis for
finding issues that can be solved with language constructs.
80
Overall conclusion The two main parts of the project, game engine and AI
library, were successfully implemented and tested. Based on the above discussion, it is our opinion that a game engine focused on TN games is not only
possible, but desirable as well, since the advantages are considerable. Creating an AI library is recommended as well, but care must be taken for choosing
techniques that will be useful for the majority of games.
81
Part V
Appendix
82
Chapter 14
Abbreviations and
acronyms
This section contains a list of abbreviations and acronyms used in the report.
• AI - Artificial Intelligence
• AS - Adversarial Search
• CBR - Case-Based Reasoning
• DSK - Domain-Specific Knowledge
• DSL - Domain-Specific Language
• FF - Ford-Fulkerson, Flow analysis algorithm
• GCC - GNU Compiler Collection (http://gcc.gnu.org)
• GUI - Graphical User Interface
• IO - Input/Output
• JVM - Java Virtual Machine
• NWN - Neverwinter Nights (http://nwn.bioware.com/)
• RM - Resource Management
• RT - Real-Time
• RTP - Real-Time Position-based
83
• RTS - Real-Time Strategy, A game genre
• SSA - Static Single Assignment (form).
• TN - Turn and Node-based
• UI - User-Interface
• VM - Virtual Machine
84
Bibliography
[1] Stuart Russell and Peter Norvig, Artificial Intelligence: A Modern Approach. Pearson Education, Inc., International Edition, 2nd Edition 1995,
2003.
[2] Jan Niestadt < [email protected] >, http://www.flipcode.
com/archives/Implementing˙A˙Scripting˙Engine-Part˙1˙Overview.
shtml, Online document / tutorial, 1999.
[3] David W. Aha, Matthew Molineaux and Marc Ponsen, Learning to Win:
Cased-Based Plan Selection in a Real-Time Strategy Game. Navy Center for Applied Research in Artificial Intelligence, Naval Research Laboratory (Code 5515), Washington, DC 20375, ITT Industries, AES Division,
Alexandria, VA 22303, Department of Computer Science and Engineering,
Lehigh University, Bethlehem, PA 18015.
[4] Ian Watson, Damir Azhar, Ya Chuyang, Wei Pan and Gary Chen, Optimization in Strategy Games: Using Genetic Algorithms to Optimize City
Development in FreeCiv. Interim Report, http://www.cs.auckland.ac.
nz/research/gameai/projects/GA%20in%20FreeCiv.pdf
[5] Tomasz Szczepański and Agnar Aamodt, Case-based reasoning for improved
micromanagement in Real-time strategy games. Department of Computer
and Information Science, Norwegian University of Science and Technology (NTNU), NO-7491, Trondheim, Norway. www.idi.ntnu.no/˜agnar/
publications/iccbr09-games-ws.pdf
85