USER MODELING IN A REAL TIME STRATEGY GAME
Romnel L. Agcaoili1, Ramon O. Ang2, Dennis O. Li3, Carlson N. Tang4, and Danny C. Cheng5
College of Computer Studies De La Salle University
2401 Taft Ave. Manila Philippines 1004
E-mail: [email protected], [email protected],
[email protected], [email protected], [email protected]
KEYWORDS
User modeling, Real-Time Strategy Game, Case
Based Reasoning
ABSTRACT
The complex environment inherent in an RTS
game makes it a good domain to apply user
modeling. User models can be used to assist
players in managing overwhelming information
and concurrent tasks within the game. This paper
looks into the possibility of modeling an RTS
game player and assigning an assistant to the
player that will help in micromanagement
throughout the game (Cheng and Thawonmas,
2003).
INTRODUCTION
In Real-Time Strategy (RTS) games, players are
faced with numerous game elements and
information to consider. Various unit types and
abilities,
resource
management,
hidden
information about the map and opponents,
simultaneous management of tasks, etc., contribute
to ever-changing world states throughout the
game. These lead to complex environments that
typically overwhelm game players. They also
make RTS games a good domain to apply user
modeling.
This paper looks into modeling an RTS game
player and focuses on using the model in order to
assist him/her. Livingstone and Charles (2003)
proposed the presence of an assistant in an RTS
game to help the player in micromanaging tasks.
This assistant is an agent that is capable of
modeling the player. In this paper, we present a
possible implementation of this proposal by
monitoring the player's in-game actions and using
case-based reasoning to determine the player's
tactics and create a user model. The aim is to let
the player focus on the strategic level or
macromanagement
by
performing
micromanagement for him/her. The research is
intended to be implemented on Wargus, a realtime strategy game similar to Warcraft II.
RELATED WORK
Some systems that are related to this research are
the COMETS (Fagan and Cunningham 2003), a
system that uses case-based plan recognition in
predicting a user's actions using the game space
invaders as its domain, and CAT (Aha et al. 2005),
a system that uses case-based plan selection in
trying to defeat its opponent using the Wargus
game as its domain, systems.
The relevance of the COMETS system to this
paper's research is the representation technique,
which makes use of plans containing different
states of the environment and the actions the user
performed during the states in the environment,
and its objective, to predict the user's actions.
Although the COMETS system provides a method
for predicting a user's actions, the domain where it
was used was a much simpler domain than this
paper's research. The COMETS systems possessed
only three possible states, the safe, unsafe, and
very unsafe states. The environment where this
paper's system will be implemented in is more
complex than the COMETS system. This is where
the CAT system comes in.
The significance of the CAT system is that
system's opponents are dynamic and that the
system uses the same domain as this paper's
research, a real-time strategy game. Another
significance of the CAT system is its usage of the
Tielt, which is a tool that integrates AI systems
with an environment. The Tielt tool is important to
this paper's research because it gives a possible
solution on integrating this paper's system with the
environment.
Even if the CAT system domain is similar to this
paper's domain, the objectives of the agents are
different. In CAT it tries to defeat its opponents by
selecting appropriate strategies to win. In this
paper's system, it tries to predict the user's actions
so that it may assist the user in executing the task.
actions needed to perform such subtasks can be
interleaved with each other.
MONITORING THE RTS GAME
The scope of the environment that the proposed
agent can view is limited to what the player can
see. In effect, the agent is like a spectator whose
point-of-view is identical to that of the person
playing the game. Therefore, game elements like
unit types and abilities, buildings, available
resources, and territorial information of the player
are immediately accessible. On the other hand,
information about the location of enemy units and
bases, hidden map information, hidden resources
or gold mines, and other non-player-related
elements will only be viewed if and when the
player has access to them in one way or another.
Analyzing the actions of an RTS game player can
be classified into three levels: operational, tactical
and strategic.
The modeling agent monitors the RTS player
throughout the game. The data available to the
agent would be the in-game commands in the form
of mouse clicks and keys entered by the player.
This is the operational level of viewing player
actions. Examples of this level would be clicking
a particular unit, issuing a command to a unit (i.e.
move or attack), and issuing a command to a
building (e.g. summon a hero).
Actions on the operational level can be abstracted
into the tactical level. This level involves analysis
from the agent to predict the subtasks that the
player intends to perform. Consider a set of
actions that include building several land attack
units and upgrading land unit attacks, the agent
could interpret this as a plan to attack by the
player. It should be noted that interpretations by
the agent would confined to the different
categories that are inherent in an RTS game that
would be predefined by the proponents (e.g.
attack, defense, etc.). Also, interpretations of
actions may vary in complexity as players often
perform different subtasks at a time and the
Figure 1. Operational to Tactical Level
Figure 1 shows an illustration of how data from
classified under operational level can be abstracted
into the tactical level. It should be noted that, in
general, the order of operational commands or
actions (circles in the figure) may vary except for
those which have prerequisite actions. For
instance, in the figure, assuming that the 12
archers that were commanded to attack in the last
square (or circle) were those that were built from
Barracks B (2nd square or circle), then the 2nd
action is a prerequisite of the 5th action. The
former should always appear before the latter.
Game situations are, of course, expected to have
more complex and interleaved actions compared to
the figure.
The tactical level of analysis by the agent can
further be abstracted into the strategic level. This
level involves establishing a relationship on the
subtasks that the player intended to perform with
the all the other information and game elements
that are available at that time. For instance, the
agent may consider a particular strategy by the
player wherein he/she used certain specific unit
types to attack the enemy taking into account the
duration of the game play, the specific race of the
enemy, and the enemy's defensive structures.
Figure 2 illustrates how data from the tactical level
can be related with current game elements to
comprise the strategic level of user data. A user’s
strategy can be described by the “tactic” he/she
employs, whether in attack, defense, etc., relative
to the current game elements or the state of the
environment.
unit (ie. does it lose health, inflict damage, build a
structure, go near an ally unit etc.) There could be
global effects to the player as a whole, such as
money gain and game time elapsed. All of this can
be obtained from a game and from each game and
can be used as a player’s data.
Figure 2: Tactical to Strategic Level
By monitoring the game player and storing data
about his/ her actions using the aforementioned
methods, user models specific to the player can be
created. This involves mapping the data into
different categories that will form the model,
which can then be used to as reference to the
player’s behavior with regards to playing the
game. Further discussions about this are specified
below.
This raw data from a game will then be analyzed
after each game. This off-line analysis would
combine the orders given throughout a game to
different units based on the time the order was
issued. This would give an exponential possible
number of possibilities in combinations. If we then
use several data from different games together, we
should be able to look for certain patterns in orders
and effects. If certain patterns appear often, then
we could generalize it to be a style of play for the
user, and this could be stored as important user
data – general impressions of a users playing
style. After gathering several general impressions,
we could then apply them in a game.
USER MODELING
We propose four steps to model the player in an
Knowledge Acquisition
1 obtain orders and effects on the
environment
2 come up with a general impression of
the player
Knowledge Application
3 do habits of the player for him when
a certain pattern is recognized
4 on new situations, the agent will do
actions based on preferences
Figure 3. Phases in Modeling a Player
RTS game (figure 3). The process is split into two
phases: knowledge acquisition, and application.
First, we obtain the actions of the player. The
actions we consider in an RTS game would be the
orders given to each unit. This is the lowest level
of information obtainable in the game and this is
what defines the basic intentions of a player. We
also get the effects of these orders for each specific
Figure 4. Cases
In the knowledge acquisition phase, we use the
user data by comparing it to the current orders in
an on-going game. Each unit will still have its
orders list in a game, as well as the effects it
encounters. When we recognize similar patterns,
then we could use the data stored to predict what
the player would do next. On perfect matches, we
can use the stored data and just replicate what the
player has done before. But it is highly probable
that the next actions may not match the current
situation that a player is in, here is where we use
the preferences of a player.
There are several aspects of an RTS game, such as
attacking, defending, using special units such as
heroes, and economic management. A player may
have preferences for each aspect in a game. For
instance, in attacking, a player may opt to spread
out the damage, or focus damage on specific weak
units. We could use these preferences to roughly
emulate a player’s style of play.
Preferences can be obtained by analysis on the
stored user data generalized from the knowledge
acquisition phase. First we define the different
conditions that a certain preference needs from
effects of actions. For example, if an order results
in doing damage on enemy units, it would be
considered as an attack preference; if an order
results in a unit progressively getting closer to a
certain structure of the player, it may be
considered as a defensive preference.
CASES
During the off-line analysis, a representation is
needed in order to form a general impression or
case from the different operational level
information. The representation of a case will
include the following information: start of the
case, end of the case, goal, classification, orders,
effects and tree level variable. This representation
can be seen in the following pseudo code:
struct Profile{
GameAspect attacking, heroes;
GameAspect defending; movement;
};
struct GameAspect{
WorldState preference[n];
Case cCases[n];
};
struct WorldState{
UnitState aState[n];
};
struct UnitState{
string unitName, unitType, hp;
boolean bInGame, isAttacking, isMoving;
…… //unit functionalities
boolean bIsCreating;
…… //building functionalities
int xLocation. yLocation;
};
struct Case{
string start. end;
WorldState goal;
string classification, orders[n];
UnitState effect;
int treeLevelVariable;
};
The start of a case will consist of an order or
command. This was decided on because this is
how attacking, defending, moving, everything is
done in an RTS game. An example would be if
one would assign a soldier to attack an enemy, that
person should click his soldier and right click that
enemy unit.
Now that the start of the case is known the end of
a case would be to determine the last command
before a unit becomes idle. For example, when
moving a unit from point A to point B, a player
would click the unit and right click the place
where the unit would go. After moving to the
location, the unit would become idle and therefore
the end of the case would be the right click to the
new location.
The next part of the representation would be the
goal of the series of actions performed. This
would be captured by getting the current world
state at the same time of the end of the case. An
example of a partial world state would be: unit a in
position x 45, y 65, barracks built in position 35,
25…..
Moving on, the classification would be the next
component of a case. The classification of a case
would just be the different aspects of the game like
attacking, defending, heroes, movement etc. The
classification will be used to group the different
cases under these different aspects of the game.
This kind of grouping is needed in order to know
what the user might do according to these different
kinds of classifications.
An essential part of each case would naturally be
the different game commands that comprise it.
This is what the orders part of the case represents.
More
complex order would of course reuse less complex
orders of other cases that it would contain.
In the case of a real time strategy game, it would
not be so practical to just store the different
commands and the goal of those commands
because the immediate effects of these commands
are not seen. If the effects of the commands are
not seen then the condition of the goal of a case
being satisfied is unknown and whether or not the
commands given were successful or not. This is
why the effects part of the case is included inside a
case. The effects part of a case will be responsible
for storing the game state visible to game units.
The last part of a case would be its tree level
variable. Each aspect of the game would contain a
tree of cases in order to be able to retrieve these
cases effectively later on for general use. What
divides the tree into levels would be a case’s tree
level variable. The tree level variable that would
be used in this case would be similar to the paper
of Aha et al. (2005) in that it would use the
buildings as the source of information as to what
the user could possibly do.
Taking everything discussed into consideration
plus the fact of user preferences within the game,
the representation would look like figure 4.
An example using the representation can be seen if
a user creates 5 units. The user begins by clicking
his already made barracks and clicking the create
button 5 times. This will be the start of the case
since the user issued an order and will be stored in
the start variable case structure. The orders and
effects of the action will be recorded as well.
When the building becomes idle after producing 5
soldiers, case 1 ends and its goal would be to
produce the world state which is also being
recorded through the different building states and
unit states within the field of vision of the user.
This case will be classified under the economy
game aspect.
Generalizations of cases are done by comparing
one case to another, given a certain threshold in
comparing. This threshold will be used to
determine the strictness of the comparisons. A
lower threshold makes comparisons stricter.
//returns a generalized case by comparing 2 cases
Case Generalize(case1, case2){
Case returnCase;
int threshold = 2;
int caseStart, caseEnd = -1;
for (int i=0; i<case1.steps.size; i++)
{
for(int j=i-threshold; j<i+threshold; j++)
{
if((case1.effect[i]== case2.effect[j])
&& (case1.steps[i] == case2.steps[j]))
{
if(caseStart == -1)
caseStart = i;
caseEnd = i;
}
}
}
if(caseStart != -1)
{
returnCase <- case1.effect[caseStart]
UPTO case1.effect[caseEnd];
returnCase <- case1.steps[caseStart]
UPTO case1.steps[caseEnd];
return returnCase
}
else
return NULL;
}
generalizeAll(){//generalize all cases
Vector generalizedCase[]
for(int i=0; i<TOTALCASES; i++)
{
for(int j=0; j<TOTALCASES; j++)
{
if(i!=j)
generalizedCase.add(generalize
(Case[i], Case[j]));
}
}
}
STRATAGUS AND WARGUS
Wargus is a real-time strategy game cloned from
the popular game Warcraft II.
Wargus is
developed from an open source engine Stratagus,
that is used to make RTS games. The choice of
Wargus is greatly influenced by the availability of
the code for manipulation thereby making
experimentation easier.
Wargus covers all basic functions of a typical RTS
game. It is focused on military command wherein
the player has a "god's point-of-view" of the
world. All information on the units and buildings
of the player are available. Map and enemy
information are hidden at the start of the game.
The player has the choice of which unit and
building type to built as well as which technology
to research on, among others.
Implementing user modeling in Wargus would
entail modification of the artificial intelligence
(AI) part of the program code. Additional features
should be added to allow for the presence of the
modeling agent, as well as capability for
knowledge acquisition and analysis by the said
agent.
CONCLUSION
The proposals mentioned in this paper have yet to
be implemented and tested. This research should
be viewed as an effort to improve AI in RTS
games as well as applying user modeling in an
environment as complex and dynamic as an RTS
game.
It should also be noted that user models of an RTS
game player can be used for other functions. For
instance, a user model can be applied to a
computer opponent making it behave in a way
similar to the modeled human player. Using the
model to assist the game player was a choice by
the proponents influenced by the proposal of
Cheng and Thawonmas (2004). By having an
agent capable of micromanagement, the player has
the chance to focus more on higher level tasks and
forming innovative strategies.
REFERENCES
Aha, D., Molineaux, M., & Ponsen, M. (2005).
“Learning to Win: Case-Based Plan Selection in a
Real-Time Strategy Game”. 2005 International
Conference on Case-Based Reasoning.
Cheng, D.C., & Thawonmas, R. (2004). “Casebased plan recognition for real-time strategy
games”.
Proceedings of the Fifth game-On
International Conference (pp. 36-40). Reading,
UK: Unknown publisher.
Fagan, M., & Cunningham P. (2003). Case-based
plan recognition in computer games”. Proceedings
of the Fifth International Conference on CaseBased Reasoning (pp. 161-170). Trondheim,
Norway: Springer.
Livingstone, D., & Charles, D. (2004). “Intelligent
interfaces for digital games”. The 19th National
Conference on Artificial Intelligence (AAAI-04),
San Jose, California, 2004.
USER MODELING IN A REAL TIME STRATEGY GAME
Danny C. Cheng
College of Computer Studies
De La Salle University
[email protected]
Romnel L. Agcaoili
College of Computer Studies
De La Salle University
[email protected]
Dennis O. Li
College of Computer Studies
De La Salle University
[email protected]
Ramon Carlos O. Ang
College of Computer Studies
De La Salle University
[email protected]
Carlson N. Tang
College of Computer Studies
De La Salle University
[email protected]
KEYWORDS
User modeling, Real-Time Strategy Game, Case Based Reasoning
ABSTRACT
The complex environment inherent in an RTS game makes it a good domain to apply user modeling. User
models can be used to assist players in managing overwhelming information and concurrent tasks within the
game. This paper looks into the possibility of modeling an RTS game player and assigning an assistant to the
player that will help in micromanagement throughout the game (Cheng and Thawonmas, 2003).
BIOGRAPHY
Danny C. Cheng is currently a faculty member of the College of Computer Studies
De La Salle University Manila Philippines. He finished his masteral degree on year
2000 and has been teaching of seven years. He is focusing on the research field of
net centric computing which includes web based technologies, mobile technologies,
networking, and applicability of other fields such as artificial intelligence to it.
Romnel Louis L. Agcaoili is on his senior year in De La Salle University taking up
the degree of Bachelor of Science in Computer Science specializing in Software
Technology. He is working on his thesis about User Modeling and has experience
in developing web-based applications, database systems, compilers, among others.
Ramon Carlos O. Ang is a fourth year student of the College of Computer Studies,
De La Salle University, Manila, Philippines. He is currently specializing in the
field of software technology and is working on his thesis on user modeling in a
real-time strategy game. He plans to graduate June 2006 and plans to finish his
master’s degree in computer science the following year.
Dennis Ivan O. Li is currently a fourth year student at the College of Computer
Studies in De La Salle University, Manila, Philippines. He is taking up Computer
Science, majoring in Software Technology. He is currently working on his thesis
on user modeling to assist in a game, and expects to finish his undergraduate
course by June 2006.
Carlson Erwin N. Tang is currently a fourth year student at the College of
Computer Studies in De La Salle University, Manila, Philippines. He is currently
specializing in the field of software technology and is working on his thesis on
user modeling to assist in a game. He plans to graduate by June 2006 and pursue a
law degree afterwards.