Knowledge acquisition for adative
game AI
Marc Ponsen et al.
Science of Computer programming
vol. 67, pp. 59-75, 2007
장수형
Outline
• Introduction
• Related work
• Adaptive Script of Wargus
• Experiment
• Result
• Alternative method
2 /24
Introduction
• Game
– Become increasingly realistic
– Graphical presentation
– Capabilities of characters ‘living’
• Game AI
– Game developers
• Encompass techniques such as pathfinding, animation, collision physics
– Academic researchers
• Intelligent behavior
– Inferior quality
• Benefit from academic research into commercial games
3 /24
Introduction
• Adaptive game AI
– Behavior of computer-controlled opponents
– Potentially increase the quality of game AI
– Incorporate a sufficient amount of correct prior domain knowledge
• Dynamic scripting
– Offline reinforcement learning technique
– Dynamic scripting in a real-time strategy game called Wargus
– Ambitious performance task
– The quality of the knowledge base is essential
4 /24
Introduction
• Knowledge base
– Manually encode
• Take a long time
• Sub-optimal due to analysis
• Not generate satisfying result
– Semi-automatically
•
•
•
•
Increase the performance
Machine learning
Added to knowledge bases
Evolution algorithm
– Automatically
• Evolutionary algorithm
• Automatically transfers the domain knowledge
5 /24
Related work
• Few studies exist on learning to win complex strategy games
• Focusing on simpler tasks
– Relational Markov decision process model to some limited Wargus
scenarios(Guestrin et al.)
– Case-bases plan recognition approach for assisting Wargus
player(Cheng and Thawonmas)
• Manual knowledge acquisition
– Typical RTS games(Age of Empires and Command & Cunquer)
• Semi-automatic knowledge acquisition
– Pattern recognition technique(Street et al.)
• Automatic knowledge acquisition
– Neural network for Backgammon, GO, Chess(Kirby)
6 /24
RTS games
• Usually focus on military combat
• Control armies and defeat all opposing forces that are situated
in a virtual battlefiled(often called a map) in real-time
• Collecting and managing resources
• Determines all decision for a computer opponent over the
course of the whole game
– Form of scripts which are list of game action that are executed
sequentially
– Constricting buildings, researching new technologies, and combat
7 /24
Wargus
• Clone of the popular RTS game
Warcraft II
• Open source
• Stratagus engine
• Strategy
– Small Balanced Land Attack
– Large Balanced Land Attack
– Soldier’s Rush
– Knight’s Rush
8 /24
Complexity of Wargus
• No single tactic dominates all others
– The rock-paper-scissors principle
• Large action space
– The set of possible actions that can be executed at a particular
moment
• In Wargus…
– A : number of assignments workers can perform
– P : average number of workplace
– T : number of troops
– D : Average number of directions that a unit can moves
– S : number of choices for a troop’s stance
– B : number of buildings
– R : average number of choices for research objectives at a building
– C : average number of choice of units to create at a building
9 /24
Complexity of Wargus
10 /24
Complexity of Wargus
• Decision complex of each state
–
– Higher than the average number of possible moves in many board
game such as chess(30)
11 /24
Dynamic Scripting for Wargus
• Game AI for complex games is mostly defines in scripts
– Contain weaknesses, which human players can exploit
– Dynamic script
• Introduced by Spronck et al.
• Ability to adapt to a human player’s behavior
• The probability that a tactics is selected for a script is an increasing function of its
associated weight value
– Requirements
• The game AI can be scripted
• Domain knowledge on the characteristics of a successful script can be collected
• Evaluation function can be designed to assess the success of the function’s
execution
12 /24
Dynamic Scripting for Wargus
• Divide the game into a small number of distinct game states
• Each state corresponds to a unique knowledge base
13 /24
Weight adaptation in Wargus
• F : The overall fitness
• Fi : the stats fitness(state i)
• Sd : the score for the dynamic player
• So : the score for the player’s opponent
14 /24
Weight adaptation in Wargus
• Sx : the score of the dynamic player state x
• Mx : the military points for player x
• Bx : building points for player x
15 /24
EA(Fitness Function)
• Md : Military points for the dynamic player
• Mo : Military points for the dynamic player’s opponent
• b : break-even point
• Ct : game cycle
• Cmax : maximum game cycle(the longest time a game is allowed
to continue)
16 /24
EA(Encoding)
• Construct, research, economy, combat genes..
17 /24
Performance evaluation
• Dynamic scripting under three condition
– Manually acquired
– Semi-automatically acquired
– Automatically acquired
• The other is controlled by a static script
• Four strategy
– SBLA, LBLA, SR, KR
• Randomization turning point
– Number of the first game in which the dynamic player statistically
outperforms the static player
– A low RTP value indicates good efficiency
18 /24
Result
19 /24
Conclusions
• Three alternative for acquiring high-quality domain knowledge
used by adaptive game AI
– Manual, semi-automatic, automatic
• Discussed dynamic scripting
• Domain knowledge is crucial factor to the performance of
dynamic scripting
• The automatic knowledge acquisition approach takes best
performance
20 /24
Alternative method
• Alternative method of script handling
– Bayesian Network
• Case study : StarCraft
• ‘Adaptive Reasoning Mechanism with Uncertain Knowledge for I
mproving Performance of Artificial Intelligence in StarCraft
21 /24
상성파악
• 전략과 유닛의 상성 파악
22 /24
베이지안 네트워크 설계
• 불확실한 지식정보
– 상대방 진영으로의 정찰 시도
– 지어진 건물들의 구성
– 생산한 유닛의 구성
– 건물과 유닛의 개수
– 위의 정보들을 얻어낸 시각
• 거짓정보는 아니지만 완벽한 정보도 아니다
– 숨겨진 유닛, 숨겨진 건물, 지어지다가 취소된 건물
23 /24
스크립트 선택
• 정보 추론 후 가장 효과적인 대응 스크립트 선택
24 /24
결과
• 실험결과
25 /26
E.N.D