Game Theoretic Validation of Air
Combat Simulation Models
Jirka Poropudas and Kai Virtanen
Systems Analysis Laboratory
Helsinki University of Technology
[email protected], [email protected]
S ystems
Analysis Laboratory
Helsinki University of Technology
Air combat simulation
• Air combat is analyzed to compare effectiveness of tactics and ways for
conducting missions as well as system performance
• Test flights are expensive and time consuming  constructive simulation
Air combat simulation model
• Aircraft, weapon systems, radars, other apparatus
• Pilot decision making and situation awareness
• Uncertainties
• Discrete event simulation models provide a controlled and reproducible
environment that may be complex and convoluted with many levels of submodels
Validation of the model?
S ystems
Analysis Laboratory
Helsinki University of Technology
Existing validation and optimization
approaches
• Simulation metamodels
– Mappings from simulation input to output
- Response surface methods, regression models, neural networks, etc.
• Validation methods
– Real data, expert knowledge, statistical methods, sensitivity analysis
• Simulation-optimization methods
– Ranking and selection, stochastic gradient approximation,
metaheuristics, sample path optimization
One-sided approaches 
Action of the adversary is not taken into account
The game theoretic approach!
S ystems
Analysis Laboratory
Helsinki University of Technology
The game theoretic approach
Definition of the scenario
–
–
–
2)
4)
Discrete tactical
alternatives x and y
Input: tactical alternatives
Output: MOE estimates
Estimation of games from
the simulation data using
statistical techniques
Use of the games in
validation
S ystems
Analysis Laboratory
Helsinki University of Technology
Discrete decision
variables x and y
MOE estimates
Payoff
RED, min
RED
Simulation of the scenario
using the simulation model
–
–
3)
Aircraft, weapons, sensory
and other systems
Initial geometry
Objectives  Measures of
effectiveness (MOEs)
Available tactics and
systems = Tactical
alternatives
y1
y2
y3
x1
-0.077
0.855
0.885
x2
-0.811
0.013
0.023
x3
0.833
0.036
0.004
Analysis of variance
BLUE, max
–
Game
Simulation
BLUE
1)
y1
y2
y3
x1
II
IV
IV
x2
I
III
III
x3
I
III
III
Regression analysis
0.7
0.5
0.6
0.4
0.5
0.3
0.4
0.2
0.3
0.1
0.2
0
0.1
15
15
0
10
Re
d
15
10
5
y
5
0
ex
Blu
15
10
Red
10
y
5
5
0
Blu
ex
Games in validation
• Goal: Confirming that the simulation model performs as intended
• Comparison of the scenario and properties of the game
• Symmetry
– Symmetric scenarios => symmetric games
• Dependence between decision variables and payoffs
– Dependence between tactical alternatives and MOEs
• Best responses and Nash equilibria
– Explanation and interpretation based on the scenario
• Initiative
– Making one’s decision before or after the adversary =>
Advantageous/disadvantageous?
– Explanation and interpretation based on the scenario
S ystems
Analysis Laboratory
Helsinki University of Technology
Validation example: Aggression level
• Two-on-two air combat scenario
– Identical aircraft, air-to-air missiles, radars, data links, etc.
– Symmetric initial geometry
– Identical tactical alternatives
- Aggression levels of pilots: Low, Medium, High
– Objectives => MOEs
- Blue kills, red kills, difference of kills
• Simulation using X-Brawler
– Many versus many air combat simulation
– Discrete event simulation methodology
– Aircraft, weapons and other hardware models
– Elements describing pilot decision making and situation awareness
S ystems
Analysis Laboratory
Helsinki University of Technology
Validation results
BLUE, max
Payoff: Blue kills RED, min
low
medium
high
Expert knowledge:
• Increasing aggressiveness 
Increasing causality rates
low
I 0.18
II 0.34
II 0.33
medium high
III 1.20 III 1.21
IV 1.50 IV 1.50
IV 1.50 IV 1.50
• Dependence
– Increasing aggressiveness
=> Increase of blue kills
• MOE: blue kills
• Best responses & Nash equilibria
 Low aggressiveness for red
– Medium or high for blue, low for red
 High aggressiveness for blue
– Medium and high leading to the same
outcome => Possible shortcoming
S ystems
Analysis Laboratory
Helsinki University of Technology
Validation results
• Increasing aggressiveness
=> Increasing causality rates
• Symmetric scenario
=> Symmetric game
BLUE, max
Payoff: Blue kills – Red kills RED, min
Expert knowledge:
low
medium
low
II -0.08
medium I -0.81
high
I -0.83
IV 0.86
III 0.01
III 0.04
high
IV 0.89
III 0.02
III 0.00
• Symmetry
– MOE estimates approximately zero when the decisions coincide
– E.g., low, high => best, worst AND high, low => worst, best
• Dependence
– Increasing aggressiveness => Increasing causality rates for both sides
– Medium and high for blue leading to the same outcome => Possible shortcoming
• Best responses & Nash equilibrium
– Low for blue, low for red
S ystems
Analysis Laboratory
Helsinki University of Technology
Conclusions
• Novel way to analyze air combat
– Combination of discrete event simulation and game theory
– Extension of one-sided validation and optimization approaches
• Validation
– Properties of games  Air combat practices
– Simulation data in an informative form
– Comparison of tactical alternatives using games
– Systematic means for analyzing air combat
- Single simulation batch
• Other application areas involving game settings
S ystems
Analysis Laboratory
Helsinki University of Technology