Computational Anti-terrorism Solutions Using a General
Purpose Evidence Fusion Engine
Pat Talbot & Dennis Ellis
TRW Systems
This paper describes our experience with the implementation of a generalpurpose evidence fusion engine for assimilating data and predicting intent in the
mission area of anti-terrorism. Data fusion is a difficult, multi-faceted problem
with many unsolved challenges, the major one being that the data fusion problem
is ill posed - data fusion means many things to many people. Fusion of evidence
provides a focused solution in cases where a decision maker is faced with discrete
bits of information that are inherently uncertain. The Dempster-Shafer Belief
Network algorithm is the basis for our approach to data fusion. Advantages are
that it: 1) provides intuitive results; 2) differentiates belief, ignorance, and
disbelief; and 3) resolves conflicts. Insights into the behavior of the belief network
were obtained from an analytical study. Mathematical and empirical properties
were explored and codified.
Belief networks have been constructed for six mission areas, leading to the
characterization of the prototype testbed as a general-purpose data fusion engine.
A novel feature of the evolving testbed is that the user is allowed to override the
belief or disbelief associated with a hypothesis and the network will self-adjust
the appropriate link values – or learn - by instantiating the override. The backpropagation algorithm from artificial neural network research is used to adjust
the links. Although the work described here is in the mission area of antiterrorism, it has broad applicability to decision making in any circumstance
where evidence is uncertain, incomplete, possibly conflicting, and arrives
asynchronously over time.
Introduction
The purpose of this paper is to describe our experience with the tailoring of a generalpurpose engine to the task of fusing evidence on terrorism, predicting enemy intent, and
quantitatively comparing allied versus enemy operations. The tool provides semiautomated reasoning support for decision makers faced with risky decisions and
unknown intents based on assessment information that is inherently uncertain,
incomplete, and possibly conflicting. Decision makers often find it difficult to mentally
combine evidence - the human tendency is to postpone risky decisions when data is
incomplete, jump to conclusions, or refuse to consider conflicting data. Those versed in
classical (frequentist) statistics realize it doesn’t apply in situations where evidence is
sparse. A data fusion engine is needed.
Technology Review Journal – Spring/Summer 2002 (Initial Draft)
1
Data fusion is a difficult, multi-faceted problem with many unsolved challenges. A
contributing factor to these challenges is that the problem is ill posed – data fusion means
many things to many people. The taxonomy shown in Figure 1 attempts to organize and
differentiate mathematical building blocks, correlation schemes, and “true” data fusion
methods. Clearly, there are many types of data fusion, and the difference between fusion
and correlation is tenuous. At the top are the fundamental building blocks, which are then
differentiated into analytical, statistical, and sub-symbolic techniques. Uncertain
reasoning, the topic of this paper, is among the statistical approaches.
Numbers
Symbols
Arithmetic
Geometry
Calculus
Algebra
Techniques:
• building blocks
• correlation
• fusion
Transforms
Nonlinear Methods
Binary Compare
Logic
NOR Gate
Set Operations
Recursion
Statistics
Classical Statistics
Non-parametric Statistics
Time Series Analysis
Sub-symbolic Processing
Emergent Computation
Search Techniques
Rule-Based
Simulation
Map-Based Reasoning
Operations Research
Utility Theory
Trade Study
Planning &Scheduling
Representation
Case-Based
Fuzzy Logic
Data Mining
Inductive Reasoning
Deductive Reasoning
Uncertain Reasoning
- Evidential Reasoning
Truth Maintenance
Rough Sets
Discovery
Concept Formulation
Story Generation
Neural Nets
Generalization
Genetic Algorithms
Artificial Life
Swarm Computing
Learning
Creativity Theory
Common Sense
General Intelligence
Figure 1. Fusion Hierarchy
Related techniques, such as Bayesian Networks and Rough Set Theory, were assessed for
applicability. The evidential reasoning approach1, which relies on the Dempster-Shafer
Combination Rule, was chosen because it provides intuitive results, differentiates
ignorance and disbelief (sometimes described as “skeptical” processing), and performs
conflict resolution. Bayesian networks also provide intuitive results, but are better suited
to causal reasoning. Rough sets differentiate between what is certain and what is possible
and are of potential future interest – a truth maintenance system appears necessary to
track the validity of hypotheses as evidence is accumulated.
Insights into the behavior of the belief network were obtained from an analytical study.
Mathematical and empirical properties were explored and codified. Discovery of a
representation for an identity and its inverse revealed fascinating properties with practical
application - e.g., fusion equations are soluble in the same way that matrix equations are.
Technology Review Journal – Spring/Summer 2002 (Initial Draft)
2
A novel feature of the evolving testbed is that the user is allowed to override the belief or
disbelief associated with a hypothesis and the belief network self-adjusts the appropriate
link values – or learns - by instantiating the override. The back-propagation algorithm
from artificial neural network research is used to adjust the links. In addition, the belief
network constructs explanations of how outcomes are obtained – this is important in risky
decision making environments. The work described in this paper has broad applicability
to risky decision-making in circumstances where evidence is uncertain, incomplete,
possibly conflicting, and arrives asynchronously over time.
Belief networks were constructed for six mission areas, leading to the characterization of
our prototype testbed as a general-purpose data fusion engine. The most sophisticated of
these consisted of a pair of six-layer networks that mapped evidence and assessments to
high-level terrorist and anti-terrorist objectives to predict terrorist intent and derive best
anti-terrorism “moves”.
Approach
Evidential Interval 0
Basic Probability Assignment (BPA):
Evidence
Source
Initialize Evidence:
• Source Name
•Value: Belief or Disbelief
• Validity: Time Interval
1
Belief
Unknown
Fog
of
War
+
Disbelief
Cartesian
Product
New Evidence
B
U
D
Previously b bB
Fused
u uB
Evidence d dB
bU
uU
dU
bD
uD
dD
Normalize: Conflict resolution
Distribute unsupported evidence
• N = 1 – dB - bD
B = (bB + bU + uB)/N
Plot results
Confidence
Compute Belief
Disbelief
Compute Disbelief
D = (dD +dU+uD)/N
Unknown
Belief
Mission Time
Figure 2. Dempster Shafer Combination Rule
The Dempster-Shafer Combination Rule (Figure 2) for fusion of evidence is the core
algorithm. Nodes in the network are represented as evidential intervals with values from
the set of real numbers ( 0 <= n <= 1 ). Three parameters specify each node: a “belief”
(B), an “unknown” (U) and a “disbelief” (D). The words “unknown”, “ignorance”, and
“don’t know” are used interchangeably throughout this paper. The “unknown” parameter
is computed as: U = 1 – B – D. New evidence is fused with existing evidence according
to the equations in the figure. Although the theory of evidential reasoning, which is a
3
“skeptical” brand of reasoning, allows “n” parameters within an evidential interval
(producing computational complexity 2n-1), we found that specifying nodes with a single
meaning and a set consisting of three values {B,U,D} was satisfactory. Disbelief (D) is
the complement of the more obtuse “plausibility (P)” parameter ( D = 1 – P) encountered
in the literature.
Having discussed how evidence is fused at nodes, we now discuss how nodes are
combined with links to form a network. We use “hierarchical directed a-cyclic graphs”
exclusively, since more complex representations are not required. The networks are
composed of layers of nodes, each having values {B, U, D}, connected by links with
constant values {L}. Three types of nodes form the networks: input or evidence nodes,
multiple layers of intermediate or hypothesis nodes, and an output layer consisting of
outcome nodes. The function of the links is to convey the influence, or impact, that one
node has on another. Link values are handcrafted for each instantiation based on domain
expertise.
As a practical matter, determining link values turns out to be the most difficult aspect of
implementing belief networks. We reasoned that it’s difficult or impossible to determine
link values for artificial neural networks, which are equivalent sub-symbol
representations of networks. Since link values in neural networks are learned
automatically from training data using a back-propagation algorithm, we adapted this
powerful approach to our use. That is, we allow a user to simultaneously change multiple
node and link values to override existing values and use the back-propagation algorithm
to correctly adjust node and link values. Constraints are imposed on which links are
adjusted. If links that influence the preceding layer of nodes are adjusted, that layer
requires modification. Typically, link values are constrained to [0 <= L <= 1], but link
values outside this interval were found to have practical significance; i.e., in some
applications, L > 1 suggests a multiplier effect, while L < 0 is interpreted as a contrary
effect.
We exploit the ability of belief networks to provide the user with explanations. We
believe that it is very important to explain the meaning of node labels; for example,
“clouds” means: “what is the belief or disbelief in evidence stating that clouds will not be
a problem?”. The value associated with links and nodes are explained in plain English
sentences. Concatenated phrases relevant to the node or link form explanations. For
example, the following explanation is computer generated to explain a link value of “1.0”
connecting a hypothesis node and an outcome node: “Belief in the hypothesis that
assessment is complete has a “certain” (1.0) impact on the outcome that the target is
neutralized”. Numerical values are replaced with linguistic variables - e.g., if the belief
associated with a node is between 0.47 and 0.53, the phrase “it may” is furnished as part
of the explanation.
The algorithm (Figure 3) combines nodes, links, and explanations and executes as
follows:
Technology Review Journal – Spring/Summer 2002 (Initial Draft)
4
User Override
• Call Backprop
b’
d’
Adjust incoming
links to get (b’,d’)
Error = b-b’ or d-d’
For each link value:
• L=L – f (Error)
• Fuse to get nodes
• Convergence?
- Max iterations
- Small error
- Minimal change
Next L
Initialize:
• Read Link Values
• Set Node Values
(B=D=0, U=1)
• Explanations
• Learning Rate
Assign to Node
1st entry?
Not
1st entry?
Initialize
ƒ
b
u
d
New Data
New Evidence
Set Belief (B), Disbelief (D)
Compute Unknown
(U=1–B–D)
Existing Data
B U
D
bB bU bD
uB uU uD
dB dU dD
N = 1 – ( dB + bD )
B = ( bB + bU + uB )/N
D = ( dD + dU+ uD )/N
Propagate
Multiply (B,U,D) values
Node values
for next layer By each outgoing link
in the network
Display results
Figure 3. Fusion Algorithm Flow Diagram
1) Following initialization, new evidence is received. If a node value is unknown
{0,1,0}, the evidence initializes the node, otherwise, it is fused with previous data.
2) Node values are multiplied by link constants for each outgoing link to produce
node values for the next layer in the network.
3) The process continues until evidence is propagated to the final layer and results
are displayed.
4) The user can request an explanation for node or link values.
5) The user has the option of overriding any or all node values and the backpropagation algorithm adjusted links and (optionally) nodes in previous layers to
correct, or reconcile, the network.
Results
Experimentation with the prototype fusion engine allowed us to confirm the
mathematical properties of the Dempster-Shafer Combination Rule and to derive useful
computational properties (Figure 4). Mathematically, the algorithm is stable, except for
the case where the denominator vanishes (1 - bD - dB = 0). If this condition, which is
interpreted as reversal of belief or disbelief, is about to occur, the new evidence is merely
perturbed by an infinitesimal amount to avoid division by zero. That the combination rule
is symmetric, bounded, commutative and associative is shown by a derivation. This
derivation elegantly produces the Dempster-Shafer Combination Rule and therefore
seems worthwhile to include. Let {b,u,d} and {B,U,D} be two pieces of evidence to be
fused. Use the property
5
1 = b + u + d and 1 = B + U + D
to obtain:
1 = (b + u + d)B + (b + u + d) U + (b + u + d)D
then combine terms to get:
(bB + uB + bU) + (uU) + (dD + uD + dU) = (1 – dB – bD)
These terms correspond to the new belief, unknown, disbelief, and normalization factors,
respectively! The Combination Rule (Figure 2) is obtained.
Mathematical Properties
•
•
•
•
•
•
•
•
•
Stable
Symmetric
Bounded
Commutative
Associative
Unit triad (0,1,0)
Inverse exists
Generalizes Bayes
Conflict resolution
Importance
No numerical instabilities (e.g., divide by zero)
Belief and disbelief are computed equivalently
Belief + Ignorance + Disbelief = 1
Same result, regardless of fusion order
Same result, regardless of fusion sequence
Fusion with pure ignorance (0,1,0) => unchanged
Can solve fusion equations
Any Bayesian model can be replicated exactly!
Explicitly provided based on normalization factor
Derived Properties
•
•
•
•
•
•
•
•
Importance
Scalable
Intuitive
Comprehensive
Slight belief is ineffective
Reversing Trends
Polarization
Network Connectivity
Link Values
Simple strategies avoid combinatorial explosion
Algorithm produces common sense results
Meaningful results produced in all cases
Make belief or disbelief either zero or > .5
Strategies: delete previous belief, add strong disbelief
Ignorance is quickly reduce in favor of strong belief
Make as sparse as possible to avoid cascading
Make link value for belief and disbelief equal
Figure 4. Summary of Belief Network Properties
In matrix theory, the identity matrix multiplied by any matrix leaves the matrix
unchanged. The equivalent for Dempster-Shafer theory is the {B, U, D} triad:
I = {0,1,0} corresponding to complete ignorance.
The inverse of a matrix, multiplied by the matrix, produces the identity (AA* = I),
where  denotes the fusion operator. For Dempster-Shafer theory, this inverse was been
determined. Fused with the corresponding evidence triad, the inverse triad produces the
identity triad {0,1,0}. The criteria for deriving the inverse is:
0 = (bB + bU + uB)/ (1 – dB – bD)
1 = uU /( 1 – dB – bD)
0 = (dD + dU + uD)/ (1 – dB – bD)
Technology Review Journal – Spring/Summer 2002 (Initial Draft)
6
Solving these equations simultaneously for {B, U, D} in terms of {b, u, d} yields,
U = 1/[ u – bd { 1 / (b + u) + 1 / (d + u)}]
Similar expressions result for B and D. The existence of an inverse means that fusion
equations can be solved arithmetically. Given triads A and C, we can find triad B such
that A  B = C. In other words, it is possible to determine how much additional evidence
is required to push existing evidence over a decision-making threshold; e.g., given fairly
strong belief {0.75, 0.10, 0.15}, what additional evidence is required to reach a decision
threshold of {0.95, 0.05, 0.0}?
Derived properties were obtained by executing the code parametrically to span the
computational space. We fused certain, moderate, weak, and null belief and disbelief to
determine if the results were intuitive. We also varied the degree and strength of link
connectivity. The algorithm produced common sense results in all cases and also
suggested rules-of-thumb to “get what we want, not what we asked for”. Four heuristics
are discussed in detail. Examples give results based on belief – they are also true for
disbelief because belief and disbelief are interchangeable.
Slight Belief (or Disbelief) is Ineffective - It is not effective to state mild belief in the
face of strong disbelief (or vice-a-versa). As shown in Figure 6, moderate disbelief (0.7)
compounded four times gives strong disbelief, even in the face of mild belief (0.3). To
make conflict meaningful, its value must be greater than ½, otherwise it may as well be
zero. Also shown is the intuitive result (which is not obtained from Bayes Rule) that
equal belief and disbelief, compounded four times, yields equal belief and disbelief!
Belief
Slight Disbelief is Ineffective
B
Unknown
=
Disbelief
C
o
n
f
i
d
e
n
c
e
U
D
1
4
Belief
.7
4
.0
.5
.3
.0
.5
Superscript means
The evidence is fused
with itself 4 times
.5
Disbelief
Mild Conflict is meaningless
Belief or Disbelief < .5 - Useless
0
1
2
3
Evidence
4
Make Belief or Disbelief > .5 or .0
Figure 6. Slight Disbelief
7
Trend of Evidence - It is not easy to reverse the trend of evidence. As shown in Figure 7,
compounding a moderate belief (0.7) produces a strong belief. Fusing strong disbelief
with this only slightly decreases earlier belief. The strategy to reduce a trend is to
decrease belief in evidence over time (we used a linear decay rate), and delete previous
evidence that is “overtaken by events” or redundant based on new evidence.
Trend of Evidence is not Easily Reversed
C
o
n
f
i
d
e
n
c
e
1
Defeat - Disbelief > Belief
B
U
D
.5
=
.7
.1
.2
3
.0
.1
.9
Derate - Lower Previous Values
0
1
2
3
4
Delete - Remove Previous Data
Evidence
Figure 7. Trend
Polarization of Assessments - Oscillations in belief based on addition of new evidence
rarely occur (Figure 8). Belief tends to grow, even in the face of weakening belief and
mild disbelief. Compounding belief above ½ also produces strong belief.
Assessments Polarize to Strong Belief or Disbelief
C
o
n
f
i
d
e
n
c
e
1
Strong Belief
Maybe
.5
Strong Disbelief
0
1
2
3
Evidence
4
4
.9
.0
.1
Fusion algorithm will not result in:
• Mild belief or disbelief
4
.5
.0
.5
.2
.7
.1
.3
• Equal belief and disbelief
.1
.0
.9
4
.3
.3
• Symmetric belief and disbelief
.6
.0
.4
•Oscillations in belief and disbelief
Figure 8. Polarization
Technology Review Journal – Spring/Summer 2002 (Initial Draft)
8
Cascading - When network connectivity is dense, belief in outcomes (last layer in the belief network) rapidly becomes strong, even if link values are not large. When link values
are large, the cascading effect is accelerated. To avoid the phenomena, only define meaningful links and avoid large link values when many links contribute to a node value.
Sparse Network Connectivity Minimizes Cascading
Findings:
C
o
n
f
i
d
e
n
c
e
• Densely connected networks
cause cascading- rapid arrival
at high belief of disbelief
1
• Large link values, combined with
dense connections accelerate the
cascading effect
Dense
Rules:
.5
• Don’t connect every node to every
other node – be sure it’s meaningful
Sparse
• Avoid large link values when many
contribute to a node value
0
1
2
3
4
Evidence
Figure 9. Sparse Network Connectivity Minimizes Casading
Display Interface
An important aspect of the testbed is providing a visual method for determining whether
the prototype is behaving as predicted or desired. To this end, we developed graphical
displays showing the results of the data fusion process.
Displays
that
present
abstractions
of
belief
networks are misleading.
The assumptions inherent in
the model are not explicitly
shown. This is a dangerous
over-simplification in belief
networks because the user
relies on the graphics to
make
potentially
risky
decisions. We designed the
displays (Figure 10) to show
explicit belief and disbelief
for all nodes. Links are
color-coded with variable
thickness to show the degree
• Goal: Explicitly represent assumptions in compatibility links
• Rules define attribution between layers
• Color code & line thickness
Evidence
Type 1
Evidence
Type 2
Evidence
Type 3
Evidence
Type 4
Compatibility links:
Strong
Moderate
Weak
Evidence
Type 5
Evidence
Type 6
Belief
(E1,I1) = .9
Indicator
Type 1
Outcome
Type 1
Indicator
Type 2
Indicator
Type 4
Indicator
Type 3
Outcome
Type 2
Outcome
Type 3
Figure 10. Belief Network Evolution
9
to which they influence nodes. Automated explanations for nodes and links are also
available. (All node labels in this paper are notional and do not reflect actual situations).
Belief Network Instantiations
Terrorist/Anti-terrorist
Terrorist attacks have historically come without warning. In retrospect, there has usually
been enough “shreds” of evidence to suggest that the attack was about to occur, but these
indicators reside in people’s heads, on paper, and in multiple disparate databases. There
has been no way to combine these various bits of information to provide timely warning.
An example belief network has been instantiated to directly address the need to
conceptually cluster information, fuse evidence, predict terrorist intent, and prioritize
anti-terrorist actions. The evidence of terrorist activity discussed here is what drives the
fusion engine. The content of this evidence is critically important to the formulation of
the belief network, which is the representation that the fusion engine works on. We
reasoned that a format for collecting evidence that captured answers to the “Reporter’s
Questions” (who, what, when, where, why, and how) as well as the original text was a
good start. We took a “learn-by-doing” approach. A newspaper article2 that contained
about 100 pieces of evidence was the basis. We parsed each piece of evidence into the
Reporter’s Template, and concurrently added fields to accommodate practical aspects of
the data set. Specifically, we added fields for how severe, how certain, and unplaced
keywords to produce a workable data structure (Figure 11).
Information ID _#1 Gazette 10/05/01
Who (people and groups)
-Information Source
1) Reporter Org: UK Government
2) Reported Affiliation_UN
3) Reporter Title: Prime Minister
4) Reporter Name: Tony Blair
-Threat
5) Enemy Organization: al-Qaida
6) Enemy Affiliation: Taliban
7) Enemy Descriptor: terrorist network
8) Enemy Name: Osama bin Laden
- Friendly
9) Ally Organization: ?
10) Ally Affiliation: ?
11) Ally Descriptor: ?
12) Ally Name: ?
- Neutral
13) Party Organization: ?
14) Party Affiliation: ?
15) Party Descriptor: ?
16) Party Name: ?
Prepared by _Pat Talbot
Date 10/08/01
When (time, duration)
- Start
21) Year: 2001
22) Month: 9
23) Date(dd): 11
24) Time (hh:mm): ?
- Duration
25) Years: ?
26) Months: ?
27) Days: ?
28) Hours: ?
29) Minutes: ?
- End
30) Year: ?
31) Month: ?
32) Date (dd): ?
33) Time (hh:mm): ?
- Future
34) timeframe (near/far): near
Persistence indefinite
Why (goal, intent, purpose)
40) Strategic: ?
41) Operational: pursue terrorist activities
42) Tactical: carry out atrocities
43) Task: ?
How (means, method)
44) “What” Action: protected by Taliban
45) Primary Descriptor: ?
46) Secondary Descriptor: ?
47) Tertiary Descriptor: ?
How Severe
48) Number of Casualties: ?
49) Number of Injuries: ?
50) Cost Estimate($US) : ?
51) Adjective: ?
52) Belief or Disbelief (B/D) B
53) Linguistic: Likely
54) Numerical (0<= P<=1): ?
Where (place)
What (activities and events)
17) “Who” descriptor: __al-Qaida_
18) Action: future attacks
19) Object (Who): US nationals, UK
nationals
20) Object (What): ?
35) Region: Western World
Unplaced keywords:
36) Nation:__US, UK
55) __targets
59) ?
37) Locale: ?
56) ?
60) ?
38) City: ?
57) : ?
61) ?
39) Structure: ?
58) : ?
62) ?
Figure 11. Evidence Form
Technology Review Journal – Spring/Summer 2002 (Initial Draft)
10
Beyond providing a simple way to organize evidence of terrorist attacks and antiterrorism actions, the completed template proved useful in deriving a belief network and
automatically producing the evidence to drive it. For example, the “Who” block
indicates to which belief network the evidence relates - if Threat content is filled in, it
goes on the terrorist belief network, while information in the Friendly section means it is
associated with the Anti-terrorism belief network. Another finding was that the content
of the “What” block provides events, activities and tactical objectives for the belief
network, while the “Why” block provides content for the operational objectives and
strategic objectives in the belief network. The “When” block provides a key for
chronologically sorting the evidence. Finally, the persistence entry is used in the belief
network to degrade the belief in the data over time, if appropriate; e.g., weather
predictions are not persistent but anti-terrorist intents are.
The evidence templates were analyzed and became the basis for deriving the Terrorist
Assessment belief network (Figure 12). Nodes are hypotheses, with the information
sources row showing low level hypotheses, the middle layers providing intermediate
level hypotheses, and the last row showing high-level objectives – also called intents.
Although the algorithmic flow of evidence is from low-level input to high-level intent,
what is of most interest in this problem is predicting (low-level) terrorist events! We have
also found that evidence may be introduced at any level in the hierarchy. To
accommodate this reality, we intend to extend the processing algorithm to allow evidence
to be associated with any hypothesis. Evidence is then fused with existing evidence for
the hypothesis and the back-propagation algorithm is invoked to reconcile the network.
Figure 12. Terrorist Assessment Belief Network
11
The evidence templates and the terrorist belief network formed the basis for the AntiTerrorism belief network (Figure 13). In many cases, our activities, tactical objectives,
and strategic objectives related directly to anti-terrorist nodes. In other cases,
asymmetries were evident: e.g., air superiority is an anti-terrorist operational objective
that is virtually unchallenged by terrorists.
Figure 13. Anti-Terrorism Assessment Belief Network
The Anti-terrorism and terrorist belief networks were then overlaid. The question was:
what strengths and weaknesses are the terrorists or the anti-terrorism forces likely to
exploit? We have provided the capability, given two linked networks with the same
topology, to determine how effective one network is compared to the other and to present
that comparison graphically to the user.
Algorithmically, we compute the ratio of anti-terrorism belief to terrorist belief for each
node and use it as a score. A ratio of unity is defined as parity, while a ratio greater than
10 corresponds to an anti-terrorism strength, and a ratio less than 1.0 corresponds to a
terrorist strength. A ratio is tabulated for each node, and portrayed on the network display
(Figure 14). This effectiveness algorithm is analogous to assessing the relative worth of
chess moves using a (one ply) ‘minimax’ algorithm, although, at the present, our
implementation is much simpler. The result of the comparison is the color-coding of the
nodes based on the value of the ratio of the beliefs at the corresponding nodes (red 
<1.0, yellow  1.0-10.0, green  >10.0).
Technology Review Journal – Spring/Summer 2002 (Initial Draft)
12
Figure 14. Effectiveness of Anti-Terrorism Assessment vs. Terrorist Assessment
Other Examples
In addition to the Terrorist/Anti-terrorist instantiation, we have tested the belief network
in a number of other mission areas in order to show its versatility. These include:





Combat Assessment for Effects-Based-Operations
Weather Prediction Impacts
Kill Chain Belief Network
Space Weather Fusion
Foreign Missile Launch Assessment (with and w/o Fog-of-War Effects)
Combat Assessment - How do we fuse battle damage assessment reports, intelligence
information, and operational knowledge over a period of weeks to determine if strategic,
operational, and tactical objectives are being met? The combat assessment instantiation
shows how weapon-target-assessment (WTA) information influences belief about task
success, which in turn influences beliefs about tactical, operational and strategic
objectives.
Weather Impact Predictions - How will the 24-hour forecast impact tomorrow’s Air
Tasking Order for sorties that employ laser-guided munitions? This instantiation shows
if weather conditions obscure an optical path between a laser designator and a target.
Kill Chain Belief Network - For Time Critical Targets (TCTs), such as mobile missile
launchers, a variety of information sources provide information on how well targets are
13
found=>fixed=>tracked=targeted=> executed=>assessed (this is referred to as the kill
chain). This instantiation allows TCT kill-chain evidence to be posted, organized, and
coherently represented in real time.
Space Weather Fusion - Over a period of hours-to-days, a satellite control officer may be
confronted by a bewildering variety of information, including communications outages,
reports of ionospheric scintillation, radio frequency interference reports, and indicators of
threat activity. This instantiation allows incomplete evidence to be fused.
Foreign Launch Assessment - This early demonstrator was based on the need to fuse
assessments from space sensors, radar, and intelligence reports to determine, as a
function of time, whether a possible foreign missile launch was hostile, deliberate, and
whether it was an intercontinental ballistic missile (ICBM) or a spacecraft launch.
Fog-of-War34 Effects - A software module that perturbs data and human cognitive
performance was integrated with the Foreign Launch Assessment belief network.
Examples of the factors that we allow the user to specify are: the frequency and severity
of lost data, data overload, human confusion, and false assumptions, among others. We
found that the result of applying Fog-of-War perturbations to the evidence is significant.
Discussion
The work described in this paper has broad applicability to decision making in
circumstances where evidence is uncertain, incomplete, possibly conflicting, and arrives
asynchronously over time. The Dempster-Shafer theory of evidence has been found to
have very useful mathematical properties - in particular, an inverse has been discovered
that allows fusion equations to be solved arithmetically. In addition, the derived
properties of these belief networks collectively suggest intuitive application of the
technique as a general-purpose fusion engine for uncertain reasoning. A novel feature of
our implementation is the addition of a back-propagation algorithm that allows the user to
override fused beliefs and disbeliefs in nodes or link values. The back-propagation
algorithm adjusts precursor node and link values to reconcile the network. Thus, the
network learns from user training data in the form of overrides.
Acknowledgements
We thank Terry Cherne for confirming the mathematical properties and deriving the
computational properties of the Dempster-Shafer Combination Rule and Terry Trout for
implementing the back-propagation algorithm that provided operator override and
network reconciliation.
Technology Review Journal – Spring/Summer 2002 (Initial Draft)
14
D. Zarley et al., “Evidential Reasoning Using DELIEF,” Readings in Uncertain Reasoning, Morgan
Kauffman, San Mateo, California, 1990.
2
The Case Against Osama bin Laden, Colorado Springs Gazette, 5 October 2001, which contained the full
text of Tony Blair, Prime Minister of the United Kingdom
3
P. Talbot, “Simulating Fog-of-War in Military Decision Making,” presented at International Conference
on Industrial & Engineering Applications of Artificial Intelligence & Expert Systems (IEA/AIE 2000),
New Orleans, Louisiana, July 2000. The paper was not included in the Proceedings.
1
Photos:
1-1/8 (1.125) in.
wide
1-3/8 (1/375) in.
high
Patrick J. Talbot has extensive experience with TRW, most of which is
in Independent Research and Development efforts. He is currently the
Principal Investigator on the Offense/Defense Integration IR&D. He has
performed on Minuteman, spacecraft projects, Battle Management
tasks, Anti-Satellite systems, and the development of decision algorithms. In addition, he has managed a Delivery Order titled “Technology Insertion Studies and Analysis” at the Joint National Test Facility in
Colorado Springs, Colorado, where his research included parallel computing, simulated human behavior, and strategic planning. He holds
Bachelors of Science degrees in Applied Mathematics and Physics from
Penn State and a Masters of Science degree in Physics from Penn State.
Email: [email protected]
Photo on File
Dennis R. Ellis was the Deputy Lead of the Technology Insertion
Studies and Analysis project at the JNTF. He worked in the fields of
parallel computing, performance benchmarking, simulated human
behavior, and strategic planning. Previously, he was Assistant Manager
in the Marketing and Benchmark group at Cray Research. Since joining
TRW five years ago, he has worked in missile defense and the
Offense/Defense Integration IR&D project. He holds a Bachelors of
Science in Chemical Engineering, from the Colorado School of Mines
Email: [email protected]
15