2013 46th Hawaii International Conference on System Sciences
Enhancing Battlefield Situational Awareness through
Fuzzy-based Value of Information
Timothy P. Hanratty
US Army Research Laboratory
[email protected]
Robert J. Hammell II
Towson University
[email protected]
Eric G. Heilman
US Army Research Laboratory
[email protected]
John C. Dumer
US Army Research Laboratory
[email protected]
Abstract
information to commanders, improving situational
awareness, and increasing knowledge. A key part of
the Battlespace Awareness FOC is the ability to
transform the information being provided by sensors
and other means into knowledge and understanding;
the Army recognizes that this process must involve “a
mixture of automation and human cognition.” [1]
The relatively recent growth in the amounts and
types of data within the military environment is
unparalleled. From sophisticated unmanned ground
acoustic sensors to open-source news feeds, the
military commander and his staff are challenged not
only by the established information overload dilemma,
but more importantly with separating the important
information from the routine. Before the dissimilar
pieces of information can be transformed into useful
knowledge, the relative value of the individual pieces
(or categories) of information must be determined.
Calculating information importance, termed the value
of information (VoI) metric, is a daunting task that is
highly dependent upon its application to dynamic
situations [2]. Solution flexibility is critical since VoI
understanding must be readily applicable across a
disparate range of information types and situational
states of affairs.
Currently, assigning a VoI assessment to a piece
of information is a multiple step process requiring
intelligence collectors and analysts to judge the
information’s value within a host of differing
operational situations.
As such, the cognitive
processes behind these conclusions resist codification
with exact precision and offer an excellent opportunity
to leverage a computational intelligence solution using
fuzzy inference. Fuzzy systems have been shown to be
effective at approximate reasoning where information
is uncertain, incomplete, imprecise, and/or vague [3,
4,5,6,7]. The situation surrounding the problem of VoI
calculation makes the selection of fuzzy logic theory
A major tenet of the US Army's data-to-decision
initiative and a primary challenge for military
commanders and their staff is the ability to shorten the
cycle time from data gathering to decisions. Today,
military operations require information from an
unprecedented number of sources resulting in an
unprecedented volume of collected data. Required are
decision support technologies to improve the synthesis
of data to decisions. Paramount to this process is the
ability to better assess the applicability and relevance
of information for decisions in complex military
environments. Towards this end, this paper presents a
soft computing approach and early results for
calculating the Value of Information (VoI) in complex
military environments using fuzzy associative memory
as an effectively framework for contextually tuning its
value based on content, reliability and latency.
1. Introduction
In March, 2008 the United States Army Training
and Doctrine Command (TRADOC) published its most
recent version of TRADOC Pamphlet 525-66,
“Military Operations Force Operating Capabilities.”
[1] The purpose of this pamphlet is to identify
“capabilities necessary of the Army to fulfill
warfighting concepts.” The document identifies eleven
Force Operating Capability (FOC) areas; two of these
are Battle Command and Battlespace Awareness.
Capability areas outlined in the Battle Command
FOC include information and decision superiority. In a
dynamic battlefield environment, timely and accurate
information is critical to the goal of providing relevant
Research was sponsored by the Army Research Laboratory and was accomplished under
Cooperative Agreement Number W911NF-11-2-0092. The views and conclusions
contained in this document are those of the authors and should not be interpreted as
representing the official policies, either expressed or implied, of the Army Research
Laboratory or the U.S. Government. The U.S. Government is authorized to reproduce and
distribute reprints for Government purposes notwithstanding any copyright notation herein.
1530-1605/12 $26.00 © 2012 IEEE
DOI 10.1109/HICSS.2013.194
Barry A. Bodt
US Army Research Laboratory
[email protected]
1400
1402
and constructs a suitable choice. In particular, the
Fuzzy Associate Memory (FAM) architecture is
suitable for creating a fuzzy-based VoI system.
The remainder of this paper is organized as
follows: Section 2 presents background information
related to VoI within the military domain. A brief
introduction to fuzzy logic, the rationale for choosing a
fuzzy-based approach, details of the FAM system that
has been constructed, and a brief description of the
knowledge elicitation process are provided in Section 3
while Section 4 presents a discussion of the current
results. Section 5 outlines the planning for a pilot
study to statistically validate the fuzzy model. Finally,
Section 6 presents conclusions and future work.
video cameras or light detection and ranging (LIDAR)
equipment will generate voluminous data feeds
measured in terabytes per hour of operation. In
addition to normal tactical missions, relatively new
activities that are part of reestablishing civil authority
within a country [10] further increases the volume of
necessary information and expands the nature of the
intelligence analysis effort.
Table 1. Military echelons with typical operational
times/areas [9]
2. Background
Echelon
Planning
time
Execution
time
Reports
per hour
Area of
Operation
Division
Week
Week/
Days
~Millions
Province
Brigade
Days
Days
170K
Province
/district
Day
56K
District
Hours
18K
Hour/Min
6K
Village
Village/
Hamlet
Battalion
This section presents information related to the
domain challenge and the military information
valuation guidelines.
Company
Platoon
2.1. Battlefield Situational Awareness
Challenges
Days
/hours
Hours
Hour/
Min
While this does not impose much of an increase in
number of collection entities, the volume of generated
data increases rapidly and the ability to merge
automated and manually collected data is not yet fully
developed. Since “the value of information is largely
subjective” [11, 12], estimating data usefulness
remains with the analyst who may not be able to
sufficiently fathom the large volume of varied data.
This prevalence of information gathering and
reporting technologies is causing an overload situation
for personnel responsible for monitoring, filtering, and
analyzing incoming data. In fact, Wilkins, et al., found
that “algorithms that alert on constraint violations and
threats in a straight forward manner inundate users in
dynamic domains” [13]. Further, Endsley [14] points
out that information overload is an additional problem
separate from information requirements, and can
certainly negatively affect situation awareness.
Accurate VoI estimation is essential to the
intelligence analysis process [10], promoting improved
situational understanding and effective decision
making.
Relevant intelligence information, that
information with proper VoI, is integral to battlefield
success. VoI is essential in the collect-assess portion of
the intelligence process. At higher echelons, VoI is a
metric useful in determining the degree of situational
estimate accuracy amidst the uncertainty of combat.
Additionally, VoI is a focusing element as a searchable
criterion, enabling analysts to find relevant information
quickly. At lower echelons, analysts can use VoI to
create an optimum course of action for immediate
mission execution.
Providing commanders and
analysts with VoI tools to filter an increasing number
On today’s battlefield, information drives action.
Personnel must know details about important persons,
places and events within their area of operations to
address issues ranging from kinetic fights to
adjudicating legal disputes to revitalizing a depleted
economy. Soldiers at the edge of conflict gather data
to support their mission. As Major General Michael
Flynn points out:
“At the battalion level and below, intelligence
officers know a great deal about their local
Afghan districts but are generally too
understaffed to gather, store, disseminate, and
digest the substantial body of crucial
information that exists outside traditional
intelligence channels.” [8]
Several factors contribute to the challenge of
analyzing VoI within military environments. As
illustrated in Table 1, lower-level echelons execute
more immediate missions and the timeliness of
information is vital; the decision cycle may be
measured in hours or less. On the other hand, as the
echelon increases, the scope of military operations and
number of information reports grows tremendously.
The ability to manage information effectively at higher
echelon levels becomes exceedingly difficult.
Today, each of these echelons can supplement
human information collection by using a host of
automated data gathering devices. For example, the
addition of a single unmanned aerial vehicle into the
reconnaissance effort with sensors such as full motion
1403
1401
of data feeds holds the potential of quickly generating
useful intelligence at all echelons.
It is clear that finding a method of accurately
determining the value of information generated from
gathered data is necessary to take some of the burden
from human analysts. Yet the nature of intelligence
requires flexible methods and tools that incorporate
measurements covering many topics. Attempting to
qualify and quantify VoI measures within these
varying military situations is a difficult challenge.
Table 2. Source Reliability [15,16]
A
Reliable
B
Usually
Reliable
C
Fairly
Reliable
D
Not
Usually
Reliable
E
Unreliable
F
Cannot
Judge
2.2. Value of Information Guidance
The procedure for alphanumerically rating the
“confidence” or “applicability” assigned a piece of
information is essentially described in the annex to
NATO STANAG (Standard Agreement) 2022 as well
as in Appendix B of US Army FM-2-22.3 [15,16]. The
NATO standard further dictates that, where possible,
“an evaluation of each separate item of information
included in an intelligence report, and not merely the
report as a whole” should be made. The weight given
each piece of information is based on the combined
assessment of the reliability of the source of the
information with the assessment of its information
credibility or content.
As depicted in Table 2 and Table 3, respectively,
the alphabetic Reliability scale ranges from A
(Completely Reliable) to E (Unreliable) while the
numeric Content scale ranges from 1 (Confirmed by
other sources) to 5 (Improbable) [15,16]. Both scales
account for the information that cannot be judged for
source reliability or content with ratings F and 6.
So as an example, a piece of information that was
received by a source that has in the past provided valid
information would be scored a Reliability Rating of
either B or C; depending on the degree of doubt in
authenticity. That same piece of information, if not
confirmed, but seeming logical, would receive a
Content Rating of either 2 or 3; again depending on the
degree the information was consistent with other
information. It quickly becomes obvious the subjective
nature of the ratings (B2 vs. C3) can easily lead to
ambiguity.
In an attempt to guide the application of composite
ratings (i.e., B2 vs. C3) to varied operational situations,
organizations have generalized the usefulness of data
by developing charts similar to the one shown in
Figure 1 [17]. Positioned along the x-axis are the
possible ratings for source reliability while the y-axis
reflects those possible for information content.
Combined, these ratings form a composite that in
general reflects the generic significance of a piece of
information to analysis efforts; that is, an information
applicability rating within a general context.
No doubt of authenticity, trustworthiness, or
competency; has a history of complete
reliability
Minor doubt about authenticity,
trustworthiness, or competency; has a
history of valid information most of the time
Doubt of authenticity, trustworthiness, or
competency but has provided valid
information in the past
Significant doubt about authenticity,
trustworthiness, or competency but has
provided valid information in the past
Lacking in authenticity, trustworthiness,
and competency; history of invalid
information
No basis exists for evaluating the reliability
of the source
Table 3. Information Content [15,16]
1
Confirmed
2
Probably
True
3
Possibly
True
4
Doubtfully
True
5
Improbable
6
Cannot
Judge
Confirmed by other independent sources;
logical in itself; Consistent with other
information on the subject
Not confirmed; logical in itself; consistent
with other information on the subject
Not confirmed; reasonably logical in
itself; agrees with some other information
on the subject
Not confirmed; possible but not logical;
no other information on the subject
Not confirmed; not logical in itself;
contradicted by other information on the
subject
No basis exists for evaluating the validity
of the information
As shown in Figure 1, a piece of information can
have three distinct value states, namely black is good,
grey is questionable, and white is not useable. This
rudimentary attempt to form a composite value shows
progress, but the three states encompass several
combined categories resulting in a misleading
depiction of the judgment on the information. Further,
the “color” rating of some possible combinations does
not make sense. For example, would an F1 rating
(where the source cannot be judged but the information
has been confirmed) really be judged as not usable?
Intuitively, one would expect that the A1 cell
would certainly be black, the E5 cell would certainly
be white, and the rest of the figure would exhibit some
sort of gradual change in color between these two
combinations. That is, the boundaries between the
individual combinations are most likely fuzzy as
opposed to crisp, thereby foreshadowing the use of
fuzzy logic to depict the combination of the two
attributes depicted in the matrix. It would seem that
three categories may not be enough to capture the
complexity of the interaction between the attributes.
Further, the lack of doctrinal guidance with respect to
1404
1402
3. Fuzzy System Prototype
how to both arrive at and use the composite ratings
means that the analyst’s intuitive and subjective
processes for utilizing these criteria will have to
somehow be captured.
A
B
C
D
E
As previously mentioned, a fuzzy-based
methodology was used to develop a prototype decision
support system for assisting with VoI determination.
Specifically, the Fuzzy Associative Memory (FAM)
model was chosen to construct the system. This
section presents a brief introduction to fuzzy logic, the
rationale for choosing a fuzzy-based approach, details
of the FAM system that has been constructed, and a
brief overview of the knowledge elicitation process.
F
1
2
3
3.1. Fuzzy Logic Background
4
In 1965, Lotfi Zadeh wrote his famous paper
formally defining multivalued, or “fuzzy” set theory
[6]. He extended the two-valued indicator function of
traditional set theory to a multivalued membership
function. The membership function is used to assign a
grade of membership, ranging from 0 to 1, to each
object in the fuzzy set. Zadeh formally defined fuzzy
sets, their properties, and various mathematical
operations on fuzzy sets. In a later paper [18] he
introduced the concept of linguistic variables which
have values that are linguistic in nature (i.e. speed =
{slow, medium, fast}).
Fuzzy logic extends conventional Boolean (twovalued) logic so that it can handle truth values other
than 0 (completely false) and 1 (completely true). That
is, fuzzy logic can work with values that indicate
partial truth. Fuzzy logic is built upon fuzzy sets and
the basic concept is easy to grasp. In reality, we input,
process, and output vague and imprecise information
every day. Suppose you are teaching your child to
drive and are discussing rules for how to handle the
approach to an intersection. Would you tell the child
“If the light has been green for 30 seconds, release the
accelerator 75 feet from the intersection”? Or would it
be better to say “If the light has been green for a long
time, let off the gas pedal as you get near the
intersection”? The precision in the first rule makes it
impossible to follow; the more vague, or fuzzy, second
rule can be easily understood and applied.
One use of fuzzy logic is to develop fuzzy
inference systems; these systems provide the ability to
perform approximate, or fuzzy, reasoning. Zadeh [7]
defines approximate reasoning as “the process or
processes by which a possibly imprecise conclusion is
deduced from a collection of imprecise statements.”
His idea of approximate reasoning uses fuzzy logic
which contains linguistic truth values (true, somewhat
true, false, etc.) and approximate rules of inference.
Linguistic variables are an important concept in fuzzy
inference.
A linguistic variable is used to
approximately characterize relationships and values.
5
6
Figure 1. Example Information Source / Reliability
Matrix [17]
Additionally, notice that the composite rating
derived by combining source reliability and
information content is most likely not enough to
represent a true “value of information”. The discussion
above hinted at this by stating that the composite
ratings reflect the “generic significance of a piece of
information to analysis efforts; that is, an information
applicability rating within a general context”.
In general, military operations are defined by their
associated operation tempo. This tempo relates to the
time it takes, or that is available, to plan, prepare, and
execute a mission. High-tempo operations typically
require the decision cycle to be measured in minutes to
hours, while slower tempo operations may allow the
decision cycle to be measured in months or longer.
Absent from the model above is the combination of
the information applicability rating with a specific
operation type. Without the specific framework of a
given operations tempo the associated impact of
information latency (or information timeliness)
requirements are lost. Restated, true VoI is dependent
upon the type of military operation to which the
information is being applied. Thus, the value of a
piece of information is not determined within a general
context, but instead it is dependent upon a specific
context as related to a particular mission. For instance,
in a high-tempo operation, where decisions are made in
short timeframes, added emphasis is assigned to
information that has high applicability and was more
recently received. Note also that the same piece of
information may simultaneously have different value
to different operations.
1405
1403
For example, numbers can be used to characterize a
person’s height, but using words instead might provide
categories such as tall, quite tall, more or less tall, not
very tall, more or less small, and so on. The
imprecision introduced by using words may or may not
be by choice. That is, the imprecision may be
intentional based on not needing to be more precise.
More often, however, the imprecision is dictated by the
lack of a means to quantitatively specify the attributes
of an object. [19]
Fuzzy rules of inference encapsulate the
approximate relationships between the input and
output, or in the terminology of rules, the antecedent
and consequent, domains. A fuzzy rule with two
antecedents has the form “If X is A and Y is B then Z is
C” where A and B are fuzzy sets over the input
domains U and V, respectively and C is a fuzzy set
over the output domain W. When using fuzzy sets in
fuzzy inference, a domain is typically decomposed into
overlapping fuzzy sets; each fuzzy set represents a
classification. An element in the domain has some
grade of membership, from 0 to 1 inclusive, in each
fuzzy set in the domain. The membership function
determines the grade of membership; the shape of the
fuzzy sets determines the membership function.
activities [25], which seems to relate quite well to the
“timeliness” characteristic of information. Further, it
was presumed that the successful development of the
VoI system would rely heavily on integrating
knowledge from subject matter experts (SMEs).
Numerous works have discussed the efficacy and
potential of using fuzzy logic in knowledge acquisition
efforts [26,27,28].
3.3. FAM System
The Fuzzy Associative Memory (FAM) model
was chosen to construct the prototype fuzzy system. A
FAM is a k-dimensional table where each dimension
corresponds to one of the input universes of the rules.
The ith dimension of the table is indexed by the fuzzy
sets that compromise the decomposition of the ith input
domain.
For the prototype system, three inputs are used to
make the VoI decision (source reliability, information
content, and timeliness). The timeliness domain is
necessary to incorporate mission context within the
final VoI determination. The impact of mission
context will be detailed further in the discussion of the
knowledge elicitation efforts below.
With three input domains, a 3-dimensional FAM
could be used. However, the decision was made to use
two, 2-dimensional FAMs connected “in series” to
produce the overall VoI result. The rationale for this
choice was presented in detail in [29] but basically it
provided a simpler knowledge elicitation process for
the SMEs, decreased the total number of fuzzy rules,
and provided the output of the first FAM as a useful
product of its own.
3.2. Rationale for Choosing Fuzzy Logic
The literature reveals that significant work has
been done with respect to “imperfect” data, especially
in the context of data and knowledge bases [4,20,21,
22,23]. Most of this work has been done from the
viewpoint of the “quality” of information; the focus of
our efforts is on the “value” of information which is
perhaps a subtle, but distinct, difference. A generally
accepted definition for information quality seems to be
as a “fitness for use” measure of the information [24].
In contrast, the value of information hinges more on
how important a piece of information should be in a
given decision-making context.
A fuzzy logic-based approach to solving the VoI
problem was chosen for several reasons. First, fuzzy
systems are known to be good at approximate
reasoning where information is uncertain, incomplete,
imprecise, and/or vague [3,4,5,6,7]. Additionally, a
fuzzy knowledge-based system was developed in [3] to
model situation and threat assessment in a littoral
environment; preliminary testing demonstrated good
performance. The author states that before deciding to
use a fuzzy approach several other methods for
handling uncertainty were studied “in depth”, including
Bayesian
methods,
Dempster-Shafer
Theory,
probability, artificial neural networks, and others.
A fuzzy logic approach was also proven effective
in detecting temporal aspects of data in data mining
Figure 2. Prototype System Architecture
The overall architecture of the prototype fuzzy
system is shown in Figure 2. Two inputs feed into the
Applicability FAM: source reliability and information
content; the output of the FAM is the information
applicability decision. Likewise, two inputs feed into
the VoI FAM: one of these (information applicability)
is the output of the first FAM; the other input is the
information timeliness rating. The output of the
second FAM, and the overall system output, is the VoI
metric.
1406
1404
The first step in the design of a fuzzy inference
system is to decompose the input and output domains
into fuzzy sets. The decomposition defines the terms
that may appear in the antecedents and consequents of
the fuzzy rules, thereby determining the language of
the rule base.
The decompositions for all five domains are
shown in Figure 3. For the Applicability FAM, the two
input domains (source reliability and information
content) are divided into five fuzzy sets following the
ratings as given in NATO STANAG 2022 (note that
the categories of “cannot judge” were omitted from
both these input domains) [16]. The output domain,
which is the composite rating of the two inputs, is
divided into nine fuzzy sets and has been labeled
“Information Applicability”. Notice that only five of
the fuzzy sets are linguistically labeled in this
decomposition; this reflects the manner in which the
scale of possible outputs was presented to the subject
matter experts (SMEs) during the knowledge
elicitation process. The labels for the Applicability
fuzzy sets are abbreviated in the figure due to space
limits, and (from left to right) represent Not
Applicable, Somewhat Applicable, Moderately
Applicable, Highly Applicable, and Extremely
Applicable.
Similarly, for the VoI FAM, the information
applicability input domain has the same nine fuzzy sets
as before, the timeliness input domain is decomposed
into three fuzzy sets, and the VoI output domain eleven
fuzzy sets. Again, not all of the fuzzy sets in the
output domain have linguistic tags for the same reason
as before. The output domain fuzzy sets are once more
abbreviated due to space limits and, from left to right,
represent Not Valuable, Minimally Valuable,
Somewhat Valuable, Moderately Valuable, Highly
Valuable, and Extremely Valuable.
The “shape” of the fuzzy sets defines the
membership functions for the system. While there are
numerous shapes for fuzzy sets (triangular, trapezoidal,
and the like), triangular membership functions are used
in the prototype system to facilitate the inference
calculations. Further, the inference process is made
even more efficient by requiring the membership
functions to be isosceles triangles with bases of the
same width; this triangular decomposition with evenly
spaced midpoints has been referred to as a TPE system
[30].
As an example, observe that the source reliability
input domain is decomposed into five fuzzy sets as
shown in Figure 3. It is clear that the TPE restriction
ensures that any input within the domain will belong to
at most two fuzzy sets; that is, any input will have nonzero membership in no more than two fuzzy sets.
Reliable
1
Usually
Reliable
Not Usu.
Reliable Unreliable
Fairly
Reliable
µ 0.5
0
Source Reliability
Confirmed
1
Probably
True
Doubtfully
True
Improbable
Possibly
True
µ 0.5
0
Information Content
NA
SA
MA
HA
EA
1
µ
0.5
0
Information Applicability
Som ewhat
Recent
Recent
Old
1
µ 0.5
0
Timeliness
NV
MinV
SV
ModV
HV
1
µ
0.5
0
Value of Information
Figure 3. Domain Decompositions
1407
1405
EV
This means that, for each input, the antecedents for at
most two fuzzy rules associated with that domain will
be satisfied. Each of the other four domains is
similarly decomposed into the appropriate number of
fuzzy sets per the previous discussion, also using the
TPE structure.
Fuzzy rules encapsulate the relationships between
the input and output (or in the terminology of rules, the
antecedent and consequent) domains. Since both
FAMs are 2-dimensional, the fuzzy rules in each will
have two antecedents and one consequent; that is, the
fuzzy rules will take on the form as outlined in Section
3.1. As a specific example for a possible rule in the
Applicability FAM, let the following define what we
will refer to as Rule 1: “if Source Reliability is Usually
Reliable (UR) and Information Content is Probably
True (PT), then Information Applicability is Highly
Applicable (HA)”.
The output from the system is determined by the
standard centroid defuzzification strategy. That is, the
degree to which each rule influences the overall output
is directly related to the degree to which its inputs
match its antecedent fuzzy sets. The degree of the ith
where mid i is the midpoint of the output fuzzy set C i
(the midpoint in a TPE decomposition is the point in
the fuzzy set that has membership equal to one).
Equation (3) implies that every rule in the fuzzy
rule base is “fired” for each set of inputs to determine
the overall output. However, for a TPE decomposition
of a 2-dimensional FAM structure it is clear that at
most four fuzzy rules will have non-zero degrees (two
rules will have “x” antecedents satisfied by input x and
two rules will have “y” antecedents satisfied by input y;
their intersection in the FAM defines the four fuzzy
rules that should be “fired”). This aspect, plus the fact
that the degrees for all rules will add to one (which is
the denominator in (3), thus eliminating the need for
the division operation), allows the TPE structure to
provide a computationally efficient defuzzification
process.
3.4. Knowledge Elicitation
In order to capture the cognitive requirements
necessary to construct the system and build the fuzzy
association rules, the authors used the Conceptual
Method for knowledge elicitation as posed by Cooke
[31]. The process will be briefly outlined here; a
detailed discussion was reported in [32].
A review of the military intelligence process
revealed the associated concepts previously discussed.
The authors then consulted with a group of three
subject matter experts (SMEs) to discuss the
relationships between data age, operational tempo, and
information applicability. These relationships were
developed into a two-part Likert survey instrument and
the final product presented to the SMEs to gather
specific values. The initial interpretation of the results
led to the fuzzy rules that were codified in the system.
The first survey was used to capture the generic
information applicability rating from the doctrinal
model presented in Section 2.2. The information
content and source reliability classes were divided into
five categories following the descriptions shown in
Tables 2 and 3, respectively (the last category, “cannot
judge” was omitted for this exercise). The composite
information applicability ratings were captures in a 5x5
matrix and expressed on a Likert scale of one through
nine, with nine being “extremely applicable” and one
being “least applicable” (see the labels for the
information applicability decomposition in Figure 3).
With the generic information applicability
ratings completed, the second step involved applying
those ratings against the aspects associated with a
specific mission type. While many different aspect
possibilities exist, the focus of this pilot survey was on
the two primary military aspects of operational tempo
and the temporal latency of the information. In this
i
fuzzy rule, deg C i , corresponding to inputs (x1, y1) is
deg iC i
mI i ( x1 )mI i ( y1 )
1
(1)
2
where C i is the output region of Rule i and
m I i is the
j
degree of membership of the input in the input region
of Rule i for the jth component. For example, the
degree of Rule 1 is:
deg1HA
mUR ( x1 )mPT ( y1 )
(2)
That is, the degree of Rule 1 is the membership value
of x1 in the fuzzy set Usually Reliable from the source
reliability input domain, multiplied by the membership
value of y1 in the fuzzy set Probably True from the
information content input domain.
The standard centroid defuzzification equation that
is used to produce the overall output from a set of
inputs (x1, y1) is:
k
y
¦ deg
i
Ci
, mid i
i 1
(3)
k
¦ deg
i
Ci
i 1
1408
1406
case the operational tempo was defined as either
‘tactical’, ‘operational’ or ‘strategic’, where the
differences between the operational tempos is defined
by the immediacy of the mission and is measured in
the amount of time it takes to plan, prepare and execute
a mission. The temporal latency of the information,
on the other hand, was measured as the degree to
which the information was recently collected,
somewhat recently collected, or old (as shown in the
“timeliness” domain decomposition in Figure 3).
The SMEs used three individual surveys to
gauge the VoI for military mission immediacy of data
use, namely one for use within a short time, one for use
within a moderate time and one for use within a long
time. Each resulting 9x3 VoI matrix would be used for
one of the specific operational tempos. Here the
composite rating is expressed on a Likert scale of zero
thru ten with ten being “extremely valuable” and zero
representing “no value” to the mission (see the labels
for the value of information decomposition in Figure
3).
The results from the knowledge elicitation process
were used to build the Information Applicability FAM,
and three VoI FAMs (one for each operational tempo).
right side of the screen in both a relative scale and as a
numerical output.
Figure 4. Example Screenshot
A regression analysis was performed to examine
the interaction between latency and applicability.
Contour plots for the three operational tempos are
shown in Figure 5.
4. Results
The system has been exercised across numerous
scenarios (that is, various combinations of input
values) to produce VoI determinations.
These
preliminary system results have been demonstrated to
the SMEs and the system performance has been
“validated” in principal and concept. That is, the
system output is consistent with expectations and has
shown the viability of eliciting and using expert
knowledge in this domain.
Note that there is no current system against
which the results can be compared. As such, the
system has not been tested comprehensively due to the
human-centric, context-based nature of the problem
and usage of the system.
Thus, the system
performance will need to be more thoroughly validated
in an experimental study which will be outlined in the
next section.
A sample screenshot from the VoI system is
shown in Figure 4. Note that the information content,
reliability, and information latency inputs are user
selectable by means of analog sliders on the screen.
The operational tempo slider is discrete in nature and is
used to pick one of the three VoI FAMs based on the
desired mission tempo, thereby associating a mission
context with the resulting VoI determination. The
actual values of the user selectable inputs are shown at
the bottom of the screen primarily for use by the
researchers. The VoI determination is shown on the
Figure 5. Regression Contour Plots
1409
1407
Note that as the tempo moves from tactical (fast
paced) to strategic (slower paced) the contribution of
latency decreases and the plot becomes driven
primarily by applicability. This is consistent with the
notion that in a fast-paced tempo there is not much
time to digest available information, so the most recent
information is likely to be the most valuable.
However, as the decision cycle becomes longer any
applicable information regardless of latency is
considered potentially valuable to the decision process.
developed to allow the quick manipulation of
information sets into their preferred order.) This
experimental approach allows for the investigation of
the “accuracy” of the measurement—the agreement
between the SME rankings and the VoI determinations.
However, in the spirit of a Gauge R&R study, it
provides the additional opportunity to explore the setto-set variability, the repeatability of an individual
analyst’s rankings, and the reproducibility of rankings
between analysts as well as the effect of operational
tempo. For repeatability, for example, the same set of
information will be provided in a different initial order
at random times during their session in an attempt to
limit any learning effect. The pilot investigation will
provide a basic assessment on the efficacy of the VoI
system approach and will potentially lead to a
subsequent study involving more analysts.
5. Validation Study
If the VoI system is capturing subject matter
expertise in the model, it is reasonable to expect that
the system, when presented the attributes of several
new pieces of information, could be used to replicate
the ranking of that information as performed by an
SME. It is this natural application that serves as the
basis for validation. For a fixed operational tempo,
consider r new pieces of information, identified only
by the attribute triple (reliability, content, and latency).
The ordered VoI scores yield one permutation among
the r! possible arrangements. An SME acting on the
same set of r attribute triples produces a competing
ranking. The difference between rankings is taken as a
single observation comparing the VoI system and
SME. In the study planned, an experimental design is
determined over the crossing of the attribute triple
elements in a manner that will fully explore the space,
producing many observed rankings for comparison.
The difference metric between the competing
rankings is fundamental to the approach. Recent work
[33] in information retrieval explores the development
of metrics to measure distances between rankings for
the purpose of comparing query effectiveness. Two
commonly used statistical measures, Kendall’s tau and
Spearman’s footrule, have long been used to compare
the difference between two rankings. The latter, for
example, forms a Manhattan distance based on
element-by-element rank comparison of the ordered
set. Both were generalized in [33] to allow the
inclusion of weights; the positional weight is of
particular interest in our study, where differences in
rankings regarding what piece of information should
receive the highest priority can weigh more heavily
than differences in rankings regarding, for example,
fourth in the order.
The planned experiment compares the rankings of
r pieces of information by three sources (two analysts
and the VoI system) over many new sets of r elements.
The VoI system ranking in each case is regarded as the
identity permutation and differences between each
analyst’s ranking and the identity permutation are
treated as response values. (A software tool has been
6. Conclusion and Future Work
The last several decades have seen an
unprecedented increase in the types and amount of
information available to the military environment. For
the military commander and his staff, separating the
important information from the routine has become a
primary challenge. This paper presented a soft
computing approach for codifying the VoI using fuzzy
associative memory as an effectively framework for
contextually tuning its value based on content,
reliability and latency. The early results are promising.
Working with SMEs the VoI prototype has been
validated in principal to work effectively on a macro
scale. That said, there are several challenges that
require further investigation.
As outlined in previous section, the next obvious
step for this effort is to validate of the system by
producing a comprehensive, statistically-relevant
experiment to compare the rankings of information by
analysts against the VoI protoype. This assessment
will provide the foundation for aligning future research
directions. Additionally, there are plans to instantiate
the VoI construct within a task network model to study
of the interaction between the characterization of
information and the intelligence analysts’ decision
making process.
As the program matures, the
capability
to
accommodate
inconsistent
or
contradictory information will need to be investigated.
As the US Army moves towards improving the
synthesis of data to decisions, the ability to efficiently
and effectively calculate VoI provides a necessary and
important step towards meeting that goal.
7. References
1410
1408
[21] Helfert, M. and Foley, O. “A Context Aware
Information Quality Framework”, Proceedings of the
Fourth International Conference on Cooperation and
Promotion of Information Resources in Science and
Technology, November 2009, pp. 187-193.
[22] Yu, B., Kallurkar, S., Vaidyanathan, G., Steiner, D.
“Managing the Pedigree and Quality of Information in
Dynamic Information Sharing Environments”,
Proceedings of the 6th International Joint Conference on
Autonomous Agents and Multiagent Systems, May
2007, pp. 1248-1250.
[23] Parsons, S. “Current Approaches to Handling Imperfect
Information in Data and Knowledge Bases”, IEEE
Transactions on Knowledge and Data Engineering, Vol
8, No. 3, June 1996, pp. 353-372.
[24] Wang, R. Y., and Strong, D. “Beyond Accuracy. What
Data Quality Means to Data Consumers”, Journal of
Management Information Systems, Vol. 12, No. 4,
1996, pp. 5-34.
[25] Vincenti, G., Hammell II, R.J., and Trajkovski, G.
"Scouting for Imprecise Temporal Associations to
Support Effectiveness of Drugs During Clinical Trials",
Proceedings of the Annual Conference of the North
American Fuzzy Information Processing Society
(NAFIPS 2005), Ann Arbor, MI, June 2005.
[26] Barnes, A. and Hammell II, R.J. “Employing Intelligent
Decision Systems to Aid in Information Technology
Project Status Decisions”, in Intelligent Systems in
Operations: Models, Methods, and Applications, B.
Nag, ed., IGI Global, Hershey, PA, 2010, pp. 1-26.
[27] McQuighan, J., and Hammell II, R.J. “Computational
Intelligence for Project Scope”, Proceedings of the 22nd
Midwest Artificial Intelligence and Cognitive Science
Conference, Cincinnati, OH, April 2011, pp. 47-53.
[28] Tolosa, J. and Guadarrama, S. “Collecting Fuzzy
Perceptions from Non-expert Users”, Proceedings of the
19th IEEE International Conference on Fuzzy Systems
(FUZZ-IEEE 2010), Barcelona, (Spain), July 2010.
[29] Hammell, R.J. II, T. Hanratty, and E. Heilman,
“Capturing the Value of Information in Complex
Military Environments: A Fuzzy-based Approach”,
Proceedings of the IEEE International Conference on
Fuzzy Systems 2012 (FUZZ-IEEE 2012), Brisbane,
Australia, 10-15 June 2012, accepted.
[30] T. Sudkamp and R.J. Hammell II, “Interpolation,
Completion, and Learning Fuzzy Rules,” IEEE
Transactions on Systems, Man, and Cybernetics, Vol.
24, No. 2, February 1994.
[31] Cooke, N.J., “Knowledge Elicitation”, in Handbook of
Applied Cognition, F. Durso, ed., Wiley, 1999.
[32] T. Hanratty, E. Heilman, J. Dumer, and R. Hammell II,
“Knowledge Elicitation to Prototype the Value of
Information”, Proceedings of the 23rd Midwest Artificial
Intelligence and Cognitive Sciences Conference
(MAICS 2012), Cincinnati, OH, 21-22 April 2012.
[33] Kumar, R., & Vassilvitskii, S., “Generalized Distances
Between Rankings,” WWW’10 Proceedings of the 19th
International Conference on the World Wide Web,
ACM Association for Computing Machinery, 2010.
[1]
Anonymous, US Army Training and Doctrine
Command (TRADOC) Pamphlet 525-66, Military
Operations Force Operating Capabilities, US Army,
March 2008.
[2] Alberts, David S., John J. Garstka, Richard E. Hayes,
and David T.Signori. Understanding Information Age
Warfare. Washington, DC: CCRP, 2001.
[3] Liang, Y. “An Approximate Reasoning Model for
Situation and Threat Assessment”, Proceedings of the
Fourth International Conference on Fuzzy Systems and
Knowledge Discovery, November 2007, pp. 246-250.
[4] Magnani,M. and Montesi, D. “A Survey on Uncertainty
Management in Data Integration”, Journal of Data and
Information Quality, Vol 2, Issue 1, July 2010.
[5] Yen, J. & Langari, R., Fuzzy Logic: Intelligence,
Control, and Information. Prentice Hall, Upper Saddle
River, NJ, 1999.
[6] Zadeh, L.A., “Fuzzy sets”, Information and Control, 8,
1965, pp. 338-353.
[7] Zadeh, L.A., “A theory of approximate reasoning” in R.
Yager, S. Orchinnikov, R. Tong, H. Nguyen (Eds.),
Fuzzy Sets and Applications (pp 367-412), John Wiley
& Sons, New York, 1987.
[8] Flynn, M. T., et.al, "Fixing Intel: A Blueprint for
Making Intelligence relevant in Afghanistan", US
Army, 5 January 2010.
[9] James, John, “Military Data”, presentation, Network
Science Center, West Point, Oct 2010.
[10] Anonymous, US Army Field Manual (FM) 3-0,
Operations, US Army, June 2001.
[11] Ahituv, N., “Assessing the value of information:
Problems and approaches”, Proceedings of the Tenth
International Conference on Information Systems,
Boston, MA, 1989.
[12] Rafaeli, S., & Raban, D. R., “Experimental investigation
of the subjective value of information in trading”,
Journal of the Association for Information Systems,
4(5), 2003, pp. 119-139.
[13] Wilkins, David E., et al, “Interactive Execution
Monitoring of Agent Teams”, Journal of Artificial
Intelligence Research, Vol 18, March 2003.
[14] Endsley, M. R., “Measurement of Situation in Dynamic
Systems”, Human Factors, 37(1), 1995, pp. 65-84.
[15] Anonymous, US Army Field Manual (FM) 2-22.3,
Human Intelligence Collection Operations, US Army,
September 2006.
[16] North Atlantic Treaty Organizaiton (NATO) Standard
Agreement 2022 (Edition 8) Annex.
[17] Hanratty, T. P., et. al., “Counter-Improvised Explosive
Device (IED) Operations Integration Center (COIC)
Data Fusion Operations and Capabilities: An Initial
Assessment”, US Army TR, December 2011.
[18] Zadeh, L.A., “Outline of a new approach to the analysis
of complex systems and decision processes”, IEEE
Transactions on Systems, Man, and Cybernetics, 3,
1973.
[19] Zadeh, L. A., “The Concept of a Linguistic Variable –
I”, Information Sciences, 8, 1975, pp. 199-249.
[20] Agrawal, P., Sarma, A., Ullman, J., and Widom, J.
“Foundations of Uncertain-Data Integration”,
Proceedings of the VLDB Endowment, Vol. 3, Issue 12, September 2010, pp. 1080-1090.
1411
1409