Institute for Parallel Processing - Bulgarian Academy of Science
Methods for Data and Information Fusion
Kiril Alexiev, Iva Nikolova
[email protected]
Tel: 9796620; 0898 898 616
25A, Acad.G.Bonchev Str., Sofia 1113,
Bulgaria
NATO ARW, Velingrad, Bulgaria, 2006
1
Institute for Parallel Processing - Bulgarian Academy of Sciences
Correct decision making (taking) in the
security sector mainly depends on
information, received from multiple
sources. Often, the information is
insufficient, unreliable and
contradictive.
Methods for Data and Information Fusion
2
Institute for Parallel Processing - Bulgarian Academy of Sciences
Architecture of sensor network
sensor
node
Communication
query
routing
data
sensor
data
sensor
data
user
Methods for Data and Information Fusion
routing
data
sensor
data
sensor
node
3
Institute for Parallel Processing - Bulgarian Academy of Sciences
Definition of Data and
Information Fusion
Wikipedia:
Sensor fusion is the combining of sensory data
such that the resulting information is in some
sense better than would be possible when these
sources were used individually.
Better = more accurate, more complete, or more
dependable
Methods for Data and Information Fusion
4
Institute for Parallel Processing - Bulgarian Academy of Sciences
Definition of Data and
Information Fusion
 Authors remark:
In definition:
“combination of data” is not very suitable
phrase. We have to find better one, for
example “simultaneously processed data”
Methods for Data and Information Fusion
5
Institute for Parallel Processing - Bulgarian Academy of Sciences
Benefits from Fusion Process
The first and the most important remark is that
fusion process is necessary most of all to reduce
(to filter) input information through its
integration (merging) and generalization.
Fusion process is necessary to improve accuracy.
Fusion process is necessary to reduce
uncertainty.
Methods for Data and Information Fusion
6
Institute for Parallel Processing - Bulgarian Academy of Sciences
Structure of Data and Information Fusion (JDL)
Level 0: Preliminary data processing – pixel or signal
level data association and characterization.
Level 1: Data alignment, association, tracking and
identification.
Level 2: Situation assessment.
Level 3: Threat assessment.
Level 4: Process Refinement includes adaptive
processing through performance evaluation and
decision or resource and mission management.
Methods for Data and Information Fusion
7
Institute for Parallel Processing - Bulgarian Academy of Sciences
Paper's classification by fusion level
11%
5%
Preprocessing
3% 2%
Level 1
Level 2
Level 3
15%
64%
Level 4
Others
Methods for Data and Information Fusion
8
Institute for Parallel Processing - Bulgarian Academy of Sciences
Paper's classification by sensor type
26%
Radar
40%
Visual
Infrared
Acoustic
15%
4%
Others
15%
Methods for Data and Information Fusion
9
Institute for Parallel Processing - Bulgarian Academy of Sciences
Paper classification by target type
Air targets
Mobile Robots
12%
36%
14%
Ground and/or mobile
targets
31%
7%
Submarine
Others
Methods for Data and Information Fusion
10
Institute for Parallel Processing - Bulgarian Academy of Sciences
Level 1
Temporal data fusion
Methods for Data and Information Fusion
Sensor data fusion
11
Institute for Parallel Processing - Bulgarian Academy of Sciences
Data association methods
 The Nearest Neighbor method associates the nearest
measurement to the track prediction. The more
complicated Global Nearest Neighbor minimizes
cluster cost function in measurement distribution.
The probabilistic data association filter (PDAF) and
its extension to multiple targets – joint PDAF (JPDAF),
solve the same task of measurement identification in a
simpler way. In the JPDAF hypotheses are built for the
measurements and targets only for the current scan. In
this way the number of hypotheses is additionally
reduced but the chance of combinatorial explosion in
dense target and clutter scenarios still remains.
Methods for Data and Information Fusion
12
Institute for Parallel Processing - Bulgarian Academy of Sciences
Data association methods
 In Multiple Hypothesis Tracking approach all measurements
received at a scan are assigned to initialized targets, new targets or
false alarms. A number of hypotheses are generated. Every one
supposes a possible assignment scheme between measurements,
received in all scans, and the targets - confirmed, new ones or false.
Pruning and gating techniques are used to retain the most likely
hypotheses and in this way to reduce their number
 Finite Set Statistics considers all measurements as measurements
from a generalized sensor and all targets as a generalized target of
interest. Fusion of information from one and the same sensor but from
different moments of time
Methods for Data and Information Fusion
13
Institute for Parallel Processing - Bulgarian Academy of Sciences
Identification
Two types of identification:
 Structural identification – more difficult
Define structure (model), which in the best way
corresponds to the observed system (process).
 Parametrical identification – a lot of algorithms
Find (calculate) values of parameters, which
characterize entirely considered system (process).
Methods for Data and Information Fusion
14
Institute for Parallel Processing - Bulgarian Academy of Sciences
Parameter identification
Mathematical description
Linear dynamic system (Markovian presentation):
x(k  1)  F (k ) x(k )  G(k )u(k )  v(k )
z(k )  H (k ) x(k )  w(k )
Kalman filter gives optimal solution for Gaussian noises
Methods for Data and Information Fusion
15
Institute for Parallel Processing - Bulgarian Academy of Sciences
Description
x(k  1)  F ( x(k ), x(k  1),..., x(0))
Markovian - semi Markovian
x(k  1)  F ( x(k ))
Linear - non-linear dynamic system
x(k  1)  F ( x(k ), v(k ))
Additive - non additive system noise
v  N (v, mv ,  v )
Gaussian - non Gaussian system noise
Additive - non additive measurement noise z(k )  H ( x(k ), w(k ))
Gaussian - non Gaussian measurement noise w  N (w, mw , w )
Methods for Data and Information Fusion
16
Institute for Parallel Processing - Bulgarian Academy of Sciences
 The simplest tracking filter, considered in the paper, is alphabeta filter. It is suitable for tracking of moving with constant
velocity targets without steady-state error. The alpha-betagamma filter has ability to track even accelerating targets without
steady-state error.
 Kalman filter is a classical optimal estimating algorithm for
dynamical linear system with Gaussian measurement and system
noise. The modification of Kalman filter - Extended Kalman filter
is developed for non-linear systems. The EKF gives particularly
poor performance on highly non-linear functions because only the
mean is propagated through the non-linearity. The unscented
Kalman filter (UKF) uses a deterministic sampling technique to
pick a minimal set of sample points (called sigma points) around
the mean.
Methods for Data and Information Fusion
17
Institute for Parallel Processing - Bulgarian Academy of Sciences
 The theoretically most powerful approach for manoeuvring
targets tracking is known to be Interacting Multiple Models
estimator. Generalized Pseudo-Bayesian (GPB) estimators
different orders, Fixed structure IMM, Variable Structure
IMM, Probabilistic Data Association IMM are variants. The
most important feature is that all these estimators use in parallel
several models for modelling of the estimated system.
 Particle filters, also known as Sequential Monte Carlo methods
(SMC), are sophisticated model estimation techniques based on
simulation. Particle filters generate a set of samples that
approximate the filtering distribution to some degree of accuracy.
Sampling Importance Resampling (SIR) filters with transition
prior as importance function are commonly known as bootstrap
filter and condensation algorithm.
Methods for Data and Information Fusion
18
Institute for Parallel Processing - Bulgarian Academy of Sciences
Temporal data fusion (Alan Steinberg)

NN
F
KF
EKF
IMM
Nearest Neighbor
Alpha-Beta Filter
Kalman Filter
Extended Kalman Filter
Interacting Multiple Model filter
Methods for Data and Information Fusion
PF
PDAF
JPDAF
Y/N
Particle Filter
Probabilistic Data Association Filter
Joint Probabilistic Data Association Filter
FISST
Finite Set Statistics
Good/Poor Choice;
19
Institute for Parallel Processing - Bulgarian Academy of Sciences
Alan Steinberg
Methods for Data and Information Fusion
20
Institute for Parallel Processing - Bulgarian Academy of Sciences
Fully Centralized Measurement
Fusion Architecture
Methods for Data and Information Fusion
21
Institute for Parallel Processing - Bulgarian Academy of Sciences
Fully Centralized Trajectory Fusion
Architecture
Methods for Data and Information Fusion
22
Institute for Parallel Processing - Bulgarian Academy of Sciences
Distributed Decision Fusion
Architecture
Methods for Data and Information Fusion
23
Institute for Parallel Processing - Bulgarian Academy of Sciences
Simple Example
When both sources are reliable, there is a consensus and
it is reasonable to find solution in the cross-section of
and - sets of corresponding sources: x  D1  D2. If the
two sources do not agree, we have x  D1  D2 . The
hypothesis for reliability sources is no longer credible
and three other hypotheses appear: 1) First source is
correct, the second is incorrect; 2) First source is
incorrect, but second is incorrect; 3) Both sources are
incorrect. How to find the correct hypothesis? As a
precaution, all available information is kept and we hold
up .
Methods for Data and Information Fusion
24
Institute for Parallel Processing - Bulgarian Academy of Sciences
Example – continue
It is obvious that the first fusion method is the most
informative because the information is refined to the
intersection of sets given by each source. It is also the
most “risky” approach because the real value of is
assumed to be inside a smaller set than the two initial
sets. The second fusion method is more reliable since all
the information given by the two sources is preserved.
The drawback of such an approach is a loss of accuracy
since the set assumed to contain , is larger than each of
the initial sets.
Methods for Data and Information Fusion
25
Institute for Parallel Processing - Bulgarian Academy of Sciences
Homogeneous sensor fusion
 AND Operator. This method transforms the output of the sensors in a binary
yes/no consensus operating with logical AND. After that thresholds are applied
to find the result. The procedure is very simple, intuitive and fast, if the values of
thresholds are determined in advance. The method does not take into account the
degree of confidence of each sensor.
 Weighted Average. This method takes a weighted average of available sensor
data and uses it as the fused value. Usually the weights are proportional to
accuracy of sensors or to credibility of sensor information.
 Voting. The voting schemes main advantage is computation efficiency. Voting
involves the derivation of an output data object from a collection of n input data
objects, as prescribed by the requirements and constraints of a voting algorithm.
The voting algorithms can be quite complex in terms of content and structure of
the input data objects and how they handle the votes (weights) at input and
output.
Methods for Data and Information Fusion
26
Institute for Parallel Processing - Bulgarian Academy of Sciences
Sensor fusion
 Bayesian Theory. The use of Bayesian inference theory is widely
spread for the fusion of redundant information. The most known
method is the Kalman Filter, that is optimal in a statistical sense (it
presents the least square error). Bayesian theory is also used to
establish the weights linking the sensors in a weighted average
fusion architecture. Moreover, some reductions of superbayesian
methods to probabilistic evidence combination formulas have been
provided. Some problems arise in a Bayesian framework: I) it
does not distinguish between “lack of evidence” and “disbelief”;
ii) practical difficulties in setting the apriori probabilities:
noninformative priors can cause a wrong bias of further reasoning;
iii) it assumes that the knowledge sources are consistent.
Methods for Data and Information Fusion
27
Institute for Parallel Processing - Bulgarian Academy of Sciences
Sensor Fusion
 Information Theory. Mutual information, in the
form of the Kullback-Leiber divergence, has been
used in [12] as a way of combining probabilistic
masses (sensor outputs). This is yet another
method of fusing two probabilities, this time with
a non-bayesian law, adding some information on
average image values (e.g. depending on lighting
conditions). The local maximum of the mutual
information is then taken as the fused value.
Methods for Data and Information Fusion
28
Institute for Parallel Processing - Bulgarian Academy of Sciences
Sensor Fusion
 Belief Theory. Dempster-Shafer evidential
reasoning is used to compute the belief of a given
event from two or more assessments provided by
different knowledge sources at a symbolic level.
This theory is based on the premise that each
source of information provides only a partial belief
about a proposition. Problem – redistribution of
conflicts.
 Dezert Smarandache Theory(DSmT). DSmT is
analogous to Dempster-Shafer evidential reasoning
theory but overcomes some drawbacks of this
theory
Methods for Data and Information Fusion
29
Institute for Parallel Processing - Bulgarian Academy of Sciences
Sensor fusion
Fuzzy Reasoning. Fuzzy sets and variables are used to
deal with real-world models where the usual ideal
mathematical assumptions are inappropriate. Under the
fuzzy framework, the possibility theory has emerged to
represent imprecision in terms of fuzzy sets and to
quantify uncertainty through four proposed notions:
possibility, necessity, plausibility, and credibility
distributions .
 Geometric Methods, e.g. using uncertainty ellipsoids.
Parametrical identification – if we know model, we can
estimate parameters;
Methods for Data and Information Fusion
30
Institute for Parallel Processing - Bulgarian Academy of Sciences
Level 2,3,4 fusion
 Belief Propagation Nets
 Markov Random Fields
 Factor Graphs
 Game theory
Methods for Data and Information Fusion
31
Institute for Parallel Processing - Bulgarian Academy of Sciences
New research direction
 New tracking filters – may be FISST, may be
new one
 Increased interest in image fusion methods improvement of existing, search for new ones.
 Increased interest on higher level fusion – not
only theoretical but engineering approach
 Decision level methods for fusion – like Dezert
–Smarandache Theory or new ones.
Methods for Data and Information Fusion
32