Simulated dataset from Ph.D.
work of Alexander Statnikov
September 2007
Background
Definition: Conditional independence. Two sets of variables X and Y are conditionally independent given a set of variables Z in the
joint probability distribution P (denoted as X  Y | Z ) if P(X=x | Y=y, Z=z) = P(X=x | Z=z) whenever P(Y=y, Z=z) > 0.
Definition: Target information equivalency of sets of variables. Two sets of variables X and Y are target information equivalent
(TIE) with respect to T if the following four conditions hold:
(1) T 
 X,
(2) T 
 Y,
(3) T  Y | X ,
(4) T  X | Y ,
where X and Y are disjoint sets of variables that are subsets of V.
Definition: Deterministic relation of variables. A variable X  V is determined by the set of variables Y  V (or equivalently, Y
determines X) if the following three conditions hold:
(1) X  Y,
(2) P(X | Y) = 1 for exactly one value of X and zero for all other values,
(3) there does not exist any strict subset Y* of Y such that (1) holds for Y*.
Definition: Parents and children set (or local neighborhood). A parents and children set of the target variable T  V (denoted as
PCT or PC when the context of target is clear) in the joint probability distribution P over variables V is a complete set of variables that
satisfies the following conditions:
(1) Every member X of PCT cannot be rendered independent of T given any subset of variables V that neither includes X, nor T,
nor variables that are TIE to X with respect to T.
(2) PCT does not include pairs of variable sets that are TIE to each other with respect to T.
Definition: Target information equivalency of sets of parents and children. Two sets of parents and children of T ( PC1 and PC2 )
are target information equivalent (TIE) with respect to T if PC1 \ PC2 and PC2 \ PC1 are TIE with respect to T.
Graphical Notation
I. Black dotted line with arrowhead indicates the following stochastic relation:
A
-
B
A is a direct cause of B (and B is a direct effect of A)
A does not determine B
B does not determine A
II. Red dotted line with arrowhead indicates the following deterministic relation:
A
-
B
A is a direct cause of B (and B is a direct effect of A)
A determines B
B does not determine A
III. Red dotted lines with arrowheads connected by another red dotted line indicate the following deterministic relation:
A
-
C
B
A and B are direct causes of C (and C is a direct effect of A and B)
{A, B} determines C (recall that from definition this implies that neither A nor B by itself determines C)
C determines neither A nor B
IV. Red dotted line with arrowhead and bold dot indicates the following deterministic relation:
A
-
B
A is a direct cause of B (and B is the direct effect of A)
A determines B
B determines A
V. Blue dotted line that joins groups of variables (surrounded by circles) and has an index “T” indicate variables that are target
information equivalent (TIE) with respect to T.
A
T
C
B
-
{A, B} is TIE to C with respect to T
A
B
C
T
-
A is TIE to B with respect to T
A is TIE to C with respect to T
B is TIE to C with respect to T
Basic Network Structure
X1
X11
X2
X17
X26
X15
X3
X16
X11
T
X27
X28
X29
X18
X15
X30
T
X5
X6
X7
X22 X23
T
X8
X10
X9
X24 X25
X12
X13
X14
X19
T
T
X20
X21
In total, there are 72 equivalent local neighborhoods of T in the network.
Example of TIE relations (1/2)
The probability distribution of discrete random variables X1, X2, X3, X11, and T is represented
graphically below. Red arrows denote nonzero conditional probabilities. Notice that variables X 1,
X2, X3, X11, and T are not deterministically related to each other.
X1
X2
X3
X11
0
0
0
0
0
1
1
1
1
1
2
2
2
2
2
3
3
3
3
3
The following TIE relations hold in the data:
TIET(X1, X2)
TIET(X2, X3)
TIET(X1, X3)
TIET(X2, X11)
TIET(X1, X11)
TIET(X3, X11)
TIEX11(X1, X2)
TIEX11(X1, X3)
TIEX11(X2, X3)
T
Example of TIE relations (2/2)
The probability distribution of discrete random variables T, X12, X13, and X14 is represented
graphically below. Red arrows denote nonzero conditional probabilities. Notice that variables T,
X12, X13, and X14 are not deterministically related to each other.
T
X12
X13
X14
0
0
0
0
2
1
1
1
3
2
2
2
1
The following TIE relations hold in the data:
TIET(X12, X13)
TIET(X12, X14)
TIET(X13, X14)
Extending Basic Network Structure
• Add more equivalent local neighborhoods, so
that not all of them are of the minimal size.
• Add more variables that can potentially be
false-positives. These variables can be both
connected & unconnected to T.