Independence, Decomposability and
functions which take values into an
Abelian Group
Adrian Silvescu
Vasant Honavar
Department of Computer Science
Iowa State University
Decomposition and Independence




Decomposition renders
problems more tractable.
Apply recursively
Decomposition is enabled
by “independence”
Decomposition and
independence are dual
notions
A
B
A
B
A
B
Conditional Decomposition and
Independence





Seldom are the two
sub-problems disjoint
All is not lost
Conditional
Decomposition /
Independence
Conditioning on C
C a.k.a. separator
A C B
A C = C B
A C B
Formalization of the intuitions



Problem P = (D, S, solP)
D = Domain, S = Solutions
solP : D S
A
solP
B
Example: Determinant_Computation(M2, R, det)
Conditional Independence /
Decomposition Formalization



(Variable Based)
P = (D = A X B X C, S, solP)
 P1 = (A X C, S1, solP1), P2 = (B X C, S2, solP2)
solP(A, B, C) = solP1(A, C)

P
P1, P 2 solP2(B,
C)
Probabilities




I(A, B|C) iff P(A, B| C) = P(A|C) P(B|C)
Equivalently P(A, B, C) = P(A, C) P(B|C)
P(A, B, C) = f1(A, C) f2(B, C)
Independencies can be represented by a
graph where we do not draw edges between
variables that are independent conditioned
on the rest of the variables.
A
C
B
The Hammersley-Clifford Theorem:
From Pairwise to Holistic Decomposability
p(V ) 
f
(C )
C
CMaxCliques( G )
Outline



Generalized Conditional Independence with
respect to a function f and properties
Theorems
Conclusions and Discussion
Conditional Independence with respect
to a function f - If(A,B|C)
 sol
f(A,
C)C)
B,C)
C) =
= fsol
P(A,B,
P1(A,
1(A,

Assumptions:
–
–
–
–
P
fsol
(B,(B,
C)C)

P1, P 2 2 P2
If(A,B|C)
S = S1 = S2 [= G]
P
P1, P 2


A, B, C is a partition of the set of all variables
Saturated independence statements – from now on
.
Conditional Independence with respect
to a function f If(A,B|C) – cont’d
A
0
B
0
C
0
f
.25
0 0 1 .3
… … … …
 If(A,B|C)
=
A
0
C
0
f1
.5
0 1 .3
… … …

B
0
C
0
f2
.5
0 1 .3
… … …
iff
 f(A, B, C) = f1(A, C)  f2(B, C)
Examples of If(A,B|C )

Multiplicative (probabilities)

Additive (fitness, energy, value functions)

Relational (relations)
Properties of If(A,B|C )

1.Trivial Independence
If(A, Φ|C)
A
C
2. Symmetry
If(A, B|C) => If(B, A|C)
3. Weak Union
If(A, B U D|C) => If(A, B|C U D)
4. Intersection

If(A, B|C U D) & If(A, D|C U B) => If(A, B U D|C)






D
B
Abelian Groups

(G, +, 0, -) is an Abelian Group iff
–
–
–

+ is associative and commutative
0 is a neutral element
- is an inversion operator
Examples:
–
–
–
(R, + , 0, - )
((0, ∞), · , 1, -¹)
({0, 1}, mod2, 0, id)
- additive (value func.)
- multiplicative (prob.)
- relational (relations)
Outline



Generalized Conditional Independence with
respect to a function f
Properties and Theorems
Conclusions and Discussion
Markov Properties [Pearl & Paz ‘87]
If Axioms 1-4 then the
following are equivalent
 Pairwise – (α,β)G =>
If(α, β|V\{α,β})
 Local If(α, V\(N(α)U{α})| N(α))
 Global – If C=V\{A, B}
separates A and B in G
If(A, B| C=V\{A, B})

α
α
A
β
V\{α,β}
N(α)
C
B
Factorization – Main Theorem
The Factorization Theorem: From
Pairwise to Holistic Decomposability
f (V ) 

CMaxCliques( G )
f C (C )
Particular Cases - Factorization

Probabilistic – Hammersley-Clifford

Additive Decomposability

Relational Decomposability
Graph Separability and Independence
[Geiger & Pearl ‘ 93]
If Axioms 1-4 hold then
SepG(A, B|C)  If(A, B|C)
for all saturated independence statements
Completeness
Axioms 1-4 provide a complete axiomatic
characterization of independence statements
for functions which take values over Abelian
groups
Outline



Generalized Conditional Independence with
respect to a function f
Properties and Theorems
Conclusions and Discussion
Conclusions (1)



Introduced a very general notion of Conditional
Independence / Decomposability.
Developed it into a notion of Conditional
Independence relative to a function f which takes
values into an Abelian Group If(.,.|.).
We proved that If(.,.|.) satisfies the following
important independence properties:
–
–
–
–
1. Trivial independence,
2. Symmetry,
3. Weak union
4. Intersection
Conclusions (2)




Axioms 1-4 imply the equivalence of the Global, Local and
Pairwise Markov Properties for our notion conditional
independence relation If(.,.|.)) based on the result from [Pearl
and Paz '87].
We proved a natural generalization of the Hammersley-Clifford
which allows us to factorize the function f over the cliques of an
associated Markov Network which reflects the Conditional
Independencies of subsets of variables with respect to f.
Completeness Theorem, Graph Separability Eq. Theorem
The theory developed in this paper subsumes: probability
distributions, additive decomposable functions and relations, as
particular cases of functions over Abelian Groups.
Discussion: Relation to Graphoids

(-) Decomposition

(-) Contraction

(+) Weak Contraction

Graphoids – No finite axiomatic charact.
[Studeny ’92]
Intersection Discussion – noninvertible elms.

Discussion – cont’d



Graph Separability  Independence
Completeness
Seems that
–
–
–
–

Trivial Independence
Symmetry
Weak Union
Intersection
Strong Axiomatic core for Independence
Applications