Workshop Conditionals, Information, and Inference
Hagen, Germany, May 13-15, 2002
January 18, 2002
Conditional independence in Gaussian vectors
and ideals of polynomials
Franti²ek Matú², Prague
Abstract. The inference among conditional independences in Gaussian vectors
is related to computational techniques of commutative algebra. A general method
for proving implications among the conditional independences is presented. Some
know implications are discussed as consequences of the method.
In a nondegenerate Gaussian vector = ( ; : : : ; n), n 2, with the covariance
matrix A = (ai;j )ni;j , the variable is conditionally independent of given ( ; : : : ; n)
if and only if the (1 2) -element of the inverse matrix A; , the concentration matrix of ,
vanishes. Denoting submatrices of A by AI;J = (ai;j )i2I;j2J , I; J N = f1; : : : ; ng, this
is equivalent by [8, p. 11] to vanishing of the determinant jAN nf g;N nf g j. All conditional
independences within the vector of the form i ? j jK , where i; j are the elements of a
two-element set ij , K N n ij and K = (k )k2K , give rise to the nite set
hAi = f(ij jK ) 2 R ; jAiK;jK j = 0g :
Here R is the family of all ordered pairs (ij jK ) with ij; K N disjoint and expressions
like fig [ K are abbreviated to iK .
The construction A 7! hAi makes sense for arbitrary complex symmetric matrix. Under
regularity conditions on A the discrete structure hAi enjoys properties of conditional
independence; for example, we shall see below that it is a semigraphoid in the sense
(1)
f(ij jkL); (ikjL)g hAi ) f(ikjjL); (ij jL)g hAi :
When studying such properties a natural problem is to recognize whether for two families
K; L R the implication
(2)
K hAi ) L \ hAi 6= ;
1
=1
1
2
3
1
;
1
2
This research was supported by the grants A 1075801 of GA AV and 201/01/1482 of GA ƒR.
Keywords: Gaussian vectors, positively denite matrices, determinats, principal minors, conditional
independence structures, ideals of polynomials, radical, semigraphoids, pseudographoids.
Permanent postal address: Institute of Information Theory and Automation, Pod vodárenskou v¥ºí 4,
182 08 Prague, Czech Republic; [email protected].
1
2
holds for all A in a given class of symmetric matrices. We concentrate on the class of real
positively denite matrices and the class A of complex symmetric matrices with principal
minors nonzero, that is, with the determinant of AI = AI;I nonzero for I N nonempty.
The aim of this note is to present a necessary and sucient condition for validity
of (2) when A 2 A . In turn, this condition is sucient for validity of (2) when A is
positively denite. Known properties of the conditional independence in Gaussian vectors
are interpreted as special instances of the latter suciency.
Let R = C [aij ; 1 6 i 6 j 6 n] be the ring of polynomials with coecients in the eld C
of complex numbers and with indeterminates aij where ij N has one or two elements.
For a symmetric matrix A with its elements corresponding to the indeterminates, the
determinant jAiK;jK j, (ij jK ) 2 R, is considered for a polynomial f ijjK of R. An additional indeterminate bI is introduced for nonempty I N and the expression 1 ; bI jAI j
is considered for a polynomial gI of R[bI ]. Given a family K R, let IK be the smallest
ideal of T = R[bI ; ; 6= I N ] containing the polynomials f ijjQ
K for (ij jK ) 2 K, and gI
for I N nonempty. The notation f ijjK is extended to fL = ijjK 2L f ijjK , L R.
(
(
(
)
)
)
(
)
(
)
Theorem. For K; L Rpthe implication (2) holds for A 2 A if a only if the polynomial
fL belongs to the radical IK of the ideal IK.
Proof. Suppose (2) is valid for the matrices A 2 A . When the polynomials f(ijjK ) for (ij jK )
in K, and gI for I N nonempty vanish after substitution of complex numbers aij and
bI then the symmetric matrix A with the elements ai;j = aij has nonzero principal minors
jQAI j = b;I 1 . Therefore A 2 A , and, using (2), (ij jK ) 2 hAi for some (ij jK ) 2 L. Hence,
jAiK;jK j equals zero and the polynomial
fL vanishes by the same substitution.
(ij jK )2L
p
Hilbert's Nullstellensatz [1, p. 170] implies fL 2 IK.
In the opposite direction, incidence of the polynomial to the radical means existence
of polynomials h(ijjK ) and hI from T such that
fLm =
(3)
X
ij jK )2K
h ijjK f ijjK
(
) (
(
)
+
X
;6=I N
hI gI
for an integer m > 1. Now, K hAi for some A 2 A implies that fLm vanishes when
substituting
elements of A and the numbers bI = jAI j; to both sides of (3). Hence,
Q
ij jK 2L jAiK;jK j equals zero and L intersects hAi , that is, (2) holds.
1
(
)
To see that (1), involving two instances of (2), can be obtained from Theorem we resort
to the following identity
jALjjAikL;jkLj = jAiL;jLjjAkLj ; jAiL;kLjjAjL;kLj
which is valid for any complex matrix A and L N , k 2 N n L, and i; j 2 N n kL. In fact,
(4)
by continuity it suces to prove (4) only when AL is invertible. In this case the classical
adjoint BL of AL satises BL = jALj(AL); (see [8, p. 11]), and the left-hand side of (4)
1
3
equals
jALj Aik;jk ; Aik;L(AL); AL;jk = Aik;jkjALj ; Aik;LBLAL;jk =
;
;
Ai;j jALj ; Ai;LBLAL;j Ak;k jALj ; Ak;LBL AL;k
;
;
; Ai;k jALj ; Ai;LBLAL;k Ak;j jALj ; Ak;LBL AL;j
2
1
by [8, Theorem 3.1.1a]. Then, the identity (4) is a consequence of [8, Theorem 3.1.3].
From (4) we obtain
f ijjL = f ijjL gkL + bkLjALjf ijjkL + bkLf jkjL f ikjL 2 If ijjkL ; ikjL g
and, making use of it, (4) with j and k switched entails
(5)
f ikjjL = f ikjjL gL + bL jAkLjf ikjL ; bL f ijjL f jkjL 2 If ijjkL ; ikjL g :
Thus, (1) is established for A 2 A .
Along these guidelines three known properties of conditional independence can be
proved. Writing (1) as f ikjL f jkjL = ;jALjf ijjkL + jAkLjf ijjL we see that ff ikjL ; jkjL g
belongs to If ijjkL ; ijjL g, thus
(6)
f(ij jkL); (ij jL)g hAi ) (ikjL) 2 hAi or (jkjL) 2 hAi ;
cf. [6, (3.34f) and pp. 129-130]. Combining (5) and (5) with j and k switched
;
f ijjL bL jAjLjjAkLj ; jAjL;kLj 2 If ijjkL ; ikjjL g :
Due to (4), the expression in parentheses equals jALjjAjkLj and by reasoning as above
f ijjL itself belongs to the ideal. Hence, the pseudographoid axiom
(7)
f(ij jkL); (ikjjL)g hAi ) (ij jL) 2 hAi ;
is established, compare [6, (3.6e) and (3.6b)] and [3, p. 107]. Finally, the equality in (5)
implies also
(8)
f(ij jL); (ikjL)g hAi ) (ikjjL) 2 hAi :
(
(
)
(
)
)
(
)
(
(
)(
)
(
) (
(
)
)
(
(
) (
)
) (
(
)
)
(
(
)(
)(
)
)
)
(
)(
)
)
(
(
(
2
)
(
)(
)
)
Example. When n = 3, the implications (1), (6), (7) and (8) for A 2 A show that hAi
equals one of the following families ;, f(ij j;)g, f(ij jk)g, f(ij j;); (ikj;); (ij jk); (ikjj )g, and
R. All ve situations are possible for the positively denite matrices (" =6 0 suciently
small)
2
3
2
3
2
3
2
3
" "
1 0 "
1
" "
1 0 0
4 " 1 " 5 40 1 "5 4 " 1 " 5 40 1 "5
" " 1
" " 1
" " 1
0 " 1
and the identity matrix, respectively.
p
Remarks. 1. Having divided the polynomial fL by any Groebner basis of IK , fL belongs
to the radical if an only if the remainder of the division is the zero polynomial. Computer
implementations of algorithms for the division and construction of Groebner bases of
radicals are available, see [1, 2]. For applications of computer algebra in statistics we refer
to [7].
2
1
2
4
2
4
3
2
3
4
2. It is an interesting open question for which K R the ideal IK is radical. Note
that in the above consequences of Theorem we had always fL in IK.
3. The attentive reader will have recognized that proving (1), (6), (7) and (8) we could
avoid the assumption jAj 6= 0. It is not clear to us whether the polynomial gN and the
indeterminate bN are idle when proving (2) in general.
4. By Theorem,
for a family K R one can write K = hAi for some A 2 A if and only
p
if fRnK 62 IK . Therefore, this condition is necessary for the representation K = hAi by
means of a positive denite matrix A. A natural question arises whether the condition is
also sucient.
5. For recent progress on semigraphoids see [4, 5].
References
[1] D. Cox, J. Little and D. O'Shea (1997) Ideals, Varieties, and Algorithms. Undergraduate Texts in
Mathematics, Springer: New York.
[2] D. Cox, J. Little and D. O'Shea (1998) Using Algebraic Geometry. Graduate Texts in Mathematics,
Vol. 185, Springer: New York.
[3] F. Matú² (1997) Conditional independence structures examined via minors. Annals of Math. and
AI 21 99128.
[4] F. Matú² (2002) Lengths of semigraphoid inference. (to appear in Annals of Math. and AI )
[5] F. Matú² (2002) Towards classication of semigraphoids. (submitted to Discrete Mathematics )
[6] J. Pearl (1988) Probabilistic Reasoning in Intelligent Systems. Morgan Kaufman: San Meteo, California.
[7] G. Pistone, E. Riccomagno and H.P. Wynn (2001) Algebraic Statistics (Computational Commutative
Algebra in Statistics). Chapman & Hall/CRC: Boca Raton.
[8] V.V. Prasolov (1994) Problems and Theorems in Linear Algebra. AMS Translations of Mathematical
Monographs, Vol. 134, AMS: Providence, Rhode Island.