SEVERAL NEW PROBLE1'1S OF THE THEORY OF MARKOFF CF.AINS
by
V. Romanovsky
Institute of Statistics
Mi.meo o Series No. 102
M9.y, 19.54
Paris
1938
QUELQUES PROBLEr.1ES NCUVEAUX DE LA THEORIE
DES CHAINES DE IJrARKOFF
Par
v
0
Romanovsky
ACTUALITIES SCIENTIFIQUES ET INDUSTRIELLES
No.
737, PPo 43-66, 1938, Paris, France
Translated by Dan Teichrow under the supervis ion of Imogene Riddick, Instructor
of Modern La~O'Uages at North Carolina State College of the University of North
Carolina, Raleigh, N. C. at the request of the Department of Experimental
Statistics, North Carolina state College. (Edited Jointly by the Translation
Service and the Deplrtment of EXJ;Brimental Statistics.)
May
Paris
17, 1954
1938
SEVERAL NEW PROBLEMS OF THE THEORY CF MARKOFF CHAINS
v.
ltezna.r,oTsky
The object of IT\Y article is to indicate several new directions which have
been pursued in the theory of Harkoff chains and to state several results
I have obtained in my research in these directions.
~hich
I am going to consider cor-
related chains, cyclic chains, and several statistical problems which are related
to probabilities in a chilin.
I.
1.
Correlated Chains
So far as I know, no one has yet considered sequences of random variablE;s
which are correlated and subject, at the same time, to a law of probability in a
chain.
But the problems which arise in the case of such sequences are not less
important than the other problems of probabi1it;ies in chains and they must be
dealt with in order to complete the theory of
the former.
To specify and simplify my article, I shall consider only the following case.
Let
be an infinite sequence of random variables connected in a simple 11arkoff chain and
associated ""lith experiments number 0, 1, 2, ••• respectively.
in the experiment number h, the corresponding variable n
h
tve shall suppose that,
can take only one of the
different values xl' x 2' • u, x n which remains the sam8 for all the experiments and
..,
have the probabilities POI' P02' ... , Pon' ~ Po..<.
= 1)
in the initial experiment,
number 0, and the transition probabilities I)~ are defined by the equations
I
, I;
..l/i...LA
,.,
(.,(,p
= P( 11.n+1 = x ~ I 11.n = X.,(
= 1,
2, ••• ,
u;
h
)
= 1,
2, ••• ),
'lr1here the symbols on the right hand side designate the probability that the v-ariable
~
+ 1 takes the value of x
B
when it is known that
~
has taken the value x.,(.
-2-
il
~ .. II¢.,(~
Thus the matrix
represents the law of the chain being considered,
more briefly of the chain Cn which connects the variable
~.
Desigilate (I) further by Pk/~ the probability P{uk • xp) calculated under
o'
the hypothesis that the values of all the other variables u
~+l' ~+2'
up ••• , uk_I'
••• , are indeterminate and consider a second infinite sequence of
chance variables v o' vI' v 2 ' ••• , associated to the same experiments and
connected to the variables uh by a correlation which is defined by the equations
rk'~"'"
(1)
where
(~
= 1,
!
P{u k .. x~,
2, •••
,n;
v k '" Yr ) c(l\:/f3)(-i/'f3r)
~
= 1, 2, ••• ,
m;
k .. 0, 1, 2, ••• ),
r
~ ~( are the conditional probabilities, the same for all the experimentE of
the equations
v k .. Y~, the condition being that uk ... xf3' and where Y1' Y2'
••• , Ymare the values which can be taken by each of the variables vo'
vI'
v 2 ' ••• , in the experiments considered.
Let
'f
=
II yrp?f It •
of the sequences
(~)
The matrice s
q> and Ifdefine
the law of correlation
and ( vIi) or of the chain Cuv • We shall study several of
the most important properties of this correlated chain.
2.
First of all we can propose the following problem:
of the chain C which connects the variables vh ?
v
For greater clarity we shall consider on the simplest
case where ~ is a pDimitive and indecomposable matrix.
what is the nature
~jd
most important
In this case its
characteristic equation
~(A)
(2)
(1)
...
I>.E-~I·o,
I will use the notation of my article in Acta Mathematica, Vol. 66,
Which will be cited below as A.M.
-j-
has aJfAa
==
I as a simple root, the absolute values of all the other roots of
(2) are < I and the probabilities
Pk/~
have for k ...,. 00 the determinate limits
p~ which represent the limiting probabilities of the equation ~
==
x~(l).
Let
then obviously
(3)
Likewise, if we designate by
~+k == ~
(4)
¢~} the probability of the equation
when it is known only that
t %) ; p( vh k
+
Further, let
the equation
==
yy
llh ...
f ~ ...
x.,(,
we shall have
xc(} ...
~ ¢!;)t~~
~
·
nJ~} be the transition probability of the equation vk_1
v
k
==
y~ to
I
0<
They satisfy the relation
== y~.
(5)
and can be obtained by the following procedure.
We have equations
a
qk-l
qk-l
P2
= P(u k
Pj : p(
~
... X(1' u k _1
...
Y1r'1
==
X) =¢ rJ,(1'
uk'" X(1} ...
r(1¥' '
of which the first is obtained by Bayes Theorem.
I~
'f~
-4It is clear that
(k) _
n~~
:
whence
(6)
n(k)
~ 1'"
~Pk-l\~Y-:13¢ 1~
0:
qk - 1 f ~
.<, (J
The probabilities n~~ define the law of chain
geneous and multiple, because
•
.J..(J '(J
If
Cv.
This chain is not homo-
nJ~ depends on k and the probability PI changes its
value according to what is known about the values of'k '-2' u
quentlyof the sequence,
k-3'
••• and, conse-
v k _2 ,
v k _ , ••• It becomes homogeneous and remains
3
multiple if the chain Cu is stabilized because then (1)
for all k and the probabilities
(7)
do not depend on k, but the information on the values of the vk_2 ' v k• ' •••
3
can be such that certain values of the uk_l become impossible and the sum on the
right hand side changes.
3.
For k 4'00 the correlation of the sequences
limit a well determined limiting correlation.
indecomposable and primitive matrix, ...j:
lim Pk I ~ ""
p~,
lim qk' yo "" q ('
lim r k ,~"(
II
r~ r
0:
•
and (vh ) has for a
In fact we have in our case of an
r
p~ t~ ~
(~)
'P~'!~ Y"
,
-5and the various moments of chain C have for their limits the corresponding moments
uv
of this limiting correlation, which remains a correlated chain.
4.
Let us pass to the important problem of the limiting law of the proba-
bility of the sums
We have for k
5-1
~ u.
h=O n
8-1
and ~ vh •
h=rQ
-.,.00 =
Eu k
=
and we shall consider the sums
s-l
~
,
~uh
=
.~
~ (uh - xO)
h=O
We can calculate the moments of the sums ~ u~ by the method which is explained
in detail in A.M., and having the moments of the sum ~ v ~ .we notice that
(7)
where
-6-
and](~)are the coefficients of the development of ¢ ~) according to powers of
~ = 1, ~, ••• , A~_l of the roots of the equation !(A) = 0, supposed for the
greatest simplicity, to be all distinct.
moments of the quantities v
We see by these relations that the
I
are obtainable by beginning with the moments
h
corresponding to the u' and replacing p ~r(k) and lk(g) by q
X(k)and x(g)
h
~'~~~
T4
'It'
~~
~'It
respectively.
We see further that
•••
•••
••• +kr )
•••
where t
= Y - YO. For the effective calculation of these moments we can apply the
procedures given in A.M.
(~k2 •••
of the moments N m ffi
l 2
that the structure
••• k )
r
••• m
r
••• m
r
In particular their asymptotic properties are identical.
We find, for example
(9)
where
(10)
is the
studied in A.M•
-7This value is entirely equivalent to that of the dispersion of ~u·~ I
(11)
(z-x-x).
o
The calculation of the moments of the products of integral and positive powers
of the sums
.~
I
~ ~
and
'X_
~v
difficulties in principal.
I
h is naturally more complicated, but does not present any
We cite only the moment for the product
(12)
where
(A ) ...
g
~p V(g)
"
(A ) ...
g
~ PI3' f~~~2lhl (f
z132 tlJ'
~p
1J!13
I
~ll
(13)
~ (Ag'~) ..
I
1fI13 2
~ll
~ (Ag'~) ...
~l ~1~2
z
ll'
yr(gLy(h)
0-< . ~l
13 113 2
~P
OJ.
reg) yr(h)
-«31
~1132
t
PI ~
,
,
z t
,
2 cr ~l ~
lfi13, ~
z
t
13 2 ~ ,
~u~ ~v~:
-8sA.
vl (A.g )
(13) continued
v2
~ (1 -
(A.g'~)
...
= r .gx '
g
A. S)
erg. A ) •
g
(1 - ~)
~ (~s
_ A S)
(1 - ~) (~g- Ag )
,
the sums on the right hand sides being taken over all the values of the greek
letters under the corresponding ~ sign.
With the aid of the formulae written above we can easily calculate the correlation coefficient of the sums ~ u~ and ~v ~ according to the general relation
r .. - - - - - - - - - •
Its final value, for k-t 00, is also obtained immediately.
5.
It is not difficult to write the generator oZ all the moments of the sums
~u~ and ~v~ it is
(14)
rI.
'P
.~
r.
s ... ~ PO-< -<)t"
u
¢
.J
1IfR , 't
~R1
iP
l
¢A
•••
~
RAt
'"'s-2'"'8-1
~s-l
8-1
e
iQ(z. + ~ ZRh)+i't"(t!+ ~ t J!
oJ..
..,
and satisfies the recurrence relation
(15)
where
(16)
>eiQ(ZR
•
"'s-h
+.,,+ZR
"'s-l
)+i't"(t~
s-h
+ ... +t~
)
6-1
-9-
(16) cont1ed
•
The relation (15) gives us
and we can show (1) that
lim s -¢s
S-"'<D [ ¢s-l
... lim log
¢ s'" - ~1 (92
2
+ 2rQ1: + 1: ),
S~CD
i f we set in (14)
Therefore,
(17)
lim
¢ s "'e
and we see that the correlation of the sums
.z % and
:::-,
I
.~
I
~ vh has for its limit
the normal correlation; r is the limiting value of the correlation coefficient
defined above.
(1)
My proof is not yet rigorous and it. is a litt,le long, tharp-fore, I have
omitted it here.
-10-
6. We have considered only the simplest case of correlated chains
characterized, in particular, by the fact that chain C
C'v.
by
But we can form chains where P(u k+1 ..
x~)
u
~ich
is
is independent of chain
depends not only on the value taken
lok-'
but also on the value of v k •
A still more general case is obtained when we suppose that the probabilities
of the simultaneous equations u k+l .. x.,(J v k+l
' \ and
11 simultaneously.
k
to formulae (1) - (17)
~ich
IS
y~
depend on the values taken by
I have calculated the fundamental formulae equivalent
are shown above for the most general case of corre-
lated chains, but I omit them here because of their complexity.
They do not
present any essentially new particulars.
II.
Cyclic Chains
1. We call a cyclic chain of indElt k a chain in which the law by an identical
permutation of lines and columns can be put in the cyclic form
-T"
(1)
j
.
0
L12
0
•••
0
0
0
~3
•••
0
• ••••••••••••• t
0
••••••••••••••••••••••
0
0
•••
0
0
•••
where L , L , ••• , L are the non zero sub-matrices, all other sub-matrices
12
kl
23
The matrix is
L~ being zero (111' ~2' ••• , L
kk should be square matrices).
said in this case to be a cyclic matrix of index k.
Let there be given a stochastic indecomposable matrix.
Then in order to say
that it is cyclic. of index k, it is necessary and sufficient that we be .able to put
-11it in the form (1) (A.M., p.165).
The simplest means of demonstrating this theorem
consists in considering the cycles of the indecomposable matrix in question, that
is, the sequence of non-.ero elements,
¢ 4 1 ' ¢ ~1~2' ...
It is evident that, for the matrix
have their orders divisible by k.
which form the cycles.
i
J
¢ ~s-l-<
presented in the form (1), all the cycles
The order of a cycle is the number of elements
Inversely, if all the cycles of an indecomposable matrix
have their orders divisible by k, we can state that it is cyclic of index k.
This last assertion is demonstrated in A.M. by the aid of the characteristic
equation of the matrix considered.
But we can give a direct and simple procedure
for reducing a matrix to cyclic form.
We shall describe this procedure because
it is new and useful.
It is simplest to consider an example of the procedure in question, which will
not diminish the generality.
Let us take for example, the following matrix;
A ..
0
1
0
0
0
0
0
0
1
0
1
0
1
0
0
1
0
1
0
1
0
0
0
0
1·
0
0
1
0
1
1
0
0
0
0
0
1
0
1
0
where the non-zero elements have, for simplicity, been replaced by unity.
To
simplify the writing we shall write the cycles and merely indicate -the indices
-12vf their ,;:lements.
For example we shall write the cycle
12, 23, 31.
We can see without difficulty that the cycles of A are the follol<.ring:
12,
12,
12,
12,
12,
12,
23,
23,
23,
25,
25,
25,
31
34,
36,
51
54,
56,
17,
17,
17,
17,
17,
17,
47, 75, 51
67, 75, 51
47, 73, 31
67, 73, 31
All these cycles have orders divisible by k
a
73,
73,
73,
75,
75,
75,
31
34,
36,
51
54,
56,
42, 25, 51
62, 25, 51
42, 23, 31
62, 23, 31
3 and we shall separate their first
indices into three groups
I:
1, 4, 6
II&
2, 7
IIH
3, 5.
In group I, one finds the indices 1,
4, 6 -
the first indices of the first
and fourth pairs of indices in the table above; in group II one finds the first
indices of the second and fifth pairs, and so on in the sequence.
It is now evi-
dent how to proceed in the general case.
Notice that to exhaust all the indices 1, 2, 7, in the groups I, II, III, it
is sufficient to apply the procedure described in the three first oyc1es.
There-
fore, we can diminish very considerably the labor of searching for the cyclic forrE
of a matrix if we write at the same time as the eyc1e, the partition of the first
indices in groups, the number of groups being defined by any means whatever.
-13-
Take now the permutation of lines and columns of the matrix defined by the
substitution
C
s ..
2
3
6
4
4
5
6
2
7
3
~
Then the matrix A will be put in th'e desir..e.C1 cyclic form;
A'"
2.
Let AI'
~,
0
0
0
1
1
0
0
0
0
0
1
1
0
0
0
0
0
1
1
0
0
0
0
0
0
0
1
1
0
0
0
0
0
1
1
1
1
1
0
0
0
0
1
1
1
0
0
0
0
... , An be independent events which we can observe in an
infinite sequence of experiments connected in a simple cyclic chain of
and let the matrix 0 of the equation (1) be their law.
be the number of lines of the L12 , ~3' ... , L K1 of~.
fulfilled, one can divide the events A
.t..
(2)
,~.
Let also t , t 2J
1
+ ••• +
#
~k-l
~
,
These conditions being
.
......
.
~
,
.
\
l' ••• , h-n )
+,
of which the fundamental property is the following.
The initial experiment can
lead us to any of these groups, say B.t..' but the succeeding experiments must
necessarily and consecutively lead us into the groups,
(3)
k
into k groups,
. . . . . . . . . . . . ..
Bk = (~#~1 + ~2
#
ind:ex~
-14the succession Bl , B2 , ••• , Bk being repeated without end. One can say that the
succession (3) of groups (2) is entirely determined by the results of one initial
experiment.
Now we can construct the cyclic law such that the events .1-'0 can still be
divided into groups, the succession of these groups not being entirely determined
by the initial experiment_
~,
. 1
...
Such is for example, the law
~
0
~
0
0
0
~3
0
0
0
L31
0
0
0
0
0
0
0
0
0
L
45
0
0
0
0
0
0
r,6
L61
0
0
0
0
0
Let t l , t 2 , t3' t , t , and t 6, be the numbers of lines of the~~Ia23~L.jllI,45,L
4 5
and L61. We ahall now divide events A.,l, into 6 groups Bl' B2 , ... , B6 including
t l , t 2 , ••• , t 6J events ~.,l, taken in the order of their indices as above.
Consider now the experiments subjected to the law ¢l. Suppose that the
initial experiment leads us into group Bl •
It is clear that the following experi-
ment can lead us either into group B or group B - In the first case the experi2
4
ments number 2 and 3 will necessarily lead us to groups B and B and in the second
l
3
case the three experiments numbered 2, 3, 4, have us pass through the succession
B , B , Ble
6
6
Thus we shall have either the cycle
Bl' B2 J By Bl
or the cycle
-15In an infinite number of experiments these cycles will be repeated without
end, but in an order regulated by chance.
We notice immediately that group B has a particular property; from this
l
group, we can pass either to group B or to group B , whereas all the other groups
2
4
B always lead us into a single determined group. Therefore, group B is a critical
l
group, a group of indetermination, a group of branching of cycles.
It is the existence of cyclic laws similar to
Jl'
the polycyclic laws, which
I should like to draw to the attention of those who are concerned with the theory
of Markoff chains.
These laws can be very complicated and show us how complex
the phenomena of the real world can be, :: e.riori. We can expect that their study
will aid
3.
US
in the understanding of these phenomena.
Of the numerous problems which are raised by the notion of a polycyclic
law of probabilities in a chain, we shall stop only for the following two'
(a)
To recognize whether a given law is polycyclic;
(b)
To study the behavior of the probabilities
Pk/~ and ¢
%) for k
-+ 00.
To simplify and shorten the article we shall consider only a few examples of
these problems which will nevertheless be sufficient to show the general methods
for their solution.
(a)
Let us take an example of the first problem.
Alii
0
1
1
0
1
0
1
0
0
0
0
0
0
0
0
1
1
0
0
0
0
0
0
0
1
0
0
0
0
0
0
0
0
1
1
0
1
0
1
0
0
0
0
0
0
1
0
0
0
0
0
1
0
0
0
0
0
0
0
0
0
1
0
0
0
0
0
1
0
0
0
0
Take as given the matrix
-16where to simplify the writing, the unit element is substituted for all non-zero
elements.
To determine whether it is cyclic, we shall look for its cycles.
These cycles are
1,3
3,2
1,3
3,10
1,4
4,2
5,3
3,2
2,9
10,9
2,9
2,9
1,6
1,8
5,6
5,8
9,1
9,1
9,1
9,5
6,7
8,7
6,7
8,7
7,1
7,1
1,1
7,1
etc.; as above we indicate only the indices of the elements of these cycles.
see two groups of cycles:
of three and of four elements.
We
Therefore, we can
suppose that A is bicyclic and take as the first indices of the cycles the six
following groups:
I
1,5
II
III
2,10
3,4
IV
V
VI
9
6,8
7
Now, by making in A. the simultaneous permutation of lines and columns defined
by the transformation
5
s ""
2
3
3
4
4
7
6
8
9
2
10
9
5
6
8
we obtain for ! the form
o
o
0
1
1
o
o o
1
1
0
1
1
o
o
o
1
1
o
0
o
0
1
1
0
o
0
o
0
1
1
o
o
0
0
o
o
0
o
o
0
1
1
o
0
o
o
0
o
0
o o
o o
o o
.0
0
o
o
1
1
o
o
0
o
o
o o
o o
1
o
I
o
o o
o o
o o
o o
which places in evidence its hicylic nature.
0
0
0
0
0
0
0
-17-
..-
The events AI' ~, ... , ~o' subject to a law ~ which has the form of A*,
are divided into groups
Bl • (AI' ~), B2 • (A3' ~), B3 •
B4
...
5 "" (Ae'
(~), B
(As'
A6 ),
~), B6 • (AIO )
which will succeed each other in two cycles B , B , B , B , B and B , B ' B ,
l
2 3 4 l
l
S 6
Bl (1).
(b)
Let us pass to the second problem and consider only bicyclic laws.
such a law events
~, ~,
For
••• , An are divided in general into groups
••• , B
m
which, in the experiments succeed each other in two cycles
(1)
We can still utilize the examination of the characteristic equation of the
chain considered for the solution of the first problem.
If, for example, the chain
is bycyclic and its cycles in B have orders k and t its characteristic equation is
necessarily of the form
d.·(1)
"" >.,r ~ C il.gk+ht "" 0,
g,h
where r ... Constant and g, h are integral, positive numbers such that gk +
h.e~n
- r.
Therefore, if the exponents of the characteristic equation of the chain considered
have the form r + gk + ht, we can conclude that we have a chain which is bicyclic
and whose cycles in B have the orders k and
-e.
That does away with the necessity
of looking for all the cycles formed by the elements ¢
-43' for us.
-18-
and
(5)
for example, the orders k and t • m - k + 1.
If k and
t, or, more generally, the
orders of the most numerous cycles (when a law is polycyclic) and similar to
cycles
(4) and (5), do not have a common divisor and i f the matrix corresponding to ~
is not decomposable, we easily see that §< does not have roots wi. th absolute value
1 other than ~
p~
= lim
Pk/~'
babilities
1.
c
Therefore, in this case, the final probabilities of.~,
will exist and be determinate, their limits for the transition pro-
¢~)
depending only on
~.
We can prove this proposition by using the
well known formulae and representing the probabilities
of the roots of
Pk/~
and ¢ ~~) as functions
i.
On the other hand, if the orders of the cycles (4) and (5) have a common
divisor, say v, the matrix
p will necessarily have among its roots the roots of
the equation >..V - 1 = 0 and the probabilities
k -+
00,
Pk/~
and
¢
5;) will not have, for
a unique limit, but will have different limits depending on the initial
PO~
probabilities
and the values taken by k.
For example, for the bicyclic law
j
we have~· (>..)
=
= >..2(>..
_ 1)
0
a
0
0
b
0
0
0
1
0
0
0
0
0
0
1
0
0
1
0
0
0
0
0
0
0
0
0
0
1
1
0
0
0
0
0
(J
+ >..2 + >.. + a) and
,
a +b
co
1,
o < a < 1,
-19¢(k) .,.
~
1.<{3 (1)
II (1)
where AI'
}.~@ (A)
+ D[
]
(X - 1) (}.) + }.2 + }. + a)
~, ~,
}.
=0
are the roots of the equations
}.3 + >..2 + }. + a ... 0
By calculating the minor~(}.), we shall verify at once that the limiting
values of
p ~)
are
¢ (00) ... ¢ (00) = ¢( 00) ...
.,(,2
.4
.,(.J
¢
(00) ..
~5
¢
Pk/~
•
3a + 2b '
b
•
- 1 + 3a + 2b '
.,(,6
= 1,
(~
The lim
(00) _
a
1 +
2, ••• , 6)
has the same values.
Again let
1=
0
a
b
0
0
1
0
0
0
0
0
0
0
1
0
0
0
O' 0
1
1
0
0
0
0
J
a +b
=1,
0< a
< 1,
We now have
4
--;t
I (}.) .,. }.(A -
and
¢(k)
..q3
..
2
a).
-
~
f (1)
b) ... }.(A2 - 1) (A2 + b)
+
_.~-l)ki='43(~l)
_f ( - 1)
}.k~~~ (}.)
+ D
(}.2 _ 1) (}.2 + b)
-20-
wh~re Al and ~ are the roots of the equation A2
Therefore,
(k)
¢
r-_
+ b
:s
O.
.:t.,(~ (1)
~ (1)
043
and the n(k) have the values given in the table below where
043
1 + (_l)k
..
' 1 _ C-l)k
= 1 + a + 36
1 + a + 3$ and Qk
Qk
Table of Values of
~
1
2
1
Q
k
aQ
2
Q
k
3
Q
k
aQ
4
Q
k
aQ
Q
k
aQ
5
t
t
,
aQ
3
t
k
k
k
I
k
k
bQ
bQ
bQ
bQ
bQ
t
k
k
k
,
k
k
n~)
4
5
bQ
bQ
bQ
bQ
bQ
bQ
k
t
k
t
k
k
t
k
bQ
bQ
bQ
bQ
t
k
k
k
,
k
k
It is evident that, in the present case, the values of the limits of the
¢ (~)for k ...,. 00 are different according to whether k passes through even or odd
.,(1"
•
or indeterminate values when k takes all the integral values.
Besides, the two
systems of limits of ¢ ~)for k increasing by even or odd values depends on
therefore the asymptotic values of the
Pk/~
which are equal to
are oscillating and depend on the initial probabilities.
.J..,
-21-
III.
1.
Statistical Problems
The statistical problems connected with the Markoff chain have not been
considered up to the present though they appear quite naturally when dependent
experiments are studied.
In fact, the simplest hypothesis about the nature of the
dependence which regulates the experiments considered, consists of supposing that
it represents a simple Markoff chain.
The first of these problems is to establish the law of the assumed chain,
that is, finding the unlmown probabilities ¢.,(~ of the chain
frequencies of the events AI'
~,
... ,
~
I
by observing the
in the experiments examined.
This can
be done in two ways.
a) We can observe N series of which each one consists of s experiments.
$ny
one series, number h, will be divided into partial series
~(h)
(h)
SAl'
of observations "after
u~' .•• ,
A:1.",
"after
S(h)
A
r
A:2" ••• ,
II
after
~" •.
be the number of observations which constitute
the repetition of the event
~
in this series.
is an empirical value of the probability ¢ 4'
The most probable value
of ¢ co<l3 is defined by the approximate equality
~ s(h)
do
'fI
.,(~
h 4
=::::---
~ (h)
~ s.,(
h
We well know how the precision in this equality is estimateq and we
shall not insist on this point.
We only note that we can judge the quality
of the adjustment obtained by comparing the numbers
s~), ~ = 1, 2, ".J
n,
-22with the numbers s,.(,
¢043' ~ '"
1, 2, ••• , n, s.,(, '"
of the test of K. Pearson
i .
(h)
~
~
(
_
ne'
8
~ s~h) , with the aid
h
)2
c43
...__,.(,..:.,,:-'P_c43
......_
S.,( ¢.(~
-.;8
b) The first procedure gives"tB series S(h) with different lengths, which is
A,.(,
To avoid this inconvenience, we can observe N
~n~nient in practice.
series such that in each of them, every partial series sih)iS of the same
length,
which we can obtain, for example, by keeping
5,
with only s observations.
e~ch
h
series si )
Go(
The series having been obtained, we proceed as
above.
Let us say several words on the determination of the limiting probabilities
p~
a
lim
Suppose N = KL series, consisting of K partial
Pk/~.
series with L sequences of observations each have been observed.
,
these partial series by Sk' k
a
1, 2, ••• , K.
,
Designate
In Sk' we find L sequences
each consisting of s observations; let n(k) be the number of cells of those
~
sequences which terminate by the observation of the event
.z
~
~J
we have
n(k)= L.
~
n(k)
7-
is evidently an empirical value approaching the probability Pk/;3':
the most probable value of this latter is
.~
(k)
fn~
•
K1
2.
kn important problem which can be set for a series of experiments whioh
are supposed to be in a chain is to recognize whether the chain is simple or not.
We can solve it as follows.
-23Let us designate by
p~~.J
the probability of events 11",/ 11 ll"
~
in the experi-
ments number k, k + 1, k + 2 respectively; evidently
(1)
P
( ~) = p.
k/~
kj""
rA.
d..
lfJ"" l'
lfJ
113 •
Let us now consider table I in which
Table I
~
~
•
•
0
~
p(l{ )
k , .p
~
(t)
Pk I 11
p( <r)
~
K
(t)
P k I nl
0 0 0
••
p( ~)
kin.
(~)
Pk I nn
0..
p( ~ )
k , .1 • o. P ( 't)
k I on
p( ~)
k I.p
=~
P
=~
""
(~ )
Pk
I""p
p( ( )
k
= Pk
I ..
~
I "" .,.( ~ ,
("IS) .p
Pk I ~
k+lf~
p( ~) _ ~ ("If)
k , •• Pk I c43
""'P
1.,(,.
p( ~)
k 11•
•
('()
P k I In
.................
p( (l')
k I aI,,.
(2)
o••
=
is given.
g~'
Pk+ll ~ •
We lmow that, in a simple chain, events A"" and
A~
¢
~
should be independent when
Therefore, the coefficient of contingency of table I,
(3)
where
2
(4)
-24
should reduce to zero for a simple chain, which we verify without difficulty for
6~i~; 0
in consequence of (1) and (2).
We now see how we would proceed to verify the simplicity of a chain from the
observations.
We must construct s series each one consisting of three observations
in the experiments number k, k + 1 and k + 2; let us call such a series a k-triad.
Of these sk-triads, there are S ( 'lJ) which may have A ~ in
and among these there may be
th~
(k + 1
s!;l) which give the sequence A...<., h
(J'
ret experiment
~.
Therefore,
for s large, we can write the approximate equalities
s(-a')
..g?
'""'"'p ( ~ )
s
-.. "k(-43 '
=
~ s ( (f )
~
~
'""'-'
,....,
sp (
(f )
kf-<..'
from which we deduce:
(5)
Therefore, in the case of a simple chain, we should have approximately
from which also
-25where K is the empirical coefficient of contingency constructed for our observations.
By the well known formulae of the theory of contingency we can estimate the precision of equations (6) and, in this manner, render our conclusions on the nature
of the chain considered more exact and more solid.
We can proceed further.
The observations which we have described just now can
be summarized by table II which correspond to table I.
Now if the chain considered
is simple, the
Table II
~
( If)
(
Al
sll
•
• • • • •
•
•
(
An
~
snl
(~
s ( is )
·~
s. I
)
)
s ( lS )
A
n
• • •
Al
.,t, •
~)
s ( l5 )
• • • sln
• •
1 •
·• .
• • • •
•
s(
•
s( ?f )
n •
ZS )
nn
s( ~)
• n
• • •
s
( CS)
rows and columns in table I are proportional as we can verify immediately.
There-
fore, if our observations relate to such a chain the rows and columns of the table
II should be approximately proportional.
This proportionality can be estimated
as follows.
We construct the sequence
,
of which the sum is
~
= 1,
2, ••• ,
s(~). In this case the criterion
_( ()
(
2=~S-43
X.,t,
~
_
(?( ) )
s.p
.
s ( ?f)
~
2
n;
-26and the table of Elderton of the corresponding probability P (we take n' ... n to
enter this table in our case) willallow us to estimate the degree of accordance
-((5 )
of the sequences s-<13
and s~ ~), ~ ... 1, 2, ... , n, for any.,(.
If we calculate
and enter this table of Elderton with
n' ... n(n - 1) + 1
we whal1 find the probability p which estimates the simultaneous discrepancy for
all the sequences
;~ ), ~
... 1, 2, ... , n; -< ... 1, 2, ... , n
and
( 2)' )
~ , ~
s.
... 1, 2, ••• , n
(See R.A. Fisher, Statistical Methods for Research Workers, 1934, p.104).
Notice that for a chain which we can suppose stable, the triads can be formed
n~~bers
by the results of the experiments
1, 2, 3; 2, 3,
4;
and so on.
In the
case of a non-stable sequence, the number k for the k triads can be any whatsoever,
but is fixed by the entirety of the triads considered.
Finally we can apply a third method.
Among the consecutive triads, for ex-
ample formed from results of the experiments 1, 2, 3; 2, 3,
those which contain A"6'
bility ~
for
A~,
~
4;
etc, we will choose
being fixed, and will determine the empirical proba-
<;:) of the A~, ~ ... 1, 2,
... , n, in the last element of our triad. and
-< ... 1, 2, ••• , n, in the first element. Then, if the observed chain is
simple, the quantities
¢( (J
1
~
should be approximately equal.
)
,
¢«f)
2
~
, ••• ,
¢('If)
n
~
- ( If )
The probabilities ¢co<j3
will be more precise if
-27we determine them by the means
n«?f) ..
'P
4
!
.e
~
f
;(Cn
'P..<I3 i '
¢~~)
being calculated to divide the partial sequences of triads. In this
case we must have tn2 in such partial sequences. The agreement of results obtained
the
with the hypothesis that the chain in question is simple is verified as follows.
The numbers
~(o )
m( t) .. ~
~ m 'P..<I3
,,,( .. 1, 2, ••• , n,
i
4
~
where m is the length of the partial sequences supposed equal, should be approximately equal; by estimating their agreement by one of the well known methods, we
will judge if our hypothesis is admissable or not.
3. Let us consider, in conclusion, a new notion which can be useful in the
theoretical or empirical examination of Markoff chains, the notion of the rigidity
of a given chain.
We call rigidity or the coefficient of rigidity of the chain
1: =/I ¢ ..(~ 11 the
number
(7)
which can also be written in the form
(8 )
This number depends, in general, on the number k, that is, on the place in
the infinite chain considered; they are not equally rigid throughout, but they
become more and more uniformly rigid for
determined limiting probabilities.
constant and equal to
(9)
k~ 00,
i f the chain considered possesses
If the chain is stabilized, its
rig~dity
is
-28-
The name given to the number K is justified by its following properties.
k
(a) For the independent events ~, ~, ... , An subject to the law
we
j,
have
K :: O.
k
We can verify in this case this equality, either directly or a consequence
of the general theory
contingence, because we perceive that K is nothing else
k
than the coefficient of contingence of events A,,( in the experiments number k and
~f
k + 1.
(b) We always have K ~ 1, and Kk"... 1 is possible only for the A..< connected
k
by an absolutely rigid chain, that is, one such when the initial experiment, which
is regulated only by chanqe, is made and given us for example, A..<; we have neces-
$rily in the following experiments
the events AI'
~,
In fact, K
k
~
••• , An succeeding themselves in an entirely defined order.
1, because in general
... n,
therefore, Kk < 1. But if the chain is cyclic of index n (in this case it is
T
evidently absolutely rigid) we can put the law 5t in the form where
¢ 12
IS
¢ 2) ... ••• "" ¢ n-l, n :: ¢ nl ... 1
and all the other ¢ -<j3 are zero.
Then
-29-
r-,
1
.. 1 \I
/,
Kk = -m-l 1-·-
L.,(~
1
rPk + 1/2
= m-l LPKlI
n
1
n-.1.
n...1.
l/~
I
J
Pk + III
+ .. ·PkJn
1
~.J .. n-l
=..,-~=l
because Pk + If~ + 1= 1, if in the experiment number k we have
if we have ~ in the experiment number k, therefore
A~,
and Pk + 1: 1= 1,
,
... ,
(c)
Conversely, if K = 0, the events A are independent and if K = 1, the
chain ~ is necessarily cyclic of indice n, therefore absolutely rigid, as we can
easily verify.
In this way the value of the number K shows us, the more or less strong
k
dependence of events AI'
~,
••• , An in the experiments connected in a chain.
At
the same time K shows also a more or Ie ss apparent, more or less temporary succesk
sion of these events and represents in this wayan essenti21 characteristic of the
processes described by the chain considered.