AUTOMATA IN ENVIRONMENTS
by
Jerren Gould
Department of statistios
University of North CaroZina at ChapeZ Hill
Institute of Statistics Mimeo Series #947
June 1974
JERREN GOULD.
Automata in Environments (Under the direction of
EDWARD J. WEGMAN).
Automata theory is the study of mathematical models of finitestate systems which admit certain inputs and emit certain outputs.
The
state transition behavior of previously considered models, such as
Turing machines, finite automata, and probabilistic automata, has been
assumed to be related only to the present input.
In this dissertation
we take the viewpoint that every system is situated within an environment which affects the initial state distribution and the state transition function.
The Rabin-Paz model of a probabilistic, automaton is
generalized so that the initial state distribution is related to the
initial configuration of the environment and that the state transition
function is not only related to the present input but also to the present
configuration of the environment.
Chapter 2 considers the case in which there is a deterministic
rule to specify the configuration of the environment at any time.
We
show that such a model is properly more general and more powerful than
the probabilistic automaton model.
Sufficient conditions are given for
an automaton with a deternrlnistic environment rule to be "simulated"
by a probabil istic automaton.
We a] so consider the behavior of automata
with environment sets that have certain metric or semi-group structure.
In Chapter 3 we consider the effects of small perturbations of
the environments on the system.
Conditions are given for automata in
environments to have stable state distribution functions.
We also give
conditions for the set of tapes defined by an automaton to remain unchanged for sufficiently small perturbations of the environments.
Finally, we consider the case of automata operating in random
environments; we assume that the realization of any environment is
governed by some probabilistic structure.
The behavior of systems with
various environment processes is investigated.
We encounter the dual
nature of the randomness involved in automata in random environments.
We have that the environments are random and that for each value the
environment assumes, the probabilistic state transitions are defined.
However, we find that there is a mean equivalent canonical representation
for automata in random environments which eliminates the randomness due
to probabilistic state transitions; for any automaton in random environments we can find another automaton in random environments with finite
environment set. deterministic assignment of the initial state. and
deterministic transitions which has a state distribution equivalent in
the mean.
ACKNOWLEDGMENTS
I wish to express my sincere appreciation to Professor Edward
J. Wegman for proposing this problem and for the illuminating suggestions he made during the course of the investigation.
I am
greatly indebted to him for his guidance and encouragement throughout my graduate career.
It has been a very valuable experience to
work under his direction.
I would also like to thank Professor W.L. Smith, the chairman
of my dissertation committee, for his helpful suggestions and for
his encouragement during my graduate study.
Thanks are also due to
Professors N.L. Johnson, G.D. Simons, S. Cambanis, G.A. Mago, and
W.R. Wogen for participating as members of my dissertation committee.
The financial support provided by the Department of Statistics
during my period of study in Chapel Hill is greatly appreciated.
The contributions of June Maxwell, who ably typed the manuscript,
are earnestly appreciated.
Finally, I wish to express my heartfelt gratitude to my wife,
Marsha, for being My Small Joy.
i1
TABLE OF CONTENTS
Chapter I:
1.1
1.2
1.3
Development of Automata Theory
The Automaton Concept ---------------------------------- 1
Finite Automata----------------------------------------- 3
Probabilistic Automata---------------------------------- 8
Chapter II:
Environments
2.1 TIle Concept of Environments for Automata----------------17
2.2 Automata in Deterministic Environments------------------18
2.3 Mathematical Generality of the ADE Model----------------20
2.4
2.5
2.6
Finite Environment Set----------------------------------2l
Reducing the Environment Set and the Induced Metric-----29
TIle Environment Set as a Semi-group---------------------32
Chapter III:
Stability Results
3.1
3.2
3.3
3.4
The Stability Problem-----------------------------------37
Stability with Tapes of Bounded Length------------------4l
Ergodic Conditions for s-Stability----------------------47
Response Intervals and a-Stability----------------------64
3.5
Round Off Considerations--------------------------------69
Chapter IV:
Automata in Random Environments
4.1
4.2
Basic Notions-------------------------------------------7l
Random Environment Sequences of Independent
Random Variables--------------------------------------75
4.3 Limit Results for Sequences of Environment Processes----77
4.4 A Canonical Representation for Automata in Random
Environments------------------------------------------81
iii
Chapter V:
Implications
5.1 Limit Results for Markov Systems-----------------------91
5.2 App1ications-------------------------------------------95
5.3 Conclusions and Summary--------------------------------97
Bibliography--------------------------------------------------100
Index-------------------------------------------------------- 102
Notation-------------------------------.---------------------- 104
iv
CHAPTER I
Development of Automata Theory
1.1
The Automaton Concept.
Automata theory is the study of the characteristics of automatic
devices.
For some time there has been research into the nature of alg-
orithms and the functions computable by following an algorithm.
Researchers in automata theory have nlade an effort to apply and extend
these results to more realistic models of physical systems.
The auto-
mata models have been motivated particularly by the behavior of digital
computers, but also by organic nerve networks, information transmission
systems, synthesis of man's languages, learning behavior, games between
automata, etc.
These models are constructed to investigate their
~~th­
ematical properties as well as to discover the characteristics and
capabilities of similar phenomena in the real world.
In considering the theoretical litnitation of the type of computation that could be carried out by an automatic calculating deVice,
A.M. Turing [18] constructed an abstract machine capable of following
a finite, consistent list of instructions.
The Turing machine is a
finite-state machine with an associated external memory.
The memory
has the form of a sequence of squares marked off on a linear tape.
tape is assumed to be infinite in both directions.
A Turing machine
has a finite alphabet of symbols and a finite internal state set.
machine is capable of five operations.
The
It can read a square of the
The
2
tape, erase a square of the tape and write a new symbol on that square,
change internal state, move to an adjacent square, and halt.
The
instructions to the machine are contained in a finite list of quintuples of the form: (present state, symbol read, new state, symbol
written, direction of motion).
The third, fourth, and fifth components
of the qUintuple are determined uniquely as functions of the first and
second components.
Thus, for any symbol read and internal machine state,
the next act of the machine is unambiguously specified.
Although
the tape is regarded as infinite in both directious, when the
t~chine
IiUn~)E;L
is started, the tape
of squares.
~~st
The Turing
De
bla~k e~cept
for a finite
machiC1e is started by placing the
machine on a square of the tape and designating an initial state.
The
output is considered to be the expression on the tape when the machine
halts.
Indeed, any computation that can be performed by a deterministic
automatic calculating device can also be performed by a Turing machine.
This fact led to "Church's thesis," ; the assumption that every algorithm
has a Turing machine which can execute it.
Thus, consideration of
Turing machines gave rise to results concerning the theory of computable functions and other equivalent logic systems based on algorithms.
The Turing machine model in many ways patterns the operation of a digital
computer.
"program."
input.
The finite state set and finite set of quintuples form a
Both systems are capable of accepting any finite amount of
The major drawback to the Turing machine model for the digital
computer is its unbounded memory capacity via its infinite tape.
This
enables the Turing machine model to consider the entire class of computable functions.
3
In order to simulate more closely the characteristics of digital
computers, the Turing machine model was amended by restricting
the external memory tape to be of bounded length.
This model.
called a linear bounded automaton, operates in the same manner as a
Turing machine except that the tape contains only a finite number of
squares equal to the length of the input.
to move beyond the bounds of the tape.
as the Turing machine.
The machine is not allowed
The new model is not as general
Linear bounded automata, however, are capable
of computing recursive functions that can be handled by a finite,
bounded memory, as in the case of digital computers.
1.2 Finite Automata
So far the discussion has centered on the transducer model of
machines.
That is, a machine acts as a transducer if its main purpose
is to generate an output sequence that is a function of the input and
the initial state of the machine.
However, for the purposes of this
paper we shall henceforth consider machines that act as acceptors.
Instead of producing an output sequence, acceptors simply answer "yes"
or "no" to questions asked of them.
The transducer model and the
acceptor model are equivalent in their computing capabilities.
We now formulate an acceptor model with
capacity.
a.p'o~nded
memory
This class of machines is called finite automata.
For finite
automata the questions take the form of finite sequences from a finite
alphabet; such a sequence of symbols is called a tape.
The machine it-
self shall be considered as a black box, capable of reading one symbol
at a time from the tape and then advancing to the next symbol on the
tape.
In order to maintain the generality of the structure of the model,
4
the concept of a finite internal state set is employed.
As with Turing
machines and linear bounded automata, this machine is capable of changing states based on the present state and latest input symbol.
With
finite automata the memory is characterized by the finite internal state
set and, hence, is finite and bounded.
To obtain the output, or answer,
from the machine its state set is partitioned into two classes, the
states for which the output is "yes" and the states for which the output
is "no."
The output is observed when the machine has read all of the
symbols on the tape.
Let us now define these notions of finite automata in a more precise
mathematical form.
let
.*
~
E be the finite input set, the alphabet, and
, the set of tapes, be the class of all finite sequences of
symbols from
that
Let
E,*
r.
Let us also include
A, the empty tape, in
*
r.
Note
together with the operation of concatenation forms a free semi-
t.
group generated by
We are now prepared to formalize the definition
of finite automata.
Definition 1.2.1.
is a system
A = (S,M,sO,F), where
internal states of
function of
A finite automaton (FA) over the alphabet
A),
A),
H
S
is a finite nonempty set (the
is a mapping
N:S
SOES (the initial state of
final, or "yes," states of
For a finite automaton
L
S
(the transition
A) , and
FcS (the set of
x
-+-
A) •
A the domain of definition of the transi-
tion function can be extended from
S x L
as follows:
M(s,A)
r
=s
VSES
M(s,xa)= M(M(s,x),a)
to
S x
t:*
by the recursion
5
The interpretation of
M(s,x) is elementary: it is that state of the
machine obtained by beginning in state
x
s
and reading through
tape
symbol by symbol, changing states according to the transition
function.
Definition 1.2.2.
M(SO,X)EF
is called
the automaton
T(A), the set of tapes accepted, or defined, by
A = (S,M'SO,F).
Definition 1.2.3.
A.
If
T,
The class of all definable sets of tapes,
T(A), for some finite auto-
is the collection of all sets of the form
maton
XEL * such that
The collection of all tapes
VET, then V is called a regular event.
In order to characterize the definable sets of tapes, we must make
the following definitions.
Definition 1.2.4.
An equivalence relation E
of tapes is right invariant if whenever
Definition 1.2.5.
xEy, then
over the set
L
*
. * •
xzEyz, vZEE
An equivalence relation E over the set of tapes
is of finite index if there are only finitely many equivalence classes
under the relation.
Hence, we now state a fundamental theorem due to A. Nerode.
Theorem 1.2.1.
Let
*
VeE.
The following are equivalent:
1.
ViTo
2.
U is the union of some equivalence classes of a right invariant
equivalence relation of finite index.
3.
The explicit right invariant equivalence relation
E defined by
6
the condition that
~X,YEE
* ,xEy
~ZE~
iff
*
whenever
XZEU,
yZEU, and conversely, is an equivalence relation of finite
then
index.
UET, then the number of equivalence classes under the relation
If
E
is the least number of internal states of any finite automaton de-
fining
U.
Theorem 1.2.1 also leads to some simple facts about the
T.
class
Theorem 1.2.2.
The class
T
is a Boolean algebra.
Theorem 1.2.3.
The class
T
contains all finite sets of tapes.
We are also interested in characterizing the set
are not told how the black box
A operates.
T(A)
when we
All that is allowed is
to supply input, or "question," the black box and observe the "answers. H
Rabin and Scott [17]
exhibit an effective procedure
1
based only on a
knowledge of a bound on the number of states of
A whereby i.n a finite
number of steps it can be decided whether
is empty, finite, or
infinite.
T(A)
It is also possible to determine an upper bound on the number
of tapes in
T(A)
when
T(A)
is finite.
Rabin and Scott further dis-
cuss this type of result in comparing finite automata.
Definition 1.2.6.
T(A)
=
T(B).
Definition 1.2.7.
symbols in the sequence
1
Two finite automata A and B are equivalent iff
The length of any tape
x.
xEE *
We denote the length of
is the number of
XEI. * by
L(x).
An effective procedure is a set of rules which dictate, from moment
to moment, precisely how to behave. Effective procedure is a synonym
for algorithm.
7
Theorem 1.2.4.
Let
al states, respectively.
exists
XE L
* such
Al
A
and
Al
Z be FA with n l
A
and
2
and
n
2
intern-
are not equivalent iff there
that L(x) < n n and x
l 2
is accepted by one machine
and not the other.
Rabin and Scott also introduced the model of a nondeterministic
finite automaton.
In contrast to a finite automaton which has only one
way to change state in any particular situation, the nondeterministic
machine allows a choice from a set of states.
A nondeterministic finite automaton over the
Definition 1.2.8.
L is a system A = (S,H,SO,F), where
alphabet
empty set,
M is a function from
subsets of
S,
SOcS, and
Definition 1.2.9.
T(A)
The set of tapes
of all tapes
x
= 00
S
is a finite non-
taking values in the set of all
SXL
FcS.
Let
A
be a nondeterministic finite automaton.
accepted, or defined, by
01 ... ryk-l
for which there exists a sequence
A such that
of internal states of
1.
SO€SO' SiEM (si-1,Oi-1)
2.
skEF.
A is the collection
for
i
= 1,2, .•• ,k
,
This appears to be a generalization of finite automata; however, Rabin
and Scott demonstrated an effective procedure to construct a finite
automaton defining exactly the same set of tapes as any given nondeterministic finite automaton.
Definition 1.2.10.
finite automaton.
V(A)
Let
A::
(S,H.SO,F)
is the system V(A)
be a nondeterministic
=
(P,Q,PO,R), where
P
8
is the set of all subsets of
that
of
Q(p,a)
S
S,
Q is a function
= U M(s,a), Po = SO'
SEp
and
Theorem 1.2.5.
If
~
P
such
R is the set of all subsets
containing at least one element of
finite automaton and is equivalent to
Q: pxE
F.
Clearly, V(A)
is a
A.
A is a nondeterministic finite automaton, then
T(A) = T(D(A) )€T.
1.3 Probabilistic Automata
So far we have assumed that our black box machine operates in an
error-free deterministic manner or in a manner equivalent to such a
machine.
In physical machines, however, malfunctions may occur as a
result of mechanical or electrical breakdown.
Rabin [16] proposed the
model of probabilistic automata to simulate the behavior of machines and
systems with certain types of random errors and to investigate the more
general class of machines.
of Rabin and Paz [13]
Definition 1.3.1.
matrices and
V
n
Here we combine the mathematical definitions
to take on the most general form.
Let
M
n
denote the set of all nxn stochastic
denote the set of all n-dimensional stochastic vectors.
Definition 1.3.2.
A probabilistic automaton (PA) over the alphabet
A = (S,U,1TO,F), where S = {sl' ••• ,sn} is a finite set
(the set of internal states of A), 1'1 is a function !:1: L ~ M (the
n
E is a system
matrix transition function) such that
changing to state
Sj
under input a
mij(a)
is the probability of
given that the machine is in state
si' 1T EV (the initial state distribution) and FcS (the set of final states).
O n
Probabilistic automata are models for systems capable of a finite
number of internal states and able to receive inputs
aEE.
When in
9
state
si
and if the input is
one of the states
mij(a).
is
a, then the system may go into any
SjES, and the probability of going into state
Sj
These transition probabilities are assumed to remain
fixed and be independent of time and other inputs.
Thus, the chance
of an "error II on any transition depends only on the present state of
the machine and the input symbol read.
We now extend the definition
of
M to the set of all tapes of finite length,
if
X ::
a
1
a
C1
2
1<.'
Definition 1.3.3.
that i f
V
n
of
* by the rule:
E.
Let
D (M
n
)
m = 0 or l.
ij
I·1ED (M ), then
n
vED(V), then
n
such that i f
denote the subset of
Also let
v
i
=0
By restricting the transition function
D(V )
M
n
such
be the subset
n
or l.
M to take values in
D(M )
and the initial distribution function
D (V )
we can now see that the class of probabilistic automata includes
n
n
to take values in
the class of finite automata.
Definition 1.3.4.
state distribution of
Let
n
F
Let
A = (S.M,nO,F)
be a PA.
A after input tape
as
be an n-dimensional column vector such that
We define the
n(x)
n: = {l
0
1
We define the probability that tape
P (x)
x
is accepted by
= nOM(x).
Si EF
si'F
A as
= If (x)n F = nOli(x)n Ii' •
As with finite automata, probabilistic automata may be used to de-
fine sets of tapes.
~ic
Because of the more general nature of probabilis-
automata, these sets of tapes will depend not only on
en additional parameter,
A, called the cut-point.
A,
but also on
10
Definition 1.3.5.
AE[O,l).
Let
T(A~A)
The set of tapes
{XIXEL* , A < p(x)}.
cut-point
A.
Let
*
VcE.
x~T(A),
then
p(x)
= O.
A,
is not true.
is accepted with
p(x)
T(A)
Hence,
Thus, every regular event is a PCE.
particular probabilistic automaton
For this
=
A with cut-point A.
V is a probabilistic cut-point
For the case of any finite automaton
AE[O,l).
x
V = T(A,A) for some PA A and some
event (PCE) if
If
T(A,A)
is defined by
is called the set defined by
Definition 1.3.6.
XET(A).
A be a real number,
xET(A~A), we say that
If
T(A,A)
A be a PA and
AE[O,l).
=1
if and only if
= T(A,A)
for any
Rabin constructed a
A to demonstrate that the converse
A, {T(A,A) IAE[O,l)} forms an uncountable pair-
wise different collection of sets.
able number of regular events.
There are, however, only a count-
Hence, this probabilistic automaton
defines a set which is not defined by any finite automaton.
the class of PCE's is a proper superset of
A
Thus,
T; that is, the class of
probabilistic automata is more general than the class of finite automata.
The class of PCE's is uncountable, but Paz showed that not every
subset of
E*
is a PCE.
For any probabilistic automaton
n
the state distribution function
leads to a right invariant equivalence relation.
x,yEE '* xEy
iff
= n(y).
iI(X)
is right invariant.
to
A,
Clearly,
n(xz)
= n(yz)
similar to Theorem 1.2.1, any set of the form
However, in a manner
T(A,A)
sented as the union of the equivalence classes of
n(x)n
* thus
VZEL;
E
E, in the case of probabilistic automata, may have up
a countable number of equivalence classes.
condition that
We say, for any
F
>
A•
can be repre-
E satisfying the
11
Under certain conditions we can guarantee that there exists a
finite automaton with the same acceptance set as a probabilistic automaton with a specific cut-point.
to
A if there exists a
Theorem 1.3.1.
for
A.
has
n
such that
0 > 0
Let
A be a PA
Then there exists an FA
states, then
A is called isolated with respect
A cut-point
Definition 1.3.7.
B
and
Ip(x) -
AI ~ 0
Vx€E * •
A an isolated cut-point
such that
T(A,A) = T(8).
B can be chosen to have n'
If
A
states, where
It seems that in passing from a probabilistic automaton to an equivalent finite automaton we may have to increase the number of states.
In
many cases a probabilistic automaton equivalent to a finite automaton
can be constructed which is more economical in the state saving sense.
Paz found other conditions for the equivalence of probabilistic
and finite automata.
His analysis in this area also introduced other
interesting mathematical results.
Definition 1.3.8.
T(A,A) = T(B,A)
(S,M), where
A probabilistic table (PT) is a system
Sand
M are as in definition 1.3.2.
ative probabilistic table (DPT) is a system
M, and
F
strongly equivalent iff
YA€[O,l).
Definition 1.3.9.
e=
A and B are
Two PA
A discrimin-
T = (S,M,F), where
S,
are as in definition 1.3.2.
Definition 1.3.10.
distributions for
T.
Let
T
be a DPT and
A tape
x
distinguishes between
~O
and
any initial
~o
and
if
12
nOM(x)n
F
~
F
P l1(x)n.
O
Two initial distributions are k-equivalent if
xEE *
they are not distinguishable by any tape
such that
L(x)
~
k.
These definitions lead immediately to results of equivalence of
probabilistic automata.
Theorem 1.3.3.
Let
T be a OPT.
If two initial distributions
are distinguishable, then there exists
XEL
which distinguishes between them, where
state set
such that
L(x)
~
n-Z
is the cardinality of the
S.
Theorem 1.3.4.
Yx€E *
PZ(x)
n
*
two PA
Given
such that
~
L(x)
n
l
and
Al
A over
Z
E, if
PI(x)
=
+ n Z - Z, then Al and AZ are
strongly equivalent.
Paz extended the concept of definiteness for finite automata to
construct a body of results for automata equivalence and approximation.
The k-suffix of any tape
last
k
symbols of
x
of 1ength at least
Let
definite set if for any
xEE *
x
is in
V.
k
~
0
and
such that
*
VeE.
L(x)
~
U is a weakly kk, then
X€V
iff the
V is k-definite if it is weakly k-definite
but not weakly (k-l)-definite.
definite for some
is the tape of the
x.
Definition 1.3.11.
k-suffix of
k
A set
V
is definite if it is
k-
k.
Clearly, every definite set is definable by some finite automaton.
Such finite automata are called definite automata.
Paz sought to gen-
eralize this concept to form a class of definite probabilistic automata.
13
Definition 1.3.12.
any
xEE * such that
and
PO' the following holds:
L(x)
e=
A PT
~
k
(S,M) is weakly
k-definite iff, for
and for any initial distributions
(TI
O
- PO)M(x)
nO
= Q.
This, indeed, does generalize the notion of weak k-definiteness.
Given any final set
F and initial distribution
definite probabilistic table, then for all
p(x)
= p(k-suffix
for a weakly k-
x€E * such that
L(x)
~
k,
of x).
Definition 1.3.13.
A PT is k-definite if it is weakly k-definite
A PT is definite if it is k-definite
but not weakly (k-l) definite.
for some
no
k.
Paz extended the result of Per1es, Rabin, and Shamir [15] to include
his probabilistic notion of definiteness.
Theorem 1.3.5.
Let
e=
it is (n-l)-definite, where
(S,11) be a PT.
n
If
e
is the cardinality of
is definite, then
S.
Note that any probabilistic automaton with a definite probabilistic
table defines a definite event and, hence, must have an equivalent definite automaton.
Paz introduced an even weaker notion of definiteness,
i.e., quasidefiniteness, to define a larger class of automata.
Members
of this class do not, in general, have equivalent finite automata;
however, this class possesses some interesting approximation properties.
Definition 1.3.14.
we define
I~I
=
Given any arbitrary n-dimensional vector
~xl~il .
Also, for any arbitrary
1
matrix
Q we define
IQj = maxlq .. 1
i,j
1J
•
n x n
~,
14
A PT
a=
there is a number
N(e)
Definition 1.3.15.
every
e > 0
that
(S~M)
is quasidefinite iff, for
such that for any
XEr *
such
and for any possible initial distributions
A with quasidefinite table
Hence, for any probabilistic automaton
let us fix
E;
if
XEE *
is any tape such that
Ip(X) - p(N(e)-suffix of x)1 < E.
and
L(x)
~
N(E), then
Thus we have a looser, approximation
notion of definiteness.
Definition 1.3.16.
every pair of states i
1
Let
MEM.
n
M is called scrambling if for
and 1 2 , there is a state
such that
j
m j > 0
i
1
satisfies the H
m
> O. A finite subset {M1' •.• '~} of M
4
n
i2j
condition of order r if there is an r such that any product of the
and
M's of length at least
that
r
is scrambling.
For a PA,any tape
x€r * such
M(x) is scrambling is called a scrambling tape.
Theorem 1.3.6.
Let
iff the set of matrices
Theorem 1.3.7.
Let
a=
a
(S,M) be a PT.
is quasidefinite
{M(a)la€r} satisfies the H condition.
4
a=
(S,M) be any PT.
There is an effective
procedure which decides in a finite number of steps whether
a
is
quasidefinite.
Definition 1.3.17.
Let
ucr*
and
UC =
r* - U.
The set of tapes
U e-approximates the set of tapes defined by a PA with cut-point
if
A
15
Theorem 1.3.8.
for every
Let
A be a PA with quasidefinite table.
£ > 0 there exists a definite automaton
T(A £ ) e-approximates T(A)A}
Theorem 1.3.9.
If
for any given
A
£
Then
such that
A.
A is a PA with quasidefinite table and if
is an isolated cut-point, then
T(A)A)
A
is a definite set.
Rabin defined the more specific concept of an actual automaton to
derive a converse for Theorem 1.3.9.
Note that every actual automaton
has a quasidefinite table.
Definition 1.3.18.
mij(o) > 0
VO€~
A PA
A is an actual automaton iff
Vi,j=l, ••• ,n.
The use of actual automata is natural in simulating certain systems
for which all of the transition probabilities are positive, no matter
how small they are.
Theorem 1.3.10.
Every definite set is definable by some actual
automaton with isolated cut-point.
Paz also extended the theorem of Rabin concerning the effects of
small perturbations of the transition probabilities of an automaton.
Results in this realm are useful in considering the stability of properties of systems subject to perturbation and approximation.
Theorem 1.3.11.
table.
every
Given any
PA
A£
=
Let
A
=
(S,M'~O,F)
0 > 0, there exists an
(S,M'~~O,F)
with
Ip(x) - p (x)
e
be a PA with quasidefinite
£
= e(o,A)
IM(o) - M' (0)1 < £
I
< 0
> 0
¥O€~
such that for
and
16
Corollary 1.3.12.
and let
small
Let
A
and
AE
A be an isolated cut-point for
E > O~
be
PA~s
A.
is an isolated cut-point for
as in Theorem 1.3.11
For any sufficiently
A€
and
T(A~A)
= T(A € ,A).
CHAPTER II
Environments
2.1
The Concept of Environments for Automata
In discussing machines subject to error or probabilistic transi-
tion, we have considered those systems whose transition probabilities
are only related to the present state and latest input symbol and are
independent of any external factors.
Now, however, we shall construct
automata models to be more general and more realistic.
The viewpoint
taken is that every machine is located within an environment and the
transition function of that machine is not only related to the input but
also to the configuration of the environment.
We denote the set of possible values that the environment may
assume by
E.
Presently, we do not make any stipulations as to the ori-
gin, form, structure, or cardinality of
E.
As each new input symbol
is read by the machine, the configuration of the environment is observed
and the transition ensues as a function of the input
condition of the environment.
a function of the environment.
onment for a machine.
s~bol
and the
We also view the initial distribution as
Thus, we shall consider the total envir-
The total environment is the sequence of environ-
ments that consists of the initial and the subsequent environments as
each symbol is read.
17
18
Definition 2.1.1.
co
E
Let
denote the cartesian product of a
countable number of copies of the environment set
~(k)
meut, or environment sequence, is a mapping
~
We may also denote
by the sequence
E.
A total environ-
= ~€E
for k=O,1,2, ••••
(e ,e ,e , ••• ).
O 1 2
2.2 Automata in Deterministic Environments
Here we shall consider the case in which the environment is completely specified.
That is, for the initial distribution and subsequent
transitions we are given the precise condition of the environment.
In
such a case we say that the environments for the initial distribution and
subsequent transitions are obtained by a deterministic rule, the total
environment.
Definition 2.2.1.
is a system
(ADE)
alphabet,
G: ExE
~
A=
An automaton in a deterministic environment
(E,S,G'~O.F,E),
S = {sl, ••• ,sn}
Mn
where
E is a finite input
is the finite state set,
(the basic matrix transition function),
initial distribution function),
G is a mapping
~O:
E ~ Vn
(the
FcS (the set of final states), and
E
is the set of environments.
The matrix transition function
M is defined on
E*xE00
by the
inductive recursion rule:
let
xEE*
such that
Hence, for all
x€E*
and
x = 01
ok
and
~€E
k
,
~o(eO)
and
=
IT G(oi,e i ), where
i=l
In considering the function
M(x,~)
e = (eO' e , •.. ) .
l
we shall use the notation
e = (eO,e , ... ).
l
co
and ~EEco ,
L(x) = k
~O(~)
'ITO
interchangeably, where
19
As with probabilistic automata we are able to compute the state
distribution of
A after input
the total environment
after input
x
e.
x.
We denote the state distribution of
in total environment
Similarly, we let
in total environment
Definition 2.2.2.
A€[O,l).
by
~
= 'IT(x~e)n F ,where
p(x,!:..)
1.3.4, denote the probability that
x
Now, however, this will depend on
= 'ITO(eO)M(x,~.
'IT(x,~)
F
n
is as in definition
A is in a state in F after input
e.
A be an ADE and
Let
T(A,~,A)
The set of tapes
A a real number,
defined by
is called the set of tapes accepted, or defined, by
ment
e
with cut-point
Let
Q'(e , ••• ,e ) = {!:..€E~I~(i)
1
k
Lemma 2.2.1.
Let
A in total environ-
A.
Definition 2.2.3.
and
Q(eO, ••• ,e k ) = {!:..€E~I!:..(i) = e i for i=O,l, ••• k}
= ei
A be an ADE.
for i=1,2, ••• k}.
For any
k
~
0 and any fixed
eO,e1, ••• ,ek€E, the following functions are constant in
M(x,o), 'IT(x,o), and p(x,o)
Proof:
such that
L(x)
Let
=r
k
~ 0;
~
k.
for each
fix
Let
=
r
IT G(oi,e ).
X€L *
such that
and fix
!:..'€Q(eO, ••• ,e ).
k
Then
k
Q(eO, ••• ,e ):
k
L(x)
eO,el, •.. ,ek€E
(e O,e 1 ,···,e k ,ek+1,e'k+2'···), where
M(x,~')
A
S
k.
x = ol, ••• ,or
e'
k
e +1 ,e + 2 , ••• €E.
=
So
Thus, M(x,·) is constant in Q(eO, ••• ,e k ). Simii
i=l
r
r
F
1ar1y, 'IT(x,e') = 'ITO(e O) IT C(o.,e.) and p(x,!:..') = 'ITo (eO) ( IT G(o.,ei»)n
i=1
1
1
i=l
1
are also constant in Q(eO, ••. ,e k ). Note that M(x,o) is also constant
o
20
2.3
i:iathematical Generality of the AOE Hodel
Let
=
A'
~'.
alphabet
(S' ,M',TIO,F')
E be any nonempty set; let us fix e€E.
Let
now construct an ADE
T(A,(e,e, .•. ),A)
be any probabilistic automaton over the
A=
for all
(~,S,G,TIO,F.E)
AE[O,l).
such
that
We simply let
We can
T(A',A) =
~=£',
S=S', F=F',
TIO(e)=TI ' and G(o,e) = M'(o) Y OE~. Thus, p(x,~) = TIO(e)M(x,(e,e, ••• »n
O
k
F
k
F
= TI '( IT G(oi,e»)n = TI '( IT M'(o»)n = p'(x). Therefore, for any AE[O,l)
O i=l
O i=l
}:€T(A' ,A) iff x€T(A,(e,e, ... ),A) and, hence, every peE can be defined
co
by an ADE with some total environment
~
Let
nF __
by
(01]
a
k
Paz [14] demonstrated that there exists Uc~*
= fa}.
::::e Uf ~S{:~:l:
::::1 ~e:l::':;~ c:::~:;r.a[~~: st:je
Let us denote a tape of
=a
a ..• o.
e€E.
Note that
a
0
Ie
k ~ 0.
for some
k
U ={o
1
Such an
k
,a
e
2
, .•• }.
Let
= A, the empty tape.
~(k)
co
E
is an element of
•
= {~
Now, for
M(o
,~)
Thus, for our
k=k
E*; then
for some
k~l
we have
i
elsewhere
k
k
a
E* are of the form
U be any subset of
We construct
,Aan:ver
repetitions of the symbol
particular situation all of the elements of
ok
y:::
such
= k.1.
i~O
for some i
=
elsewhere •
if k=k for some i
i
elsewhere.
Therefore,
if ki=O for some i
elsewhere •
Let
A = 0, then
our particular
U=
,,*
T(A,~,A).
Since
is definable by
U is arbitrary, every subset of
A under some environment.
But
F
21
there exists
maton.
UcE*
which is not definable by a
Thus, the class of automata in deterministic environments
properly contains the
Let
~
~EE
probabilistic auto-
A
c1as~
of probabilistic automata.
eEE~.
be an ADE and let
, the state distribution function
equivalence relation on
*
E.
x, yEE *
For any
n(·,~)
such that
is a right invariant
As with probabilistic automata, for any
there may be up to a countable number of equivalence classes
fixed
n(·,~).
of the relation
For any
AE[O,l),
T(A,~,A)
is the union of
the equivalence classes satisfying the condition that
The greater generality of the ADE model manifests itself in the fact
that the equivalence relation and, hence, the equivalence classes may
depend on
e.
A,
Depending on
unique equivalence relations.
there may be an uncountable number of
The above two state ADE has an uncountable
number of unique equivalence relations.
2.4 Finite Environment Set
Let
A
be an ADE
with a finite environment set.
We shall con-
struct an effective procedure to find a probabilistic automaton which
simulates the operation of
A PA
Definition 2.4.1.
ADE
A=
1)
(E,S,G,nO,F,E)
for any
A.
A'
over alphabet
if there is a relation
(x,~)EE
*xE
00
there exists
E'
simulates the
R between
x'EE'
*
such
E'*
t~at
and
22
2)
if
[x'
A'
If a PA
x'€T(AI
x'ET(A'
~,
have
\'le
rule
a
,~),
x
total environment
e.
is an input to.re
X'EL'*
tape
A
x
acceptance of
x'
x I.
Xl
by
of the form
x
A'
and
(x,~),
is zero.
A' for any
By lemma 2.2.1 we see that the probXEL*
depends only on the environ-
= L(x).
k
eO,el, ••. ,e ,
k
where
A under
is not admissible, then the probability of
where
only depend on
for
for
By condi tion 3 of def ini tion 2.4.1 we see
ability of acceptance of any tape
ments
x'
is admissible if and only if there
that tapes which are not admissible are not accepted by
that is, if
A under
A under some total environment e from tJ!:ich
for
P will yield
~E[O,l);
for
to find an input tape
R
x' has the same acceptance properties as
such that
,~)
then there exists
A, given any input
simulates an ADE
total environment
the rule
~E[O,l)
if for some
3)
~E[O,l),
then for each
xET(A ,~,~);
iff
AI
,(x,~)]ER,
~EQ(eO, •.•
Hence, the relation
where
,ek)
and
= L(x).
k
need
Thus, all pairs
= L(x),
k
F
can be consid-
ered equivalent.
Theorem 2.4 .l.
II (E) <
ex>
I
•
we shall let
L'
=
A
(L,S,G;Il"O,F ,E)
Al
There exists a PA
which simulates
Proof:
Let
be an ADE such that
over some finite alphabet
L'
A.
Let
L
m = II (L)
= {1,2, ••. ,m}
= {O,l, •.• ,E(m+l)-l}.
define a mapping
E = If (E) •
and
Let
g: (Eu{$})xE
E = {O,l, ..• ,E-l}.
and
$
+
Without loss of generality
Let us consider
be any symbol such that
L'
as fellows:
1 #(E) denotes the car.dinality of the set
E.
$¢L.
We now
23
=e
g(o,e)
+ oE
V eeE
This mapping is a one-to-one correspondence.
*,
«Eu{~})xE)
We extend
g
to
the set of all finite sequences of symbols of
(Eu{~})xE
,
by component-wise application of the mapping and concatenation of the
results.
That is.
al a
k
2
•.• a
al,a2, ••• ,akE(Eu{¢})xE. then
~«Eu{.})xE)*
g(a l a 2 ···B )
k
and
= g(al)g~a2) ••. g(Bk)'
remains a one-to-one correspondence between «ru{~})xE)*
g
Let
be any nonempty element of «Lu{~})xE)*, then
b
b
and E'*
is isomorphic
to the form (y,(eO, •••• e ». where for some
k
k+l, and
oo
eO, •.• ,ekEE.
We define the relation
F between
E'*
and
E*xE
to be the set of all elements of the form: [g(~x,(eO, ••• ,ek»'(x,~)],
where
XEr*, L(x)
then
L(A)
=0
= k,
and
and
~A
=
e
= (eO, .•• ,e k , ••• ).
S
= {Sl, ..• ,sn}
o
~ 0'
~
E-l.
be a PA over
')
A'
E'.
is the state set of
So for any
x
= Aer*,
~.
We shall now construct a PA
A' = (S' ,M' ,TI O$
Note that when
0'
to simulate
Let
A.
such that
SI
= Su{sn+l,sn+2}'
o'eEc~',
For any
0
A. Let
~ 0'
~
where
then
E-l. we define
o
1
o
o
1
o
1
o
M'(o')
is the i-th component of
where
implies
=
o'EE.
These values of
distribution function of
A
0'
o
Note that
o~
0'
~
E-l
are used to incorporate the initial
into the PA model
A'.
The values
24
a'€L'
E :s;
such that
:s; E(m+l}-l
(J
AI
are used in order that
imitate the basic matrix transition function of
A.
can
The set
{a I IE :s; a' :s; E(m+l)-l}
is in one-to-one correspondence with the set
LxE
g.
under the mapping
M'(a')
where
'0 -
E :s; a' :s; E(m+l)-l,define
So for
=
o
0
o
o
o
o
o
1
o
o
o
1
G is the basic matrix transition function of
(0, ••• ,0,1,0)
=F.
and F'
of' -
Therefore
[x' ,
(x,~)]E:
x'
= g(~x,(eO, ••. ,ek»'
where k
= L(x),
such that
R.
Suppose [x', (A ,!.) ] ER.
Therefore,
pl(X l ) be the probability that
ability that
pi (x')
[fl .
Also, let
R that for any
It is clear from the definition of
there exists a unique
A.
A accepts
x
AI
Xl
accepts
= g(~,eO) = eO'
x'
Let
and p(x,e) be the prob-
under total environment
e.
Hence,
= pi (eO)
=
.01
(0, ... ,0,1,0)
0
.01
(1)
(n)
~O
(eO)"'~O
o
Thus, for each
AE[O,1), pl(X')
if [xl,(A,!.)]ER, then for each
= p'(e O)
>
A iff
(eO)' 0 0
O. 0 1
p(A,!.) > A.
AE[O,l), XlET(A',A)
iff
That is,
A€T(A,~,A).
25
Suppose
[x',(x,~)]€R,
the definition of
x
= a1 , ... ,ak
= eOai, ... ,ak.
where
x
R we have that
has length
Thus,
k.
p'(x l
Xl
x'
So
~
A.
k
= IT
i=l
o .
o .
0 I
·
o0
·· ··
o0
o1
o0
= i=1
• 0
o
o . . .
0
0
0 1
o • • • COl
0 0
i=l
G(o.,e.)
.
0
0
1
1
···
· ·· ·· 00
·
o0
0 1
0 1
0 0
=
H(x,~)
. ··.
0 ·
· · · 00
0
···
Hence, we have
O.
By
where
= g($,eO)g(a1,e1), .•. ,g(ak,ek)
• 0 I o 1
k
II
=
>
= p'(eOai,···.ak) = ~OM'(eoai
)
G(g-l(a'» I
i
I
k
II
L(x)
= g(<jlx,(eO, .. .,ek »,
But
k
IT !1' (0 r )
i=l
i
Hence,
··
0 0
0 1
0 1
... ~k)n
F'
.
x # A, then for each
So, if [x',(x,e)]ER and
p(x,~ >
A.
Thus,
x'ET(A',A)
for each
A€[O,l), x'ET(A' ,A)
such that
x'
since
x'
is not admissible.
g
*,under
where
However;
x'
k
= L(x).
= g(b)
and
x€E*
so
~€Eoo
and
Therefore, such
bE«LU{~})xE)*
for some
= 0 0' ...
x'€E'*
ok.
E'*.
A', is the image of AO, the empty tape in
g.
= (0, .... 0.1.0)
Let
I
if [x',(x,~)]€R, then
is a one-to-one correspondence between «EU{~})xE)* and
«Eu{~})xE)
'ooF'
A iff
>
)
x€T(A,~,A).
iff
= g(~~,(eO, .•• ,ek»'
A', the empty tape for
X
¥ e€E~
[x' ,(x,~)]€rr. Hence, there does not exist
that
an
and
such that there does not exist
Let
p'(x i
xET(A'~JA).
iff
¥ X€L*
We have verified that
AE[O,l)
Clearly,
[tJ
= O.
A' is not adwissible.
Hence
A"T(A',>.)
p'(A')
for any
be any tape which is not admissible and
is the image under
g
of some element
=
>'dO.1),
Xl
# A'.
*,
b€«Eu{~})xE)
where
h~s
h
the form
(y.(eO ••.•• e
*
YE(Eu{~}).
Note that
Let
x'
is not admissible iff
i=1,2 .... ,k.
If
LO
E
~ 0
0
~
'E(n+l)-l.
...
input a' a'
0 1
sn+2
SO
a'k
then
TjH.
Then
enters state
definition of
state
sn+2~EbF'.
Hence.
or
LiH
we see that
1'1'
'i~L
If
Hence,
for some
A'
under
~
sn+2
and
E-1.
a'
a' a'
0 I
Again.
p'(x') = 0
i=1,2, •••• k; say
0 ~ a~
J
and. hence,
But
= o.
p'(x')
for some
A' under input
we see that
fer
'fidU{</>}
at the first transition.
at the (j+l)th transition.
sn+2
state and
M'
= k+1.
g('O,e O) and, hence
sn+2 ~FmF'.
= ~. a'j = g(Tj,e j )
J
,J
sn+2
~ [0 , 1) .
I\E
f or any
T.
00
By our definition of
where
~ ~
TO
,=
'a EL
is an absorbing state and
X '!T(A',~)
II1\
and L(x') = L(y)
y = TO TI ••. Tk •
i=O,I •••• ,k.
~ ~>
27
»
k
By our
enters
k
is an absorbing
x'ET(A',A)
for any
AdO.I).
Thus. if for some
oo
that is. there exists
Consequently.
AE[O.I) x'ET(A'.A). then
{x,e)EE*xE
such that
x'
is admissible;
[x'. (x.e)]E R .
A', as constructed. simulates
A.
0
The results of section 2.3 and theorem 2.4.1 seem to present a
paradox.
We have found for a particular
which is not a PCE, that is, for any
automaton
A
p
such that
E* that there exists
AE[O.!) there is no probabilistic
V=T(A ,X). However. we showed that there
p
exists an ADE with finite envirocment set over
for some
and
ilistic automaton
simulates
A.
ment whereby
Let
V
VcE *
E such that
V=T(A,~.A)
By theorem 2.4.1 there exists a probabA'
VI
over an expanded alphabet
= {g(~X,~)IXEV},
= T(A.~.A).
where
E'
e
such that
A'
is the total environ-
U' = T(AI. A) and. hence, is a PCE.
paradox is resolved by noting that we not only have expanded
The
E to
E'
28
but also enriched the alphabet with a structure that had been included
in the environment.
VeL * remains because V'
The problem for
different structure within
E'*.
has a
In fact, we can find a subset of
E'*
which is not a peE.
As an example of the application of theorem 2.4.1,
Example 2.4.1.
let us consider the ADE discussed in section 2.3.
ADE over
E
= {a}
E
with
= {O,l}.
'lfO(e)
A is a two-state
= (l-e,e),
G(o,e)
=(l-e
1-e
and
F
0
n = (1) • By theorem 2.4.1 A' has alphabet L' = {0,1,2,3}
and
-,
,
1-1
is defined on
H' (0) -
=
H' (2)
The mapping
E'
as follows
[
000 1
000 1
1 000
000 1
]
H' (1)
[
100
100
000
000
]
H' (3)
g
0
0
1
1
=
=
[
000 1
000 1
o 100
o0 0 1
]
[
010 0
o 100
000 1
000 1
]
is described by:
= e and g(o,e) = 2+e •
g(~,e)
With
'0 - (0,0,1,0)
F
n -
and
[~],
it is easily verified by the same
o
procedure as the proof that for any
and
x€T(A,~,A)}
oo
e€E
exists a PA
such that
A'
A'
and that
Corollary 2.4.2.
fixed
:),
Let
E
hE[O,l)
A•
simulates
A=
T(A' ,h) = {g($x,e)I~€E~
(L , S,G, 1T O,F , E)
= {e.li=O,l,
••. }
l.
over some alphabet
L'
be an ADE.
For any
is a finite set, then there
such that for any
x€E *
29
and
for the ADE
2.5
xET(A,~,A)
AE[O,l),
=
A
E
gE(X,~)ET(A' ,A),
iff
where
(E,S,G,nO,F,E) as the function
gE
is defined
in theorem 2.4.1.
g
Reducing the Environment Set and the Induced i1etric
Let
A
=
(E,S,G,nO,F,E)
be an ADE
and let
distinct configurations of the environment.
= G(a,e ' )
G(a,e)
for all
aEL. then
on the transition functions.
e
If
and
e
and
e'
be two
nO(e)
=
nO(e')
and
e'
have identical effects
That is, the configurations
are equivalent in the sense that for any
mains constant if we substitute
e
Thus, we do not need to include
e'
not augment the structure of
A.
for
e
and
and
e'
e'
re-
whenever
e'
occurs in
in the environment set since
Moreoever, we can reduce
e.
it does
E so that
no two distinct environments are equivalent in the above sense.
Definition 2.5.1.
Let
A = (E,S,G.nO,F,E)
say
E is irreducible i.f for all
and
G(a,e)
= G(a,e')
for all
Let us assume now that
the ADE
A
=
= e'
iff
nO(e)
= nO(e')
aE~.
Recalling definition 1.3.14, we construct
Consider the function
naxIG(a,e) - G(a,e')I),
e
E is an irreducible environment set for
(E,S,G,nO,F,E).
E.
a metric on
e, e'EE
be an ADE, then we
where
d(e,e')
e, e'EE.
= max(l~o(e)-no(e')I,
Clearly,
d(e,e')
= d(e',e)
aE:E
and d(e,e')
~
0
for all
d(e,e')
e,e'EE.
For any
= max(lno(e)-~o(e')I,
e,e', e"EE, then
maxIG(a,e)-G(a,e')I)
aE:E
s max(lno(e)-~o(e")1 +lno(eil)-~o(e')I,
max(IG(cr,e)-G(a.e")
,ad:: ..
s max(I~O(e)-~O(e\1)
I,
I
+ IG(o,el')-G(cr.e) I)
max/C(a,e)-G(a,e ll )
ad
I)
30
+ max(I'Tro(eV)-'lTO(e a ) I, maxIG(a,e')-G(a,e
ll
)
I)
aEE
So
d
satisfies the triangle property.
'Tro(e)
=
'lTO(e i
when
e
= e'.
d(e,e') > O.
G(a,e)
and
)
Since
So
the function
If
then the function
is defined for all
d
e
= e',
aEL.
for all
E is irreducible, if
Theorem 2.5.1.
A,
G(a,e V)
c
Also, if
~
e
then
So
d(e,e')
e', then by definition
as defined above is a metric on
~
E.
E is an irreducible environment for an ADE
= max(I'lTo(e)-'lTo(e')I ,
d(e,e')
e,e'EE
and is a metric on
maxIG(a,e)-G(a,e')I)
aEE
E.
Since irreducibility is needed only to show that if
d(e,e')
=0
0, the function
d
e
~
e', then
is a pseudo-metric on any arbitrary environ-
ment set.
Let us now consider an arbitrary ADE
A
=
(E,S,G,'lTO,F,E)
Et.
the environment set is reducible to a finite set
for which
That is, we can
t
t
define an onto mapping 11: E -+ E c E, where i f h(e) = e , then
t
d(e,e ) = O. The mapping h is unique only when Et is irreducible.
h
oo
can be extended to
h.
h: E -+ (Et)oo
That is, h(e) ::: (h(e ) ,h(e ), ••• ).
O
l
and, hence, we knou that
for all
aEE.
and
Theorem 2.5.2.
II
(h(E»
<
= T(A,h(e) ,A)
CXl,
then
So for all
= 1r O(h(e»
So by extension, for all
'IT(x,e) = 'Tr(x,h(e»,
T(A,~,A)
'lTO(e)
by component-wise application of
and
A
for all
=
00
CXl
and
=
=0
G(a,h(e»
= M(x~h(~»,
for all xEE * •
(L,S,G,'lTO,F,E)
eEE
G(a,e)
~€E, M(x,~)
p(x,e) = p(x,h(e»
Let
eEE, d(e,h(e»
be any ADE, then
AE[O,I).
A can be simulated by a PA.
Furthermore,
if
31
Proof:
00
Let
*
xd.
e€E.
So for any
x€T(A,~,A) i f f
p(x,~)
We have seen that
AdO,l)
p(x,!:.) > A iff
xET(A,h(e) .A); that is,
=
p(x,h<,~»
fraIl
p(x,h(!:.» > A.
T(A,~,A)
=
Hence,
T(A,h(~)
A).
then by theorem 2• . 1 there
exists a PA
A' over alphabet
(E,S,G,nO,F,Et ).
E'
which simulates the ADE
Thus, there exists a relation
t
R
[x'
t
,(x,h(~)]€R.
x'€E' * , sueh that
exists
E' * and
R between
t
[x', (x,h(e» ]€R.
«Eu{$})xE).*
Hence, for any
t
AdO,l)
x' €T(A' ,f\)
T(A,~,A)
= T(A ,h(e) ,A)
for all
AE[O,l).
[x' , (x,~) ]ER,
then for each
AdO,l)
x' €T(A I ,A)
But
for any
[Xl
x'd' * there does not exist
,(x,~)JER,
*
{X,~')EE x(E
+
x'
then since
.) *
T
such that
admissib~e
is not
h
under the relation
iff
iff
oo
f
If
such th t
is an onto oapping, there does not exist
t
[Xl,(X,~ )]ER
Hence,
2.5.3.
such that
Let
A'
t ,and conversely.
R iff
simulates
x'
That is,
is not admissible
A via the r lation
XEL * and
AdO,l),
A = (I.,S,G,nO,F,E) be an ADE.
E = {eili=O,l, ... }
set, then there exists a PA
any
x€T(A t ,h(e) ,A).
R
o
Corollary
e€E
E*xE 00 , there
x€T( ,!:.,A).
as defined.
fixed
rule:
then
Therefore,
(x,e)d *xE00
under the relation
at.
(x,~)
[x' ,(x,e)]€R,
If
and
We now
by th
Clearly, for all
[x' , (x,!:.)]E:R.
E'*
betweel
«EU{$})xET.) * satisfying the conditions of definition 2.4.
define a relation
At =
A'
over some alphabet
xET(A,~,A)
is defined in theorem 2.4.1 and
is reducible to
h
iff
E'
sue
g(x,h(e»ET(A' ,A),
For any
finite
that for
here
is defined in theorem 2• . 2.
g
32
2.6 The Environment Set as a Semi-group
Definition 2.6.1.
The pair
(W,o), where
any binary operation, is a semi-group if
tion
0
o
and
of all
nxn
operation forms a semi-group.
W is closed under the opera-
stochastic matrices and the multiplication
In Section 1.2 we observed that any
with the operation of concatenation forms a semi-group.
PA
and
* then
x,yd,
H(xy)
= r1(x)i-1(y).
Let
(W,o)
a semi-group iff the mapping
M
n
M
n
A be any
Let
is associative under
We shall now generalize this result.
Lemma 2.6.1.
Proof:
Since
*
E
A ctefines a subsemi-group
the operation of multiplication,
*
{I1(x) IxEE}.
is
0
is associative.
M
n
The set
W is any set and
be any semi-group.
h: w
+
M(w)
{1'I(w) EM
n
IWEW}
is
is a homomorphism.
Obvious from the definition of homomorphism and the fact that
is a semi-group.
Suppose we have an ADE
binary operation
M
0
+
set
= {G(cr,e) IeEE}
G(cr)
be a
form a semi-group.
G(cr,·) : E
n
homomorphism.
is
associative.
For each
crEE
E and some
let the mapping
By lemma 2.6.1 for each
crEE
the
is a semi-group under the operation of multi-
Also, since
G(cr)
A = (E,S,G,nO,F,E), where
G(cr)
eM,
n
multiplication of elements of
Under certain conditions an ADE on an environ-
ment set which is a semi-group may be simulated by a probabilitic automaton.
33
1.
A=
Let
Theorem 2.6.1.
E = S(E+),
(~,S,G,nO,F,E)
where E+
be an ADE
is any finite set and
the semi-group generated by
E+
such that:
S(E+)
denotes
and the binary operation
° ,.
2.
the mapping
for any
Go(e) = G(o,e),
3.
E = S(E+)
i=l, •.• ,k
M
+
defined as
n'
is a homomorphism;
A'
which simulates
implies
E+
c
E.
as a finite product of elements of
+ +
eiE:E
°
is any fixed constant vector in
nO(o)
Then there exists a PA
Proof:
G : E
and
is finite.
k
eE:E
eE:E.
Qan be expressed
That is,
Since for any
Go is a homo+
+'
e ', ,
e = e ' 0 ••• 0
(JE~
+
k
for all
A.
Also, any
E+.
V
n
Clearly, if
G(o,e) = IT G(a,e ).
i
k
1
i=l
where e+, E: E+ i = 1, .•. ,k' and k' is finite, is any other decompoi
k'
k
+
sition of e, then
IT G(o,e ) =
IT G(0,e1')·
i
i=l
i=l
morphism,
Let
then
XE:E * and
i=l, ... ,r
e.
1.
M(x,~)
j = 1, ... ,k •
i
ij
i=l j=l
1.
o f Q'( e+ , ••• ,e+
1k
1
11
Let
+
kr
x
... a
,e+ , •.• ,e+ ).
21
rk r
ki
+
F
+ +
IT G(ai,eij»n = p(x ,~).
r
n ( IT
O i=1 j=1
+ ') f or a 11
X+ E: T(A ,~,A
iff
e~lo
••• oe~
k'
k'
x'
= r.
L(x)
has a finite decomposition
ki
+
IT G(a.,e ).
IT
Suppose
For each
ei€E
+
+
e. = eilo
••. oeiki
1.
where
r
e+.. E:E+ for
~J
co
eE:E .
i
r
IT G(oi,e ) =
=
+ i=l
E:E, and
1
T'len
)
p(
x,~
e
+
i
be any element
) nF =
= nO~'"' (
x,~
Hence, for all
AE:[O,I),
XE:T(A,~,A)
+, '(~+
+
+
+
~11,···,elki···,e21, ... ,erkr)'
~ EQ
be some other decomposition
of
e
i
Let e i =
i = 1, ••• ,r. Let
k'
= 01 1 , ••• ,or r EE *
and let
I
~ , EQ' ( ell""
"
,elk'
,e 21 ,···,e 'k') •
1
r
But
34
again, for all
~ , €Q'
AE[O,l),
xET(A,~,A)
iff
x'ET(A,~',A)
for all
Thus, f or a 1 1 'I\€ [0)
,1,
( ell'"
,
" ,e ,· .. ,e ' )
• ,elk'
21
rk r, ·
I
x€T (A ,!:..,A)
x+ET(A,~+,A) for all e+EQ'(etl, .•. ,e~k ,e;l, ••• ,e;k) and for any
iff
1
decoUlposition of e.
i=l, .. "r.
J.
osition of
= l, ••• ,r.
i
e.
J.
A~
Let us now consider an ADE
= r,
L ( x)
where
+ )
rk
+ co
••• e
r
n (E)
+
+
•
For any
for i
that for any
AE[O,l)
e
ij
( E,S,G'~OF, E+) •
=
we now restrict the choice of
€E
since
x€E
*
1, .. _,r,
=
But
!~.r (E+) < ~"
_
we have a relation
For any
xEE ~ ,
,+
+
+
€Q (ell"" ,elk ,e 21 ,
e
+ must exist
I
By the above He see
+
+
x €T(A,~ ,h),
iff
Hence, t here i s a PA
R+
+
00
= l, ••• ,k i .
j
XET(A,!:..,A)
.§:.
!:..€E , such an
and
where
e
+
is any
x+ is the derived input
appropriate decomposed total environment and
tape.
r
Note that x+ depends on x and the decomp-
A' Vll1J.cn
,.,
.
SU1.U 1
a t es A+ .
So
E'* and E*x(E+)co satisfying the pro-
between
perties of definition 2.4.1.
We now define a relation
[x',(x,~)]ER
R
and
between
iff for any appropriate
oo
as follows:
decomposed total environment
+
,++
+
x , [x ,(x ,~ )]ER.
and derived input tape
E*x E
For any
e+
*00
(x,e)EE xE
we can find an associated decomposed total environment and derived input
+
x.
tape
[x'
Hence, for any
,(x,~)]ER..
+ ,~+ ,A).
xET(A
x'ET(A',A)
R+
that
iff
A'
iff
(x,~)EE
*xE00
Also, we have for all
Hence, if
[x'
xET(A,~,A).
Example 2.6. 1.
AE[O,l)
,(x,~]ER,
R.
that
then for any
And, clearly,
x' is not admissible by
simulates
there exists
Xl
x'EE' * such that
xET(A,~,A)
iff
AE[O,l)
is not admissible by
Thus, we have the desired result
o
A •
As an example of theorem 2.6.1 let us consider
where
is a constant and
35
E = , {h,2h, ••• } for some constant h > O. Clearly, E and the operh
h
ation of addition form a semi-group. Let G be such that for any
OEL:
e€E
G(o,e) = (G(O,h»e/h. Thus for any OEL and
(el+eZ)/h
e1/h
eZ/h
= (G(o,h»
G(0,e )G(0,e ) = (G(a,h»
(G(o,h»
1
Z
and
h
= G(a,e +e )'
1 Z
Hence,
E = S({h}).
h
For any
G is a homomorphism for each
eEE ,
h
OEL.
Clearly,
e = h+ ••. +h; there are e/h occurrences of
e. Let XEL * such that L(x) = r and
e Ih
e !h
+
r
+
1
and e = (h,h, •.. ) . Thus,
Cf'nsequent1y,
•••
C1
x
=
~E~.
°1
r
r
r
ei/h
r -ei /h
M(x,e) = i~1G(ai,ei) = II (G(o.,h»
= II
II G(ai,h) :: ~1(x+,~+).
i=l
~
i=1 j=l
h
in the decomposition of
Now define a PA
to simulate
Ab+ =
A' = (8' .:H ' ,7T ,F')
O
O::,S,G,7T O,F,{h}).
Hence,
over
L'
p(x,e) =
as in theorem 2.4.1
1TOl1(x,~)11
F
=
r ei/h
F
e /h
e Ih F'
e1/h
e Ih
1
1T O( II
II G(0ih » 11 = 1T ,M'( ¢T
••. T r ) 11
= P '( ~Tl
••. Tr r )
r
O
l
i=1 j=l
Consequently, xET(A ,~,A)
where Ll,.·.,TrEL' and g(oi,h) = Li •
e1/h
erlh
X'=~T
... L
ET(A',A) foral1 AdO,l).
iff
r
By our definition we may regard probabilistic automata as only
being able to change state within a discrete time scale
t
= 1,2, ..••
The concept of the continuous time probabilistic automata as introduced
by Knast ['S] is similar to the
r~rkov
chain with a continuous parameter.
The next state function of such an automaton is defined for
as well as
OEL.
tE[O,oo)
A continuous time probabilistic automaton may change
state at any time.
Definition 2.6.2.
(CTPA)
A continuous time probabilistic automaton
is a system A = (L,S,G,1T O,F,T), where
L,S,1T O' and
Fare
as in definition 1.3.Z, T:: [0,00), and G(o,t) is a matrix transition
36
function satisfying:
1.
G(cr,t)€M
3.
Every
n
for all
gij
crEL
and
tET.
is a measurable function in
lim gii(cr,t) = 1
vEE
for all
(o,~)
and
and i = l, ••• ,n
and
t~O
j = l , ••• ,n.
The element
to + t
glj(cr,t) is the probability of being in state
when the input is
at time
to
for any
cr
tO€T.
given that the system is in state
An infinitesimal matrix of the semi-group
A(o)
= lim(G(o,t)-In )/t, where
t-+O
The elements aij(o) of A(a)
denotes th2 nxn identity matrix.
n
called the rates of transition from state
symbol
si
cr.
({G(O,t)ltET}, multiplication) is the limit
I
at time
Clearly, the transition properties depend
on the time interval for input
Definition 2.6.3.
Sj
to state
s.
J
are
via input
o.
Knast proved that the transition function for every CTPA has
~
the canonical form
G(o,t)
= eA(o)t = L
(tA(o»i
i!
i=O
It is readily seen from the definition of the CTPA that it is a
special case of the ADE model with a semi-group environment set whose
structure is preserved by the homomorphisms
there does not exist a finite set
E+
for all
G
a
OEE.
Since
such that T=S(E+), theorem 2.5.1
may not be used to show the existence of a PA
Ai
to simulate a CTPA.
Knast showed, however, that under certain conditions a CTPA can be
approximated by an ADE of the form
Ah
as above.
For further results
concerning this special case of the ADE mvdel see Knast [8].
CHAPTER III
Stability Results
3.1
The Stability Problem
The stability problem is a natural consequence of the inevitable
impreciseness of the measurement of the environment configuration.
We
shall consider the effects of small perturbations of the environmental
condition on the acceptance sets and transition functions.
Implicit in the notion of perturbation of the environment is
consideration of a natural metric on the environment set by which the
size of the perturbation is measured.
metric on the environment set by
N
d •
We shall denote the natural
For example, let us consider a
system whose transitions are related to temperature.
E
=
[0,100]; dN(e,e ' )
=
le-e'l, where
e,e'EE, may be a useful natural
metric to consider perturbations of the environment.
point concerning
d
N
That is, suppose
The salient
is that it is computed directly from the environ-
ments without necessarily considering the transition functions.
In Section 2.5 we considered the function
d(e,e')
based on the ADE
= max(I~O(e)-~O(ei)l,
A=
(L,S,G'~O,F,E),
maxIG(cr,e) - G(cr,e')I)
crEE
where
e,e'EE.
The function
was, in general, found to be a pseudo-metric on the environment set
We shall now establish the notion of continuity of the transition
functions with respect to the natural metric.
37
d
E.
Definition 3.1.1.
A = (L,S,G,nO,F,E)
Let
be a natural metric on the environment set
transition function
G is continuous at
if for every
there exists
E >
0
1y continuous with respect to
and
at
N
d
with respect to
d
N
G
if
e.
if for every
<5
d
N
can be chosen independent of
38
U
N
such that
~( e,E ) • G is uniform-
u
is continuous at each
Similarly,
E > 0
e€E
is continuous
no
there exists
n
uniformly continuous with respect to
and
d
d (e,e') < <S(e,e).
such that
E€E
with respect to
d N(e,e') <
can be chosen independent of
e
e
d
The basic matrix
6(e,e) > 0
maxIG(cr,e) - G(cr,e')I < E whenever
O'€L
E.
be an ADE and
<5(e,E) > 0
n
o
is
is continuous at each
if
e.
Continuity conditions will allow us to bound the perturbations of
the transition probabilities given the perturbation of the environment
with respect to the natural metric.
Lemma 3.1.1.
d N.
metric
If
we can find
ever
Let
G and
<S (e,e)
>
0
A = (L,S,G,nO,F,E) be an ADE with natural
are continuous at
nO
such that for any
E > 0,
e, given any
e'€E,
d(e,e') < e:
when-
dN(e,e') < <S.
Proof:
Clear from the definition of continuity and
Thus, i f
ations of
G and
e
G and
are continuous at
with respect to
d
N
e, for small perturb-
we can bound the perturbation of
Furthermore, by making the perturbation of
small we can.make the perturbation of
Also, if
G and
nO
d.
G and
nO
e
sufficiently
as small as desired.
are uniformly continuous, then the choice of
can be made independently of
e.
<S
39
For notational convenience, let
An ADE
Definition 3.1.2.
metric
on
stable) at
e
A
E and cut-point
=
J
=
= {O,1,2, ••• }
(E,S,G,nO,F,E)
with natural
is tape-acceptance stable (a-
(e ,e , ••• ) iff there exists
O 1
sup dN(ei,ei) < 0 implies
iE:J
and
0 > 0
such that
T(A,~,A) = T(A,~',A).
In other words, an ADE with natural metric
d
N and with cut-point
A is a-stable if the set of tapes defined by A under total environment
e
A is not changed by sufficiently small per-
with cut-point
turbations of the environments.
N
d
any
on
A = (E,S,G,no,F,E)
An ADE
Definition 3.1.3.
~
E is strictly stable (s-stab1e) at
0
€ >
there exists
In(x,e) - n(x,~')1
< €
Theorem 3.1.1.
metric
d
N
there exists
E.
on
0
>
0 > 0
0
iff given
N
such that
A = (E,S,G,nO,F,E) be an ADE with natural
A is s-stab1e at
such that
Ip(x,~) - p(x,~')1 < y
= (eO,e 1 , ••• )
sup d (ei,ei) < 0 implies
*
i€J
Yx€E , where n is the state distribution function.
Let
If
with natural metric
~,
then given any
y > 0
N
sup d (ei,ei) < 0 implies
* i€J
Vx€E, where p is the acceptance probability
function.
= n(x,~)n F , where nF
xE:E * , Ip(x,e) - p(x,~') I = I (n(x,e)
Proof:
For any
p(x,~)
In(x,~) - n(x,~')I#(F).
is
s-stab1e at
~,
Given any
there exists
y > 0
0 > 0
is as in definition 1.3.4.
- n (x,~' »n
let
€
such that
F
I~
= Y/ff (F) •
Since
A
sup dN(ei,ei) < 0
i€J
11T(Xt~) - 1T(X,~V)
implies
<
=y
e:#(F)
I
40
= y/tl(F).
< e:
Tp(xt~) - p(x,~') I
Thus,
0
VXEL*.
Thus,
s-stabi1ity at
e
guarantees a bound on the perturbation
of the acceptance probability function.
A cut-point
Definition 3.1.4.
to an ADE
A at
e
A is called isolated with respect
y > 0
if there exists
Ip(X,~)-AI ~ y
such that
'rJXEL * •
Theorem 3.1.2.
N
d
metric
if
y > 0
A
=
(L,S,G,1T ,F,E) be an ADE 1;dth natural
O
A is an isolated cut-point for
If
A is s-stab1e at
Proof:
then
~,
A with cut-point
A is an isolated cut-point for
VXEL * •
such that
choose
o>
E.
on
Let
E
0
such that
sup dN(ei,e~) < 0
iEJ
such that
e.
e.
So there exists
A is s-stab1e at
Then by theorem
0 < E < Y •
and
e
A is a-stable at
A at
Since
A at
~,
3.1.1 there exists
implies
Ip(x,~) - p(x,~')1
A.
A is isolated, so
< E < Y
VXEL.
If
;::: A + y.
p(X,~
If
so
xET(A,~,A),
x i
then
Thus,
T(A,~,A),
p (x ,.!:.) ::;; A - y.
p(x,~) >
p(x,~');:::
then
Thus,
T(A,~,A)
=
A+ Y -
p(x,~) ~
e: > A.
A.
Again,
p (x,~') ::;; A -y+e:::;; A.
Consequently, there exists
implies
But
(A,~',A).
0
>
0
That is,
Hence
XE:T(A,~' ,A).
A is isolated,
And, hence, x
~ T(A,~,A).
such that
sup dN(e.,e ') < 0
i
iEJ
J.
A with cut-point A is
a-stable.
o
The concept of a-stability differs from s-stability in that
a-stability allows a large perturbation of the acceptance probability
function as long as the cut-point is not crossed.
41
3.2 Stability with Tapes of Bounded Length
Definition 3.2.1.
P
is an arbitrary
=
~
If
matrix, then
(~i)
is an arbitrary vector and
II f, II
=
? I~i I
and
II p II =
].
max
i
L Ip .. 1
j
•
J.J
Lemma 3.2.1.
IIB-B' II + Ilc-c'
a) Let
II.
b)
B,B'
Let
IIBc-B'c' II ~
,C,C'EM,
then
n
then
V,V'€V,
n
IlvB-v'B'
II : ; I lv-v' II
+ II B-B' II·
Proof~
1/·1 I
We must first show that
property on the set of all
nxn
satisfies the triangle
matrices.
IIH-H'I/ = max L Ihij-htjl
i
j
= max
I
Ihij-hij+hij-htj 1
~ max
I
{Ihij-h~jl + Ihij-hij I}
I
Ih. j-hi.'.1 + max
i
i
::; max
i
j
j
j
].
1J
"
J.
= IIH-w'lI + IIH"-H'II
H,H' ,
and H".
B'c'l! ~
L Ihij-hij I
J
for any
Let us now consider II BC·-B' C'
II(B-B')CII + I!B'(C-C')II . But
~ max
j k
= max
i
k j
i
L L ckjlb'k-b~kl
1].
L L c kJ· Ib'k-b~kl
].
J.
= II B-B' II·
II
nxn
=
matrices
IIBc-B'c + B'e -
42
Similarly,
= rr~x
i
::; max
i
I b~l~~ jIlckj-ck'·1
J
k
L b 1k
~ II c··c II
i
k
= II c-c' II .
Thus,
IIBc-B'c' II ::; liB-BIll + Ilc-c' II·
For b) 11·1 I similarly satisfies
set of all n-ciimensional vectors.
II vB-v' B' II
=
the triangle property on the
Thus,
II vB-v' B+v
1
B-v 'B' II
::; II (v-v' ) BII
+
II v ' (B- B') Ii .
We see that
I l(v-v')BII=
~I~(vi-Vi)bijl
:; ~ f
=
=
b ij vi-v ~
I
L
I
i j
b .. lvi-v i'
1J
LIv
-v' I
I
I
iii
= I lv-v' II .
Also,
I /v'(B-B')1
I = III
j
i
V~(bi·-bij)1
J
: ; ?J ?Vilbij-bijl
:I.
=
i v~ i
:S
L
v .11 B-B II
i
1.
=
I IvB-v'B'1 I
Hence,
Lemma 3.2.2.
I lv-v' II
:S
Proof:
I
II B-B II
II B-B' II .
I
0
k > 0, let
and let
V,WEV ,
n
B1, •.• ,B
and
k
then
By induction on the results of lemma 3.2.1.
Lemma 3.2.3.
nl B-C I and
+
Ibij-bi j
I
For any integer
M
n
43
Let
Iv-wl:s
Proof:
IB-ci
B,CEM
I Iv-wi I
and
n
V,WEV,
then
n
IB-cl:s
I IB-ci I
:s
:s nIV-\II.
= ~xlbij-Cijl.
i * and
Let
j '*
be the indices
1.,j
where the maximum is obtained.
Thus, IB-ci
Ibi*j*-ci*j*1 :s ~Ibi*j-ci*jl :s ~x
f
max Ilb .. -ci.1 :s max Ilbi*j* - c.~j*1
i
j
J.J
i
J
1.
j
=
Ibi*j*-ci*j*l.
Ibij-Cijl
= nIB-~I.
= I IB-ci I·
But
And also,
And similarly for the
0
result concerning vectors.
Keeping in mind the above results, let us now consider an ADE
A = (L,S,G'~O,F,E) with natural metric dN on E. For any fixed K,
XE~* be an arbitrary input tape such that
let
x
is input under
p(x,~)
k
~ =
=k
:S
K.
If
(eO,e , ••• ), we evaluate
l
F
= ~O(eO)(i~lG(cri,ei»)n.
probability when
than
some environment
L(x)
o.
Let us now consider the acceptance
N by less
are perturbed with respect to d
eO, ••• ,e
k
That is,
max
dN(ei,e~) <
·0
1.=
, ••• ,K1 .
o.
If we assume that
~O is
44
continuous at
eO
and
G is continuous at each
l~o(eo)-~o(eo)1 < e: 1 (e O)
i = 1, •.. ,k.
and
max IG(o,e i ) - G(o,ei)I < e: 2 (e i ),
OE}";
e: * = max(e: (e )'
max e: 2 (e.», then
1 O 1=
.1 , ••• , T.r
l.
I\.
Thus, if we let
maxIG(o,e i ) - G(o,ei)I < e: * , i = l, ..• ,k.
oEE
N
that
max
d (ei,ei) < 0 implies
i=O, ..• ,K
max IIG(o,e i ) - G(o,ei)\
o€:}";
I
e , i=l, ... ,K, then
i
*
< ne: , i
Using lemma 3.2.3 we see
<
Ile: * and
= 1, ••• ,K.
By lemma 3.2.2 we have
I I~(x,~) -
(x,~')1
I = I I~o(eo)
k
k
1.=1
i =1
!TG(oi,e i ) - ~O(eO) TTG(oi,ei)I
~
I I~o(eo)
~
fie: * + kne: *
- ~o(eo)1 1+
S (K+l)ne: *
and
Ve'EE
00
L IIG(oi,e i )
<
-
i=l
VXEL * such that
N
max
d(ei,ei)
i=O, ... ,K
such that
k
L(x)
~
G(oi,e~)1 I
K
o.
Now, consider
Ip(x,~ - p(x,!:.') I
=
S
I (~(x,e)
I
- ~(x,~Y»nFI
I~(i)(x,e) - ~(i)(x,~')1
i
Thus,
max
dN(ei,ei) < 0
i=O, .•. ,k
.
II ~(x,~
for anv
.
_·tr(x,~)
II
S
implies
.
(K+l)ne:*and
XEL * such that
L(x) S k.
I~(x,~ - ~(x,~')1 s
Ip(}~,e)
' I
- p(x,!:.)
S
I
(K+l)ne: *
45
Since
~ve
°
let
at each
e
<
and
"7
J....
E*
are fixed finite numbers, given any
n
N
max
d ( e.,e.') < u~
1.
1.
i=O, .•. ,K
maxIG(a,ei) - G(a,ei)
aEL
Theorem 3.2.1.
metric
E.
on
L(x)
for i : ;,; 1, ••• ,K.
A=
~
2.
G is continuous with respect to
implies
L(x)
~
K.
be an ADE with natural
= (eO,el, ••• )EE
TI
O
such that
such that
(L,S~G,TIO,F,E)
For any fixed
G
In turn, this implies
E:
1.
is continuous with respect to d
then given any
<5
Let
and
00
and finite non-
K, if
negative integer
<
< E
0,
and
Ip(x,~) - p(x,~')1 ~ (':+l)m.: * <
that
0
at
implies
i,
I
o>
there exists
i = 1, ••• ,TZ
i
By the continuity of
E/(K+l)n.
<
E >
E:
>
°
o>
there exists
Ip(x,~) - p(x,~ I.) I <
E:
°
d
N
N
at eO;
at each e
such that
for all
xEE *
i
i = 1, ••• ,K
max dN(ei,ei)
i=O, •.• ,K
such that
K.
~
When the continuity conditions are satisfied, we can apply theorem
3.2.1 to say that for any
p(x,~')
+
p(x,~)
as
used to approximate
K and any
XCL *
such that
L(x)
~
K,
max
dN(ei,ei) + 0. Thus, p(X,~i) can be
i=O, .•• ,K
p(x,~) and we have a bound on the maximum error of
the approximation as a function of the largest perturbation of
Example 3.2.1.
S = {sl,s2,s3} ,
is
and
Consider the ADE
E
=
[0,1].
A with L = {0,1, ••• ,m-1} ,
The basic matrix transition function
46
G(o,e) =
m-oe-l
m
11m
oe
m
1
°
°
°1
°
and the initial distribution function is 11"0 (e) = (e,l-e,O).
F
F = {s3}; hence, n =
Also let e = (1,1, .•• ) •
[~]
.
Suppose Euclidean distance is the natural metric on
we have
l11"o(e') - 11"0(1)1 = Il-e'l = dN(l,e')
~ Il-e'l = dN(l,e').
So then
11"0
and
Given any
E >
satisfies
max dN(l,e.)
i=O, ••• ,K
1
0, we can find
8
L(x) = k ~ K,
0
>
and we have
I
i=O
le~ 1
I
i=O
I<
=
I
k
= eO'
I
i=l
E,
where
M(o) = G(o,l).
1
mi
1
0.
~.
i=l m
0 < E/(K+1)
So we choose
k
0ie~
i=l
m
k
I-i-I
0.
~
i=l m
I~
And
< E.
A operating under the environment (1,1, ••• ), we
see that such a system is a probabilistic automaton
over
that
o.e~
This last inequality follows from lemma 3.2.2.
If we consider
K.
8 we have
F
- p(x,(l,l,···»1 = leo
lei-II ~ 8(K+l)
~
In fact, we can see that for
k
p(x,(l,l, ••• »
co
~'EE
such that for every
p(x,e') = 11"O(e ')( IT G(o.e~»)n
O i=l
1 1
Ip(x,~')
11 .
k
clearly,
xEE * such that L(x)
max
Il-e~
i=O, ••• ,K
1
k
And similarly,
Thus,
maxIG(o,e') - G(o,l)1
oEE
are continuous at e = 1.
=,
Ip(x,~') - p(x,(l,l, •.• »1 < E.
k
g
E.
and
K be any nonnegative integer and
Let
Let
Al = (S,M,11"O,F)
S,F, and
E are in
A and
11"0 = 11"0(1)
So for any
AE[O,l),
T(Al,A)
= T(A,(l,l, ••• ),A)
and
since
pl(x) = p(x,(l,l, ••• » VXEE*.
Hence, we can use the probabilistic automaton
1
A
to approximate
47
the acceptance probability of any input for
nonnegative integer
K,
1
p (x), where
~ p(x~~') ~
K+l times
(1-8, ••• ,1-0,0,0, ••• ).
T(A,e( 61 K) ,A)
But
by
T(A 1 ,A).
lim
0-+-0
AE[O~l),
any
max d
i=O, ••• ,K
Hence~
A.
Moreover, for any
*
XEL,
and any
N
(e~,l)
p(x,~(8,K»
=
~(o,K)
< 0 and
I
T(A9~(o,K),A) c T(A,~',A) c
is strictly increasing as
6 -+-
T(A ,A).
° and bounded above
So the limit exists.
T(A,~(o,K),A)
= {XEL *1 xET(A 1 ~A)
and
lim T(A,~(o,K),A) is strictly increasing
0-+-0
1
above by T(A ,A); hence~ this limit exists.
Also,
lim lim
8-+-0
~ K}
L(x)
as
K
+ ~
L(x)
~
K}
•
and bounded
T(A~~(o,K),A)
K~
= lim
{XEL * IXET(A 1 ,A) and
K~
= T(Al,A)
Similarly, lim lim
0-+-0 K~
T(A,~(8,K),A)
= T(A 1 ,A).
Now we can say that
lim
K~
for all
lim N
T(A,~' ,A)
max
d (ei,l)-+-O
i=O ~ •••.,K
= T(A1,A)
AdO,l).
3.3 Ergodic Conditions for s-Stability
In the previous section we produced sufficient conditions for a
stability result on sets of input tapes of bounded length.
We shall
now consider the asymptotic properties of long products of stochastic
48
matrices to generate sufficient conditions for s-stabi1ity.
under such conditions for some fixed
find
0 >
°
€
> 0, we can
such that if the largest perturbation of the environment
is less than
0, then the largest perturbation of the state distribution
function is less than
for any input tape.
E
Definition 3.3.1.
and
<Xl
eEE, given any
That is,
a- = m1n(a,0).
For any real number
a+ = max(a,O)
a, let
For any
d(P) =
n
- L (p
For any
Lemma 3. 3 .1.
i=l
i
-q )
i
Proof:
N'uN" = {1,2, ••• ,n} and N'nN" =
n
° = i=l
Lp
n
1
L qi =
-
1=1
+
n
n
= L (Pi- q ·) +.L
i=l
Hence,
n
I
i=1
1
(p -q)
i
4>
i
+
1=1
(Pi-qi)
n
= - I (p -q )
i=1
i
Also,
i
+
n
n
= L (p.-q) = - L (P.-q.)1=1
1
i
i=1
1
1
o
49
Lemma 3.3.2.
1.
0
2.
a(AB)
a(A)
~
Proof:
o~
L(a
j
i
j-a
1
~
A,Bd~ •
For any
n
1
~
a(A)a(B) •
i
Let us consider any fixed indices
i
j)+
2
~
L ai
j
j
= 1.
Hence
1
0 ~ a(A)
=
N' = {jIL(a. k_ai k)b k .
k 1.1 '. 2
.J
~
max
i
Again, let us consider any fixed indices
i
1
and
1
1
,i
2
i
2
, then
L(a. j-ai j)+~ 1.
1. 1
2
j
2.
and
i
Let
and
i , but
2
OJ. Now we have
By 1ennna 3.3.1
Note that the elements of
S
Nl
depend on
11
max \
+
k L(b .-b ·)
k
k
J
K1 , 2 j
'"1 J
2
,
= a(B)
50
which is independent of
a(AB)
i
and
l
=
i •
2
l ( l(ai
max
i ,i k
l 2
~ max
i
=
Lemma 3.3.3.
I
such that
~i
l
,i
2
1
k-ai k)b kj )+
2
+
l
(ailk-aizk) o(B)
k
BEM
o
and any
n
I I~BI
then
k
d(A) a(B) •
For any
= 0,
Therefore,
I
~
n-dimensional vector
~
I I~I IO(B).
i
Proof:
~
If
is the zero vector, then
0
=
I I~BI I = I I~I lo(B)
and the result is true.
Let
vectors
~
1
and
~
2
L ~i =
I;
1
2
,1; €V
n
Let
rows are
and
2~+
i
II~ II
r'
+
0, we have that L ~.J.
i
~
1
-
I;
2
=
2
and
~
=
i
=-
l
i
So
2o(RB)
-2E;~.
Define the
.
II E.:II
~~ = 1/2 LI~il
Rence
2~
II ~ II
REMn be.
' s:!lch that its first row is
1;2.
= O.
as
~
i
~i
i
I;~ =
Since
l
be any nonzero vector such that
~
= 2 I(I(z;t j k
Z;;)b
kj
,..1
.,
and all of its other
)+. By lemma 3.3.1,
51
2a(HB)
= rII(~k1 j k
Thus,
~2)b
k
.
kJ
ll£~lL = a(HB) ~
II t; II
I IE;BI I
the result
IlL
2
k kj
II E;; II
j k
a(H)a(B)
~
I
a(B)
=
For any
BEM
Let
a
i
And therefore,
o
n
denote the
and any
nxn
i-th row of
i
A.
i
i
= a(B) max Lla.jl =
A such that
rIa .1
.
j
matrix
in lemma 3.3.3, then
E;;
~ d(B) max II a~ II = a (B) max
i
II E; II .
k kj
by 1ennna 3.3.2.
all of its rows satisfy the condition for
Proof:
11\t;b 11=2J.illll
2
II t; II ~
I lE;I la(B).
~
Lemma 3.3.4.
I=
t; b
j
J
o
IIAIld(B) •
1
Lemma 3.3.5.
IIAB-B II
Proof ~
and
I IA-In II
~ 2.
II (A-lh)B II.
=
Hence, by lemma 3.3.4
Definition 3.3.2.
(w-ergodic) iff for any
a(Z1(x,e»
<
£
Each row of
VeEE00
A-In
I IAB-BI I
sums to
~ 2a(B).
0
0
A = O::,S,G,1f O,F,E) is ''1eakly ergodic
there exists N(£) such that
An ADE
£
and
>
0
•
* such that
vXEE
L(x)
~
N(£).
The weakly ergodic condition is a generalization of quasidefiniteness for probabilistic automata.
That is, for any two stochastic vectors
52
1T
and
~,e have
p,
*
~xEE
s: 2E
I (1T-P)i'1(X,~) I
such that
N(E).
a(M(x,~»
weakly ergodic, then
Definition 3.3.3.
~ =
~
L(x)
Let
(ek,ek+l, ••• )€E""
£
An ADE
0 < 15 < 1
=
L(x)
= k.
Proof:
N(E)
k-tailof
= {~ Ik€J}
e.
'IeEE""
< 15
Ve€E""
0 < E < 1,
E* (~)
e.
and
A is w-ergodic, then for any
If
The total environment
A is w-ergodic iff there exists
such that
Choose any
""
(eO,e1, ••• )Ef..
d(H(x,~)
such that
2a(M(x,e»
If we have an ADE that is
is called the
Theorem 3.3.1.
S
as L(x)+oo uniformly with respect to e.
+ 0
is called the set of all tails of
and
II (1T-p)r:l(x,~) II
s:
0 = E and
then let
E >
VXEL *
and
such that
0, there exists
such that
~
L(x)
N(E).
k = N(E).
For the converse, suppose there exist
k€J
and
ifxEL * such that
that
YXEL*
kEJ
0
L(x)
= k.
such
1
< 0 <
For any
n
f;
> 0
we can find
that
L(x)
~
L(y )
i
=k
and
11(x~~)
=
()(H(x,~»
kn O'
such that
Then
L(y)
~
x
O.
(~l E(yi '~(i-l»
15 0 < £ .
has the form
x
Let
)11(y
'~Ok)'
where
Also by lemma 3.3.2, 0 S a(M(y,e k»
-nO
0, choose
N(E)
eEE""
And by lemma 3.3.2,
nO
£ >
tape such
Thus, for any fixed, but arbitrary
(i~l ()(l1(Yi'~(i-l»»d(l1(Y'~Ok» s: 0
Thus, given any
be any
= Yl""YnOY'
nO
S
x
S 1, so
= kn O'
.
()(ivr(Y'~Ok»
S
0
nO
o
•
< E •
53
A such that for each
Suppose we have an AIE
given Any
e >0, we can find
such that
L(x)
~
tHe ,~).
N(e,~)
such that
a(M(x,~»
VX€L *
e
<
\lIe present the following example to demon-
strate that such an ADE need not be w-ergodic.
G(o,e) =[l-e
e
A vlith
Let
Example 3.3.1.
e ]
l-e
The matrix function
=
=
G(o,e)
E = {o}, E
=
(0,1), and
G(o,e)
1/2]
[1/2
1/2
can be written as
and
1/2
U
2
=[
-1/2]
1/2
-1/2
112
k
k
IT G(o ,e ) = U + U2 ( IT(l - 2e »)·
i
l
i
i=l
1=1
Thus
k
k
M(x,~)
L(x) = k,
where
L(x) = k.
~
for any
N(e),
close to
1.
N(e).
such that
k
xd *
is not
I
However, this choice of
So for some
such that
w-ergodic.
L(J{)
e'EE
0
<
e
~
N(e)
such that
k
IT
11 -
i=l
<Xl
So given any eEE
i~111-2eil < e.
N(e>
we can find
XEL *
=
VeEf.
N(e)
II ll-2e.1 ~ II Il-'2e
i=l
1
i=l
i
<Xl
and
and any
Hence,
G(o,e ).
N(e)
a(H(x,5:,» =
L(x) = k
(eO.el, .•. )EE<Xl
i
i=l
But 11-2el < 1
<Xl
~EE,
k
IT
=
0, we can find
>
=
~
Now, for any
e
k
aITT G(o ,e i ) = (U 1 + U2 ( IT(l - 2e i »)) = Tr 11 - 2eil
i=l
i=l
i=l
2e.1
1
and any
Thus, for this
*
such that
< e:
\1xd:
N(e)
depends on
~.
And
N(e)
II 11-2e~1 is arbitrarily
i=l
1
<Xl
1 and any N(e), there is e'EE
such that
<
whereby
a(rI(x,~'»
>
€.
Hence,
A
54
Dafinition 3.3.4.
00
at
e€E
An ADE
~ > 0
iff for any
A
=
(E,S.G'~OF,E)
N(~)
there exists
such that
Theorem 3.3.2.
00
~€E
each
A is an ADE that is
If
such that
w-ergodic, then for
A is
If
But for any
00
we have
E* (~) c EllO •
as above; and hence,
N(e)
such that
~
L(x)
property is a pointwise property.
~
)
o
~
N(€).
"".~€E *(~
d(M(x,~ ) < ~
IlO
E
and
while the ever-ergodic
If the choice of
N(€)
is not de-
A is ever'-ergodic at each point of
and i f
L(x)
Then, clearly, given any
N(€).
The w-ergodic property is global in
pendent on
0, there exists
such that
~€E,
0, choose
~ >
w-ergodic, then given any
such that
Yx€E *
~ N(~).
L(x)
, A is ever-ergodic at e.
Proof:
~ >
is ever-ergodic
00
E , then
A is w-ergodic.
Lemma 3.3.6.
If
A
Further, if there exists
such that
A is ever-ergodic at
Proof:
Let
> 0, the choice
~k'
need not depend on
Suppose there exists
Therefore, given any
'lfjEJ
e.
k€J
~
and
~k€J.
c
A is ever-ergodic at
N1
such that
,(
(e::)
L(x)
~.
such that
~
E* (e).
Also, given any
k.
such that
e > 0, there exists
k€J
k€J, E* (~)
For all
A is ever-ergodic at
N(~)
then
A is ever-ergodic at e.
then
A be ever-ergodic at
Clearly, this implies
~
~,
A is an ADE that is ever-ergodic at
Nk(e::).
For this
55
to prove ever-ergodicity at
Let
xd
*
such that
and
~
L(x)
N(e)
~ Nk(e:).
L(y)
we simply choose
~
= Nk(e:)
Thus,
+ k.
So
=
x
~y,
where
fl(X,~) = H(~1~)tl(y,~).
Now
by 1ennua 3.3.2
a(M(x,e»
:;; a(II(~,~) a(H(y ,~»
o
:;; a (U(y,~» < e .
Lemma 3.3.7.
o<
0
1
<
\vhere
k
" *
VXi€L
An ADE
such that
Proof:
there exists
L(x )
i
o
<
such "that
N(d
~
= k i +1 -
N(e).
j
-J
*€J
o = £,
0 < 0 < 1
3uch that
j
then given any
~,
"'~€E * (e)
< e
K
= N(e) ,
and
0
»
<
and
:;; 0
k
i
=
e > 0, there exists
:tl
nO
e < 1,
'fX€L
*
iK.
and a sequence (k ,k , ... )
O 1
We wish to examine the behavior of
e, e., where
-
a (H(xi'~
K, such that
k •
i
a (l1(X,~»
Choose
Therefore given any
e.
iff there exists
i
A is ever-ergodic at
If
L(x)
for some
S K
Suppose now we have
above.
nO
e
and an increasing sequence of nonnegative integers (k ,k , ..• ),
O 1
O = 0 and ki+f- k i
such that
A is ever-ergodic at
as
such that
at an arbitrary tail of
is any fixed, but arbitrary positive integer. We can find
k.*~j
J
and
kj*~l:;;j.
So by lemma 3.3.2 we deduce
Let
X€L
*
such that
L(x)~(nO+l)K.
Thus,
56
Since
is an arbitrary positive integer, given any
j
find
'Ie
and
ifxd
e •
1
Hence, by lemma 3.3.6
such that
~
L(x)
Definition 3.3.5.
such that a(M(x,~»
nO
and
N(E).
That is,
iff for any pair of rows
i
~~j'
j=1,2, •.•
0
e.
0 > 0, a matrix A€M
and
l
< E
A is ever-ergodic at
A is ever-ergodic at
For some
0, we can
E >
n
i , there is a column
2
is
a-scrambling
such that
j
and
Definition 3.3.6.
the ever-H (o)
4
Vx€E *,where
An ADE
condition at
~
L(x)
N(o),
e
A = (E,S,G,nO,F,E) is said to satisfy
iff there exists
M(x,~)
is
;,](0)
o-scrambling
such that
*
'11~€E (~).
These definitions are extensions of the concepts concerning probabilistic automata.
a(A) <
1-0.
Note that if
A€M
n
is
We now extend theorem 1.3.6.
Theorem 3.3.3.
A is ever-ergodic at
A satisfies the ever-H (0) condition at
4
e.
A is ever-ergodic at
~,
Proof:
there exists
such that
o-scrambling, then
If
N(E)
L(x)
arbitrary row
~
such that
N(E).
M(x,~).
a (N(X,~»
For some
Since
e
iff for some
then given any
M(x,~)€Mn'
i
E
>
0,
and Irfxd
< E
k€J, let
0 > 0,
l
'Ie
be the index of an
then there exists
57
s.€S
mi ,(x,~)
lJ
such that
J
Imi
i 2fi 1 ,
2J
is
~
L(x)
= 1/2n,
€
The choice of
Consequently,
is arbitrary.
k
and
1.1
*
.X€L
and
n
such that
N(1/2n) •
4
such that
~
so for any
€,
o = 1/2
So, choose
i.
l/2n-scrambling
ever-H (o)
L(x)
<
Hence, if we choose
€.
for any
a > 0
Suppose now that we have
Hence,
d ([i(x ,~,»
'"
1/2n
N(1/2 n) = N(1/2n ).
But
lin.
.(x,~~)-mi ,(x,~,)1 <
lJ
mij(x,~) >
then
~
condition at
L(x)
~ N(o)
aC1(x'~k»
N(o).
Let
e.
implies
=
i
N(a) such that VX€L *
Thus, there exists
~1(x'~k)
V~ €E
s 1 - a < 1
k
A satisfies the
such that
and
i~(o)
0
* (e)
<
*
t'~<.€E (e).
is a-scrambling
and
0*
Vx€E *
= l~o
<
such that
1.
By lemma 3.3.7
o
A is ever-ergodic at e.
We shall now see that under a certain continuity condition, if
A is ever-ergodic at
some neighborhood of
~,
A is ever-ergodic at each e'
then
with respect to the pseudo-metric
~
in
sup dN(e"e ')
i
, J
~
1.E:
+
on
Theorem 3.3.4.
€ >
0
A is ever-ergodic at
If
such that for any
maxIIC(cr,ei) - G(cr,e~)1
I
~i
<
oo
€E
00
~€E
, then there exists
A is ever-ergodic at
p
e'
if
for all i€J+.
€
crE:E
Proof;
Let
k€J
note the matrix in
of
N(x,~).
max
L
Imi,-ro,
j
J 1.
i
M
n
all of whose rows are equal to the io-th row
Then by lemma 3.3.1
jl
0
=2
de-
be fixed. but arbitrary.
max I(m, ,-m
i
j
1.J
1
0
1111(x'~k) - H (x,~)
,)+ , where
i oj
II =
58
Hence,
i
I IM(x,~)-M O(x,~)1 I
Since
s 2 max
A is ever-ergodic at
independent of
k
l(m, ,-ro, j)+
J.1J
i 1,J.. J.
2
for any
~~
a(n(x'~k»
such that
J. 2
0
= 2a(t1(x,~)
.-
< 0 <
< 0/6,
•
1 there exists
*
nO
,*
and \fxEE
V~€E (~)
Thus, by the above proved inequality, we have
and
Let
Hence,
x be a
IIH(x,~) Now let
o<
¥XEL * such that
L(x)
L(~)
= nO.
fixed, but arbitrary tape such that
i
11 0 (x,~)
II
e: < 0/3n •
O
< 0/3
vS i
ES.
o
Hence, for any
no
i~ll IG(oi,e k +i )
S
- G(vi,e
no
by lemma 3.2.2
<
I
1=1
0/3no
= 0/3
For any
= max I(mi' J~-mi'
J.. 1 , i 2 J.
1
,)+
2J
= 1/2 max Ilmi' j-m~ J·1
J.•1 •
,J. 2 J'
1
2
k+i )I I
~
nO.
59
Thus, for any
si €S
o
i
H
a(H(~'~t» ~ 1It::(~,~~) ~
I IM(i,~)
+1 IM(X»~)
+llu
<
But
.
x
and
are arbitrary.
k
Vx€E *
by lemma 3.3.7,
d
N
E(~) = {ej
and
iff given any
The choice of
A
6
L(x) = nO.
=
Ijd+}.
uniformly continuous on
G
If
=6
k
.
< 0 < 1
i = inO.
Hence,
o
e V•
be an ADE with natural
is uniformly continuous on
0 > 0
E(~)
such that
< 0
for any
eEE(~.
e€E(~).
A is ever-ergodic at
E(~),
I
O(~,~) II
Let
(E,S,G'~O,F,E)
is independent of
Corollary 3.3.5.
i
_ H
~nlenever dN(e,e')
< £
•
O
a(H(x.~»
Therefore,
e > 0, there exists
maxIG(cr,e) - G(cr,e 1 )I
crEE
i
I
M (x,~)1
-
O(~.~)
A is ever-ergodic at
Let
- M(x,~)1
6/3 + 6/3 + 6/3
such that
Definition 3.3.7.
metric
i
O(i,~» II
then there exists
ex>
eEE
0 > 0
and
G
is
such that for
any
co
~'E:E
60
N
sup d ( ei,e V) < 0,
i
iEJ+
llhere
,
£*> 0
By theorem 3.3.4 there exists
Proof:
max II G(a ,e ) - G(a,e~)11 < £* lfiEJ+
i
l.
e v•
A is ever-ergodic at
such that i f
A is ever-ergodic at
and
~,
ad
then
A is ever-ergodic at
e' .
Uniform continuity of
G on
there exists
0
>
0
such that when
I
ergodic at
e.
€ <
£'* , we can find
A=
Let
For any
II
<
< n(£/n)
=£
€
Vxd
(l.,S,G'~O,F,E)
'*
I
< £
such that
'*
e'.
be an ADE
'*
0, there exists
>
€
0 > 0
maxi IG(a,e i ) - G(a,ei)I
aEl.
A is ever-ergodic at
11I:1(x,e) - H(x,~')
I
ad:
implies
Theorem 3.3.6.
we have
Hence, by lemma 3.2.3
~ n maxIG(a,ei) - G(a,ei)
Thus, if we choose 0 <
By theorem 3.3.4
implies that given any £ > 0,
sup dN(ei,e~) < 0,
iE:J+
~iEJ+.
maxIG(a,ei) - G(a,ei>1 < £/n
aEl.
maxi !C(a,e i ) - G(a,ei)1
aE:L
E(~)
whenever
£ >
0
that is ever-
such that
maxi IG(a,e i ) - G(a,ei)1
I
< £
ad:
Proof:
---
By theorem 3.3.4 there exists
we can find
YkE:J
such that
A is
max II G(a ,e ) - G(a,e ) II < £1' '1iEJ+.
i
t
aE:l.
A is ever-ergodic at both e and e' • So, given any £ > 0,
ever-ergodic at
Hence,
£1 > 0
and
e'
llhenever
such that
such that
a(l1(x .~»
L(x)
;::?;
no'
< £/6
and
*
61
0 < £2 < £/3n O•
Choose
I IM(x,~)
kEJ
whenever
- M(x,~)1
£*
= min(£l'£2)'
maxIIG(a,e i ) - G(a,e~)
aEI:
=~
Since ~
*
II
and
<
£*
I
ViE~+.
< £2
The above results remain true when
ifiEJ+.
~ = ~i
III:Hx,~) - M(x.~')
we have
,
L(y)
L(x) ~ nO
XEL * such that
= nO
and
L(x)
=k
tJe can represent
L(x)
~
~ 0
for some
nO'
+llrI(x ,~9 )I1(y ,~)
kEJ.
+
~ 2a(M(y~~»
a (H(y ,~» < £/6
IIH(y,~) - H(y,~)
II
<
e:/3.
such that
L(x) ~ nO
II
II
and
II
by lemma 3.3.5.
d 01(y,~»
<
£/6.
Also
Thus
IIH(x,e) - H(X,~9)
-};
II
2a(M(Y,~»
+IIH(y.~) .. H(y,~)
L(y) = nO' then
.
x = xy,
Hence
- H(y .~)
+IIH(y,~) _. H(Y'~k)
VXEI:
< £/3 < £
whenever
::; IIM(x,~~1(y '~k) - H(y ,~)
Since
II
maxi IG(a,e i ) - G(a,e~)l I < £*
O'EI:
*
Thus, we have the desired result
VXEI:
such that L(x) < nO .
such that
Now let
where
~XEL* such that L(x) ~ nO
£/3
<
maxi IG(a,e i ) - G(a.e~)1
aEI:
Now let
~xd
I
By lemma 3.2.2 we see that for any
II
<
2(£/6) + 2(£/6) + e:/3 = £
maxi IG(O',e.) - G(O',e~)1 I < £
1
1
O'Ei.o~
whenever
Hence, the result is true
*•
~x€E
o
62
Theorem 3.3.6 is a useful stability result.
certain continuity conditions we have
Corollary 3.3.7.
metric
then
d
N on
E.
A
Let
=
s-stabi1ity.
(L,S,G,TIO,F,E)
~EE
If for some
co
A is ever-ergodic at
2.
G is uniformly continuous on
3.
TI
A is a-stable at
e
E<,~) =
{eili€J+}
,
eO
e.
e;'* > 0
By theorem 3.3.6, given any e; > 0, there exists
Proof:
--such that
be an ADE with natural
,
1.
O is continuous at
Noreovcr, under
IIH(x,e) - H(X,~I)
II
<
e;/2
\tXEL
*
uhenever
< e; * "IiEJ+. Since G is uniformly continumaxIIG(cr,e.)
-G(cr,e!)11
~
~
crEL
ous on E(e) , then there exists (\ > 0 such that
maxi IG(cr,e ) - G(cr,e )I
i
i
crEL
I
<
e;*
whenever
The initial distribution function
so there exists
dN(eO,e
O) <
02.
02 > 0
such that
l7T
sup dN(ei,ei)
iEJ+
TI
01.
O is continuous at
o(e o) -
Hence by lemma 3.2.3
<
TIo(e
IITIo(e o)
-
o)I
<
TIo(e
o)I I
e;/2n
eO'
when
< e;/2 when
N
d (eO,e O) < 02·
Hence, whenever sup dN(e.,e~)
iEJ
~
~
by lemma 3.2.3
by lemma 3.2.1
<
Hence,
A
£/2 + e;/2
is· a-stable at
e.
= e;
IIXEL * •
u
o
<
0,
Example 3.3.1-
L
= {a},
A = ( L: , S, G~ 'IT
Let
and
S = {s1,s2,s3}
= [-1/2,1/2].
E· that is, for any
be the natural metric on
The function
E
°'F , E) be an
~
63
Euclidean distance
Let
N
1/3+e
=
5
0
4
e /2
e 4 /2
2/3-e
1_e
" / 5 (2-e 5 ) /5
(3+e~
'lTO(e)
=
Let
eEE~
such that
y4/32-scramb1ing
N(y4/ 32 )
0
1
VeEE
such that
lei ~ y/2.
~~.
the ever-H 4 (y /32) condition at
A is ever-ergodic at
The components of
Hence, for
4
= 1, A satisfies
By theorem 3.3.3
4
lim infle.1 ~ y for some y > O. Hence,
i-+oo
1
le.1 ~ y/2 > 0 Vi~N. But G(a,e)
such that
N
5
4
4
4
(1/6+e , 1/3+e , 1/2-2e ).
We define
is
e.
G are continuous at each
eEE.
But
a compact set with respect to the natural metric topology.
.~
~EE.
for any
Hence, given any
A
=
Hence,
G is uniformly continuous on
The function
is also continuous at any
€
>
0, we can find 0 > 0
\fxd * whenever
Let
E is
Clearly.
is uniformly continuous.
E(e)
Ie-e' I·
e, e' EE, d (e,e') =
G is defined
G(a,e)
there exists
\-'ith
ADE
(E,S,G,'lTO,F,E)
be an ADE
G
eEE.
such that
= sup Ie
iEJ
with natural metric
i
-e'
i
d
1'1
I
<
on Eo
E be a compact set with respect to the natural metric topology.
Let
G(a,e) are continuous at each eEE, then G is
N
uniformly continuous with respect to d • Iloreover, if there exists
If the components of
o>
0
such that all of the components of
o.
G(a,e) are bounded below by
64
A is w-ergodic and, hence, ever-ergodic at each
y, then
An ADE satisfying the above conditions is s-stable at every
Furthermore, the choice of
0
need not depend on
co
eEE.
co
~EE
•
e.
3.4 Response Intervals and a-Stability
Now we shall establish sufficient conditions for an ADE to be
tape-acceptance stable under environment sequence
A for
y > 0
A under total environment
A under total environment
e
~.
(eO,e , ••. ).
l
Recall that a cut-point
is isolated if there exists
*
such that
=
A is an isolated
We shall consider the problem when we have that
cut-point for
e
In view of theorem 3.1.2
'itxd: •
and corollary 3.3.7, we shall not require the automaton to be s-stable
or ever-ergodic at
e.
We shall tolerate perturbations of the trans-
ition functions as long as the acceptance
~robability
function does
not cross the cut-point.
A = (E,S,G,TIO,F,E)
Let us consider an ADE
vironment
Define the response intervals
R
i
= [0,1]
CO
e€E
and
= 1,2, ••• ,
as
L
[L
and
'i7(a,i.j) = max{gl.(a,e )} .
k
are the components of
Theorem 3.4.1.
R.,
i
1.
n { u
~(a.i,j),
'i7(a,i,j)]L
jEF
crd: j EF
where
The values
and some total en-
Let
A
=
(E,S,G,TIO,F.E)
AE[O,l), T(A.~.A) = T(A,~',A)
G(a,e.).
1.
.iJ
lve also define
be an AllE.
whenever
i
For any
A
L
~
'R
U
65
•
~~i
iE:J
IA-ril
inf
riER i
2.
where
> IG(a,e ) - G(a,e~)1
IHF)
i
'lad
~
~iE:J+
and
.
A under total environment
R , i=O,l, ••• , are defined for
i
e
as above.
Proof:
A(
u
iE:J
R
i
Consider
AE:T(A,~,A) ,
then
p (A,e)
>
A
~
o.
p(A,~'
Nm-T
p(A,~i) - p(A,e) ~
p(A,e) - Ip(A,~) - p(A,~')1 •
3, p(A,~')
- IA-p(A,~)1 ~ A.
> p(A,~)
.
A :f p(A,e).
implies
If
R = {p(A,e)}
O
A, the empty tape.
) = p (A,~) +
Hence, by condition
p(A,~') > A~
Thus,
consequently
AET(A,~' ,A).
If
A
¢
T(A,~,A),
p(A,~')
=
p(A,~)
< p(A,e)
S p(A,~) <
0
then
+ p(A,e') -
A.
Now
p(A,~
+ 1)..-p(A,e)I
by condition 3
::; A •
Thus
p(A,~i)::::;
Now consider
impossible for
Let
and
any
XOEL * ,where
the form
v
A~T(A,~' ,A)
A; consequently
other
element of
XEL * and
crEL.
•
* We can write it in
L.
Since
crEE, then it is
xa=A.
and
w be arbitrary vectors in
are defined so that for any
VEV
n
vn .
The functions
and i = L(x) + 1, we have
l
6(o,i,j) S VG(o,ei)n
F
jEF
where
l.I(o,i,j) _ 0 If (F)
i
0i
IA-ril
= inf
and
wG(o,ei)n
F
p(xo,!.) > A.
that
n
F
I
S
~(o,i,j) +
jEF
o I!(F) ,
i
ViEJ+ .
= p(xo,~').
p(xo,~) ~
A ¢ R , then
i
Since
WEV
wG(o,ei)n
S
> 0
/; (F)
ri€R i
p(xo ,~)
•
jEF
Condition 2 implies that for any
I
L ~(a,i,j)
S
jEF
66
But by condition 2,
A.
I
A<
xOET(A,~,A),
If
l.I(o,i,j) - o/(F)
jEF
p(xo,~') >
Hence,
p(xo,~)
L
S
then
p(xo,!.)
<
V(o,i,j) + 0 #(F) < A •
i
jEF
S
p(xo,~').
xOET(A,~' ,A).
A; therefore
xo¢T(A,~,A),
If
then
A.
So
Again by condition 2,
p(xo,!.') < A; hence,
xo~T(A,!.',A).
xOET(A,~,A)
Therefore,
arbitrary, nonempty tape.
empty tape, we have
metric
d
l.
N
on
A¢
E.
iEJ
then
Let
= T(A,~',A).
A=
If for some
-U R
i
00
eEE
and
is any
G is uniformly continuous on
3.
1T
is continuous at
E(!.) ,
eO'
A is a-stable at
be an ADE with natural
AE[O,l),
( --- denotes closure),
A with cut-point
xo
o
(~,S,G'~O,F,E)
2.
O
But
Together with the considerations of the
T(A,£,A)
Corollary 3.4.2.
xOET(A,~' ,A).
iff
e.
We are interested in finding
Proof~
sup dN(ei,ei)
iE:J
Clearly
0
<
A
~
U R
implies there exists
£
= Y1 /IJ(F).
1
exist
implies
i
Y > 0
1
A 4:
such that
u R •
i
iEJ
= yl/#(F)
o€~
whenever
A 4:
sup dN(ei,ei)
iEJ+
u R
i
iEJ
IA-p(A,~)1 > O.
Let
eO'
there exists
>
Ip(A,~
oZ'
-
O
2
p(A,~)1 ~
(,)
dN
eO,e
O
Now choose
are verified.
0
£
2
<
A4:R '
O
such that
Now
•
E(~),
G on
there
IA-r.1
1
~ inf
riERi
# (F)
Hence
A ~ p(A,e).
So let
By the continuity of
ITIo(e o ) - TIo(e
o)\
ITIo(e o) - TIo(eo)I#(F) < £Z#(F)
<
£2
TI
O
at
whenever
J:
u
= Y2
IA-p(A,~)1
Z'
° = min(ol'oZ)'
The conditions of theorem 3.4.1
Hence, there exists
Corollary 3.4.3.
~EE
= Y1
01 .
= Y2/#(F).
sup dN(ei,e~) < 0, then T(A,~,A)
iE:J
J.
point A is a-stable at e.
co
u R
i
iE:J
Therefore
=
Wh enever
<
implies
Yz =
O) <
A 4:
inf inf IA-ril
iE:J+ riER i
By the uniform continuity of
maxIG(o,e i ) - G(o,ei)I < £1
dN(eO,e
Also,
such that
01 > 0
Again,
such that
> 0
implies the conditions of theorem 3.4.1 hold.
iEJ
let
°
67
Let
A
=
IS > 0
such that
= T(A,~' ,A).
V~'EEro, where
That is,
A with cut-
0
(L.S,G,TIO.F,E)
, A is an isolated cut-point for
be an ADE.
For any
A under total environment
e
68
Proof:
Clearly, {p(x,~ IXd:*}
U Ri • But A ~
c
i.EJ
is a closed set.
U R
iEJ i
Hence, there exists
~YXEL * •
Example 3.4.1.
L
= {0'1'O'Z}'
S
u
R
iEJ i
and
such that
y > 0
o
Let
A = (L,S,G,1T O,F,E)
= {s1,sZ,s3,s4},F = {s2,s3} ,
Let
Euclidean distance be
for
0'1
be an ADE
and E
the natural metric on
E.
=
\'lith
[-1/2,1/Z].
We define
G
as
1l/12-e
=
4
4
1/12+e
0
0
1
0
0
0
o
0
0
1
o
0
1/6+e
4
5/6-e
4
and for 0'2 as
1/12
=
o
o
4
Ze
1/12-e
1/4+e4
4
o
o
4
o
3/4-e4
o
o
5/6+e
1
3/4-2e
4
1/4
Define
E is a compact set with respect to the natural metric topology.
Since for each
eEE, then
O'EL
the components of
G(O',e) are continuous at each
G is uniformly continuous ~nth respect to
is also continuous at
e
with respect to
d N•
N
d •
For any
lXl
Let ~ = (eO,el, ••• )€E.
R
i
R
Thus
O
= [0,1/4 + 2e 4i ] u [3/4 - 2e 4i ,1].
[5/8,1].
lXl
e€E.
Clearly,
U
id
A € (3/8,5/8),
Hence, for any
a-stable at any
=
R
i
C
[0,3/8] U
A is
A with cut-point
A € (3/8,5/8)
Note that any
69
4
{1/4 - ZeO} and
is an isolated
cut-point.
For any
0 > 0
such that
k > 0
lXl
and any
.
k
M(crl'~)
=
e€E,
there does not exist an integer
k
IT G(crl,e ) is
o-scrambling.
i
i=l
does not satisfy the ever-H (o) condition at any
4
A
Thus,
~€E""
A
Hence,
for any
0 > O.
~€E"" .
is not ever-ergodic at any
3.5 Round Off Considerations
Suppose we have an ADE
Eo
on
A=
(t,S,G,TIO,F,E)
with natural metric
We, however, are not able to measure the environment con-
figuration exactly; the environment can only be measured correctly up
to a certain approximation and the remainder is subject to round off.
That is, if
eEE
is the true condition of the environment, we can only
observe a unique round off estimate
for some
y > O.
Fy = {f y (e) IeEE}
f
y
that is
E.
y-dense in
such that
That is, for any
y
component-wise rounded value of
= {(f O,f 1 , ..• )lf i EFy }
e.
Thus
f (e)
y-
f y (e)
-
Clearly, F""
y
sup dN(ei,f (e » ~ y. Hence,
i
iEJ
N
respect to the metric sup d •
i€J
e€E,
N
d (e,f) ::; y.
For notational convenience, we shall let
F~
~
Thus, we shall assume that we have a subset
there exists a unique
Let
N
d (e,f (e»
y
such that
(e)
is
=
denote the
(f (eO),f (e ), ••• ).
y
= {fy (e)\eEE
- -
y
lXl
}.
lXl
y-dense in E
l
For any
",ith
70
Now we can use the stability results of the previous sections
to find round off schemes which maintain stability of an acceptance
set or stability of the acceptance probability function.
Suppose we have an ADE
A which is a-stable at some cut-point
fE('O •
y
A for some environment sequence
T(A,~,A)
that
y
o ~ then
<
= T(A,i~A) whenever sup dN (ei,f i )
¥eEE
00
= T(A,~,A)
o.
<
id
<
= i,
o>
0
such
Clearly, i f
N
sup d (f 0 ,e 0)
1 1
1E J
Thus for sufficiently small y > 0,
such that
N
sup d (f (e.) ~ e ) s y
i
Y 1
iEJ
T(A~!.,A)
There exists
o.
00
\IeEE
f (e)
y-
we have
=
o
such that
f y (~
= -f.
So i f we have a-
A at the rounded value of the environ-
stability for some cut-point
ment sequence and if the round off error is sufficiently small, then
the set defined by the ADE with cut-point
A remains unchanged by the
round off procedure.
We also obtain this result under the conditions of
corollary 3.4.2 when
y
Now suppose that
e:
any
*
VXE~
>
0, there exists
whenever
is sufficiently small.
A is
oo
s-stable at some -fEF y •
o(e:)
~up dNCfi,e i )
such that
<
O(e:).
Hence, given
11T (x ,f) - 1T (x ,~) I < e:
Clearly, i f
y < o(e:), then
1EJ
00
"ieEE
For
such that
fy(~)
= i,
we have
11T(x,fy(~»
- 1T(x~~)1 < e:
*
\fXE~ •
sufficiently small, the conditions of corollary 3.3.7 also
y
imply this result.
The round off procedure is also applicable to approximation of
the acceptance probability function for sets of tapes of bounded length.
In this case we need only consider
any
e:
>
0, by theorem 3.2.1 if
Ip(x,~-p(x,f)1 < e:
i
= l, ... ,k.
y
fO,fl, ••• ,f , for some
k
ke:J.
For
is sufficiently small, then
VX~E* such that L(x)sk, whenever f (e.)
y
1
= fl'
for
CHAPTER IV
Automata in
4.1.
Rando~
Environments
Basic Notions •.
We shall now consider the case of automata operating within random
environments.
In contrast to the case of the ADE t where we suppose we
can control the environment t we shall assume that the realization of
the environment is governed by some probabilistic structure.
This
formulation is useful when we are only able to make certain probabilistic
assumptions about the occurrence of any environment configuration or
when the environment configuration can only be measured by statistical
techniques.
Let
A=
environment.
jE{O,l, ••• }
where
B'
(L,StG'~O,F,E)
Let
=Jt
(Q,BtP)
let
z.
J
be an
auto~~ton
in a deterministic
be a probability space.
For each
be a measurable function from (QtB) to
is a a-field of subsets of
E.
~Je
shall use
(E,E!),
to denote
the random variable for the initial configuration of the enviI:onmcnt
and
Zjt j=l,2 t ••• , to denote the random variable fc..-.:
of the environment for the j-th input symbol.
variables
the
('onfiguration
The family of random
Z = {z. IjEJ} is an environmental stochastic process (ESP).
J
lrJ"hen we wish to emphasize the probability structure, \"1e shall use
(QtB,P,Z) to denote the process.
71
72
An automaton in a random environment (ARE) is
Definition 4.1.1.
a system (A,Z), where
A=
Let
ExE~
that G:
A is an ADE and
(E,S,G'~O,F,E)
be an ADE and (n,B,p,Z) be an ESP.
and Z.: n+E
n
Z is an ESP.
for all jEJ+= {1,2, ••• }. Now for all oEE, we
J
define G (e) = G(o,e) and assume G (e) is measurable.
°
such that G ·z.(w) = G(o,Zj(W»
call Go.z
~O:
j
is measurable.
0
j
n~~
J
by the name G(o,Zj)'
E+Vn is also measurable.
~O·zO(w)
=
~O(ZO).
Note that
matrix and
Thus, G ·z :
n
For convenience we shall
0
°
Note
~O(zO(w»
~O(zO)
Similarly, zO:n+E and we assume that
Hence,
~O·zO:n+Vn
is measurable.
such that
We shall call
~O·zO
by the name
G(o,Zj) for all jEJ+ and all oEE is a random stochastic
is a random stochastic vector.
counter no ambiguity in using the notation G and
We shall, however, en~O
in both the ADE and
A..1(E contexts.
Let
(n,B,p.z)
be an ESP with environment set
z (w) :; (zO (w) ,zl (w) , ••• ) •
map from (n, B)
00
00
to (E • B' ),
where
E.
The mapping
8,00
Define
is measurable
z
is the smallest o-field
generated by the measurable cylinders.
We can use this to define the
random matrix transition function. Let
XEE * such that
= k.
L(x)
k
Now,
M(x.z(w»
= IT G(o"zi(w»,
i=l
l.
~O(ZO(w»M(x,z(w»
and p(x,z(w»
T(A,z,A) for any
= ~(x,z(w»n F •
AE[O,l) is a set function of
The functions
and
~(x,z)
=
Furthermore,
WEn
for any
vIf is a
any x~l:* •
It is not difficult to see that
E 7f(X,Z)EV
for
11
convex set; hence
~(x,z(w»
taking values
L*•
in the set of all subsets of
vn .
Similarly, we have
E G(o,zj)dA
n
for all
OEE
XEL *
take values in
~onve~ se~; thus,
Similarly,
and for all
M
n
is a
jEJ+ and
73
EH(x,z)Eivl
so
XEL * •
for all
n
Note also that [0,1] is a convex set;
Ep(x,z)E[O,l] for all XEL * •
AE[O,l).
.~lE
and
A a real number,
ET(A,z,A) defined by ET(A,z,A) =
The set of tapes
{XIXEL * ,A<Ep(x,z)}
(A,Z) be an
Let
Definition 4.1.2.
is called the expected set of tapes accepted, or
defined, by (A,Z) with cut-point A.
Suppose
Z
(1)
,2
(2)
, ...
are independent identically distributed
(lID) sequences of randon environments.
arbitrary
*
XEL ,{p(x,z
Ip(x,z(i~1 ~
since
(i)
co
)}i=l
is a sequence of lID random variables
1 a.s. for all
law of large numbers to obtain
Thus, for any fixed, but
i=1,2, ••• , we may apply the strong
lim;
y
p(x,z(i» = Ep(x,z(l» a.s.
i=l
We can obtain similar almost sure convergence for the components of
N-+<>.>
'rT
and
Ii.
Let
A = (E.S,G,'rTO.F,E) be all ADE.
probability space ([0,1],
tive Borel field and
variable
~(a)
=
e,
~
B([O,l]).~),
Let us consider the
where
is Lebesgue measure.
B([O,l]) is the relaWe define the random
aE[O,l], on this probability space.
but arbitrary XEL * and
~EE<X> ,
For any fixed,
we define the random variables
if
~(e) ~ p(x,~)
otherwise •
74
Clearly, for any
~(eII(x,~,~(e»
X€L
*
~€E
and
= 1) = p(x,~).
00
,
l(x,~,¢)
is measurable and
E I(x,~~,~)
Also,
=
f I(x,~,~(e»d~(e) = p(x,e). The random variable l(x,~,~) has the
[0,1]
same relevant probability structure as an indicator which is 1 when A
F after input
is in a state in
x
under total environment
e
and
o otherwise.
Let
fined
z:
(Q,B,P,z)
00
n~ E•
l(x,z(c),~(.»
be an ESP
with environment set
E.
We have de-
Hence, we can consider the composition
as a function defined on the product space [0,1] x Q,
where
~(e) ~
if
-- { 1
0
l(x,z(w),~(e»
So for any fixed, but arbitrary
X€L
otherwise •
*,
l(x,z,~)
defined on the product probability space.
~nth
p(x,z(w»
is a random variable
We now obtain
E l(x,z,W)
respect to the product measure.
E
I(x,z,~) =
f
[O,l]xQ
l(x,z(w),~(e»d(~xp).
By Fubini's theorem,
E
l(x,z,~)
=
J [0,1]
f l(x,z(w),W(e»d~(e)dp(w)
Q
=
I p(x,z(w»dp(w) = E p(x,z)
n
,
where the last expectation is taken with respect to (n,B,p).
00
Suppose
{l(x,z(i)
{(z(i) ,~(i»}i=l are independent copies of (z,W).
,~(i»};=l
Hence
is a sequence of random variables which are lID
acccrding to the ·product measure
~xP.
By the strong law of large numbers,
75
1
N
(i)
lim N I I(x,z
N-+«>
i=l
input tape
,~
(i)
)
=E
p(x,z) a.s.
~xP.
Thus, if we repeatedly
under lID random environment sequences, the relative
x
frequency of acceptance of
x
is a strongly consistent, unbiased
E p(x,z).
estimator of
4.2 Random Environment Sequences of Independent Random Variables
(A,Z)
Let
be an ARE such that
of independent random variables.
L(x)
= k,
z
=
(zO,zl' •.• )
is a sequence
XEL *
Thus, for any
such that
Ep(x,z)
And, similarly, the expectations of the other automata functions are
products of the expectations of their factors.
Let Z(z)
= {ZjljEJ}
z.
be the set of all mappings in the sequence
For our purposes here, we shall consider two environmental random variables
to be equal if their distribution functions are equal.
set which is in one-to-one correspondence with Z(z).
g: Z(z)
functional
any
Z€Z(z)
EG(o,z).
and
Thus,
+
E', which is
OEL, define
nO(g(z»
= EnO(z)
and
is an ADE.
k
k
(Eno(zo»( .TTEG(oi,zi»)n
= Ep(x,z).
J.=1
F
be a
That is, we have a
For
G1(o,g(z»
Also, for any
= no(g(ZO»(i~G(Oi,g(Zi»)nF
=
£'
a one-to-one correspondence.
A' = (E,S,G',nO,F,E')
p'(x,g(z»
Let
=
XEL * ,
76
Therefore. for any
if
Z(z)
E'
is a finite set.
"(E')::;
Furthermore.
= ET(A.z,~)
A' can be simulated
is a sequence of lID random
1 and t-le can find a PA N' over
for all
E
where
a sequence of lID random variables.
= {a}
such that
and
zl'zZ""
is
The following results hold for state
A of any finite cardinality, but for the sake of simplicity of
exposition we shall restrict consideration to
N successive inputs of
denote the tape of
N
asymptotic moments of
H(a ,z).
Let
S
a.
(CLN,SN)
= {sl'sZ}'
(~+l,S~+l)
= (CLN'Sw)G(a,zN+l)'
Let
aN€t*
We wish to find the
be the first rO\'l of
~1(aN,z). Clearly, for every U)€~, (CLN,Su)€V Z for all
Clearly,
!:
~dO,l).
Suppose we have an ARE (A,Z)
sets of
ET(A,z,~).
Hence,
z
In the particular case when
variables, then
T(AIi.~)
T(A 1 ,B(Z),~) ::;
contains only a finite number of random variables with distinct
distributions, then
by a PAc
~€[O.l),
N
= 1,Z, . . . .
By the lID property,
(ECLN+l.EB N+l )
= (ECLN.EBN)EG(a.z N+l ) =
matrix A€M,.
is ergodic i.f it can be considered as a transition matrix
n
(ECLl,EB l ) (EG(a,zl»N.
We say a
of an irreducible ergodic Markov chain as defined in Feller [6].
Thus, if
is an ergodic matrix, then (EG(a,zl»
EG(a,zl)
to a limit matrix L .
l
The rows of
converges to (ECL ,E8 )L
I
Let
k
1
and
k
= t l , where i 1 is any row of Ll ,
= 1.
N::; 1, Z • • • ••
converges
are identical; thus (ECLN,EB )
N
be any fixed positive integer.
N ::; 1,2, ••• , (CLN+B N)
W€~
l
L1
N
For every
W€~
k k k-l
k
Thus (CL N'(1)CL N BN ••••• BN)EVk+ l
From the above. we
CL N+l
= (lngll(a,zN+l)
SN+l
=
knO\l
that
+ 8NB2l (a,zN+l)
CL NB12 (a.z N+I ) + Bug22(a,zN+l) •
and
for every
77
where
gij(a,zN+l)
is the (i,j) component of
G(a,zN+l).
By taking
the appropriate products we can derive an expression for each component
k
k
k k-l
of (aN+l'(l)aN+l6N+l, ... ,SN+l) in terms of the components of
k k k~l
k
(aN'(l)a N eN, .•• ,SN) and G(a,zN+l). This reduces to the relation
k
k k-l
k
k k k-l
k
(aN+l'(l)aN+lSN+l,···,SN+l) = (aN'(l)a N SN,···,SN)G(a,zN+1)[k]'
where
G(a,zn+1)[k]
is the k-th Kronecker power of
G(a,zN+1) as
defined in Bellman [3]. Observe that G(a,zN+l) [k]EMk+1 • Again, by the
lID property
k
k k-l
k
N
(Eal,E{l)a l Pl,···,E8 l )(EG(a,zl)[k])
=
If
EG(a,zl)[k]
limit
Lk
is an ergodic matrix,
with identical rows.
k
k k-l
k
(EaN,E{l)a N BN, ••• ,Ee N)
(EG(a,zl)[k])N
converges to a
Thus as before,
converges to
~,
a row of
So ~
M{a N,2).
Lk .
is a vector of asymptotic moments of the first row of
4.3 Limit Results for Sequences of Environment Processes
A=
Let
space.
Let
defined on
2
(r,s,G'~O,F,E)
be an ADE and (n,B,p)
be a probability
l a (n,B,p;E) be the space of all environment valued sequences
n
such that for any
= (zO,zl' ... )Ela(n,B,PjE),
for almost every
WEn.
LElzj(w)l
jEJ
a
<
00
For the sake of generality, we do not give an
explicit expression for the norm of the environment configuration.
78
z(1) ,Z(2)E! (n,B,p;E), if we consider
For
= z(2)
z(l)
CL
z(l)(w)
iff
=
z(2)(w) a.s., then! (n,B,p;E) is a metric space.
CL
The metric for this space When
(2»
d CL «z1)
,z
where
(l) = «1)
Zo
Z
(1)
,zl
arbitrary elements of
Let
!
"..
CL
-
an
d
e,e'EE.
= «2)
Zo ,zl(2) , ••• )
(2)
Z
A. We say G satis-
G be the basic transition function of
K such that
CLE(O,l]
is not continuous or that
z
l a (n,B,p;E)
(1)
,z
(2)
a
as
I
~ Kle-e'I
CL
for all
since other values imply that
G(o,o) is a constant for each OEE.
, ••. be a sequence of environment sequences in ,,"-
that converges to
d (z(i) ,z) + 0
if there exists a positive
a
maxi IG(cr,e) - G(cr)e')1
crEE
We shall only consider
Let
are
(~2,B,p;E).
fies the Lipschitz condition of order
constant
is
~r;'1 (1)_ (2)I CL
j;J'~ Zj
Zj
,
_
)
CLE(O,l]
i +
00.
l CL (n,B,p;E).
in
z
Let us now suppose that
a.
Lipschitz condition of order
For any
That is,
G satisfies the
XEL * and any
WE~,
by lemma
3.2.2, we have
I !M(x,z(i)(w»
- M(~,z(w»1
I
~
k
(')
L IIG(Oj,Z.1
j=l
(w»
- G(oi,zj(w»II.
J
-
However, by the Lipschitz condition
rI
j=l
IG(Oj,Z(i)(W)-C(O"Zj(W»I
J
I~
Thus
EII14(x,z(i»-u(x,z)11
= JIIH(X,z(i)(W»-M(X,Z(w)lldP(W)
r.
~ fK
n
I
j=l
Izj(i)(w)-z.(w)/CLdP(W)
J
79
d (z(i) z) -+- 0
But
a.
as
'
for all
XEL * •
Hence,
formly for all
*
XEL.
Since
i -+-
00
Z6 i) -+-
z0
(1)
1T O(ZO
in probab ili ty •
) -+- 1T O(ZO)
in probability uni-
z (i) -+If
1T 0
in probability.
is continuous, then
Thus, by the fact that the transition
functions are bounded, we have for all
lT
O
(z6 i )H1(X,z(i»<-+-1TO(Zo)r.-1(x,Z)
=
l ex (Q,B,p;t), we have that
in
Z
xEE*
IT(x,z)
that
1T(x,z(i»
=
in probability and
p(x,~(i) = IT(x,z(i))n F -+- 1T(x,z)n F = p(x,z)
in probability as
i -+-
00
Consequently, we have a stability and approximation result under the
above conditions.
It is also possible to obtain an ever-ergodic type of stability for
automata in random environments.
Definition 4.3.1.
€ > 0, there exists
An ARE (A,Z) is ever-ergodic a.s. iff for any
WCU, where
p(W)
=
"dzk(W)t:E* (z(w»
that
Note that
Zk(w)
A is ever-ergodic at
is the k-tai.1 of
z(w)
0, and there exists
whenever
z(w).
and the choice of
WfW and
N(€)
L(x)~N(€).
WfW,
Hence, for every
N(€)
such
is independent of
W.
Suppose there exists
0 < £1 < 1
such that there exists
W
€1
eQ,
80
P(W ) = 0, and there exists N(£l) whereby d (H(x,zk (w») < £
£1
*
Vzk(w»EE (z(w» whenever W(W
and L(X)~N(£l)' For any £ such
£1
nO
that 0 < £ < £1' we can find nO such that £1 < £ . By lemma
nO
*
3.3.2 we have that 3(H(x,Zk(W») < £1 < £ VZk(W)EE (z(w»
when-
where
wcH~
ever
be an ARE.
component of
G(a,zi)
almost every
WEn, then
'tad.
and
Vxd. *
0 > 0
If there exists
and
Vi€J+
< 1-0 a.s. Yad:
d(G(a,zi»
such that
such that each
is greater than
By application of lemma 3.3.1 we see that
VZkEE *(z)
W to be inde-
Thus, we can choose
£ •
(A,Z)
Let
L(x)~nON(£l)'
and
£1
pendent of
for
and
'lfiEJ+ •
nO
a. s.
< (1-0)
d(l1(x,zk»
L(x)~nO'
0/2
Thus, we have generated a
sufficient condition for (A,Z) to be ever-ergodic a.s.
Theorem 4.3.1.
E.
If
nO
is uniformly
o>
0
Let
(A,Z)
be an ARE with natural metric
(A,Z) is ever-ergodic a.s.,
continuous~
G is uniformly continuous and
then given any
such that for any ARE (A,Zi), where
satisfying
~up
N
d (zi(w),zi(w»
< 0
on
£ >
Z'
0, there exists
is defined on (n,B,p),
for almost every
WEn, we have
l.EJ
VXEL * and for almost every
WEn.
The proof of theorem 4.3.1 follows in a similar way as corollary
3.3.7 and its antecedents except that the results apply uniformly for
almost every
WEQ.
Furthermore. if we have a sequence of
of random environment sequences such that
z(i)
+ Z
Z
(1)
,Z
(2)
, •••
a.s., then under
the continuity conditions of theorem 4.3.1 we obtain that
81
TI(X,Z(i»
+ TI(x,z)
a.s. VX€E*.
And, consequently,
p(x,z(i»
+
p(x,z)
Vx€E.
*
a.s.
4.4 A Canonical Representation for Automata in Random Environments
In section 4.1 we encountered the dual nature of the randomness
involved in automata in random environments.
We have that the environ-
ment sequence is random and that for each value the environment sequence
assumes, the probabilistic state transitions are defined.
Hence, the
state of the machine after an input tape under a random environment
sequence is a function defined on a product probability space.
shall show that for any ARE (A,Z)
We
there exists an ARE (An,Y) over
the same alphabet whose transition probabilities are equivalent in the
mean to those of (A,Z), but the state of (An,Y) obtained after an input under a random sequence is only related to the value the random
environment sequence obtains.
To accomplish this we need that the
initial distribution function and the basic matrix transition functions
of
An
take on values which, in addition to being stochastic vectors
and stochastic matrices, respectively, must also have components that
are either 0 or 1.
Thus, the only randomness in the system
is the randomness of the environment sequence.
~je
(~,Y)
shall also con-
struct (AD,Y) to have a finite environment set.
Theorem 4.4.1.
is an ADE
in
E.
and
Z
Let
(A,Z)
= {z.] Ij€J}
be an ARE, where
is an ESP
A
=
(E,S,G,TIO,F,E)
defined on (n,B,p) taking values
There exists an ARE (An,Y), where
AD
=
(E,S,H,PO,F,E') is an
82
ADE and
Y
= {yj!j€J}
is an ESP defined on some (Q'
E', such that (An?Y)
values in
1.
H: I;xQ' -+ D (Mn )
2.
PO:Q' -+ D(Vfi)
3.
IF(E') <
4.
E'IT(x,z) = Ep(x,y)
5.
ET(A,z,A) =
satisfies the following properties:
00
Here we are using
'IT
VX€L, where
ET(~,y,A)
and
p
is the set of all
or 1 and that
D(Vri )
S.
l
to denote the state distribution functions
Recall from definition 1.3.3 that
stochastic matrices with components 0
is the set of all n-dimensional stochastic vectors
with components 0 or 1.
set
nxn
y = (YO,Y , ••• )
VAE[O,l) •
of (A,Z) and (AD,Y), respectively.
D(~)
,B' ,P') taking
Note that
n
is the cardinality of the state
This property implies that knowledge of the input and environ-
ment sequence determines the state
Proof:
For any
aEL
and
of(~,Y)
eEE
obtained.
we may decompose
finite convex linear combination of elements of
G(u,e) into a
D(M) by the following
n
algorithm:
l.
Set
G(a,e)
2.
Let
i
gjk
j
and
= G1
and set
i = l.
be the smallest nonzero component of
k
are its coordinates.
If there is a tie, choose
any one of the contenders.
3.
Set
4.
Construct
a.
aia(e)
(B
i
= gjk
•
Bia(e)€D(~h)
ia (e» jk
= 1.
G , where
i
by the rule:
83
b.
r~j,
For
row of
find the largest component of the r-th
Gi , say
grt' then set
(Bio(e»rt = 1.
If there is a tie within any row, choose anyone
of the contenders.
Set all of the remaining components of
c.
B. (e)
1.0
equal to zero.
5.
Set
6.
If
Gi +l = Gi - aiO(e)Bto(e).
is a matrix of all 0
G+
i l
not, increase the value of
i
components, stop.
by
1
If
and go to step 2.
2
This algorithm must terminate in at most p=n -n+l iterations regard-
less of
of
e
and
because at each iteration at least one component
0
is zeroed and the rows contain nonnegative components summing
i
to 1 - I a (e). Clearly for each i, Hi (e)ED(M,).
If for some 0
a
n
j=l j a
and e this process terminates before i = p, we define aia(e) = 0
G
i
for the remaining values of
any arbitrary value in
So we now have
crEE
and each
fa
and assign the corresponding
Bia(e)
D(M n ).
1=1 i a
eEE.
i
(e) = 1
and
Hence for each
r
a (e)B (e) = G(a,e) for each
ia
i=l i a
eEE, we have generated the follow-
ing arrays of numbers and matrices:
B
(e)
la
l
a
where
pa
(e)
m
m is the cardinality of
B
(e)
la
ill
For any
B
pa
(e)
m
il, ••• ,i =l, •.. ,p, let
m
84
a (11 ~ ... ~ i
e
)
m
=
m
ofT a.
j=l 1
(e) .
j
We now consider
lil~ ••• ~im=l~ ••• ,p}L as a list of numbers.
in that redundancy is preserved.
are not equal because
the second list.
a (il, ••• ,t )
e
m
~
0
A list differs from a set
That is, the lists
rOlL
and
{O,O}L
occurs once in the first list and twice in
Clearly for any
O.
A(e) = {a (i , ... ,1 )
e l
m
Oj
eEE
il, ••• ,im=l~ ••• ,p,
and any
Now for any eEE,
m-l
i
r j~
U
m-l
i
For each
Al
ok
(e) •.•• ,A
A
iOk
(e)
O'kEL, we may partition
POk
r
m-l
ijO . (e) /
J
m-l
IT
a
(e)
j=l i jO'
=
J
A(e) into
p
sublists
(e), where
= {ae(i 1 ,··· ,ik._I,i,ik+I ,··· ,im)
lil~ ... ,ik_pik+l, ... ,im
m
= 1, ...
lJe obtain the sum of the elements of the list
Ai
,p}L •
ok
(e)
=
1.
85
There arep
m
elements in the list
list
A(e).
For i
a~(e)
1
lists
AI'
ok
=1
+
m
L (i
• 1
-l)pJ-
j=l j
This uniquely defines each element of
= a e (i 1 , ... , im) .
and A'(e) = A(e).
We now construct the
For each
0kEL , we partition
(e), ••• ,A' (e), where
POk
a: (e)EA ' (e)
j Ok
1
ae(i1,···,ik_1,j,ik+1,· •• ,im)EAjOk (e)
and
, set
A' (e)
A'(e) into
p
sub-
iff
i = I + (j-l)p
k-l \'
r-l
;7~ir-I)P
(e) = A. (e).
Thus, we have reordered the set
JOk
JC1k
A(e) and maintained the properties of the partitioning.
Actually,
A~
We now construct a list
B'(e) = {B
to
(e)li=l,C1EL, ••• ,pm}L
by the
rule:
i = 1 and k
= 1.
1.
Set
2.
ai(e) must belong to at least one of the sublists Aio (e), •.• ,
k
A' (e), say A' (e) • Set B ' (e) = B. •
j Ok
i
pOk
JOk
°
3.
Delete one occurrence of
ai(e) from the list
A:JO
but let the amended list retain the name
4.
Increase the value of
5.
If
i -< pm, go to step 2.
6.
If
k < m, then set
and go to step 2.
by
i
A'
(e) ,
j Ok
(e) •
k
1.
If not, go to step 6.
i=l, increase the value of
If
k
~
k
by
1,
m, stop.
AI' (e), ••• ,A' (e)
C1
po
This algorithm must terminate because for each
OEL,
is a partition of the
Furthermore for each
C1EE
pm
elements of
A'(e).
each of these sublists will be emptied by the deletion process.
As a result of the construction of
for each
eEE
A'(e) and
B'(e)
we obtain
•
86
Pm
I a~(e)
i=1 ~
Vi=I, ••• ,pm,
1.
2.
For each
oEL,
A'(e) can be partitioned into
I
Ai (e), ••• ,A' (e) such that
o
= 1.
a~ (e)EA~ (e)
po
~i=I, ••• ,pm
Bto(e)ED(Mn)
when
i
Also, for any
and
Pm
I
i=l
and
VOEL
and
A~
is an index of an element in
oEL
a~ (e)B~ (e) =
~o
ai(e)
sublists
= a..JO (e)
•
Ja
1.
3.
p
Bto(e)
JO
= B.JO (e)
(e).
eEE,
p
I
I
B
(e)ai(e)
j =1 a' (e) EA ~ (e) j 0
i
JO
=
=
Hence, for each
r B.
j=l JO
(e)a.
j0
(e)
I
(e)
m
B ' (e)
la
Bv
B'
B'
m
(e) •
m
D(A~)
product of
set
with
n
n
with itself
systematic way.
Since
For each
m times.
D
n,m
k
(e)
= D(Mn ) x ••• xD (Mn),
distinct elements, then
distinct elements.
m
P om
D
n,m
Consider now the finite set
(e)
la
P 01
P
.
eEE we have the following arrangement
1
a
= G(a,e)
D
Since
n,m
the Cartesian
D(~~)
is a finite
is a finite set with
n
nm
is a finite set, we can order it in a
= l, .•• ,nnm ,
take the k-th element of
87
by our ordering and call it
D
n 1m
k = l, ••• ,n
that
nm
Ik(e) = {il (B~
, define
n
II (e)
(;1
I
k2
D • Also, for each
k
10"1
(e) = 0
kl~k2
for
eEE
and
(e), ••• ,B ' (e» = Dkl Note
i 0"
n
m
m
and k~l Ik(e) = {l ••.• ,p I.
Thus,we set
if
if
n
nm
l
k=l
n
Bk(e)
nm
l
=l
iElk(e)
k=l
Also for each
and for all
for k
Hence,
a~(e)
1
m
I
=
a'(e) = 1
i=l i
and
n
nm
l
k=l
n
nm
Bk(e)D k = l
k=l
=!
m
i=l
a~(e)(B~
(e), ... ,B ' (e»
i O"m
1
10"1
We are now ready to define <A.-- ,Y) •
u
(A, z) • Let
in
Dk •
For
E' = {1, ... ,nnm }.
k = l, .. o,n, let
Define
PO(k)
E,S, and F from
lve retain
H(O" . ,k)
1
to be the i-th matrix
be an n-dimensional row vector
of all zeroes except for a 1 in the k-th component.
let
PO(k)
be the n-dimensional vector (1,0, ... ,0).
defined for all
O"EE
and all
e'EE'
and
For
n+l~k~nnm,
Hence,
H is
is defined for all
e'EE.
where
~~k)(e)
= l, ••• ,n
k
= n+l, .•• ,n nm
~O(e).
is the k-th component of
Existence Theorem we define the process Y
~nsional
where
88
k
By Kolmogorov's
= {yo Ij€J}
J
by the finite dim-
distributions
jl, ••• ,jr€J
are all distinct.
If some
ji
= 0,
replace
Bk (Zj )
i
i
Note that the integral is taken with respect to the
Yk (zO)
i
probability space defining z.
by
0
Thus, for any
Let
r€J,
x€E * be an input tape of arbitrary length, say
°r .
Hence
where each sum is taken from 1 to n
Ep(x,y)
= kr"'kI
Bk (zr)dP
r
•
By our construction of
Po(ko)If(°l,kl) .. ·U(Orkr)l
r
j
nm
Y
ko
(ZO)Sk (zl)'"
l
Y,
89
( kI
H {a ,Itl 13
r
r
r kr
(z
r
») dP
= J no(zO)G(a 1 ,zl)···G(a r ,zr)dP
n
=
J n(x)z)dP = En(x,z)
by construction
•
n
11oreover, if we denote the probability of acceptance of
(~,y)
under random environment sequence
Ep' (x,y)
= Ep{x,y)n F =
=
(En{x,z»n
F
y
by
x
by
p'{x,y), we have
F
{Ep{x,y»n-
= En(x,z)n F = Ep(x,z)
•
Thus, since the expected probability of acceptance remains unchanged,
ET (An,y ,A)
= ET (A , z , A)
o
VA dO, I) •
Example 4.4.1.
S
= {sl,s2}
, and
E
=
[0,1].
Let
G be defined as
e
l-e
and
G(a 2 ,e)
=
e
2
2
(1_e ) (~
2
1_e
sin e
i) + sin e(l1
0)
0
°
e
]
I-sin e
2
1 0
+ (e -sin e)(O 1)
such that
(I-sir: e) (0 1)
I +
2
°
ieEE
e
;?:
eO
2(1 0) + (sin e-e 2)(0,.1)
1
VeEE
I
such that e < eO '
°
90
where
of
The approximate value
eO
is .88.
We now define
ai(e) for i=1, •.• ,6
to be the functions
0 =:; e < eO
2
e
ai(e)
sin e-e
ai(e)
=:;
1
sin e
2
0
e - sin e
3
a (e)
e
=:;
eO
e-e
2
a;' (e)
1-e
as (e)
0
e - sin e
a (e)
0
l-e
6
Hence for all eEE,
Bi a (e) =
1
ai'(e) ~ 0
0
2
6
and
I
i=l
a~(e) = 1.
We now define
1.
(~ ~)
Bi a (e) =
(~ ~)
=
(~ ~)
2
Bi a (e) =
Bi a (e)
B' (e)
3a
(i ~)
= (i g)
B
= (~
B
=
B
=
(~ i)
B
(e)
Sa1
= (~ ~)
4a 2 (e)
B
Sa2 (e)
=
(~ ~)
B
=
(~ ~)
B6a (e)
=
1
l
4a l (e)
6a1 (e)
2
3a 2 (e)
(~ i)
i)
OJ
2
(g i)
6
Clearly,
I
i=l
ai(e)Bi (e)
a
=
G(a,e)
for all
aEL:
and
eEL
CHAPTER \I
Implications
5.1
Limit Results for Markov Systems.
Automata theory is generally concerned with input tapes of finite
length.
Because of its application to :larkov systems in environments,
we shall now allow input tapes that are countable sequences of elements
of
.. .
~
E denote the set of all countable ,sequences of
We shall let
E.
elements of
If
tape of the first
k
symbols of
Definition 5.1.1
A
= 0lo2 ..• EE,
x
then let
X
k
= 0loZ ••• ok
be the
x.
Cn denote the class of all nxn stochastic
Let
matrices for which there is an eigenvector corresponding to the eigenvalue 1 that is a stochastic vector.
Thus, i f
A€C ,
there exists
n
VEV
n
such that
is an ergodic matrix as defined in section 4.2, then
A
Suppose we have a countable input sequence xEL
ment sequence
rheorem 5.1.1.
= 0loZ .•• EL
and
k
=
00
If
A
AEC •
n
and an environ-
.'
A = (E,S,G.nO,F,E)
Let
~
-+
(eO,el, ..• )EE
co
0
A is ever-ergodic at
e
2.
G(oi,ei)€C
= 1,2, .••
n
for all i
L I Ivi+1-vil I
<
co
,
where
iEj
91
be an ADE.
Let
If
1.
J.
= v.
The following theorems give conditions for
eEE.
ri(~t ,~) to have a limit as
x
vA
viEVn
such that
Vi G(oi,e i )
= Vi'
92
then tJ:-.ere exists
H(i,e)
II
limIII1(~,~) - H(x:~)
k-+oo
such that
=
=0
a(l1(x,e»
and
O.
n
Proof.
The space of n-dimensional vectors with norm
i=l
So by condition 3, there exists an n-dimensional vector
is complete.
I Iv.-vl
I~
1
such that
the space of
Let
V. EM
1
n
I Ivl I = L Ivil
0
as
i ~
00.
Since
Vn
is a closed subset of
n-dimensional vectors and since each
be the matrix each of whose rOllS is
VEV •
n
ViEV , then
n
v.'
Similarly, let
1
A
~
11(x,~)
be the matrix each of whose rows is
Note that for any
is any row of
AEM
n
v.
with identical rOtlS,
v
Clearly,
a(D(X,~»
II A II = II a II,
= O.
\'rhe rea
A.
For notational convenience, we shall denote
Thus,
~ IIG~-'Vkll + IIVk-l~(i,~)11
~ IIGk1-VjG~11
+ Ilv .G~-V, II + IIVk-H(x,~) II .
J
J J ~
Note that
r. k
Vj Okj -- VjJ
j +1
k - l-
k
IlvJ.G~-Vl
II = I li~j (V t -V i +1 )Gi +1 II
J K
k":l
s
.
i~j I Iv i -vi +1 1 la(G~+l)
k-l
L. II vi-vi +111
i=J
by lemma 3.3.3
00
~
L II vi+l-vi II .
i=j
93
It follows from condition 3 that for
can be made less than
j
~ ~J(e)
this last expression
e/3.
= II (Gi-l-Vj)G~1I ~
Also,
. 1
k
IIGi- - Vj I13(.1T G(oi,e i »)
1.
by lemma 3.3.3.
exists
for
N (e)
l
~,
A is ever-ergodic at
Since
=j
for fixed
there
j
such that the above expression can be made less than e/3
k ~ 1\ (e) •
And finally
I iVk-H(x,e) II =
sion can be made less than
11~1(~,~
Hence,
cient1y large
-
e/3
H(x,e)
Ilvk-vll.
k
for
II
~
By condition 3, this expres-
N2 (e).
can be made less than
for suf£!-
E
o
k.
Note that theorem 5.1.1 remains true if we replace the ever-ergodic
condition by the weaker condition:
E>O, there exists
given any
j+!itE)
G(oi~ei») <
i=j
that G(oi,e ) ~ C
n
i
a( I I
Let
~
1.
G(oi,ei)€C
2.
there exists
Vi€Vn
and
3.
k
and
N(e)
= 0lo2 ••• €E
and
oo
e€E
such that
We nay also weaken condition 2 to require
n
viG(oi,e i )
=
A = (E,S,G'~O,F,E)
(eO,el, ••• )€E
for all
V€V n
i
oo
•
be an ADE.
Let
If
= 1,2 ••••
such that
I lVi-vi I
~ 0
as
i ~
00.
where
= Vi
there exists a constant
= 1,2, ....
x
for only a finite number of times.
Theorem 5.1.2.
x = 0lo2 ••• €L
~j€J+.
£
for
k
y
such that
L 3(Gkj )
j=1
~ y for all
94
then there exists
}!(x,~)
limlll'1(~,~) - H(x,~)
k~
Proof.
L d(Gkj )
It
j=l
large
=0
and
II = O.
Construct
~ y,
a(H(x,~»
such that
Vi
N(~,!0
and
as in theorem 5.1.1.
Since
k
d(G _ ) lesq than e for sufficiently
k l
k
k-I
k
k
But by lemma 3.3.2, d(Gj) ~ d(G j )a(Gk _ l ) ~a(Gk_I)' Hence,
k.
we can make
a(G~) ~ 0 as k ~ ~ for all jEJ+.
Now,
IIH(~ ,e)-U(x,e) II = II G~-H(x,e) II
~ IIG~-Vkll
IIVk-H(x,~) II·
+
But,
k
IIGl-Vk II
But
I IG(ol,el)-vll I ~
I IG(ol,el)-vll la(G~)
By condition 2,
there exists
For all
k(e) >
k~k(e) + 1,
there exists
k
N2 (c)
k
.I I lVi_I-Vii la(G i )
~=2
I IG(ol,el)-vll la(G 2)
a(G~) ~
and
less than
i=2
as
0
e/4
for
L (Vi_l-Vi)Gil I
+
k
k
L I lVi_I-Vii la(G i )
i=2
k
~
=.
k
l
k~Nl (e).
k
\Ivt_I-v.lldCG.) ~c/8.
~
k
~
00
k
l
such that
i=2
~
for all i.
k
d(G i ) ~ c/16.
~ c/8.
•
Hence, we can make
I lVi_I-Vii I ~ 0 as i ~ =. So given e >
1 such that I lVi_I-Vii I ~ e/8y for i~k(c).
0,
Also, we have seen
Hence, given
I lVi_I-Vi' I
But
Hence, for
c > 0,
~ 2;
k~N2(e) we have
~ e/4.
I IG~-Vkl I ~ e/2
2, I IVk-M(x,~)1 I ~ e/2 for
IIM(~;e)-M(x,e) II ~ c for k
Hence,
+
k
I I lVi_I-vii la(G~)
i=2
k
~
2
k
k
II (G(ol,el )-VI )G2
i=k+l
k
a(G ) ~ 0 as
i
earlier that
so we have
k
~
for
k
~ max(NI (c),N 2 (c».
k ~ N3 (c).
So given any
~ max(N l (e:) ,N Z(e:) ,N 3 (c» •
By condition
e > 0,
0
95
We may again weaken condition 1 to require that
G(cri,ei)¢C n
for only a finite number of times.
By these theorems under the appropriate conditions, we have
demonstrated a consistent way to define the transition function of
~
xeL
under the environment sequence
eeE.
5.2 Applications.
Consideration of automata in environments has allowed us to generalize the finite-state system model to incorporate the various external
factors which may influence the behavior of the system.
A probabilis-
tic automaton may be constructed to account for the effects of the
environments only when the system can be simulated by a probabilistic
automaton.
Thus, the interesting case for application of the new auto-
maton in environments model is the case that the system cannot be
simulated by a probabilistic automaton.
In such a case the environment
cannot be reduced to a finite set by the induced' pseuffo:"metric or
does not have the appropriate semi-group properties.
Moreover, we may
incorporate a random structure to the occurrence of the environments.
This formulation is particularly useful where there is no control over
the occurrence of the environments and we can only make probabilistic
assumptions about the occurrences of the environments.
A traditional motive for automata models has been the behavior
of digital computers.
These systems are indeed situated in environ-
ments which affect their mechanical and electrical properties and,
hence, their output.
Lightning storms, power fluctuations, earth
96
tremors, and late evening personnel are often blamed as the source of
errors.
We admit that our sophisticated technology has produced
digital computer systems which rarely have transitions other than the
ones anticipated.
However, thera are many less reliable systems which have probabilistic transitions affected to a greater extent by the configuration
of the environment.
Information transmission systems may be modeled
by automata in environments.
The inclusion of the environments
will allow us to consider the various types of noise which may occur
nonhomogeneously during a transmission.
Another application for automata in environments is modeling
learning and pattern recognition.
The input represents a stimulus which
will encourage some desired behavior.
be some external circumstances.
The environment may be taken to
An alternative interesting possibility
for the environments would be their use as records of penalties for
previous behavior.
The transition functions may then incorporate this
history to modify the transition ?robabilities due to reinforcement of
desired behavior.
Decision procedures may be modeled by automata with the input
symbols representing the data collected
and with the environments
added to account for the cost of various actions of the system or to
account for certain external phenomena which may be controlled or
subject to random variation.
See Ash [1]; Chandrasekaran and Shen [5], Krylov and Tsetlin [9],
Tsetlin [19], Varshavskii and Vorontsova [21], and Yakowitz [22] for
references appropriate to this discussion.
97
5.3 Conclusions and Summary.
We have taken the viewpoint that every system is situated within
environments which affect its behavicr.
Accordingly, we have general-
ized the Rabin-Paz model of a probabilistic automaton to account for
the effects of the environments.
The initial state distribution is no
longer considered as a constant, but rather as a function of the initial
configuration of the environments.
Similarly, the subsequent probab-
ilistic transitions become functions not only of the present input, but
also the present configuration of the environment.
We first discussed the case of automata with deterministic environment rules
ro
e€E.
That is,
e
=
(eO,e , ••• )
l
specifies the configuration
of the environment for the initial distribution and the subsequent transitions.
Such a system is called an automaton in deterministic environ-
ments (ADE).
We might have considered the environments as a nuisance,
but actually they enable us to construct a model with greater
c~pacity
and flexibility.
It is not always necessary to consider the ADE model for systems
with environments.
Theorem 2.4.1 demonstrates that a system may be
simulated by a probabilistic automaton
finite.
~lhen
the environment set is
This involves construction of a probabilistic automaton over
a larger input alphabet which incorporates the role of the environments.
Theorem 2.4.1 led to further simulation results when the environment
set can be reduced to a finite set or when the environment has the
appropriate semi-group structure preserved by the system.
In section
2.3 we find that there are sets of tapes which can be defined by some
98
ADE but which cannot be defined by allY probabilistic automaton.
However,
it is possible that such ADE's may be simulated by probabilistic automata.
In Chapter III we tackled the stability problem.
because of
pos~ible
It is important
perturbations of the environments or some imprecise-
ness of the measurements of the environments.
In section 3.3 we discussed
sufficient conditions for an ADE to be s-stable for general perturbations
of the environments or when the perturbations are measured with respect
to a natural metric.
We needed to extend the notion of quasidefiniteness
to derive the ever-ergodic condition for the proofs.
Theorem 3.4.1 and
corollary 3.4.2 give sufficient conditions for a-stability without requiring s-stability or the ever-ergodic condition.
In section 3.5 we explored
the application of the stability results when round off procedures were
used in measuring the configuration of the environment.
Finally, we considered the case of automata operating in random
. environments (ARE).
In section 4.1 we defined the new model and dis-
cussed procedures to estimate the expected properties of the system.
In
section 4.2 we discovered that when the random environments are independent then the properties in the mean can be found by consideration of a
derived ADE.
When
r
= {a}
and
z
= (zO,zl"")
is a sequence of lID
random variables, this derived ADE is actually a probabilistic automaton; moreover, we found sufficient conditions to compute the asymptotic
moments of the components of
We observed the dual nature of the randomness of automata in random
environments.
However, theorem 4.4.1 demonstrates that there is a mean
equivalent canonical representation which eliminates the randomness due
99
to probabilistic transition; for any ARE, we can find another ARE with
finite environment set, deterministic assignment of the initial state,
and deterministic transitions which has a state distribution equivalent
in the mean.
Since the ADE is a special case of the ARE, we find that
every ADE has a cauonical representation as an ARE with the above
properties.
There are some interesting and still open problems which merit further
investigation.
At various points we have assumed that we could determine
whether or not a certain cut-point was isolated.
Beyond corollary 3.4.3
we have not given any conditions to locate isolated cut-points.
In
certain regions of the environment set it may be that the transition
functions
region.
G(o,o)
have certain components which remain 0 for the entire
In such cases there may be weaker conditions needed for stability.
There are also many opportunities for application of the new models to
various systems as suggested in section 5.2 •
BIBLIOGRAPHY
Infor.mation Theoy.y.
[1]
Ash, R. (1967).
[2]
Bachman, G. and Narici, L. (1966).
Press, New York.
[3]
Bellman, R. (1954). Limit theorems for non-commutative operations
I. Duke Mathematical Journal 21~ 491-500.
[4]
Booth, T.L. (1967).
Wiley, New York.
[5]
Chandrasekaran, B. and Shen, D.W.C. (1969). Stochastic automata
games. IEEE Transactions on Systems S~~ence and cybepnetics 5~
145-149.
[6]
Feller, W. (1968, 1966).
and its
Wiley, New York.
Functional Analysis.
Sequential Machines and Automata
Academic
Theo~J.
An Introduction to Probability Theory
Applications~ 1~2.
Wiley, New York.
[7]
F1achs, G.M. (1967). Stability and cut points of probabilistic
automata. Ph.D. dissertation, Michigan State University.
[8]
Knast, R. (1969).
ation and Control
[9]
Continuous-time probabilistic automata.
15~ 333-352.
Infor.m-
Krylov, V. Yu. and Tsetlin, M.L. (1963). Games between automata.
Automation and Remote Control 24~ 975-987.
[10]
Madsen, R.W. and Isaacson, D.L. (1973). Strongly ergodic behavior for non-stationary Markov processes. Annals of Probability
1, No.2, 329-335.
[11]
Minsky, M.L. (1967). Computation: Finite and Infinite Machines.
Prentice Hall, Englewood Cliffs, N.J.
[12]
Paz, A. (1965). Definite and quasi-definite sets of stochastic
matrices. Proceedings of the American Mathematical Society 16~
634-641.
[13]
Paz, A. (1966).
Some aspects of probabilistic automata.
9~ 26-60.
Infor.m-
Probabilistic Automata.
~ew
ation and Control
[14)
Paz, A. (1970).
[15]
Perles, M., Rabin, M.O., and Shamir, E. (1963). The theory of
definite automata. IEEE Transactions on Electronic Computers
EC-12~
233-243.
lCO
Academic Press,
York.
101
InfoTmation and
[16]
Rabin, M.O. (1963). Probabilistic automata.
Control 6~ 230-245.
[17J
Rabin, M.O. and Scott, D. (1959). Finite automata and their
decision problems. I.B.M. Journal of Research and Development 3~ 114-125.
[18J
Sa1omaa, A. (1969).
York.
[19]
Tset1in, M.L. (1961). On the behavior of finite automata in a
random media. Automation and Remote Control 22, 1345-1354.
[20J
Turing, A.M. (1936). On computable numbers, with an application
to the entscheidungsproblern. Proceedings of the London Mathematical Society 42~ 230-265.
[2lJ
Varshavskii, V.I. and Vorontsova, I.P. (1963). On the behavior
of stochastic automata with a variable structure. Automation
and Remote Control 24~ 353-360.
[22]
Yakowitz, S. (1974). Multiple hypothesis testing by finite memory
algorithms. Annals of Statistics 2~ No.2, 323-336.
Theory of Automata.
Pergamon Press, New
102
WDEX
5,10,19,72,73
acceptance set
acceptor
3
15
actual automaton
22
admissible
automaton in a det.:rministic environment (ADE) 18
automaton in a random environment (ARE)
72
continuous time probabilistic automaton (CTPA)
continuity
cut-point
definite
38, 59
9
12,13
discriminative probabilistic table (DPT)
a-scrambling
11
56
6
effective procedure
17
environment
environmental stochastic process (ESP)
environment set
71
17
ever-ergodic a.s.
79
ever-ergodic at e
54
ever-H (o) condition
4
56
4
final states
finite automaton (FA)
4
H4 condition 14
independent, identically distributed (lID)
29
induced pseudo-metric
29
irreducible
isolated cut-point
k-definite
11, 40
12,13
12
k-suffix of x
k-tai1 of e
52
length of x
6
linear bounded automaton
Lipschitz condition
78
3
73
35
103
list
84
natural metric
37
nondeterministic finite automaton
probabilistic automaton (PA)
7
0
0
probabilistic cut-point event (peE)
probabilistic table (PT)
quasidefinite
11
14
random environment
regular event
10
71
5
response interval
64
right invariant equivalence relation
round off
69
scrambling
14
semi-group
4, 32
simulate
21
strictly stable (s-stable)
strongly equivalent
tape
39
11
1,4
tape-acceptance stable (a-stable)
total environment
transducer
18
3
Turing machine
1
weakly ergodic (w-ergodic)
weakly k-definite
12,13
51
39
5
104
i'-lOT.l\TION
A 4,8,18
G(o,e)
18
T(A,~,A)
19
(A,Z)
G(o,z)
72
T(A,z,A)
72
72
B 71
I (x,~~1P)
73
V
n
B'
71
I (x,z ,~)
74
x 4
C
91
J
n
29
d
D(M )
n
N
d
L(x)
M
37
n~m
D(V )
n
E 17
foo
18
e
59
E(e)
* (~)
E
..
':k
E
y
oo
F
y
f
y
Z
71
z
71
z
72
F
9
11(x,z)
72
A
4
A
9,11
II
73
71
9
8
p(x,~)
19
lT
O
p(x,z)
72
1T
(e )
O O
19
19
18
72
O(ZO)
1T(X) 9
1T
R
21,64
1T (x,~)
19
S
4
1T(X,Z)
72
p (x,Y)
82
69
So
69
T 5
4
(e)
69
T(A)
(e)
69
T(A,A)
y-
82
n
Q'(e ,ooo,e )
k
1
73
Y
18
Q(eO,ooo,e )
k
52
4
F
f
52
T(A,z,A)
F
4,8
p(x)
18
91
r1(x,e)
P
18
e
6
H(x)
9
77
8
n
86
D
"
l (l (n ,B,P)
9
91
~
J+ 39
78
d (l
~
39
8
E
*
E
5
4
4
A
10
E
91