Outline
CDM
Closure Properties
2
1
Nondeterministic Machines
2
Determinization
3
Closure Properties
4
State Complexity
5
Alternating Automata
Klaus Sutner
Carnegie Mellon University
Fall 2015
Recall: Regular Languages
3
The Logic Link
4
As already mentioned, we are primarily interested in using automata to solve
decision problems in logic.
We have a definition of RegΣ , the class of regular languages over alphabet Σ,
in terms of deterministic finite automata (DFAs).
To this end we have to construct machines for various languages, quite often
over large alphabets of the form
Alternatively, we can consider monoid morphisms from Σ? to a finite monoid
(recognizable languages).
Γ = Σ × Σ × ... × Σ
In order to get a better understanding of RegΣ , it turns out that other
characterizations can be very useful, both from the theory perspective as well
as for the construction of algorithms.
This turns out to be the right environment for checking validity of logical
formulae over certain structures.
Language Operations
The key to many of these algorithms is the fact that regular languages are
closed under a great many more operations. To begin with, there is closure
under Boolean operations:
5
More Operations
Here are some more operations on languages that do not affect regularity:
reversal
concatenation
Kleene star
union
homomorphisms
intersection
inverse homomorphisms
complement
symmetric difference
This is useful to deal with propositional logic over structures that can be
described by finite state machines.
In a while, we will see how to handle quantifiers.
Alas, it is difficult to establish these properties within the framework of DFAs:
the constructions of the corresponding machines become rather complicated.
Ditto for recognizability.
One elegant way to avoid these problems is to generalize our machine model to
allow for nondeterminism, and show that the general machines still only accept
regular languages.
6
Nondeterministic FSMs
7
Traces and Runs
8
Here is a straightforward generalization of DFAs that allows for
nondeterministic behavior. Recall that transition systems may well be
nondeterministic.
It is straightforward to lift the definition of acceptance from DFAs to NFAs (it
all comes down to path existence, anyway).
Definition
A nondeterministic finite automaton (NFA) is a structure
Recall that in any transition system h Q, Σ, τ i a run is an alternating sequence
A = h Q, Σ, τ ; I, F i
π = p0 , a 1 , p 1 , . . . , a r , p r
where h Q, Σ, τ i is a transition system and the acceptance condition is given
by I, F ⊆ Q, the initial and final states, respectively.
where pi ∈ Q, ai ∈ Σ and τ (pi−1 , ai , pi ) for all i = 1, . . . , r . p0 is the source
of the run and pr its target. The length of π is r.
The corresponding trace or label is the word a1 a2 . . . ar .
So in general there is no unique next state in an NFA: there may be no next
state, or there may be many. Of course, we can think of a DFA as a special
type of NFA.
Some authors insist that I = {q0 }. This makes no sense.
The Fatal Definition
9
Sources of Nondeterminism
The acceptance condition is essentially the same as for DFAs, except that
initial states are no longer unique (and even if they were, there could be
multiple traces).
10
Note that nondeterminism can arise from two different sources:
Transition nondeterminism:
a
a
there are different transitions p −→ q and p −→ q 0 .
Definition
An NFA A = h Q, Σ, τ ; I, F i accepts a word w ∈ Σ? if there is a run of A with
label w, source in I and target in F . We write L(A) for the acceptance
language of A.
Initial state nondeterminism:
there are multiple initial states.
In other words, even if the transition relation is deterministic we obtain a
nondeterministic machine by allowing multiple initial states. Intuitively, this
second type of nondeterminism is less wild.
But note that now there may be exponentially many runs with the same label.
In particular, some of the runs starting in I may end up in F , others may not.
There is a hidden existential quantifier here.
Again: all that is needed for acceptance is one accepting run.
Autonomous Transitions Epsilon Moves
11
Σε
12
While we are at it: there is yet another natural generalization beyond just
nondeterminism: autonomous transitions, aka epsilon moves. These are
transitions where no symbol is read, only the state changes. This is perfectly
fine considering our Turing machines ancestors.
We will encounter several occasions where it is convenient to “enlarge” the
alphabet Σ by adding the empty word ε:
Definition
Of course, ε is not a new alphabet symbol. What’s really going on?
A nondeterministic finite automaton with ε-moves (NFAE) is defined like an
NFA, except that the transition relation has the format τ ⊆ Q × (Σ ∪ {ε}) × Q.
Σε = Σ ∪ {ε}
Σ freely generates the monoid Σ? , and ε is the unit element of this monoid.
We can add the unit element to the generators without changing the monoid.
We could even add arbitrary words and allow super-transitions like
aba
Thus, an NFAE may perform several transitions without scanning a symbol.
Hence a trace may now be longer than the corresponding input word. Other
than that, the acceptance condition is the same as for NFAs: there has to be
run from an initial state to a final state.
p −−→ q
Exercise
Explain why this makes no difference as far as languages are concerned.
Closure under Homomorphisms
13
Recall that any map f : Σ → Γ? gives rise to uniquely determined monoid
homomorphism f : Σ? → Γ?
f (x) = f (x1 )f (x2 ) . . . f (xn )
2
Theorem
Regular languages are closed under homomorphisms and inverse
homomorphisms.
Proof.
For homomorphisms, consider a NFA for L ⊆ Σ? and replace
a
Nondeterministic Machines
Determinization
Closure Properties
State Complexity
Alternating Automata
f (a)
transitions p −→ q by p −→ q.
For inverse homomorphisms, let K ⊆ Γ? . Then L = f −1 (K) is regular by our
monoid characterization of regular languages.
2
Conversion to DFA
15
Our first order of business is to show that NFAs and NFAEs are no more
powerful than DFAs in the sense that they only accept regular languages. Note,
though, that the size of the machines may change dramatically, so one needs to
be a bit careful.
Terminology
16
For the time being, we will refer to DFAs, NFAs and NFAEs simply as finite
automata.
Strictly speaking, all three types are distinct, but there are natural inclusions
The transformation is effective: the key algorithms are
DFA ⊆ NFA ⊆ NFAE
Epsilon Elimination Convert an NFAE into an equivalent NFA.
Determinization Convert an NFA into an equivalent DFA.
NFAE to NFA
The heart of the OO fanatic now beats faster . . .
17
ε-Closure
18
A transitive closure problem: we have to replace chains of transitions
Epsilon elimination is quite straightforward and can easily be handled in
polynomial time:
a
ε
ε
introduce new ordinary transitions that have the same effect as chains of ε
transitions, and
remove all ε-transitions.
Since there may be chains of ε-transitions this is in essence a transitive closure
problem. Hence part I of the algorithm can be handled with the usual graph
techniques.
by new transitions
a
a
a
a
ε
Epsilon Elimination
19
Determinization
20
Theorem
For every NFAE there is an equivalent NFA.
Proof.
Conversion of a nondeterministic machine to a deterministic one appeared first
in a seminal paper by Rabin and Scott titled “Finite Automata and Their
Decision Problem.” In fact, nondeterministic machines were introduced there.
This requires no new states, only a change in transitions.
Suppose A = h Q, Σ, τ ; I, F i is an NFAE for L. Let
A0 = Q, Σ, τ 0 ; I 0 , F
Theorem (Rabin, Scott 1959)
For every NFA there is an equivalent DFA.
where τ 0 is obtained from τ as on the last slide.
I 0 is the ε-closure of I: all states reachable from I using only ε-transitions.
2
The idea is to keep track of the set of possible states the NFA could be in.
This produces a DFA whose states are sets of states of the original machine.
Again, there may be quadratic blow-up in the number of transitions and it may
well be worth the effort to try to construct the NFAE in such a way that this
blow-up does not occur.
General Abstract Nonsense to the Rescue
21
Proof of Rabin-Scott
22
Suppose A = h Q, Σ, τ ; I, F i is an NFA. Let
A0 = P(Q), Σ, δ; I, F 0
The transition relation in a NFA has the form
τ ⊆Q×Σ×Q
where δ(P, a) = { q ∈ Q | ∃ p ∈ P τ (p, a, q) }
By GAN we can think of it as a function:
F 0 = { P ⊆ Q | P ∩ F 6= ∅ }
τ : Q × Σ → P(Q)
It is straightforward to check by induction that A and A0 are equivalent.
2
and this function naturally extends to
The machine from the proof is the full power automaton of A, written
powf (A), a machine of size 2n .
τ : P(Q) × Σ → P(Q)
The latter function can be interpreted as the transition function of a DFA on
P(Q). Done.
Accessible Part
This is as good a place as any to talk about “useless” states: states that cannot
appear in any accepting computation and that can therefore be eliminated.
Definition
A state p in a finite automaton A is accessible if there is a run with source an
initial state and target p. The automaton is accessible if all its states are.
Now suppose we remove all the inaccessible states from a automaton A
(meaning: adjust the transition system and the set of final states). We obtain
a new automaton A0 , the so-called accessible part of A.
Lemma
The machines A and A0 are equivalent.
Of course, for equivalence only the accessible part pow(A), the power
automaton of A, is required.
23
Coaccessible/Trim Part
There is a dual notion of coaccessibility: a state p is coaccessible if there is at
least one run from p to a final state. Likewise, an automaton is coaccessible if
all its states are.
An automaton is trim if it is accessible and coaccessible.
It is easy to see that the trim part of an automaton is equivalent to the whole
machine. Moreover, we can construct the coaccessible and trim part in linear
time using standard graph algorithms.
Warning: Note that the coaccessible part of a DFA may not be a DFA: the
machine may become incomplete (a partial DFA). The accessible part of a DFA
always is a DFA, though.
24
Keeping Trim
25
Of course, in implementations we would like to keep machines small, so making
them accessible or trim is a good idea. There are really two separate issues
here.
26
The right way to construct the Rabin-Scott automaton for A = h Q, Σ, τ ; I, F i
is to take a closure in the ambient set P(Q):
clos( I, (τa )a∈Σ )
First, we may need to clean up machines by running an accessible or trim
part algorithm whenever necessary–this is easy.
Here τa is the function P(Q) × Σ → P(Q) defined by
Much more interesting is to avoid the construction of inaccessible states of
a machine in the first place: ideally any algorithm should only produce
accessible machines.
a
τa (P ) = { q ∈ Q | ∃ p ∈ P p −→ q }
This produces the accessible part only, and, with luck, is much smaller than the
full power automaton.
While accessibility is easy to guarantee, coaccessibility is not: while
constructing a machine we do not usually know the set of final states ahead of
time. So, there may by need to eliminate non-coaccessible states.
Power Automaton
Smart Power Automata
27
Virtual Graphs
28
Here is a slightly more hacky version of this construction.
Of course, this is nothing but a graph exploration algorithm, except that we are
computing in a virtual graph (the full power automaton). Depending on how
we implement the extraction operation on the set of active elements, we get
Depth-First-Search or Breadth-First-Search.
active = QQ = {I};
while( active != empty )
P = active.extract();
foreach a in Sigma do
compute R = tau(P,a)
keep track of transition P to R
if( R notin QQ ) then
add R to QQ and active
Again, we do not have the standard explicit representation in form of adjacency
lists or adjacency matrices. Instead, we have an implicit representation: we
enumerate out-edges whenever needed.
Since the ambient set has size 2n this is the only reasonable way to compute a
Rabin-Scott automaton (except for little toy examples).
Upon completion, QQ ⊆ P(Q) is the state set of the accessible part of the full
power automaton. We write pow(A) for this machine.
Example: La,−3
29
Rabin-Scott
30
Here is a useful family of regular languages over {a, b}? :
La,k = { x ∈ {a, b}? | xk = a }.
Applying the Rabin-Scott construction we obtain a machine with 8 states
For negative k this means: −kth symbol from the end.
{1} , {1, 2} , {1, 2, 3} , {1, 3} , {1, 2, 3, 4} , {1, 3, 4} , {1, 2, 4} , {1, 4}
It is trivial to construct an NFA for La,−3 :
where 1 is initial and 5, 6, 7, and 8 are final. The transitions are given by
a
b
a, b
0
a
1
a, b
2
a, b
3
1
2
1
2
3
4
3
5
6
4
7
8
5
5
6
6
7
8
7
3
4
8
2
1
Note that the full power set has size 16, our construction only builds the
accessible part (which happens to have size 8).
The Diagram
31
Acceptance Testing
32
Here is the corresponding diagram, rendered in a particularly brilliant way. This
is a so-called de Bruijn graph (binary, rank 3).
aab
aaa
Recall one of the key applications of FSMs: acceptance testing is very fast and
can be used to deal with pattern matching problems.
abb
aba
bab
baa
How much of a computational hit do we take when we switch to
nondeterministic machines?
bbb
We can use the same approach as in determinization: instead of computing all
possible sets of states reachable from I, we only compute the ones that
actually occur along a particular trace given by some input word.
bba
Exercise
Explain this picture in terms of the Rabin-Scott construction.
Membership Testing
33
Running Time
Here is a natural modification of the DFA acceptance testing program.
P = I;
while( a = x.next() )
P = tau_a(P);
The loop executes |x| times, just as with DFAs.
Unfortunately, the loop body is no longer constant time: we have to
update a set of states P ⊆ Q.
// next input symbol
This can certainly be done in O(|Q|2 ) steps though smart data structures
may sometimes give better performance.
return ( P intersect F != empty );
The update step uses the same maps τa : P(Q) → P(Q) as in the Rabin-Scott
construction.
Actually, it seems that in practice (i.e. in NFAs that appear naturally in
some application such as pattern matching) one often deals with overhead
that is linear in |Q| rather than quadratic.
Exercise
At any rate, we can check acceptance in an NFA in O(|x| |Q|2 ) steps. For
fixed machines this is still linear in x, but the hidden constant may be
significant.
Think of dirty tricks like hashing to speed things up.
A Better Mousetrap?
35
Nondeterministic Machines
Determinization
3
Closure Properties
Acceptance testing is slower, nondeterministic machines are not simply
all-round superior to DFAs.
Advantages:
Easier to construct and manipulate.
Sometimes exponentially smaller.
Sometimes algorithms much easier.
Drawbacks:
Acceptance testing slower.
Sometimes algorithms more complicated.
Which type of machine to choose in a particular application can be a hard
question, there is no easy general answer.
State Complexity
Alternating Automata
34
Products
37
Suppose we have two transition systems T1 = h Q1 , Σ, τ1 i and
T2 = h Q2 , Σ, τ2 i over the same alphabet Σ.
Product Machine
38
To get a machine we need to define an acceptance condition.
The new initial state set is I1 × I2 .
Construct a new transition system T = T1 × T2 , the so-called (Cartesian)
product as follows.
By selecting final states in the product, we can get union and intersection
L(A1 ) and L(A2 ):
Q = Q1 × Q2
τ ((p, q), a, (p0 , q 0 )) ⇐⇒ τ1 (p, a, p0 ) ∧ τ2 (q, a, q 0 )
union
intersection
It is often helpful to think of the new transition system as running T1 and T2 in
parallel.
F = F1 × Q2 ∪ Q1 × F2
F = F1 × F2
Alas, the size of T is quadratic in the sizes of T1 and T2 . This causes problems
if a product machine construction is used repeatedly.
It’s a DFA
39
Absolute Complements
40
In the special case where A1 is universal we get the plain complement
Σ? − L(A2 ). Of course, we do not need a product construction here, all we
need to do is switch final and non-final states in the given DFA.
In the case where T1 and T2 come from DFAs A1 and A2 the product is again
a DFA: the new transition function δ = δ1 × δ2 looks like
F 0 = Q − F.
Q = Q1 × Q2
δ((p, q), a) = (δ1 (p, a), δ2 (q, a))
More importantly, we can also construct a machine for the complement
L(A1 ) − L(A2 ).
difference
Dire Warning: Determinism is essential here, we will see shortly that
complementation for nondeterministic machines is much harder.
F = F1 × (Q2 − F2 )
Exercise
Construct a counterexample that shows that the switch-states construction in
general fails to produce complements in NFAs.
And back to closures . . .
41
Closure Properties of RegΣ
For the umpteenth time: a real algorithm for product machines should not
construct the full Cartesian product.
At any rate, we have established a critical closure property for regular
languages:
Instead, one should compute closures of the appropriate initial states under the
transition function/relation of the product machine.
Lemma
For example, for a product DFA construction we need
clos( (q01 , q02 ), δ1 × δ2 )
Exercise
Figure out in detail how to do these constructions producing the accessible part
only.
Regular languages form a Boolean algebra: they are closed under union,
intersection and complement. Moreover, the closure is effective and even
polynomial time for DFAs.
Effective here means that, given two machines A1 and A2 , we can compute a
new machine for L(A1 ) ∩ L(A2 ), L(A1 ) ∪ L(A2 ) and L(A1 ) − L(A2 ). If the
machines are DFAs all operations are polynomial time (but complement may
blow up for NFAs).
We will have more to say about the complexity of the corresponding
algorithms; this is critical for applications.
42
Déjà Vu All Over Again
43
Deciding Equivalence
44
We can now deal with the Equivalence problem from last time.
The last lemma should sound very familiar by now: there are analogous results
for semi-decidable sets, decidable set and primitive recursive sets.
Equivalence
Two DFAs A1 and A2 .
Are the two machines equivalent?
Problem:
Instance:
Question:
Of course, for semi-decidable ones we have to omit complements.
In all cases, we can effectively construct machines/programs that recognize the
sets obtained by Boolean operations. The real challenge is state complexity: in
many applications one needs to deal with very large machines. The
corresponding algorithm may require a lot of ingenuity.
Lemma
A1 and A2 are equivalent iff L(A1 ) − L(A2 ) = ∅ and L(A2 ) − L(A1 ) = ∅.
Note that the lemma yields a quadratic time algorithm. We will see a better
method later.
Deciding Inclusion
45
46
Product constructions are important even for relatively simple languages, it can
be quite difficult to build automata for, say, the intersection of two regular
languages directly by hand.
Observe that we actually are solving two instances of a closely related problem
here:
Inclusion
Two DFAs A1 and A2 .
Is L(A1 ) ⊆ L(A2 )?
Problem:
Instance:
Question:
Aside: Complicated Intersections
Here is an example: build a DFA for the language of all words that contain the
scattered subword (not factor) ab 3 times, and a multiple-of-3 number of a’s.
Building the two component machines and taking their product we get
which problem can be handled by
Lemma
1
L(A1 ) ⊆ L(A2 ) iff L(A1 ) − L(A2 ) = ∅.
a
2
b
4
a
7
b
10
a
a
b
b
a
6
b
Note that for any class of languages Equivalence is decidable when Inclusion is
so decidable. However, the converse may be false – but it’s not so easy to
come up with an example.
Sizes of Product Machines
More generally, suppose we have DFAs Ai of size ni , respectively.
3
47
b
8
a
9
Here is the Emptiness Problem for a list of DFAs rather than just a single
machine:
Problem:
Instance:
Question:
A = A1 × A2 × . . . × As−1 × As
has n = n1 n2 . . . ns states.
Alas, there are cases where exponential blow-up cannot be avoided.
5
Bad News: DFA Intersection
Then the full product machine
The full product machine grows exponentially, but its accessible part may
be much smaller.
a
DFA Intersection
AT
list A1 , . . . , An of DFAs
Is L(Ai ) empty?
This Q
is easily decidable: we can check Emptiness on the product machine
A = Ai . The Emptiness algorithm is linear, but it is linear in the size of A,
which is itself exponential. And, there is no universal fix for this:
Theorem
The DFA Intersection Problem is PSPACE-hard.
48
Example
49
Full Product Automaton
Of course, sometimes the accessible part of the full product is much smaller.
As an example, consider the product automaton for Aaa and Abb .
50
b
⊥0
a
a
1
b
b
20
2
b
ab
21
10
⊥
a
b
51
⊥0
ab
b
20
b
21
10
01
a, b
a, b
a
22
a
11
a
01
a, b
b
2⊥
a
b
a
1⊥
a
02
b
b
a
0⊥
Concatenation and Kleene Star
52
a
Definition
⊥⊥
2⊥
a
b
a
02
L1 · L2 = { xy | x ∈ L1 , y ∈ L2 }.
Let L be a language. The powers of L are the languages obtained by repeated
concatenation:
b
L0 = {ε}
k+1
L
1⊥
a
Given two languages L1 , L2 ⊆ Σ? their concatenation (or product) is defined by
a, b
b
12
a
00
⊥2
b
a
b
b
a
b
⊥1
a
⊥⊥
a, b
a
b
22
12
a
00
a, b
a, b
a
11
a, b
The Accessible Part
⊥2
b
b
a, b
a
b
⊥1
b
0
a, b
a
b
The Kleene star of L is the language
L? = L0 ∪ L1 ∪ L2 . . . ∪ Ln ∪ . . .
a
Kleene star corresponds roughly to a while-loop or iteration.
0⊥
Star Examples
Example
{a, b}? : all words over {a, b}
Example
{a, b}? {a} {a, b}? {a} {a, b}? : all words over {a, b} containing at least two a’s
Example
= Lk · L
53
Concatenation is Easy
Suppose we have two NFAs A1 and A2 for L1 and L2 . To build a machine for
L1 · L2 it is easiest to go with an NFAE A:
Let Q = Q1 ∪ Q2 , keep all the old transitions and add ε-transitions from F1 to
I2 . I1 are the initial states and F2 the final states in A.
It is clear that L(A) = L1 · L2 . But note that this construction may introduce
quadratically many transitions.
{ε, a, aa} {b, ba, baa}? : all words over {a, b} not containing a subword aaa
Example
{0, 1} {0, 1}? : all numbers in binary, with leading 0’s
{1} {0, 1}? ∪ {0}: all numbers in binary, no leading 0’s
Exercise
Find a way to keep the number of new transitions linear.
54
Concatenation is Hard
55
Now suppose we have two DFAs A1 and A2 for L1 and L2 .
Kleene Star and More Nondeterminism
56
The situation gets worse if we try to construct a DFA for the Kleene star of a
language:
We can get a DFA from the previous machine, but can we build a DFA for
L1 · L2 directly?
L? = L0 ∪ L1 ∪ L2 . . . ∪ Ln ∪ . . .
The problem is that given a word w we need to split it as w = xy and then
feed x to A1 and y to A2 . But there are |w| + 1 many ways to do the split,
and we have a priori no idea where the break should be.
Not only do we not know where to split the string, we also don’t know how
many blocks there are.
Moreover, the number of blocks is unbounded (at least in general), and it is far
from clear how this type of processing can be handled by a DFA.
One can also think of this as a guess and verify problem: guess x and y, and
then check that indeed A1 accepts x, and A2 accepts y.
Of course, we can simply start with an NFA or even and NFAE, and then
convert back to a DFA.
Of course, there is a slight problem: DFAs don’t know how to guess.
NFAEs
57
Pebbles
58
Another way of thinking about this is to place pebbles on the states.
If we are willing to deal with ε-transitions closure under concatenation and
Kleene star is not hard at all.
Initially, each state in I has a pebble.
Under input a, a pebble on p multiplies and moves to all q such that
a
p −→ q.
Exercise
Given two NFAs, construct an NFAE for the concatenation of the two machines.
Multiple pebbles on a state are condensed into a single one.
We accept whenever a pebble appears in F .
Exercise
Given a NFA, construct an NFAE for the Kleene star of the machine.
Pebble Automaton for Concatenation
Pebbles are very helpful in particular in the direct construction of DFAs: the
movement of the set of all pebbles is perfectly deterministic.
59
Quoi?
60
We start with one copy of DFA A1 , the master, and one copy of DFA A2 , the
slave.
Place one pebble on the initial state of the master machine.
Move this and all other pebbles along transitions according to standard
rules.
Whenever the master pebble reaches a final state, place a new pebble on
the initial state of the slave automaton.
The composite machine accepts if a pebble sits on final state in the slave
machine.
So the number of states is bounded by |A1 | 2|A2 | : the A1 part is deterministic
but the A2 part is not.
First, the machine just described is deterministic: we place and remove a
collection of pebbles according to an entirely deterministic rule.
The states are of the form (p, P ) where p ∈ Q1 and P ⊆ Q2 , corresponding to
a complete record of the positions of all the pebbles.
Now let ni be the state complexity of Ai . Then the number of states is at
most
n1 2n2
Of course, the accessible part may well be smaller.
Exercises
61
Reversal Closure
62
Here is yet another example of an operation that is difficult to capture within
the confines of DFAs.
Exercise
Let
There are several gaps and inaccuracies in the outline above, fix them all.
Lop = { xop | x ∈ L }
Exercise
Carry out this construction for the languages Ea = even number of a’s and
Eb = even number of b’s and run some examples.
be the reversal of a language, (x1 x2 . . . xn−1 xn )op = xn xn−1 . . . x2 x1 .
The direction in which we read a string should be of supreme irrelevance, so for
regular languages to form a reasonable class they should be closed under
reversal.
Exercise
Explain why the pebbling construction really defines a DFA.
Exercise
How would we go about constructing a machine for Lop ? Well, flip all the
transitions around and interchange I and F . Of course, the result will almost
never be a DFA.
Carry out a pebbling construction for Kleene star.
It is noteworthy that pebbles don’t seem to help much either in this case.
Example: Third Symbol
63
But Lop
a,3 = { x | x−3 = a } is hard for DFAs: we don’t know how far from the
end we are.
By flipping transitions (and the whole machine to get the initial state over on
the left) we obtain a machine that looks like so:
It is very easy to build a DFA for La,3 = { x | x3 = a }.
We omit the sink to keep the diagram simple.
a, b
a, b
0
a, b
1
a, b
2
a
3
a
2
a, b
1
a, b
0
3
Now 3 is the initial state and 0 is final.
Applying the power automaton construction (accessible part only) to this NFA
we obtain the de Bruijn machine with 8 states from above.
Constructing a DFA by Hand
65
Nondeterministic Machines
Determinization
Closure Properties
Challenge: can we construct this DFA by hand, without any automata
conversions?
Note that we need to remember the last 3 symbols of the input: if we’ve
reached the end we have enough information to decide whether we should
accept.
So we use Q = {a, b}3 and transitions
s
xyz −→ yzs
4
State Complexity
Final states are {aaa, aab, aba, abb}.
Initial state is bbb (a clever hack, otherwise we would have to add some more
states ε, a, b, aa, ab, . . . ) and cone them on top of the strongly connected
part.
Alternating Automata
Algorithmic Issues
67
So we have three increasingly complicated types of machines: DFAs, NFAs and
NFAEs, that all accept exactly the regular languages. There are two conversion
algorithms:
Exponential Blow-Up
68
From an algorithmic point of view, ε-elimination is no problem: we can
compute all the ε-closures in time at most O(n3 ).
But the powerset construction is potentially exponential in the size of A.
Elimination of ε-moves: conversion from NFAE to NFA.
Of course, it may happen that the accessible part is small, but sometimes (a
large part of) the full powerset is accessible. Even worse, it can happen that
this large power automaton is already minimal, so there is no way to get rid of
these exponentially many states.
Elimination of nondeterminism: conversion from NFA to DFA.
The first one is comes down to computing transitive closure of the ε-transitions
and can be handled efficiently using standard graph algorithms.
Exercise
Determine the running time of a reasonable implementation of the Rabin-Scott
construction. Make sure to build only the accessible part.
But nondeterminism is more difficult to get rid of: there may be an exponential
blow-up in the state complexity of the deterministic machine.
Blow-Up Example 1
69
Blow-Up Example 2
70
Recall the languages
a
L(a, k) = { x | xk = a }
Here is a 6-state NFA based on a
circulant graph. Assume
I = F = Q.
Proposition
L(a, −k) can be recognized by an NFA on k + 1 states, but the state
complexity of this language is 2k .
However, for X = a the power
automaton has maximal size 26 .
We have already seen the de Bruijn DFA for the language. One can show that
this machine has the smallest possible number of states.
b
a
b
b
a
a
2
Tip of an Iceberg
a
X
If X = b than the power
automaton has size 1.
Proof.
b
b
a
71
Decision Problems
The example generalizes to a whole group of circulant machines on n states
with diagram C(n; 1, 2).
Start with a labeling where the edges with stride 1 are labeled a and the edges
with stride 2 are labeled b.
Then change exactly one of these edge labels: the resulting nondeterministic
machines have power automata of size 2n and the power automata are already
reduced.
Exercise
Full blow-up means that for any subset P ⊆ [n] there is some word x such that
δ([n], x) = P . Determine such a word x.
Exercise
Prove that full blow-up occurs for all these NFA.
The paper by Rabin and Scott also introduced the study of the
computational complexity of various decision problems associated with
finite state machines.
We have already seen some of these: Emptiness, Finiteness, Universality,
Equality and Inclusion. For DFAs they are all easily solvable (linear or
quadratic time).
For NFAs the situation is more complicated (NFAEs are not relevant here
since they can be converted to NFAs in quadratic time).
72
Decision Problems for Regular Languages
Problem:
Instance:
Question:
Emptiness Problem
A regular language L.
Is L empty?
Problem:
Instance:
Question:
Finiteness Problem
A regular language L.
Is L finite?
Problem:
Instance:
Question:
Universality Problem
A regular language L.
Is L = Σ? ?
73
More Problems
74
Problem:
Instance:
Question:
Equality Problem
Two regular languages L1 and L2 .
Is L1 equal to L2 ?
Problem:
Instance:
Question:
Inclusion Problem
Two regular languages L1 and L2 .
Is L1 a subset of L2 ?
Equality and Inclusion are polynomial time for DFAs.
Both problems are PSPACE-complete for NFAs.
Emptiness and Finiteness are easily polynomial time for DFAs and NFAs.
Universality is polynomial time for DFAs but PSPACE-complete for NFAs.
Predicting Blow-Up
75
Many algorithms for example in the area of pattern matching naturally produce
nondeterministic machines. Exponential blow-up makes it somewhat difficult to
decide whether it is advantageous to compute the corresponding power
automaton: the actual matching process is faster but the machine may be too
large.
Likewise, it is not clear in general whether minimization is preferable: the cost
of minimization is significant, the speed-up in acceptance testing essentially nil.
It would be nice if one could perform a simple, cheap test to determine what
the size of the power automaton would if one were to go ahead with
conversion. Unfortunately, the following problem is PSPACE-hard:
Problem:
Instance:
Question:
Power Automaton Size
A nondeterministic machine A, a bound B.
Is the size of the power automaton of A
bounded by B?
State Complexity of Operations
intersection
union
concatenation
Kleene star
reversal
complement
DFA
mn
mn
(m − 1)2n − 1
3 · 2n−2
2n
n
76
NFA
mn
m+n
m+n
n+1
n
2n
Worst potential blow-up starting from machine(s) of size m, n and applying
the corresponding operation.
Note that we disregard the number of transitions entirely.
Weird Automata
Nondeterministic Machines
Determinization
Closure Properties
In a DFA there is exactly one trace for each input.
In an NFA there may be exponentially many, and acceptance is determined by
an existential quantification: is there a run . . . . So here is a tempting question:
Is there a useful notion of acceptance based on
“for all runs such that such and such”?
State Complexity
One problem is whether these “universal” automata are more powerful than
ordinary FSMs. As we will see, we only get regular languages.
5
Alternating Automata
Of course, this raises the question of how the state complexities compare.
78
Forall Automata
79
How would one formally define a type of FSM where acceptance means all runs
have a certain property?
Logical And
80
As an example consider the mod counting languages
The underlying transition system will be unaffected, it is still a labeled digraph.
L1 = { x | #a x = 0 (mod m1 ) }
L2 = { x | #b x = 0 (mod m2 ) }
The acceptance condition now reads:
Membership in each can be tested by a DFA of size m1 and m2 , respectively.
For the union L1 ∪ L2 we have a natural NFA of size m1 + m2 . However, for
the intersection L1 ∩ L2 we only have a product machine that has size m1 m2 .
A accepts x if all runs of A on x starting at I end in F .
Let’s call these machines ∀FA .
More importantly, note that nondeterminism does not seem to help with
intersection: there is no obvious way to construct a smaller NFA for L1 ∩ L2 .
But we can build a ∀FA of size m1 + m2 : just take the disjoint union and
declare the acceptance condition to be universal.
Read: universal FA. Actually, don’t: this collides with our previous use where
universal means “accepting all inputs.” Just look at the beautiful symbol and
don’t say anything.
By the same token, a NFA would be a ∃FA.
A Hidden Product
81
What is really going on here?
For example if some word x crashes all possible computations so that
δ(I, x) = ∅ then x is accepted.
{p, q}
Likewise we can modify the Rabin-Scott construction that builds an equivalent
DFA: as before calculate the (reachable part of the full) powerset and adjust
the notion of final state:
where p ∈ Q1 and q ∈ Q2 . In the end we accept iff p ∈ F1 and q ∈ F2 .
This is really no different from a product construction, we just don’t spell out
all the product states explicitly. Choosing clever representations is sometimes
critically important. Trust me.
We can think of the transitions in a NFA as being disjunctions:
δ(p, a) = q1 ∨ q2
We can pick q1 or q2 to continue. Likewise, in a ∀FA we are dealing with
conjunctions:
82
Note that acceptance testing for a ∀FA is no harder than for an NFA: we just
have to keep track of the set of states δ(I, x) ⊆ Q reachable under some input
and change the notion of acceptance: this time we want δ(I, x) ⊆ F .
Let’s assume that Q1 and Q2 are disjoint. Starting at {q01 , q02 } we update
both components. So after a while we are in state
An Analogy (pace Banach)
That’s Not Insane
F0 = {P ⊆ Q | P ⊆ F }
83
Alternating Automata
84
In an alternating automaton (AFA) we admit transitions of the form
δ(p, a) = ϕ(q1 , q2 , . . . , qn )
where ϕ is an arbitrary Boolean formula over Q, even one containing negations.
δ(p, a) = q1 ∧ q2
meaning: We must continue at q1 and at q2 . So how about
δ(p, a) = (q1 ∨ q2 ) ∧ (q3 ∨ q4 )
Does this make any sense? Sure, we must spawn to computations: the first
may start at q1 or q2 , the second at q3 or q4 .
How would such a machine compute? Initially we are in “state”
q01 ∨ q02 ∨ . . . ∨ q0k
the disjunction of all the initial states.
One Step and Acceptance
85
Suppose we are in state Φ, some Boolean formula over Q. Then under input a
the next state is
Why Alternating?
86
The name “alternating automaton” may sound a bit strange.
The original paper by Chandra, Kozen and Stockmeyer that introduced these
machines in 1981 showed that one can eliminate negation without reducing the
class of languages.
Φ[p1 7→ δ(p1 , a), . . . , pn 7→ δ(pn , a)]
Thus, all variables q ∈ Q are replaced by δ(q, a), yielding a new Boolean
formula. In the end we accept if
One can then think of alternating between existential states (at least one
spawned process must succeed) and universal states (all spawned processes
must succeed).
Φ[F 7→ 1, F 7→ 0] = 1
Also note that alternation makes sense for other machines. For example, one
can introduce alternating Turing machines (quite important in complexity
theory).
Exercise
Verify that for NFA and ∀FA this definition behaves as expected.
nil novis sub solem
87
Why Would Anyone Care?
Theorem
Because an AFA can be much, much smaller that the minimal DFA. In fact,
n
the 22 bound is tight: there are AFAs on n states where the minimal
equivalent DFA is doubly exponential in n.
Alternating automata accept only regular languages.
Proof.
We can build a DFA over the state set Bool(Q) (the collection of all Boolean
formulae with variables in Q) where one representative is chosen in each class
n
of equivalent formulae (so there are at most 22 states). For example, we
could choose CNF.
The transitions are described as above, except that after the substitution we
must bring the formula back into the chosen normal form.
So we have a very concise representation for a regular language but one that
still behaves reasonably well under the usual algorithms. Avoiding the standard
DFA representation is often critical for feasibility: in reality we cannot actually
construct the full DFA. Laziness is a good idea in this context.
BTW, this is even true in pattern matching: DFAs should be avoided unless
they are absolutely necessary (because the pattern contains a negation).
The final states are the ones that evaluate to true as above.
2
88