Measuring DFA complexity
91.304 Foundations of
(Theoretical) Computer Science
 Suppose:
 you have a DFA with states named
00000000 .. 11111111 (28 = 256 unique
states)
 an LCD attached to the thing showing
the current state name
 Σ={ c } (for “clock pulse”)
 δ(q, c) = (q + 1) & 0xFF
 This is a simple counter machine – feed
it clocks and it counts upwards
David Martin
[email protected]
This work is licensed under the Creative Commons Attribution-ShareAlike License.
To view a copy of this license, visit http://creativecommons.org/licenses/bysa/2.0/ or send a letter to Creative Commons, 559 Nathan Abbott Way, Stanford,
California 94305, USA.
1
2
Measuring DFA complexity
Measuring DFA complexity
 Time complexity
 Space complexity: the amount of memory
used
 A DFA always takes one transition per input
character
 So time complexity is not useful here
 Program complexity
 A DFA’s “program” is (mostly) its δ
 The model specifies no particular “programming
language” for δ; it’s just a table mapping (state,
input) pairs to (state) outputs
 Though it can sometimes be specified concisely,
as in: δ(q, c) = (q + 1) & 0xFF
 Reprogram the clock for any permutation of
{0,1}8 and δ’s table remains just as big
 But a DFA has no extra memory; it only
remembers what state it is in
 Can’t look back or forward
 So a DFA always uses the same amount of
memory, namely the amount of memory
required to remember what state it’s in
 Needs to remember current element of Q
 Can write down that number in log2 |Q| bits
3
4
DFAs as real computers
Recognition model for functions
 Consider a 256 MB computer that
takes a finite input and produces a
finite output
 Can still sort of be modeled by a DFA
 Inputs: clock pulses, interrupts, hard
drive, keyboard, mouse, network, etc.
 Outputs: video, hard drive, network, etc.
 Can code everything in binary
 But DFA only accepts or rejects input
5
 PC = { x # y | x,y 2{0,1}* and the
input x produces the
output y }
 Note ‘#’ character is just a separator
 DFA plays the role of equipment verifier
 Verifying correctness seems easier than
computing the output, but at least it’s
related
6
1
Are DFAs reasonable?
Are DFAs reasonable?
 One issue is that the programs don’t seem
to reflect much about the problem being
solved
 Similarly: An 8-bit counter is
structurally very different than a 9-bit
counter
 If you can figure out how many bits of memory
are needed for the solution, then you can
always build a DFA based on that knowledge –
could be tedious and really large
 No difference in program complexity between
same amount of memory means DFAs don’t help
us see the difference between programs very
easily
 Neural nets??
 More memory needed
) totally different δ program needed
 Not very modular!
7
8
Are DFAs reasonable?
Is REG reasonable?
 Another issue is that DFAs “prefer”
the beginning of their inputs to the
end of their inputs
 We should be able to combine
computations as subroutines in simple ways
 logical OR (A [ B)
 logical AND (A Å B)
 concatenation (A · B) and star (A* )
 hard to prove!! motivation for NFA
 complement (Ac )
 reversal (AR)
 All above are easy to do as logic circuits
L5 = { x2{0,1}* | first 5 digits of x are 0 }
L6 = { x2{0,1}* | last 5 digits of x are 0 }
 DFAs know where the input begins but
not where it ends
 Will discuss further as closure under
language operations
9
10
Nondeterministic Finite Automata
NFAs
 Will relax two of these DFA rules:
 Thus the NFA rejects the input if
there doesn’t exist any way of
reading the input that winds up in an
accepting state at the end of the
string
1. Each (state, char) input must produce exactly
one (state) output
2. Must consume one character in order to
advance state
 Example: L6 = Σ* ·{bob}·Σ*

See M6
 The NFA accepts the input if there exists
any way of reading the input that winds up
in an accepting state at the end of the
string

Otherwise it rejects the input
11
 In other words: every way of reading
the input leads to a nonaccepting state
 Example: M7
 L7 = ?
a
1
b
ε
2
c
ε
3
12
2
Ways to think of NFAs
Ways to think of NFAs
 NFAs want to accept inputs and will
always take the most advantageous
alternative(s)
 fork() model
a a
 Input string is in a variable
 fork() at every nondeterministic choice point
 subprocess 1 (parent) follows first transition
 subprocess 2 (child) follows second
 subprocess 3 (child) follows third (if any), etc.
 A process that can’t follow any transition calls
exit() -- and gives up its ability to accept
 A process that makes it through the whole string
and is in an accepting state prints out “ACCEPT”
 A single ACCEPT is enough
 Because they will accept if there exists
any way to get to an accepting state at
the end of the string
 The quickest way there may be just one
of many ways, but it doesn’t matter
13
14
Syntax of DFA (repeat)
Syntax of NFA
 A deterministic finite automaton
(DFA) is a 5-tuple (Q,Σ,δ,q0,F) such that
 A nondeterministic finite automaton
(NFA) is a 5-tuple (Q,Σ,δ,q0 ,F) such that
1. Q is a finite set of states
2. Σ is an alphabet
3. δ:Q£ Σ
! Q is the transition function
4. q0 2 Q is the start state
5. F µ Q is the set of accepting states

a
1. Q is a finite set of states
2. Σ is an alphabet
3. δ:Q£(Σ [ {ε})!P(Q) is the transition function
4. q0 2 Q is the start state
5. F µ Q is the set of accepting states
Usually these names are used, but others
are possible as long as the role is clear

Usually these names are used, but others
are possible as long as the role is clear
15
16
Syntax of NFA
NFA computation
 Definition: Σε = Σ [ {ε}
 This definition is different from but equivalent to the
one in the text
 Books definition may be easier to understand at first,
but that makes its version of Theorem 1.39 (subset
construction) harder
 Goal: a function δ*:Q£Σ*! P(Q)
where δ*(q,x) is the set of all states reachable in
the machine after starting in state q and reading
the entire string x
 Then for an NFA M, we will define:
 We’ll use this frequently enough
 Differences on state-transition
diagram





δ(1,a) = { 1 } (not δ(1,a) = 1)
δ(1,ε) = { 1, 2 }
a
b
δ(3, c) = { 2, 3 }
ε
1
2
δ(2,a) = ;
δ(3,ε) = { 3 }
Example M
c
ε
3
L(M)
c
8
17
= { x2Σ * | δ*(q0,x) contains
some accepting state }
18
3
NFA computation
NFA computation
 Let M=(Q,Σ,δ,q0,F) be an NFA. We define
some auxiliary functions:
 Thus E(q) is the set of all states you can
get to from q without reading any input
 In M8, E(3) = ? E({2,1}) = ?
 We define a simple extension of δ that
takes a set of states as input :
 E : Q ! P(Q) by
("ε-closure")
E(q) = { p2 Q | p is reachable from q by
following a chain of 0 or
more ε transitions }
 Although E takes elements of Q as input, we'll
also use it as a function that takes subsets of Q
as input (that is, elements of P(Q)). So
E : P(Q) ! P(Q) by
In other words, given a set
 δ: Q £Σε! P(Q) (this comes with the NFA)
 σ:P(Q)£Σε ! P(Q) defined by
Again, given a set as
input, just process each
element independently...
as input, just process each
element independently...
19
20
NFA computation
NFA computation
 We have a function E() that follows εtransitions and a function σ that
behaves like δ but takes sets as input
 δ*:Q£Σ*! P(Q) is defined inductively:
For all q2 Q,
δ*(q,ε) = E( q )
If w2Σ* and c2Σ, let
δ*(q,wc) = E(σ(δ*(q,w),c))
 Finally, we define
L(M) = { x2Σ* | δ*(q0,x) contains
some accepting state }
= { x2Σ* | δ*(q0,x) Å F ≠ ;
}
 δ*(1,ac) = E(σ(δ*(1,a),c))
 δ*(1,a)=E(σ(δ*(1,ε),a))
a
b
c
 δ*(1,ε) = ?
ε
ε
1
3
2
 δ*(1,ac) = ?
c
Example M8
21
22
Question
NFAs are good at union (or)
 "How do I know when to follow ε
transitions and when not to?"
 L2={x2{0,1}*| the binary number x is
a multiple of 2 }
 L3={x2{0,1}*| the binary number x is
a multiple of 3 }
 Let A = L2 [ L3
 NFA for A using guess-and-verify
strategy
 If you're talking about δ, then don't--it's
the program itself. δ can express that
"there is an ε transition here" but you
never go any further than that one hop.
 If you're talking about δ* , then do-because it includes E() as part of its
definition, which is there precisely in
order to follow ε transitions
23
 Preview of Theorem 1.45
24
4
The Subset Construction
The Subset Construction
Theorem 1.39 For every NFA M1 there
exists a DFA M2 such that
L(M1) = L(M2)
Proof idea: Well, how does fork() work
on a uniprocessor machine?
Proof: Let M1=(Q1,Σ,δ1,init1,F1) be the NFA and define
the DFA M2=(Q2,Σ,δ2,init2,F2) as follows:
1. Q2 = P(Q1).
 Each state of the DFA records the set of states that
the NFA can simultaneously be in
 Can compare DFA states for equality but also look
"inside" the state name to find a set of NFA state
names
Remember: in an NFA,
2. Define
δ1: Q1 £Σε ! P(Q1 ) from def
δ 2 : Q2
£Σ ! Q2
δ2 : P(Q1)£Σ ! P(Q1) σ1:P(Q1)£Σε ! P(Q1 ) extend to sets
by
E1:P(Q1)
!P(Q1) ε-closure
δ2(S,a) = E1(σ1(S,a))
Go to whatever states are reachable from the states
in S and reading the character a
25
26
The Subset Construction
Subset construction example
3.
4.
1. Q2= {;,{1},{2},
a
b
{3},{1,2},{1,3},
ε
ε
1
2
{2,3},{1,2,3}}
2. (On board)
Example M (think of this as M
3. init2={1,2,3}
the construction)
4. F2={{3},{1,3},{2,3},{1,2,3}}
init2 = E(init1)
F2={q 2 Q2 | q Å F1≠ ;}, in other words
F2={S µ Q1 | S Å F1≠ ;}
The effect is that the DFA knows all states that are
reachable in the NFA after reading the string so far.
If any one of them is accepting, then the current DFA
state is accepting too, otherwise it's not.
8
c
3
c
1
in
If you believe this then that's all it takes to see that the
construction is correct. So, convince yourself with an
example. QED
27
28
Be methodical
Subset construction conclusion
 Need to compute
δ2({1,2,3},c) = E1(σ1({1,2,3},c))
 Adding nondeterminism makes programs
shorter but not able to do new things
 Remember: regular languages are defined
to be those "recognized by a DFA"
 We now have a result that says that every
language that is recognized by an NFA is
regular too
 By definition,
σ1({1,2,3},c)= δ1(1,c) [ δ 1 (2,c) [ δ 1 (3,c)
=;
[;
[ {2,3}
 Then take E1( {2,3} ) = { 2,3 }
 Save intermediate results for reuse
 So if you are asked to show that a language is
regular, you can exhibit a DFA or NFA for it and
rely on the subset construction theorem
 Sometimes questions are specifically about DFAs
or NFAs, though... pay attention to the precise
wording
 It's OK to eliminate unreachable states in
practice, even though that's not what the
construction really does
29
30
5
More NFA examples
 Write an NFA for {ab,abc}* with 3
states
 NFA and DFA for { ε } over Σ={0,1}
 Rule: ε 2 L(M) , ?
 NFA and DFA for ; over Σ={0,1}
31
6