CDT314
FABER
Formal Languages, Automata
and Models of Computation
Lecture 3
School of Innovation, Design and Engineering
Mälardalen University
2012
1
Content
Finite Automata, FA
Deterministic Finite Automata, DFA
Nondeterministic Automata NFA
NFA  DFA Equivalence
2
Finite Automata FA
(Finite State Machines)
Based on C Busch, RPI, Models of Computation
3
There is no formal general definition for
"automaton". Instead, there are various kinds of
automata, each with it's own formal definition.
Generally, an automaton
• has some form of input
• has some form of output
• has internal states
• may or may not have some form of storage
• is hard-wired rather than programmable
4
Finite Automaton
Input
String
Output
Finite
Automaton
String
5
Finite Accepter
Input
String
Finite
Automaton
Output
“Accept”
or
“Reject”
6
Finite Automaton as Directed Graph
Nodes = States
q0
a
q1
Edges = Transitions
An edge with several symbols is a short-hand
for several edges:
a
q0 a, b
q1
q0
q1
b
7
Deterministic Finite Automata DFA
8
DFA
• Deterministic
there is no element of choice
• Finite
only a finite number of states and arcs
• Acceptors
produce only a yes/no answer
9
Transition Graph
abba -Finite Acceptor
a, b
Alphabet = {a, b}
q5
b
q0 a
a
a
b
q1 b q2 b q3 a
initial
state
transition
state
a, b
q4
final
state
“accept”
10
Formal Definition
Deterministic Finite Accepter (DFA)
M  Q, ,  , q0 , F 
Q : set of states
 : input alphabet
 : transition function
q0 : initial state
F : set of final states
11
Set of States Q
Q  q0 , q1, q2 , q3 , q4 , q5 
a, b
q5
b
q0 a
a
a
b
q1 b q2 b q3 a
a, b
q4
12
Input Alphabet 
  a, b
a, b
q5
b
q0 a
a
a
b
q1 b q2 b q3 a
a, b
q4
13
Initial State q0
a, b
q5
b
q0 a
a
a
b
q1 b q2 b q3 a
a, b
q4
14
Set of Final States F
F  q4 
a, b
q5
b
q0 a
a
a
b
q1 b q2 b q3 a
a, b
q4
15
Transition Function 
 :Q  Q
a, b
q5
b
q0 a
a
a
b
q1 b q2 b q3 a
a, b
q4
16
 q0 , a   q1
a, b
q5
b
q0 a
a
a
b
q1 b q2 b q3 a
a, b
q4
17
 q0 , b   q5
a, b
q5
b
q0 a
a
a
b
q1 b q2 b q3 a
a, b
q4
18
 q2 , b   q3
a, b
q5
b
q0 a
a
a
b
q1 b q2 b q3 a
a, b
q4
19
Transition Function

q0
q1
q2
q3
q4
q5
a
q1
q5
q5
b
q5
q4
q5
q5
q5 b
q5
q5
q2
q3

a, b
q0 a
q5
a
a
b
q1 b q2 b q3 a
a, b
q4
20
Extended Transition Function  *
 * : Q  *  Q
a, b
q5
b
q0 a
a
a
b
q1 b q2 b q3 a
a, b
q4
21
 * q0 , ab   q2
a, b
q5
b
q0 a
a
a
b
q1 b q2 b q3 a
a, b
q4
22
 * q0 , abba   q4
a, b
q5
b
q0 a
a
a
b
q1 b q2 b q3 a
a, b
q4
23
Observation: There is a walk from q0 to
with the label abbbaa
 * q0 , abbbaa  q5
a, b
q5
b
q0 a
q5
a
a
b
q1 b q2 b q3 a
a, b
q4
24
Recursive Definition
 * q ,    q
 * q, wa    ( * (q, w), a )
a, b
q5
b
q0 a
a
a
b
q1 b q2 b q3 a
a, b
q4
25
 * q0 , ab  
  * (q0 , a ), b  
   * q0 ,  , a , b  
  q0 , a , b  
 q1 , b   q2
a, b
q5
b
q0 a
a
a
b
q1 b q2 b q3 a
a, b
q4
26
String Acceptance
Definition:
A string w is accepted by DFA M if w
drives M to a final state from the initial
state.
Formally: M accepts w iff
 * q0 , w  F
27
 * q0 , abbbaa  q5
abbbaa is NOT accepted
a, b
q5
b
q0 a
a
a
b
q1 b q2 b q3 a
a, b
q4
28
 * q0 , abba   q4
abba is accepted
a, b
q5
b
q0 a
a
a
b
q1 b q2 b q3 a
a, b
q4
29
Language Accepted by DFA
Take a DFA
M
Definition:
The language LM  contains
all input strings accepted by M
LM  = strings that driveM
state
to a final
30
Example
LM   abba
Alphabet = {a, b}
M
a, b
q5
b
q0 a
a
a
b
q1 b q2 b q3 a
a, b
q4
accept
31
Another Example
LM    , ab, abba
Alphabet = {a, b}
M
a, b
q5
b
q0 a
accept
a
a
b
q1 b q2 b q3 a
accept
a, b
q4
accept
32
Formally
For a DFA
M  Q, ,  , q0 , F 
Language accepted by
M:
LM   w  * :  * q0 , w  F 
alphabet
transition
function
initial
state
final
states
33
Observation
Language accepted by
M
LM   w  * :  * q0 , w  F 
Language rejected by
M
LM   w  * :  * q0 , w F 
34
More Examples
LM   {a b : n  0}
n
a, b
a
q0
b
q1
accept
a, b
q2
trap state
Alphabet = {a, b}
35
LM = { all strings with prefix ab }
a, b
q0
a
q1
b
a
q3
Alphabet = {a, b}
b
q2
accept
a, b
36
Alphabet = {0,1}
LM  = { all strings without substring 001 }
0
1
0,1
1

0
0
00
1
001
0
37
Regular Languages
L is regular if there is
M such that L  LM 
A language
a DFA
All regular languages form a language family
38
Example
The language
L  awa : w  a, b*
a
b
b
is regular
q0
a
q2
q3
a
b
q4
Alphabet = {a, b}
a, b
39
Nondeterministic Finite Automata NFA
40
NFA
• Nondeterministic
there is an element of choice: in a given state
NFA can act on a given string in different
ways. Several start/final states are allowed.
-transitions are allowed.
• Finite
only a finite number of states and arcs
• Acceptors
produce only a yes/no answer
41
Nondeterministic Finite Accepter (NFA)
Alphabet =
{a}
Two choices
a
q0
q1
a
q2
a
q3
42
First Choice
a a
a
q0
q1
a
q2
a
q3
43
First Choice
a a
a
q0
q1
a
q2
a
q3
44
First Choice
a a
a
q0
q1
a
q2
a
q3
45
First Choice
a a
a
q0
q1
a
q2
“accept”
a
q3
46
Second Choice
a a
a
q0
q1
a
q2
a
q3
47
Second Choice
a a
a
q0
q1
a
q2
a
q3
48
Second Choice
a a
a
q0
q1
a
q3
a
q2
No transition:
the automaton hangs
49
Second Choice
a a
a
q0
q1
a
q2
a
q3
“reject”
50
Observation
An NFA accepts a string if
there is at least one computation
of the NFA that accepts the string
51
Example
aa
is accepted by the NFA:
a
q0
q1
a
q2
a
q3
52
Lambda Transitions
q0
a
q1 
q2
a
q3
53
a a
q0
a
q1 
q2
a
q3
54
a a
q0
a
q1 
q2
a
q3
55
(read head doesn’t move)
a a
q0
a
q1 
q2
a
q3
56
a a
q0
a
q1 
q2
a
q3
57
a a
“accept”
q0
String
aa
a
q1 
is accepted
q2
a
q3
58
Language accepted:
q0
a
q1 
L  {aa}
q2
a
q3
59
Another NFA Example
q0
a
b
q1
q2

q3

Alphabet = {a, b}
60
a b
q0
a
b
q1
q2

q3

61
a b
q0
a
b
q1
q2

q3

62
a b
q0
a
b
q1
q2

q3

63
a b
“accept”
q0
a
b
q1
q2

q3

64
Another String
a b a b
q0
a
b
q1
q2

q3

Alphabet = {a, b}
65
a b a b
q0
a
b
q1
q2

q3

66
a b a b
q0
a
b
q1
q2

q3

67
a b a b
q0
a
b
q1
q2

q3

68
a b a b
q0
a
b
q1
q2

q3

69
a b a b
q0
a
b
q1
q2

q3

70
a b a b
q0
a
b
q1
q2

q3

71
a b a b
“accept”
q0
a
b
q1
q2

q3

72
Language accepted
L  ab, abab, ababab, ...

 ab
q0
a
b
q1
q2

q3

Alphabet = {a, b}
73
Another NFA Example
Alphabet = {0,1}
0
q0
1
q1
0, 1 q2

74
Language accepted
L   , 10, 1010, 101010, ...
 10*
0
q0
1
q1
0, 1 q2

75
Formal Definition of NFA
M  Q, ,  , q0 , F 
Q:
Set of states, i.e.
q0 , q1, q2 
:
Input alphabet, i.e.
a, b
:
Transition function
q0 : Initial state
F:
Final states
76
Transition Function

 q0 , 1  q1
0
q0
1
q1
0, 1 q
2

77
 (q1,0)  {q0 , q2 }
0
q0
1
q1
0, 1 q
2

78
 (q0 ,  )  {q0 , q2}
0
q0
1
q1
0, 1 q
2

79
 (q2 ,1)  
0
q0
1
q1
0, 1 q
2

80
Extended Transition Function  *
(Utvidgad övergångsfunktion)
 * q0 , a   q1
q5
q4
a
q0
a
a
b
q1
q2

q3

81
 * q0 , aa   q4 , q5 
q5
q4
a
q0
a
a
b
q1
q2

q3

82
 * q0 , ab   q2 , q3 , q0 
q5
q4
a
q0
a
a
b
q1
q2

q3

83
Formally
q j   * qi , w
if and only if
there is a walk from
with label w
qi to q j
84
The Language of an NFA M
F  q0 ,q5 
q5
q4
a
q0
a
a
b
q1
q2

q3

 * q0 , aa   q4 , q5
Alphabet = {a, b}
aa  L(M )
85
F  q0 ,q5 
q5
q4
a
q0
a
a
b
q1
q2

q3

 * q0 , ab   q2 , q3 , q0 
ab LM 
86
F  q0 ,q5 
q0
q5
q4
a
a
a
b
q1
q2

q3

 * q0 , abaa   q4 , q5
abaa  L(M )
87
F  q0 ,q5 
q5
q4
a
q0
a
a
b
q1
q2

q3

 * q0 , aba   q1
aba  LM 
88
q5
q4
a
q0
a
a
b
q1
q2

q3


LM   aa  ab*  ab aa
89
Formally
M
The language accepted by NFA
is:
LM   w1, w2 , w3 ,...
where
 * (q0 , wm )  {qi , q j ,...}
and there is some (at least one)
qk  F
(final state)
90
w LM 
 * (q0 , w)
qi
w
q0
qk
w
w
qk  F
qj
91
NFA  DFA Equivalence
92
Equivalence of NFAs and DFAs
NFAs

DFAs ?
The same power?
Accept the same languages?
YES!
93
We will prove:
Languages
accepted
by NFAs

Languages
accepted
by DFAs
NFAs and DFAs have the same
computation power!
94
Step 1
Languages
accepted
by NFAs
Proof

Languages
accepted
by DFAs
Every DFA is also an NFA
A language accepted by a DFA
is also accepted by an NFA
95
Step 2
Languages
accepted
by NFAs

Languages
accepted
by DFAs
Proof Any NFA can be converted to an
equivalent DFA
A language accepted by an NFA
is also accepted by a DFA
96
Procedure NFA to DFA
Idea:
Define new states in DFA that collect the states
with the same transition
{1,3}
1
a
a
start
start
0
a,b
{0}
b
3
{3}
97
Procedure NFA to DFA
1. Initial state of NFA: q0
Initial state of DFA: q0 
98
Example
NFA
a
q0
a

q1
q2
b
Step 1
DFA
q0 
99
Procedure NFA to DFA
2. For every DFA’s state
{qi , q j ,..., qm }
Compute in the NFA
 * qi , a ,
 * q j , a ,
}
 {qi , qj ,..., qm
...
Add transition
}
 {qi , q j ,..., qm }, a   {qi , qj ,..., qm
100
Example
a
NFA
q0
a
q1

Alphabet = {a, b}
q2
b
 * (q0 , a)  {q1, q2}
Step 2
DFA
q0 
a
q1,q2 
 q0 , a   q1, q2 
101
Procedure NFA to DFA
Repeat Step 2 for all letters in alphabet,
until no more transitions can be added.
102
Example
a
NFA
q0
a
Alphabet = {a, b}

q1
q2
b
a
b
Step 3
DFA
q0 
a
q1,q2 
b

a, b
103
Procedure NFA to DFA
3. For any DFA state {qi , q j ,..., qm }
If some
q j is a final state in the NFA
Then {qi , q j ,..., qm }
is a final state in the DFA
104
Example
a
NFA
q0
a

q1
Alphabet = {a, b}
q1  F
q2
b
a
b
Step 4
DFA
q0 
a
q1,q2 
b

q1, q2  F 
a, b
105
Theorem
Take NFA
M
Apply the procedure to obtain DFA
Then
M
M and M  are equivalent :
LM   LM 
106
We have proven (proof by construction):
Languages
accepted
by NFAs

Languages
accepted
by DFAs
Regular Languages
END OF PROOF
107
Nondeterministic
vs.
Deterministic Automata
108
Formal Definition of NFA
NFA is a mathematical model defined as a quintuple:
M  Q, ,  , q0 , F 
Q:
:
:
q0 :
F:
Set of states, i.e.
Input alphabet, i.e.
Transition function
q0 , q1, q2 
a, b
Initial state
Final (accepting) states
109
Deterministic Finite Automata
A deterministic finite automaton (DFA) is a
special case of a nondeterministic finite
automaton (NFA) in which
1. no state has an -transition, i.e. a transition
on input , and
2. for each state q and input symbol a, there
is at most one edge labeled a leaving q.
110
Example: NFA
a
start
0
a
b
1
2
b
3
b
A nondeterministic finite automaton
INPUT SYMBOL
STATE
0
1
2
a
b
{0, 1}
-
{0}
{2}
{3}
Transition table for the finite automaton above
111
Example
a
1
a
2

start
0
b

3
b
4
NFA accepting aa* + bb*
112
Example
b
b
start
0
a
1
b
b
2
3
a
a
a
a
NFA accepting (a+b)*abb
113
Example
start
a
1
start
2
a
3
b
4
a
start
6
b
b
7
b
5
8
(a) NFA for a, abb, and a*b+.
a
1
2

start

0
7
b
4
a

(b) Combined NFA.
a
3
5
b
6
b
b
8
NFA recognizing three different patterns.
114
Ways to Think of Nondeterminism
• always make the correct guess
• “backtracking” (systematically try all possibilities)
For a particular string, imagine a tree of possible
state transitions:
q4
a
a
q1
a
q0
a
b
q2
a
q0
q3
115
Advantages of Nondeterminism
• An NFA can be smaller, easier to construct and
easier to understand than a DFA that accepts
the same language
• NFAs are useful for proving some theorems
• NFAs are good introduction to nondeterminism
in more powerful computational models, where
nondeterminism plays an important role
116
DFA vs. NFA Space and Time Complexity
Tradeoffs
AUTOMATO
N
SPACE
TIME
NFA
DFA
O(|r|)
O(2|r|)
O(|r||x|)
O(|x|)
Space and time taken to recognize regular expressions:
- NFA more compact but take time to backtrack all choices
- DFA take place, but save time
Time-Space Tradeoffs Goal: Given reg. exp. r and input string x, determine whether x is in
L(r) Method #1: Build NFA N from r using Thompson's construction, then run previous
algorithm construct NFA in O(|r|) time. N has at most twice as many states as |r|, and at
most two transitions from each state, so transition table is O(|r|) space. Previous algorithm
accepts or rejects x in O(|r|×|x|) time
(Where r is regular expression, and x is input string)
117
Equivalent automata
Two finite automata M1 and M2 are equivalent if
L(M1) = L(M2)
that is, if they both accept the same language.
118
Equivalence of NFAs and DFAs
To show that NFAs and DFAs accept the same class of
languages, we show two things:
– Any language accepted by a DFA can also be accepted
by some NFA (As DFA is a special case of NFA)
– Any language accepted by a NFA can also be accepted
by some (corresponding, specially constructed) DFA
119
Proof Strategy
To show that any language accepted by a NFA is
also accepted by some DFA, we describe an
algorithm that takes any NFA and converts it into
a DFA that accepts the same language.
The algorithm is called the “subset construction
algorithm”.
We can use mathematical induction (on the
length of a string accepted by the automaton) to
prove the DFA that is constructed accepts the
same language as the NFA.
120
Converting NFA to DFA
by Subset Construction
http://www.math.uu.se/~salling/Movies/SubsetConstruction.mov
121
Subset construction
Given a NFA construct a DFA that accepts the
same language.
The equivalent DFA simulates the NFA by
keeping track of the possible states it could be
in. Each state of the DFA is a subset of the set
of states of the NFA -hence, the name of the
algorithm.
If the NFA has n states, the DFA can have as
many as 2n states, although it usually has many
less.
122
Steps of subset construction
The initial state of the DFA is the set of all states the
NFA can be in without reading any input.
For any state {qi,qj,…,qk} of the DFA and any input a,
the next state of the DFA is the set of all states of the
NFA that can result as next states if the NFA is in any
of the states qi,qj,…,qk when it reads a. This includes
states that can be reached by reading a, followed by
any number of -moves. Use this rule to keep adding
new states and transitions until it is no longer possible
to do so.
The accepting states of the DFA are those states that
contain an accepting state of the NFA.
123
Example
a
a

b b
b
a
Alphabet = {a, b}
b
Here is a NFA that we want to convert to an
equivalent DFA.
124
a
Alphabet = {a, b}
a
NFA

0
DFA
{0,1}
1
b b
a
b
2
b
The start state of the DFA is the set of states the NFA can be
in before reading any input. This includes the start state of
the NFA and any states that can be reached by a -transition.
125
DFA
a
a
a
NFA

0
{0,1}
b b
a
b
1
b
2
b
{2}
For start state {0,1}, make transitions for each possible
input, here a and b. Reading b from start {0,1}, we reach
state {2}. Means from either {0}, or {1} we reach {2}.
126
NFA
DFA
a
a
a

{0,1}
a
{1,2}
b
0
b b
a
b
1
b
2
b
{2}
For state {2}, we create a transition for each possible input,
a and b. From {2}, with b we are either back to {2} (loop) or
we reach {1}- see the little framed original NFA. So from
{2}, with b we end in state {1, 2}. Reading a leads us from
{2} to {0} in the original NFA, which means state {0, 1} in
the new DFA.
127
DFA
a
b
NFA
a
a
{0,1}
a

{1,2}
b
a
0
b b
a
b
1
b
2
b
{2}
For state {1, 2}, we make again transition for each
possible input, a and b. From {2} a leads us to {0}. From {1}
with a we are back to {1}. So, we reach {0, 1} with a from {1,2}.
With b we are back to {1,2}.
At this point, a transition is defined for every state-input pair.
128
NFA
a
{0,1}
DFA
a
b

0
{2}
a
b
1
b b
{1,2}
b
a
a
a
b
2
b
The last step is to mark the final states of the DFA.
As {1} was the accepting state in NFA, all states containing
{1} in DFA will be accepting states: ({0, 1} and {1, 2}).
129
Subset Construction Algorithm
130
Subset Construction
States of nondeterministic M´ will correspond to
sets of states of deterministic M
Where q0 is start state of M, use {q0} as start
state of M´.
Accepting states of M´ will be those state-sets
containing at least one accepting state of M.
131
Subset Construction (cont.)
For each state-set S and for each s in alphabet
of M, we draw an arc labeled s from state S to
that state-set consisting of all and only the ssuccessors of members of S.
Eliminate any state-set, as well as all arcs
incident upon it, such that there is no path
leading to it from {q0}.
132
The power set of a finite set, Q,
consists of 2|Q| elements
The DFA corresponding to a given NFA with Q states
have a finite number of states, 2|Q|.
If |Q| = 0 then Q is the empty set, | P(Q)| = 1 = 20.
If |Q| = N and N  1, we construct subset of a given
set so that for each element of the initial set there are
two alternatives, either is the element member of a
subset or not. So we have
2 · 2 · 2 · 2 · 2 · 2 · 2…. ·2 = 2N
N times
133
From an NFA to a DFA
Subset Construction
Operation
 - closure(s)
 - closure(T)
Move(T,a)
Description
Set of NFA states reachable from
an NFA state s on  -transitions
Set of NFA states reachable from
some NFA state s in T on  transitions
Set of NFA states reachable from
some NFA state set with a transition
on input symbol a
134
From an NFA to a DFA
Subset Construction
Initially,  -closure (s0) is the only states in D and it is
unmarked
while there is an unmarked state T in D do
mark T;
for each input symbol a do
U:=  -closure(move(T,a));
if U is not in D then
add U as an unmarked state to D
Dtran[T,a]:=U;
end(for)
end(while)
135