Kleene’s theorem
1
CHAPTER
1
INTRODUCTION
1:
INTRODUCTION
Unification
This theorem was proved by kleene in 1956.
We introduced three separate ways of defining a language: generation by regular
expression, acceptance by finite automata, and acceptance by transition graph. in
this book we present a theorem proved by Kleene in 1956.which says that if a
language can be defined by any one of these three ways, then it can also be defined
by the other two ways.
Theorem: Any language that can be defined by
a regular expression, or
a finite automaton, or
a transition graph
can be defined by all 3 methods.
All of these three methods of defining language are equivalent.
If we were trying to prove the mathematical theorem that the set of all ZAPS, the set
of all ZEPS and the set of all ZIPS are same. To do this, we need three parts. In part
1,we show that all ZAPS are ZEPS.In part 2,we shall show that all ZEPS are
ZIPS.Finaly in part three ,we shall show that all ZIPS are ZAPS. Taken together,
these three parts will establish the equivalence of the three sets:
[ZAPS ZEPS ZIPS ZAPS]=[ZAPS=ZEPS=ZIPS]
This is the fundamental theorem of finite automata.
Proof
 Part 1: Every language that can be defined by an FA can be
defined by a TG.
 Part 2: Every language that can be defined by a TG can be
defined by a regular expression.
2
Kleene’s theorem
CHAPTER
1
INTRODUCTION
 Part 3: Every language that can be defined by a regular expression can
be defined by an FA.
When we have proven these three parts, we have finished our
theorem.
Kleene’s theorem
3
CHAPTER 2 PROOF(KLEENE’S THEOREM PART I)
2
PROOF(KLEENE’S THEOREM PART I)
FINITE AUTOMATA
As we know that A finite automata is a collection of three things:
1.
2.
3.
A finite set of states, one of which is designed as the initial state, called start state, and some
(may be none) of which are designed as final states.
An alphabet ∑ of possible input letters.
A finite set of transition that tell for each state and for each letter of the input alphabet which
state to go to next.
Clause three in the definition means that every edge labelled by some string of only one letter
or strings of letters.
TRANSITION GRAPH
We also know that a transition graph is a collection of three things:
1.
A finite set of states at least one of which is start state and some of which are
designed as final state.
2. An alphabet ∑ of possible input letters.
3. A finite set of transition(edges,labellesls)that show how to go from some states
to another, based on reading specified substring of input letters(possibly even
the null string ).
Clause three in the definition means that every edge labelled by some string or strings of
letters. Not necessarily only one letter. Transition graph were invented by John Mayhill in 1957.
It is extremely important for us to notice that every FA is also a TG.This means that any picture
that represents an FA can be interpreted as a picture of a TG.Of course, not every TG satisfies
the definition of an FA. Since every FA can be viewed as a TG that accepts the same
language therefore, any language that has been defined by a finite automaton has
already been defined by transition graph. But let see some examples that proof this
statement true.
Example
4
Kleene’s theorem
CHAPTER
2
PFOOF(KLEENE THEOREM PART I)
Let we have a language L={^} then the FA that accept this language is
and the TG that accept this language is
So both FA and TG accept this language.
Example
Let we have a language L={baa} then both FA and TG accept this language
A FA that accept this language is
A TG that accept this language is
Example
The FA below accepts exactly the two strings baa and ab.
5
Kleene’s theorem
CHAPTER
2
PROOF (KLEENE’S THEOREM PART I)
The TG below accept exactly the two strings baa and ab.
When an edge is labelled with the string ^ it means can take a ride it offers free(without consuming
any letters from the input string).Remember that we do not have to follow that edges, but we can if we
want to.
Example
Let we have a regular expression (a+b)*b.Language of this regular expression is all words ending on b.
A FA that accept this language is
And the same language also accepted by a TG.TG that accept this language is
6
Kleene’s theorem
CHAPTER
2
PROOF(KLEENE’S THEOREM PART I)
Hence it is proved that every language that can be defined by finite automaton can also be defined by a
transition graph.
Kleene’s theorem
7
CHAPTER
3
3
PROOF(KLEENE’S THEOREM PART II)
PROOF(KLEENE’S THEOREM PART II)
PROOF (KLEENE’S THEOREM PART II)
To prove part II of the theorem, an algorithm consisting of different steps, is explained showing how a
RE can be obtained corresponding to the given TG. For this purpose the notion of TG is changed to that
of GTG i.e. the labels of transitions are corresponding REs.Basically this algorithm converts the given TG
to GTG with one initial state alone with a single loop, or one initial state connected with one final state
by a single transition edge. The label of the loop or the transition edge will be the required RE.
 Step 1
If a TG has more than one start states, then introduce a new start state connecting the new state to
the old start states by the transitions labelled by Λ and make the old start states the non-start states.
This step can be shown by the following example
Example
8
Kleene’s theorem
CHAPTER
3
PROOF(KLEENE’S THEOREM PART II)
The above TG can be converted to
 Step 2:
If a TG has more than one final states, then introduce a new final state, connecting the old final states
to the new final state by the transitions labelled by Λ.This step can be shown by the previous example of
TG, where the step 1 has already been processed
Example
The above TG can be converted to
9
Kleene’s theorem
CHAPTER
3
PROOF(KLEENE’S THEOREM PART II)
If a state has two (more than one) incoming transition edges labelled by the corresponding RE ,from the
same state (including the possibility of loops at a state), then replace all these transition edges with a
single transition edge labelled by the sum of corresponding RE.
This step can be shown by a part of TG in the following example
Example
CHAPTER
3
PROOF(KLEENE’S THEOREM PART II)
10
Kleene’s theorem
 Step 4 (bypass and state elimination)
If three states in a TG, are connected in sequence then eliminate the middle state and connect the first
state with the third by a single transition (include the possibility of circuit as well) labelled by the RE
which is the concatenation of corresponding two REs in the existing sequence. This step can be shown
by a part of TG in the following example
To eliminate state 5 the above can be reduced to
Consider the following example containing a circuit
Example
Consider the part of a TG, containing a circuit at a state, as shown below
CHAPTER
3
PROOF(KLEENE’S THEOREM PART II)
11
Kleene’s theorem
To eliminate state 4 the above TG can be reduced to
CHAPTER
3
PROOF(KLEENE’S THEOREM PART II)
12
Kleene’s theorem
Note:
It is to be noted that to determine the RE corresponding to a certain TG, four steps have been discussed.
This process can be explained by the following particular examples of TGs
Example
Consider the following TG
To have single final state, the above TG can be reduced to the following
To eliminate states 2 and 3, the above TG can be reduced to the following
CHAPTER
3
PROOF(KLEENE’S THEOREM PART II)
13
Kleene’s theorem
To eliminate state 1 the above TG can be reduced to the following
Hence the required RE is (ab+ba)(aa+b)*(aaa+bba)
Example
Consider the following TG
To have single initial and single final state the above TG can be reduced to the following
14
Kleene’s theorem
CHAPTER
3
PROOF(KLEENE’S THEOREM PART II)
To obtain single transition edge between 1 and 3; 2 and 4, the above can be reduced to the following
To eliminate states 1,2,3 and 4, the above TG can be reduced to the following TG
15
Kleene’s theorem
CHAPTER
3
PROOF(KLEENE’S THEOREM PART II)
To connect the initial state with the final state by single transition edge, the above TG can be reduced to
the Following
Hence the required RE is (b+aa)b*+(a+bb)a*
Example
Consider the following TG, accepting EVEN-EVEN language
It is to be noted that since the initial state of this TG is final as well and there is no other final state, so
to obtain a TG with single initial and single final state, an additional initial and a final state are
introduced as shown in the following TG
To eliminate state 2, the above TG may be reduced to the following
16
Kleene’s theorem
CHAPTER
3
PROOF(KLEENE’S THEOREM PART II)
To have single loop at state 1, the above TG may be reduced to the following
To eliminate state 1, the above TG may be reduced to the following
Hence the required RE is (aa+bb+(ab+ba)(aa+bb)*(ab+ba))*
17
Kleene’s theorem
CHAPTER
3
PROOF(KLEENE’S THEOREM PART II)
 PROBLEM
Using PROOF(KLEENE’S THEOREM PART II) convert the following TG in to RE.
1. Let us suppose a TG that has some state(called state x) inside it (not the – or + state)that has
more than one loop circling back to itself:
Where r1,r2,r3 are all regular expressions or simple string .In this case we replace the three
loops by one loop labelled with a regular expression.
The meaning here is that from state x we can read any one string from the input that fits the
regular expression r1 + r2 +r3 and return to the same state.
2. Suppose two states are connected by more than one edge going in the same direction:
We can replace this with a single edge that is labelled with a regular expression
18
Kleene’s theorem
CHAPTER
3
PROOF(KLEENE’S THEOREM PART II)
3. Suppose the following TG:
We can convert this TG into the following RE by using bypass and state elimination operation.
4.
Suppose the following TG:
This TG becomes
5. Suppose the following TG:
The above TG becomes
19
Kleene’s theorem
CHAPTER
3
PROOF(KLEENE’S THEOREM PART II)
6. Suppose the following TG:
The following RE defines the same language that TG defines.
7. Suppose the following TG:
This becomes
8. Suppose the following TG:
20
Kleene’s theorem
CHAPTER
3
PROOF(KLEENE’S THEOREM PART II)
It becomes
9. Suppose the following TG:
In this TG the incoming states are 1 and 3 and the outgoing state is only 1.Therefore, we must
add edges connecting state 3 to 1 and 1 to 1.so the machine becomes
Kleene’s theorem
21
CHAPTER
4
4
PROOF(KLEENE’S THEOREM PART III)
PROOF(KLEENE’S THEOREM PART III)
Converting the Regular Expression to FA
Proof (kleene’s theorem part 3)
If the language can be expressed by a RE then there exists an FA accepting the language. As the regular
expression is obtained applying addition, concatenation and closure on the letters of an alphabet and
the Null string, so while building the RE, sometimes, the corresponding FA may be built easily, as
shown in the following examples
Example
Consider the language, defined over Σ = {a,b}, consisting of only b, then this language may be
accepted by the following FA.
which shows that this FA helps in building an FA accepting only one letter
Example
Consider the language, defined over Σ = {a,b}, consisting of only Λ, then this language may be
accepted by the following FA.
22
Kleene’s theorem
CHAPTER
4
PROOF(KLEENE’S THEOREM PART III)
As, if r1 and r2 are regular expressions then their sum, concatenation and closure are also regular
expressions, so an FA can be built for any regular expression if the methods can be developed for
building the FAs corresponding to the sum, concatenation and closure of the regular expressions along
with their FAs. These three methods are explained in the following discussion
Method 1 (Union of two FAs):
Using the FAs corresponding to r1 and r2 an FA can be built, corresponding to r1+ r2. This method can
be developed considering the following examples
Example
Let r1 = (a+b)*b defines L1 and the FA1
And r2 = (a+b )*aa(a+b )* defines L2 and FA
Let FA3 be an FA corresponding to r1+ r2, then the initial state of FA3 must correspond to the initial
state of FA1 and the initial state of FA2. Since the language corresponding to r1+ r2 is the union of
23
Kleene’s theorem
CHAPTER
4
PROOF(KLEENE’S THEOREM PART III)
corresponding languages L1 and L2, consists of the strings belonging to L1or L2 or both, therefore a final
state of FA3 must correspond to a final state of FA1 or FA2 or both.
Since, in general, FA3 will be different from both FA1 and FA2, so the labels of the states of FA3 may be
supposed to be z1,z2, z3, …, where z1 is supposed to be the initial state. Since z1 corresponds to the
states x1 or y1, so there will be two transitions separately for each letter read at z1. It will give two
possibilities of states either z1 or different from z1. This process may be expressed in the following
transition table for all possible states of FA3
24
Kleene’s theorem
CHAPTER
4
PROOF(KLEENE’S THEOREM PART III)
RE corresponding to the above FA may be r1+r2 = (a+b)*b + (a+b)*aa(a+b)*.
FA1 contains two states while FA2 contains three states. Hence the total number of possible
combinations of states of FA1 and FA2, in sequence, will be six. For each combination the transitions for
both a and b can be determined, but using the method in the example, number of states of FA3 was
reduced to five.
Example
Let r1 = (a+b)*a and the corresponding FA1 be
also r2 = (a+b)((a+b)(a+b))* or ((a+b)(a+b))*(a+b) and FA2 be
FA corresponding to r1+r2 can be determined as
25
Kleene’s theorem
CHAPTER
4
PROOF(KLEENE’S THEOREM PART III)
Method 2 (Concatenation of two FAs):
Using the FAs corresponding to r1 and r2, an FA can be built, corresponding to r1r2. This method can be
developed considering the following examples
Example
Let r1 = (a+b)*b defines L1 and FA1 be
and r2 = (a+b )*aa (a+b )* defines L2 and FA2 be
26
Kleene’s theorem
CHAPTER
4
PROOF(KLEENE’S THEOREM PART III)
Let FA3 be an FA corresponding to r1r2, then the initial state of FA3 must correspond to the initial state
of FA1
and the final state of FA3 must correspond to the final state of FA2.Since the language corresponding to
r1r2 is
the concatenation of corresponding languages L1 and L2, consists of the strings obtained, concatenating
the
strings of L1 to those of L2 , therefore the moment a final state of first FA is entered, the
possibility of the
initial state of second FA will be included as well.
Since, in general, FA3 will be different from both FA1 and FA2, so the labels of the states of FA3 may be
supposed to be z1,z2, z3, …, where z1 stands for the initial state. Since z1 corresponds to the states x1, so
there will be two transitions separately for each letter read at z1. It will give two possibilities of states
which correspond to either z1 or different from z1. This process may be expressed in the following
transition table for all possible states of FA3
Hence the required FA will be as follows
27
Kleene’s theorem
CHAPTER
4
PROOF(KLEENE’S THEOREM PART III)
Example
Let r1 = ((a+b)(a+b))* and the corresponding FA1 be
also r2 = (a+b)((a+b)(a+b))* or ((a+b)(a+b))*(a+b) and FA2 be
FA corresponding to r1r2 can be determined as
28
Kleene’s theorem
CHAPTER
4
PROOF(KLEENE’S THEOREM PART III)
Hence the required FA will be as follows
Method3: (Closure of an FA)
Building an FA corresponding to r*, using the FA corresponding to r. It is
to be noted that if the given FA already accepts the language expressed by the closure of certain RE,
then the given FA is the required FA. However the method, in other cases, can be developed
considering the following examples Closure of an FA, is same as concatenation of an FA with itself,
except that the initial state of the required FA is a final state as well. Here the initial state of given FA,
corresponds to the initial state of required FA and a non final state of the required FA as well.
Example
Let r = (a+b)*b and the corresponding FA be
then the FA corresponding to r* may be determined as under
29
Kleene’s theorem
CHAPTER
4
PROOF(KLEENE’S THEOREM PART III)
The corresponding transition diagram may be as under
Example
Let r = (a+b)*aa(a+b)* and the corresponding FA be
then the FA corresponding to r* may be determined as under
CHAPTER
4
PROOF(KLEENE’S THEOREM PART III)
30
Kleene’s theorem
The corresponding transition diagram may be
Example
Consider the following FA, accepting the language of strings with b as second letter
then the FA corresponding to r* may be determined as under
CHAPTER
4
PROOF(KLEENE’S THEOREM PART III)
31
Kleene’s theorem
If at then the FA corresponding to r* may be determined as under he initial state of the given FA, there
is either a loop or an incoming transition edge, the initial state corresponds to the final state and a non
final state as well, of the required FA, otherwise the initial state of given FA will only correspond to a
single state of the required FA (i.e. the initial state which is final as well).
CHAPTER
5
NONDETERMINISTIC FINITE AUTOMATA
32
Kleene’s theorem
5
NONDETERMINISTIC FINITE AUTOMATA
Nondeterministic Finite Automaton (NFA)
Definition
An NFA is a TG with a unique start state and a property of having single letter as label of transitions. An
NFA is a collection of three things
1. Finite many states with one initial and some final states
2. Finite set of input letters, say, Σ= {a, b, c}
3. Finite set of transitions, showing where to move if a letter is input at certain state (Λ is not a
valid transition), there may be more than one transition for certain letters and there may not be
any transition for certain letters
Observations
It may be observed, from the definition of NFA, that the string is supposed to be accepted, if there
exists at least one successful path, otherwise rejected. It is to be noted that an NFA can be considered to
be an intermediate structure between FA and TG. The examples of NFAs can be found in the following
Example
It is to be noted that the above NFA accepts the language consisting of a and ab.
Example
CHAPTER
5
NONDETERMINISTIC FINITE AUTOMATA
33
Kleene’s theorem
It is to be noted that the above NFA accepts the language of strings, defined over Σ = {a, b}, containing
aa.
Note
It is to be noted that NFA helps to eliminate a loop at certain state of an FA. This process is done
converting the loop into a circuit. But during this process the FA remains no longer FA and is converted
to a corresponding
NFA, which is shown in the following example.
Example
Consider a part of the following FA with an alphabet Σ = {a,b,c,d}
To eliminate the loop at state 7, the corresponding NFA may be as follows
Note that
Definition of NFA is different from that of FA since




A FA must have from each state an arc labelled with each letter of alphabet, while NFA
does not.
A FA is deterministic, while a NFA may be nondetermisic.
An NFA can have repeated labels from any single state.
NFA allows for human choice to become a factor in selecting a way to process an input
string.
CHAPTER
5
NONDETERMINISTIC FINITE AUTOMATA
34
Kleene’s theorem
Definition of NFA is different from that of TG since



A TG can have arcs labelled with substrings of letters while a NFA has arcs labelled with
only letters.
A TG can have arcs labelled with ^ while a NFA cannot.
A TG can have more than one start state while a NFA can only have one.
Converting an FA to an equivalent NFA
It is to be noted that according to the Keene’s theorem, if a language can be accepted by an FA, then
there exists a TG accepting that language. Since, an NFA is a TG as well, therefore there exists an NFA
accepting the language accepted by the given FA. In this case these FA and NFA are said to be
equivalent to each others.
. Note that



Every FA is an NFA since we can consider an FA to be an NFA without the extra possible
features.
Every NFA is a TG.
Kleene’s theorem states that every TG has an equivalent FA.
Following are the examples of FAs to be converted to the equivalent NFAs.
Example
Consider the following FA corresponding to (a+b)*b
CHAPTER
5
NONDETERMINISTIC FINITE AUTOMATA
35
Kleene’s theorem
Can the structure of above NFA be compared with the corresponding RE ?
 NFA useful because
i.
ii.
Applications in artificial intelligence (AI).
Given two FA’s for two languages with regular expressions r1 and r2, it is easy to
construct an NFA to accept language corresponding to regular expression r1 + r2.
iii.
This works when neither of the original FA’s has any arcs going into their original
initial states.
If one or both of the original FA’s has an arc going into its original initial state, the
newly constructed FA for the language corresponding to regular expression r1 + r2
may be incorrect. This is because the new FA may process part of the word on one of
the original FA’s and then process the rest of the word on the other FA, and then
incorrectly accept the word.
iv.