0100001101101111011011010
1110000011101010111010001
100101011100100010000001
0100110111010001110101011
001000110100101100101011
10011001000000100010001
1010010111011001101001011
10011011010010110111101101
110
[Computer Studies Division]
# Cor Jesu College #
Non-regular languages
(Pumping Lemma)
Fall 2006
010000110110111101101101011100000111010101110100011001010111001000100000010100110111010001110101011
Costas Busch - RPI
1
001000110100101100101011100110010000001000100011010010111011001101001011100110110100101101111011011
10
{a b : n  0}
n n
Non-regular languages
{vv : v {a, b}*}
R
Regular languages
a *b
b*c  a
b  c ( a  b) *
etc...
Fall 2006
010000110110111101101101011100000111010101110100011001010111001000100000010100110111010001110101011
Costas Busch - RPI
2
001000110100101100101011100110010000001000100011010010111011001101001011100110110100101101111011011
10
How can we prove that a language
is not regular?
L
Prove that there is no DFA or NFA or RE
that accepts L
Difficulty: this is not easy to prove
(since there is an infinite number of them)
Solution: use the Pumping Lemma !!!
Fall 2006
010000110110111101101101011100000111010101110100011001010111001000100000010100110111010001110101011
Costas Busch - RPI
3
001000110100101100101011100110010000001000100011010010111011001101001011100110110100101101111011011
10
0100001101101111011011010
1110000011101010111010001
100101011100100010000001
0100110111010001110101011
001000110100101100101011
10011001000000100010001
1010010111011001101001011
10011011010010110111101101
110
[Computer Studies Division]
# Cor Jesu College #
The Pigeonhole Principle
Fall 2006
010000110110111101101101011100000111010101110100011001010111001000100000010100110111010001110101011
Costas Busch - RPI
4
001000110100101100101011100110010000001000100011010010111011001101001011100110110100101101111011011
10
4 pigeons
•
3 pigeonholes
Fall 2006
010000110110111101101101011100000111010101110100011001010111001000100000010100110111010001110101011
Costas Busch - RPI
5
001000110100101100101011100110010000001000100011010010111011001101001011100110110100101101111011011
10
A pigeonhole must
contain at least two pigeons
•
Fall 2006
010000110110111101101101011100000111010101110100011001010111001000100000010100110111010001110101011
Costas Busch - RPI
6
001000110100101100101011100110010000001000100011010010111011001101001011100110110100101101111011011
10
n pigeons
•
...........
m
pigeonholes
nm
...........
Fall 2006
010000110110111101101101011100000111010101110100011001010111001000100000010100110111010001110101011
Costas Busch - RPI
7
001000110100101100101011100110010000001000100011010010111011001101001011100110110100101101111011011
10
n pigeons
•
m
The Pigeonhole Principle
pigeonholes
nm
There is a pigeonhole
with at least 2 pigeons
...........
Fall 2006
010000110110111101101101011100000111010101110100011001010111001000100000010100110111010001110101011
Costas Busch - RPI
8
001000110100101100101011100110010000001000100011010010111011001101001011100110110100101101111011011
10
0100001101101111011011010
1110000011101010111010001
100101011100100010000001
0100110111010001110101011
001000110100101100101011
10011001000000100010001
1010010111011001101001011
10011011010010110111101101
110
[Computer Studies Division]
# Cor Jesu College #
The Pigeonhole Principle
and
DFAs
Fall 2006
010000110110111101101101011100000111010101110100011001010111001000100000010100110111010001110101011
Costas Busch - RPI
9
001000110100101100101011100110010000001000100011010010111011001101001011100110110100101101111011011
10
•
Consider a DFA with
b
q1
states
b
b
a
q2
a
a
Fall 2006
4
q3
b
q4
a
010000110110111101101101011100000111010101110100011001010111001000100000010100110111010001110101011
Costas Busch - RPI
10
001000110100101100101011100110010000001000100011010010111011001101001011100110110100101101111011011
10
Consider the walk of a “long’’ string:
(length at least 4)
aaaab
A state is repeated in the walk of
q1
a
b
q1
Fall 2006
q2
a
q3
a
q2
a
q3
q2
q4
b
b
a
b
aaaab
a
a
q3
b
a
q4
010000110110111101101101011100000111010101110100011001010111001000100000010100110111010001110101011
Costas Busch - RPI
11
001000110100101100101011100110010000001000100011010010111011001101001011100110110100101101111011011
10
The state is repeated as a result of
the pigeonhole principle
Walk of
Pigeons:
(walk states)
q1
a
q2
a
q3
a
aaaab
q2
a
q3
b
q4
Are more than
Nests:
q1
(Automaton states)
Fall 2006
q2
q3
Repeated
state
q4
010000110110111101101101011100000111010101110100011001010111001000100000010100110111010001110101011
Costas Busch - RPI
12
001000110100101100101011100110010000001000100011010010111011001101001011100110110100101101111011011
10
Consider the walk of a “long’’ string:
(length at least 4)
Due to the pigeonhole principle:
A state is repeated in the walk of
q1
a
b
q1
Fall 2006
q2
a
q3
b
q4
b
q2
aabb
q4
b
b
a
aabb
a
a
q3
b
a
q4
010000110110111101101101011100000111010101110100011001010111001000100000010100110111010001110101011
Costas Busch - RPI
13
001000110100101100101011100110010000001000100011010010111011001101001011100110110100101101111011011
10
The state is repeated as a result of
the pigeonhole principle
Walk of
Pigeons:
q1
a
q2
a
q3
b
aabb
q4
b
q4
(walk states)
Are more than
Nests:
(Automaton states)
Fall 2006
q1
q2
q3
Automaton States
q4
Repeated
state14
010000110110111101101101011100000111010101110100011001010111001000100000010100110111010001110101011
Costas Busch - RPI
001000110100101100101011100110010000001000100011010010111011001101001011100110110100101101111011011
10
In General: If | w |  # states of DFA ,
by the pigeonhole principle,
a state is repeated in the walk w
Walk of w  1 2  k
q1  1
 2 ....  i q  i 1 ....
i
j
qi
 j 1....  k
qz
Arbitrary DFA
q1  1
Fall 2006
k q
......
qi
z
Repeated state
 2 ......
010000110110111101101101011100000111010101110100011001010111001000100000010100110111010001110101011
Costas Busch - RPI
15
001000110100101100101011100110010000001000100011010010111011001101001011100110110100101101111011011
10
| w |  # states of DFA  m
Pigeons:
Walk of
(walk states)
q1
....
qi
....
w
qi
....
qz
Are
more
than
Nests: q1
q2
(Automaton states)
Fall 2006
....
qi
....
A state is
repeated
qm1
qm
010000110110111101101101011100000111010101110100011001010111001000100000010100110111010001110101011
Costas Busch - RPI
16
001000110100101100101011100110010000001000100011010010111011001101001011100110110100101101111011011
10
0100001101101111011011010
1110000011101010111010001
100101011100100010000001
0100110111010001110101011
001000110100101100101011
10011001000000100010001
1010010111011001101001011
10011011010010110111101101
110
[Computer Studies Division]
# Cor Jesu College #
The Pumping Lemma
Fall 2006
010000110110111101101101011100000111010101110100011001010111001000100000010100110111010001110101011
Costas Busch - RPI
17
001000110100101100101011100110010000001000100011010010111011001101001011100110110100101101111011011
10
Take an infinite regular language L
(contains an infinite number of strings)
There exists a DFA that accepts
L
m
states
Fall 2006
010000110110111101101101011100000111010101110100011001010111001000100000010100110111010001110101011
Costas Busch - RPI
18
001000110100101100101011100110010000001000100011010010111011001101001011100110110100101101111011011
10
Take string
w L
with
| w|  m
(number of
states of DFA)
then, at least one state is repeated
in the walk of w
Walk in DFA of
w  1 2  k
1
 2 ......
q
......  k
Repeated state in DFA
Fall 2006
010000110110111101101101011100000111010101110100011001010111001000100000010100110111010001110101011
Costas Busch - RPI
19
001000110100101100101011100110010000001000100011010010111011001101001011100110110100101101111011011
10
There could be many states repeated
Take
q
to be the first state repeated
One dimensional projection of walk
1
First
Second
occurrence
occurrence
 2 ....  i q i 1 ....
j
q
w:
 j 1....  k
Unique states
Fall 2006
010000110110111101101101011100000111010101110100011001010111001000100000010100110111010001110101011
Costas Busch - RPI
20
001000110100101100101011100110010000001000100011010010111011001101001011100110110100101101111011011
10
We can write
w  xyz
One dimensional projection of walk
1
First
Second
occurrence
occurrence
 2 ....  i q i 1 ....
x  1 i
Fall 2006
j
y   i 1 j
q
w:
 j 1....  k
z   j 1 k
010000110110111101101101011100000111010101110100011001010111001000100000010100110111010001110101011
Costas Busch - RPI
21
001000110100101100101011100110010000001000100011010010111011001101001011100110110100101101111011011
10
In DFA:
w x y z
contains only
first occurrence of q
y
...
j
...
1
Fall 2006
x
i
 i 1
q
 j 1
...
...
z
k
010000110110111101101101011100000111010101110100011001010111001000100000010100110111010001110101011
Costas Busch - RPI
22
001000110100101100101011100110010000001000100011010010111011001101001011100110110100101101111011011
10
Observation:
length
| x y |  m number
of states
of DFA
y
...
Unique States
j
...
1
Fall 2006
x
i
 i 1
q
Since, in xy no
state is repeated
(except q)
010000110110111101101101011100000111010101110100011001010111001000100000010100110111010001110101011
Costas Busch - RPI
23
001000110100101100101011100110010000001000100011010010111011001101001011100110110100101101111011011
10
Observation:
length
| y | 1
Since there is at least one transition in loop
y
...
j
 i 1
q
Fall 2006
010000110110111101101101011100000111010101110100011001010111001000100000010100110111010001110101011
Costas Busch - RPI
24
001000110100101100101011100110010000001000100011010010111011001101001011100110110100101101111011011
10
We do not care about the form of string
z
may actually overlap with the paths of
z
x and y
y
...
z
...
Fall 2006
x
q
010000110110111101101101011100000111010101110100011001010111001000100000010100110111010001110101011
Costas Busch - RPI
25
001000110100101100101011100110010000001000100011010010111011001101001011100110110100101101111011011
10
Additional string:
Do not follow loop
The string
is accepted
xz
y
...
j
...
1
Fall 2006
x
i
 i 1
q
 j 1
...
...
z
k
010000110110111101101101011100000111010101110100011001010111001000100000010100110111010001110101011
Costas Busch - RPI
26
001000110100101100101011100110010000001000100011010010111011001101001011100110110100101101111011011
10
Additional string:
Follow loop
2 times
xyyz
y
...
j
...
1
Fall 2006
The string
is accepted
x
i
 i 1
q
 j 1
...
...
z
k
010000110110111101101101011100000111010101110100011001010111001000100000010100110111010001110101011
Costas Busch - RPI
27
001000110100101100101011100110010000001000100011010010111011001101001011100110110100101101111011011
10
Additional string:
The string
is accepted
Follow loop
3 times
y
...
j
...
1
Fall 2006
xyyyz
x
i
 i 1
q
 j 1
...
...
z
k
010000110110111101101101011100000111010101110100011001010111001000100000010100110111010001110101011
Costas Busch - RPI
28
001000110100101100101011100110010000001000100011010010111011001101001011100110110100101101111011011
10
In General:
The string
is accepted
Follow loop
times
xy z
i  0, 1, 2, ...
y
i
...
j
...
1
Fall 2006
i
x
i
 i 1
q
 j 1
...
...
z
k
010000110110111101101101011100000111010101110100011001010111001000100000010100110111010001110101011
Costas Busch - RPI
29
001000110100101100101011100110010000001000100011010010111011001101001011100110110100101101111011011
10
Therefore:
i  0, 1, 2, ...
x y z L
i
Language accepted by the DFA
y
...
j
...
1
Fall 2006
x
i
 i 1
q
 j 1
...
...
z
k
010000110110111101101101011100000111010101110100011001010111001000100000010100110111010001110101011
Costas Busch - RPI
30
001000110100101100101011100110010000001000100011010010111011001101001011100110110100101101111011011
10
In other words, we described:
The Pumping Lemma !!!
Fall 2006
010000110110111101101101011100000111010101110100011001010111001000100000010100110111010001110101011
Costas Busch - RPI
31
001000110100101100101011100110010000001000100011010010111011001101001011100110110100101101111011011
10
The Pumping Lemma:
• Given a infinite regular language
• there exists an integer
• for any string
• we can write
• with
w L
m
L
(critical length)
with length
| w| m
w x y z
| x y |  m and | y |  1
• such that:
Fall 2006
xy z  L
i
i  0, 1, 2, ...
010000110110111101101101011100000111010101110100011001010111001000100000010100110111010001110101011
Costas Busch - RPI
32
001000110100101100101011100110010000001000100011010010111011001101001011100110110100101101111011011
10
In the book:
Critical length
Fall 2006
m
= Pumping length
p
010000110110111101101101011100000111010101110100011001010111001000100000010100110111010001110101011
Costas Busch - RPI
33
001000110100101100101011100110010000001000100011010010111011001101001011100110110100101101111011011
10
0100001101101111011011010
1110000011101010111010001
100101011100100010000001
0100110111010001110101011
001000110100101100101011
10011001000000100010001
1010010111011001101001011
10011011010010110111101101
110
[Computer Studies Division]
# Cor Jesu College #
Applications
of
the Pumping Lemma
Fall 2006
010000110110111101101101011100000111010101110100011001010111001000100000010100110111010001110101011
Costas Busch - RPI
34
001000110100101100101011100110010000001000100011010010111011001101001011100110110100101101111011011
10
Observation:
• Every language of finite size has to be regular
(we can easily construct an NFA
that accepts every string in the language)
Therefore, every non-regular language
has to be of infinite size
(contains an infinite number of strings)
Fall 2006
010000110110111101101101011100000111010101110100011001010111001000100000010100110111010001110101011
Costas Busch - RPI
35
001000110100101100101011100110010000001000100011010010111011001101001011100110110100101101111011011
10
Suppose you want to prove that
An infinite language L is not regular
1. Assume the opposite:
L
is regular
2. The pumping lemma should hold for
L
3. Use the pumping lemma to obtain a
contradiction
4. Therefore,
Fall 2006
L is not regular
010000110110111101101101011100000111010101110100011001010111001000100000010100110111010001110101011
Costas Busch - RPI
36
001000110100101100101011100110010000001000100011010010111011001101001011100110110100101101111011011
10
Explanation of Step 3: How to get a contradiction
1. Let
m be the critical length for L
2. Choose a particular string w L which satisfies
the length condition | w | m
3. Write
w  xyz
4. Show that
w  xy z  L
i
for some
i 1
5. This gives a contradiction, since from
pumping lemma
Fall 2006
w  xy z  L
i
010000110110111101101101011100000111010101110100011001010111001000100000010100110111010001110101011
Costas Busch - RPI
37
001000110100101100101011100110010000001000100011010010111011001101001011100110110100101101111011011
10
Note:
It suffices to show that
only one string w L
gives a contradiction
You don’t need to obtain
contradiction for every w L
Fall 2006
010000110110111101101101011100000111010101110100011001010111001000100000010100110111010001110101011
Costas Busch - RPI
38
001000110100101100101011100110010000001000100011010010111011001101001011100110110100101101111011011
10
Example of Pumping Lemma application
Theorem: The language L  {a nb n : n  0}
is not regular
Proof:
Fall 2006
Use the Pumping Lemma
010000110110111101101101011100000111010101110100011001010111001000100000010100110111010001110101011
Costas Busch - RPI
39
001000110100101100101011100110010000001000100011010010111011001101001011100110110100101101111011011
10
L  {a b : n  0}
n n
Assume for contradiction
that L is a regular language
Since L is infinite
we can apply the Pumping Lemma
Fall 2006
010000110110111101101101011100000111010101110100011001010111001000100000010100110111010001110101011
Costas Busch - RPI
40
001000110100101100101011100110010000001000100011010010111011001101001011100110110100101101111011011
10
L  {a b : n  0}
n n
Let
m
be the critical length for L
Pick a string
w such that: w  L
and length
We pick
Fall 2006
| w| m
wa b
m m
010000110110111101101101011100000111010101110100011001010111001000100000010100110111010001110101011
Costas Busch - RPI
41
001000110100101100101011100110010000001000100011010010111011001101001011100110110100101101111011011
10
From the Pumping Lemma:
we can write w  a b
m
with lengths
m
x y z
| x y |  m, | y | 1
m
w  xyz
a b
m m
m
 a...aa...aa...ab...b
x
y
z
k
y

a
, 1k m
Thus:
Fall 2006
010000110110111101101101011100000111010101110100011001010111001000100000010100110111010001110101011
Costas Busch - RPI
42
001000110100101100101011100110010000001000100011010010111011001101001011100110110100101101111011011
10
y a , 1k m
k
x y za b
m m
From the Pumping Lemma:
xy z  L
i
i  0, 1, 2, ...
Thus:
Fall 2006
xy z  L
2
010000110110111101101101011100000111010101110100011001010111001000100000010100110111010001110101011
Costas Busch - RPI
43
001000110100101100101011100110010000001000100011010010111011001101001011100110110100101101111011011
10
y a , 1k m
k
x y za b
m m
From the Pumping Lemma:
xy z  L
2
mk
m
xy z  a...aa...aa...aa...ab...b  L
2
x
Thus:
Fall 2006
a
y
y
z
m k m
b L
010000110110111101101101011100000111010101110100011001010111001000100000010100110111010001110101011
Costas Busch - RPI
44
001000110100101100101011100110010000001000100011010010111011001101001011100110110100101101111011011
10
a
BUT:
m k m
b L
k ≥1
L  {a b : n  0}
n n
a
m k m
b L
CONTRADICTION!!!
Fall 2006
010000110110111101101101011100000111010101110100011001010111001000100000010100110111010001110101011
Costas Busch - RPI
45
001000110100101100101011100110010000001000100011010010111011001101001011100110110100101101111011011
10
Therefore:
Our assumption that L
is a regular language is not true
Conclusion: L is not a regular language
Fall 2006
END OF PROOF
010000110110111101101101011100000111010101110100011001010111001000100000010100110111010001110101011
Costas Busch - RPI
46
001000110100101100101011100110010000001000100011010010111011001101001011100110110100101101111011011
10
Non-regular language
{a b : n  0}
n n
Regular languages
* *
L( a b )
Fall 2006
010000110110111101101101011100000111010101110100011001010111001000100000010100110111010001110101011
Costas Busch - RPI
47
001000110100101100101011100110010000001000100011010010111011001101001011100110110100101101111011011
10