Unit 9
More Pushdown Automata
Context-free Languages
Pumping Lemma for CFL
Reading: Sipser, chapter 2.3
1
Properties of PDAs
• An NFA can only distinguish between |Q|
different characterizations.
• A PDA can distinguish between an unlimited
number of characterizations.
• A PDA can recognize non-regular languages
because the stack can ‘count’.
• A PDA can count ‘more than once’. But it can
not mix counters. Only one active counter can
be used each time.
2
Example: L = {aibicjdj |i,j0}
Construct a PDA to recognize:
L = {aibicjdj |i,j0}
We will use the empty stack model.
The basic idea:
• Push to the stack an A for each a, pop an A
for each b.
• Push to the stack a C for each c, pop a C for
each d .
3
Properties of PDAs
• We had two ways to describe regular
languages:
Regular-Expressions
DFA / NFA
syntactic
computational
• How about context-free-languages?
CFG
syntactic
PDA
computational
4
CFG=PDA
Theorem: A language is context-free iff
some pushdown automaton recognizes it.
Proof:
• CFLPDA: we show that if L is CFL
then a PDA recognizes it.
• PDA CFL: we show that if a PDA
recognizes L then L is CFL.
5
From CFG to PDA
• Proof idea: Use PDA to simulate leftmost
derivations.
• Leftmost derivation : A derivation of a string is
a leftmost derivation if at every step the leftmost
remaining variable is the one replaced.
• We use the stack to store the suffix that has not
been derived so far.
• Any terminal symbols appearing before the
leftmost variable are matched right away.
6
Different derivations for the same
parse tree
CFG: EEE | E+E
E0 | 1 | 2 | … | 9
leftmost
derivation
E
E  EE
 E  EE
 5 EE
 5  3 E
 5  3 2
E  EE
E
E
5
E
E
+
3
x
2
rightmost
derivation
 E2
 E  E2
 E  3 2
 5  3 2
7
Starting configuration:
control
input: 5 + 3 x 2
stack: E
$
E
control
E  EE
input: 5 + 3 x 2
EE
stack: E
x
E
$
control
E  EE
 E  EE
input: 5 + 3 x 2
E+EE
stack: E
+
E
x
E
$
control
E  EE
 E  EE
 5 EE
input: 5 + 3 x 2
5+EE
5
stack: +
E
x
E
$
control
E  EE
 E  EE
 5 EE
input: 5 + 3 x 2
EE
E  EE
 E  EE
 5 EE
 5  3 E
stack: E
x
E
$
control
input: 5 + 3 x 2
3E
stack: 3
x
E
$
E  EE
control
 E  EE
 5 EE
 5  3 E
input: 5 + 3 x 2
stack: E
$
E
E  EE
 E  EE
 5 EE
 5  3 E
 5  3 2
control
input:
5+3x2
2
stack: 2
$
E  EE
 E  EE
 5 EE
 5  3 E
 5  3 2
control
input: 5 + 3 x 2
stack: $
The string ‘5 + 3 x 2’ is accepted
From CFG to PDA
Informally:
1. Place the marker symbol $ and the start variable
S on the stack.
2. Repeat the following steps:
– If the top of the stack is a variable A:
Choose a rule A→1…k and substitute A with 1…k
– If the top of the stack is a terminal a:
Read next input symbol and compare to a
If they don’t match, reject (die)
– If top of stack is $, go to accept state
13
From CFG to PDA
• For a given CFG G=(V,,S,R),
we construct a PDA P=(Q,,,,q0,F) where:
– Q={qstart, qloop, qaccpt}
–  = V{$}
– q0=qstart
– F={qaccpt}
14
From CFG to PDA
• We define  as follows (shorthand notation):
–
–
–
–
(qstart,,)={(qloop,S$)}
(qloop,,A)={(qloop, 1…k) | for each A1…k in R}
(qloop,a,a)={(qloop,) | for each a }
(qloop,,$)={(qaccpt,)}
{,A1…k | for rules A 1…k}
{a,a | for all a}
qstart
,S$
qloop
,$
qaccpt
15
Example:
• Construct a PDA for the following CFG G:
SaTb | b
TTa | 
qstart
,S$
L(G)= a*b
,SaTb
,Sb
,TTa
,T
a,a
b,b
qloop
,$
qaccpt
16
From PDA to CFG
✓
context-free grammar
pushdown automaton
• First, we simplify the PDA:
– It has a single accept state qf
– $ is always popped exactly before accepting
– Each transition is either a push, or a pop, but
not both
17
From PDA to CFG
• single accept state qf:
,
,
18
From PDA to CFG
• $ is always popped exactly before accepting:
{,A | A, A$}
,$
19
From PDA to CFG
• Each transition is either a push, or a pop:
,ab
,a
,b
,
z
,z ,z
20
From PDA to CFG
• For any word w accepted by a PDA
P=(Q,,,,q0,qf) the process starts at q0 with an
empty stack and ends at qf with an empty stack.
• Definition: for any two states p,qQ we define
Lp,q to be the language that if we starts at p with
an empty stack and run on wLp,q we end at q
with an empty stack.
• We define for Lp,q a variables Ap,q s.t.
Lp,q = {w | Ap,q * w}
• Note, that L(P)=Lq0,qf
21
From PDA to CFG
• Consider a word wLp,q
• While running w on P, the stack is empty at p and
at q but what happens in the middle?
• Two possibilities:
– Option 1: The stack also empty in the middle
– Option 2: The stack never empty in the middle
stack
height
p
r
q
p
q
22
From PDA to CFG
Option 1: The stack also empty in the middle
• If the stack become empty at some state r then the
word wLpq can be reconstructed by a
concatenation of a word from Lpr and a word from
Lrq, thus Lpr Lrq  Lpq
• In the CFG we express this by a rule: Apq  AprArq
p
generated by Apr
q
r
generated by Arq
23
From PDA to CFG
Option 2: The stack never empty in the middle
• The symbol that has been pushed at p is the
symbol that is popped at q.
• Thus, if at p we read a symbol a and moved to r,
while from state s we read a symbol b and moved
to q, aLr,sbLp,q and in CFG we have Apq  aArsb
r
s q
b
p
a
generated by Ars
24
From PDA to CFG
Let P=(Q, , , , q0, qf) a given PDA.
We construct a CFL G=(V,,S,R) as follows*:
• V = {Ap,q | p,qQ}
• S=A q0,qf
• R is a set of rules constructed as follows:
* Proof of correctness and further reading at the supplementary
material in the course web page .
25
From PDA to CFG
• Add the following rules to R:
1. For each p,q,r,sQ, t, and a,b,
if (r,t)(p,a,) and (q,)(s,b,t) add a rule
Apq  aAr,sb
a,t
p
r
b,t
s
push t
q
pop t
2. For each p,q,rQ, add a rule Ap,q Ap,r Ar,q
p
r
1. For each pQ, add the rule Ap,p
q
p
26
Example:
qs
qs
0,A
1,A
,$
#,
,$
q0
q1
0,A
,$
#,z
q0
q3
q2
1,A
,z
,$
L(P)=0n#1n
q1
q2
27
Example:
qs
0,A
,$
#,z
q0
q3
1,A
,z
,$
q1
q2
start variable: AS2
productions:
ASS → ASSASS
ASS → AS0A0S
ASS → AS1A1S
ASS → AS2A2S
AS1 → ASSAS1
AS1 → AS0A01
AS1 → AS1A11
...
ASS → 
A00 → 
A11 → 
A22 → 
A33 → 
AS2 → A01
A01 → 0 A011
A01 → #A33
28
CFG=PDA
• We have shown that a language is context-free
iff some pushdown automaton recognizes it.
• In particular all regular languages can be
generated by CFGs and so can be recognized
by PDA.
• The class of languages accepted by nondeterministic PDAs is larger than those
accepted by deterministic PDAs.
29
The Context-free Languages
DPDA
context-free languages
the regular languages
30
Non context-free Languages
• Consider the language L={aibici |i0}.
• When trying to build a push-down automaton
that recognizes L, we can compare the number
of a-'s with b-'s or c-'s but not both;
• If we compared the number of a-'s to the
number of b-'s then we can't compare c-'s with
any of them, as at this stage the stack (or
counter) is empty.
31
Non context-free Languages
• So some languages seem to be not CFL.
• The question is which?
• This can be determined using the pumping
lemma for context-free languages.
32
The Pumping Lemma - background
• Let L be a CFL and let G be a simple
grammar (no unit/ rules) generating it.
• Let wL be a long enough word (we will say
later what is long).
• The parsing tree of w contains a long path
from S to some leaf (terminal).
• On this long path some variable R must
repeat (remember, w is long).
33
The Pumping Lemma - background
S
• Divide w into uvxyz
according to the parse
tree, as in the figure.
R
• Each occurrence of R
has a subtree under it.
R
u
v
x
y
z
34
The Pumping Lemma - background
• The upper occurrence of R has a larger subtree
and generates vxy.
• The lower occurrence of R has a smaller
subtree and generates only x.
• Both subtrees are generated by the same
variable R.
• That means if we substitute one for the other we
will still obtain valid parse trees.
S
R
R
u
v
x
y
z
35
S
S
R
R
R
x
u
v
v
y
x
z
u
z
y
Replacing the smaller by
the larger repeatedly
generates the string
uvixyiz at each i>0.
Replacing the larger by
the smaller generates the
string uxz or uvixyiz
where i=0.
Therefore, for all i0, wi = uvixyiz is also in L
36
The Pumping Length
• That means that every CFL has a special
value called the pumping length such that all
strings longer than the pumping length can
be "pumped".
• The string can be divided into 5 parts
w=uvxyz.
• The second and fourth can be pumped to
produce additional words in L.
• for all k0, wk = uvkxykz can also be
generated by the grammar.
37
Pumping Lemma for CFL
Lemma: Let L be a context-free language.
There is a positive integer p (the pumping
length) such that for all strings wL with
|w|p, w can be divided into five pieces
w=uvxyz satisfying the following conditions:
1. |vy|>0
2. |vxy|p
3. for each i0, uvixyizL
38
Proof - value of p
• First we find out the value of p.
• Let G be a CFG for CFL L.
• Let b be the maximum number of symbols in
the right side of any rule in G.
• So we know that in any parse tree of G a node
can't have more than b children.
• So if the height of a parsing tree for wL is h
then |w|< bh
(h>logb|w|).
A
A123 b

1 2 3
b
39
Proof - value of p
• Let |V| be the number of variables in G.
• We set p = b|V|+2 .
(h>logb|p|).
• Then for any string of length p the parse tree
requires height at least |V|+2 (Note, b>1 since
there are no unit rules).
• Given a string wL, s.t. |w|  p , since G has
only |V| variables, at least one of the variables
repeats (height |V|+2  |V|+1 variables +
terminal).
• W.l.o.g. assume this variable is R
40
Proof – condition 1
• To prove condition 1 (|vy|>0) we have to show it
is impossible that both v and y are .
• We use a grammar without unit rules.
• But the only way to have v=y=, is to have a
rule R  R, which is a unit rule. Contradiction.
• So condition 1 is satisfied.
S
R
R
u
v
x
y
z
41
Proof – condition 2
• To prove condition 2 (|vxy|p) we will check
the height of the subtree rooted in first R =
the subtree that generates vxy.
• Its height is at most |V|+2 (R was selected as
a variable that has two occurrences within
the bottom |V|+1 levels of the parsing tree).
• So it can generate a string of length at most
b|V|+2.
• Since p= b|V|+2, condition 2 is satisfied.
42
Proof – condition 3
S
S
R
R
R
u
v
R
v
x
x
y
z
y
Replacing the smaller by
the larger repeatedly
generates the string
uv ixy iz at each i>0.
u
z
Replacing the larger by
the smaller generates
the string uxz or uv ixy iz
where i=0.
Therefore, for all i0, wi=uvixyiz is also in L
43
Usage of the lemma
• We use the pumping lemma to prove that a
language is not context-free.
General Structure:
• Assume (by contradiction) that L is contextfree and therefore should fulfill the lemma.
• Let p be the pumping length for L.
• Select a word wL s.t. |w|>p.
• Show that for any partition of w into five parts
uvxyz such that |vy|>0 and |vxy|p, there
exists an i such that wi= uvixyiz  L
• Contradiction!
44
Usage of the lemma - Example
• We use the pumping lemma to prove that the
language L={anbncn |n0} is not context-free.
Proof:
• Assume, by contradiction, that L is context-free
thus satisfying the lemma.
• Let p be the pumping length for L.
• We select the string w= apbpcp . wL and |w|>p,
so it can be pumped.
45
L={anbncn |n0}
• Divide w into five parts uvxyz such that |vy|>0
and |vxy|p.
• There are two cases:
1. u and y are both homogeneous.
2. v or y is heterogeneous.
46
L  {a nb n c n | n  0}. Let w  a p b p c p  uvxyz.
ap
Case 1:
v
v and y are
homogeneous
bp
cp
y
v
y
v
y
v
y
v
y
Case 2
v or y is
heterogeneous
or
v
y
47
L={anbncn |n0}
Case 1: v and y contain only one type of
alphabet symbol. By choosing i=2 we get
w2=uv2xy2z in which the number of
appearances of one or two symbols
increased while the third symbol remain
unchanged. So w2 cannot contain the same
number of a's, b's and c's and w2L.
48
L={aibici |i0}
Case 2: v or y contain two types of alphabet
symbols. (Cannot have three because
condition 2 holds). So if we choose i=2 then
the order of the symbols in v2 or y2 was
destroyed and w2=uv2xy2z L.
Conclusion: The assumption that L is contextfree is false. and L={anbncn |n0} is not CFL.
49
Pumping Lemma – another example
Let L={ww| w{0,1}*}
[ 00, 110110L ; 010, 0010L ]
Prove that L is not context-free
Proof: In class.
50
Intersection 1
CFL are not closed under intersection.
Proof: by contradiction
L1={anbnck |n,k>0}
L2={akbncn | n,k>0}
Both L1 and L2 are CFL but
L1L2 = {ajbjcj | j>0} is not a CFL.
51
Intersection 2
• CFL and RL are closed under intersection.
Proof:
Build a product automaton of a PDA and a DFA:
Note that the resulting automaton is a PDA.
DFA
input
AND
accept/reject
PDA
stack
52
Complement
Theorem: CFL are not closed under complement.
Proof: by contradiction.
• Take two CFLs L1 and L2 and assume that ~L1
and ~L2 are CFL. (~L denotes complement)
• CFL are closed under union operation so
~L1~L2 is a CFL.
• Using our assumption again we get that
~(~L1~L2) is a CFL as well.
• But ~(~L1~L2) =L1L2 and we already know
that CFLs are not closed under intersection.
53