Chapter 1
Introduction to the Theory of
Computation
1
Theory of Computation

Can we mathematically model the computation?
 Automata Theory
 Regular languages: DFA, NFA, Regular expressions
 Context-free languages: Context-free grammar,
pushdown automata
 Turing machines: the most powerful model of
computation

Can the computer solve all problems?
 Computability Theory
 Halting problem

How efficient the solution is?
 Complexity Theory
 Time and space complexity (Big O notation)
2
1.1 Mathematical Preliminaries
Sets (1)

A set is a collection of objects
 Order and repetition is irrelevant
 X = {1, 2, 3}
 X = {n | 1  n  3, n  N }
 Y = { } =   Y is an empty set

The power set of X is the set of all subsets of X
 (X) = 2X = { Y | Y  X}
 |(X)| = 2|X|
Example X = {1, 2, 3}
 2X = (X)={{},{1},{2},{3},{1,2},{1,3},{2,3}, {1,2,3}}
 {1,2}  X
is a proper subset of X
 {1,2,3}  X is NOT a proper subset of X
 {1,2,3} is a superset of3{1, 2}

Sets (2)

The  symbol signifies membership (belong to):
 x  X indicates that x is a member or element
of the set X
 x  X indicates that x is not a member or
element of the set X
 Assume X = {1, 2, 3}, then
 1  X, but 4  X

Two sets are equal if they contain the same
members (elements)
 If X = {1, 2, 3}, Y = {1, 2, 3}, and Z = {2, 3, 4}
 Then X = Y, but X  Z, and so Y  Z
4
Sets (3)

Assume X = {1,2,3} and Y = {2,4}

Union
 X  Y = {z | z  X or z Y} = {1,2,3,4}

Intersection
 X  Y = {z | z  X and z Y} = {2}

Difference
 X - Y = {z | z  X and z  Y} = {1,3}
5
Sets (4)



Complement
 Let X be a subset of Y
 The complement of X ( X ) with respect to Y is
the set of elements in Y but not in X
 X = { z | z  Y and z  X } = Y - X
Assume X = {1,2,3} is a subset of Y = {1,2,3,4}
 X = {4}
DeMorgan’s Laws

(X  Y) = X  Y

(X  Y) = X  Y
6
Inductive Proofs

Prove a statement S(X) about a family of objects X
(e.g., integers, trees) in three parts:
 Basis Step:
 Prove for one or several small values of X directly

Inductive step:
 Assume S(Y) for Y smaller than X

Prove S(X) using that assumption
7
Inductive Proofs (EX1)


Prove by induction that
 S: 1 + 2 + 3 + ……+ n = n (n + 1) / 2
Basis step:
 n=1
 1 ( 1 + 1) / 2 = 1


Induction step (assumption step):
 Assume that S is true for all series of length < n
Prove that S is true for a series of length n?
 The series of length n is
 1+2+3+…+n-1+n
 By the assumption 1+2+…+n-1 = (n-1)((n-1)+1)/2
 Add n to the summation proves S.
8
Inductive Proofs (EX2)


Example
 A complete binary tree with n leaves has 2n - 1
nodes
Proof:
 Formally, S(T): if T is a binary tree with n
leaves, then T has 2n – 1 nodes
 Induction is on the size = number of nodes of T
9
Inductive Proofs (EX2)



Basis:
 If T has 1 leaf, it is a one-node tree. 1 = 2*1 - 1, so
OK
Induction:
 Assume S(U) for trees with fewer nodes than T. In
particular, assume for the subtrees of T
 T must be a root plus two subtrees U and V.
 If U and V have u and v leaves, respectively, and T
has t leaves, then u + v = t.
Proof:
 By the inductive hypothesis, U and V have 2u - 1 and
2v - 1 nodes, respectively.
 Then T has 1 + (2u -1) + (2v - 1) = 2(u + v) - 1 = 2t 1
 Proving the inductive step
10
If-And-Only-If Proofs





Often, a statement we need to prove is of the form
 “X if and only if Y”
We are then required to do two things:
 Prove the if-part: Assume Y and prove X
 Prove the only-if-part: Assume X, prove Y
Remember:
 The if and only-if parts are converses of each
other
One part, say “if X then Y” says nothing about
whether Y is true when X is false.
An equivalent form to “if X then Y” is “if not Y
then not X”; the latter is the contra-positive of the
former.
11
Equivalence of Sets


Many important facts in language theory are of the
form that two sets of strings, described in two
different ways, are really the same set
To prove sets S and T are the same, prove:
 x is in S if and only if x is in T. That is:
 Assume x is in S; prove x is in T .
 Assume x is in T ; prove x is in S.
12
Functions

Let f be a function defined on a set A and taking values in
a set B. Set A is called the domain of f; set B is called the
range of f.

Function f is said to be one-to-one if, whenever f(x) = f(y),
it must be the case that x = y.
 In other words, f is one-to-one if it maps distinct
objects to distinct objects.

Function f is said to be onto if, for any b  B, there exists
an a  A for which f(a) = f(b).

A function is bijection if it is one-to-one and onto.

If function f is defined for each element of A, then f is said
to be a total function; otherwise, f is said to be a partial
function.
13
Relations

Given two sets, X and Y, a relation R is a set of
ordered pairs (x, y) where x  X and y Y

Equivalence relations
 Reflexive
 R is reflexive if for every a  A, then (a, a)  R

Symmetric
 R is symmetric if for every a and b in A, if (a, b) 
R, then also (b, a)  R

Transitive
 R is transitive if for any a, b, c, if (a, b) and (b, c) 
R, then also (a, c)  R

R is an equivalence relation if R is reflexive,
symmetric and transitive
14
1.2: Strings and Languages (1)

Alphabet: Finite, nonempty set of symbols
 Examples:
  = {0, 1}: binary alphabet
  = {a, b, c, …, z}: the set of all lower case letters
 The set of all ASCII characters

String: Finite sequence of symbols from an
alphabet 
 Examples:
 01101 where  = {0, 1}
 abracadabra where  = {a, b, c, …, z}
15
1.2: Strings and Languages (2)

Empty String: The string with zero occurrences of
symbols from  and is denoted e or 

Length of String: Number of symbols in the string
 The length of a string w is usually written |w|
 |1010| = 4
 |e| = 0
 |uv| = |u| + |v|

Reverse : wR
 If w = abc, wR = cba
16
1.2: Strings and Languages (3)

Concatenation: if x and y are strings, then xy is the
string obtained by placing a copy of y immediately
after a copy of x
 x = a1a2 …ai, y = b1b2 …bj
 xy = a1a2 …aib1b2 …bj
 Example: x = 01101, y = 110, xy = 01101110
 xe = ex = x
17
1.2: Strings and Languages (4)

Power of an Alphabet: k = the set of strings of length k
with symbols from 
 Example:  = {0, 1}
 1 =  = {0, 1}
 2 = {00, 01, 10, 11}
 0 = {e}

Question: How many strings are there in 3?

The set of all strings over  is denoted *
 * = 0  1  2  3  …
Also
 + = 1  2  3  …
 * = +  {e}
 + = * - {e}

18
1.2: Strings and Languages (5)

Substring: any string of consecutive characters in
some string w
 If w = abc
 e, a, ab, abc are substrings of w

Prefix and suffix:
 if w = vu
 v is a prefix of w
 u is a suffix of w
 Example
 If w = abc
 a, ab , abc are prefixes of w
 c, bc, abc are suffixes of w
19
1.2: Strings and Languages (6)

Suppose: S is the string banana
 Prefix : ban, banana
 Suffix : ana, banana
 Substring : nan, ban, ana, banana

Proper prefix, suffix, or substring cannot be all
of S
20
1.2: Strings and Languages (8)


Language: set of strings chosen from some alphabet
A language is a subset of *
 Example of languages:
The set of valid Arabic words
The set of strings consisting of n 0’s followed by n 1’s
– {e, 01, 0011, 000111, …}
 The set of strings with equal number of 0’s and 1’s
– {e, 01, 10, 0011, 0101, 1010, 1001, 1100, …}



Empty language:  = { }
The language {e} consisting of the empty string
Note:   {e}
21
1.2: Strings and Languages (9)

Can concatenate languages
 L1L2 = {xy | x  L1, y  L2}
 Ln = L concatenated with itself n times
 L0 = {e}; L1 = L

Star-closure
 L* = L0  L1  L2  ...
 L+ = L* - L0

Languages can be finite or infinite
 L = {a, aba, bba}
 L = {an | n > 0}
22
1.2: Strings and Languages (10)
OPERATION
union of L and M
written L  M
concatenation of L
and M written LM
Kleene closure of L
written L*
DEFINITION
L  M = {s | s is in L or s is in M}
LM = {st | s is in L and t is in M}
L* =  Li , i=0,.., 
L* denotes “zero or more concatenations of “ L
positive closure of L
written L+
L+ =  Li, i=1,.., 
L+ denotes “one or more concatenations of “ L
L+ = LL*
23
1.2: Strings and Languages (11)
L = {A, B, …, Z, a, b, …z}
D = {1, 2, …, 9}
L  D = the set of letters and digits
LD = all strings consisting of a letter followed by a digit
L2 = the set of all two-letter strings
L4 = L2 L2 = the set of all four-letter strings
L* = { All possible strings of L plus e }, L+ = L* - e
D+ = set of strings of one or more digits
L (L  D ) = set of all strings consisting of a letter
followed by a a letter or a digit
L (L  D )* = set of all strings consisting of letters and
digits beginning with a letter
24
1.2: Strings and Languages (12)

The language L consists of strings over {a,b} in
which each string begins with an a should have an
even length




aa, ab  L
aaaa,aaab,aaba,aabb,abaa,abab,abba,abbb  L
baa  L
AL
25
1.2: Strings and Languages (13)

The language L consists of strings over {a,b} in
which each occurring of b is immediately
preceded by an a






eL
aL
abaab  L
bb  L
bab  L
abb  L
26
1.2: Strings and Languages (14)

Let X = {a,b,c} and Y = {abb, ba}. Then
 XY = {aabb, babb, cabb, aba, bba, cba}
 X0 = {e}
 X1 = X = {a,b,c}
 X2 = XX = {aa,ab,ac,ba,bb,bc,ca,cb,cc}
 X3 = XXX =
{aaa,aab,aac,aba,abb,abc,aca,acb,acc,baa,bab,ba
c,bba,bbb,bbc,bca,bcb,bcc,caa,cab,cac,cba,cbb,
cbc,cca,ccb,ccc}
27
1.2: Strings and Languages (15)

The language L = {a,b}*{bb}{a,b}* =(a|b)*bb(a|b)*
=*bb*
 consists of the strings over {a,b} that contain
the substring bb
 bb  L
 abb  L
 bbb  L
 aabb  L
 bbaaa  L
 bbabba  L
 abab  L
 bab  L
 bL
28
1.2: Strings and Languages (16)

Let L be the language that consists of all strings that begin
with aa or end with bb
 L1 = {aa}{a,b}*
 L2 = {a,b}*{bb}
 L = L1  L2 = {aa}{a,b}*  {a,b}*{bb}=aa* | *bb
 bb  L
 abb  L
 bbb  L
 aabb  L
 bbaaa  L
 bbabba  L
 abab  L
 bab  L
 ba  L
29
1.2: Strings and Languages (16)

Let L1 = {bb} and L2 = {e, bb, bbbb} over b

The languages L1* and L2* both contain precisely
the strings consisting of an even number of b’s.

e , with length zero, is an element of L1* and L2*
30
1.2: Strings and Languages (17)

What is the language of all even-length strings over
{a,b}
 L = {aa, bb, ab, ba}* = (aa|bb|ab|ba)*

What is the language of all odd-length strings over
{a,b}
 L = {a,b}* - {aa, bb, ab, ba}* or
 L = {a,b}{aa, bb, ab, ba}*
or
 L = {aa, bb, ab, ba}* {a,b}
or
 L = (a|b)(aa|bb|ab|ba)*
31