Introduction to the Theory of
Computation
Qiuyue Wang
[email protected]
[email protected]
Goals
 What are the fundamental capabilities
and limitations of computers?
 What is a computer?
 What can and can not be computed?
 How quickly can a problem be computed?
Computational Models
 What is a computer?
 Formal definition of computation
 Computational models
 Turing machines (= real computers)
 Simpler computing devices: finite state
automata, push-down automata
Computability
 What can and can not be computed?
 Some problems have no algorithms
(we will prove this)
 Halting problem
Complexity
 How quickly can a problem be
computed?
 Many problems probably have no
efficient algorithms (no one knows
how to prove this yet)
 NP-hardness (= what can not be
computed efficiently)
 factoring an integer into primes
 determining the shortest tour of given n
cities
What is Theory Good for?
 Elegant way of thinking
 Expanding your minds
 Useful tools in practice
 finite automata, regular expressions:
text processing
 grammars: programming language
design and specification
 NP-hardness: approximate solutions or
randomized computation
Text Book and References
 Michael Sipser, Introduction to the
Theory of Computation, 计算理论导论，
机械工业出版社，2002.
 J. E. Hopcroft, R. Motivani, and J. D.
Ullman, Introduction to Automata
Theory, Languages, and Computation,
Second
Edition,
Addison-Wesley,
2001.
 http://info.ruc.edu.cn/wangqiuyue/comp_t
heory.htm
Contents (1)
 0. Introduction
 Mathematical notations, proofs
 1. Regular Languages
 1.1. Finite Automata
 1.2. Regular Expressions
 1.3. Properties of Regular Languages
 2. Context-Free Languages
 2.1. Context-Free Grammar
 2.2. Pushdown Automata
 2.3. Properties of Context-Free Languages
Contents (2)
 3. The Church-Turing Thesis
 3.1. Turing Machines
 3.2. Variants of Turing Machines
 3.3. The Definition of Algorithm
 4. Decidability
 4.1. Decidable Languages
 4.2. The Halting Problem
 5. Reducibility
 5.1. Undecidable Problems
 5.2. Mapping Reducibility
Contents (3)
 7. Time Complexity
 7.1. P and NP
 7.2. NP-Completeness
 7.3. NP-Complete Problems
 Final Exam
Grading
 Assignments: 20%
 Gradiance Online Accelerated Learning
 Project: 20%
 Final Exam: 60%
Mathematical Notations





Sets
Sequences and Tuples
Functions and Relations
Graphs
Strings and Languages
Set
 No order, no repetition among
members of a set.
 Subset: A  B
 Union: A  B
 Intersection: A  B
 Complement: A
Power Set
 Power set: the set of all subsets
 P (A) = {S| S  A}
 E.g. A = {0, 1}
P (A) = { , {0}, {1}, {0, 1} }
 |P (A)| = 2|A|
Sequences and Tuples
 A sequence of objects is a list of
these objects in some order.
 A tuple is a finite sequence of objects.
 k-tuple, pair
 Cartesian product of sets: AB = {(a,
b)| aA, bB}
 A = {0, 1}, B={3, 4}, AB = {(0, 3), (0,
4), (1, 3), (1, 4)}
Functions
 f: DR
 f is a function, D: domain, R: range
 E.g. f: {0, 1}  {2, 3}
f(x) = x+2
 A function takes an input and produces an
output
 The same input always produces the same
output
 k-ary function
 D = A1… Ak, i.e. k arguments
 k=1: unary function, k=2: binary function
Relations
 A set of k-tuples S…S is called an
k-ary relation on S
 E.g. binary relation “<“ on N
 R is an equivalence relation if and
only if
 Reflexive: xRx for all x in S;
 Symmetric: xRy implies yRx;
 Transitive: xRy and yRz imply xRz;
 E.g. m is an equivalence relation
Closures of Relations
 P-closure of a relation R is the smallest
relation R’ that includes all the pairs of R
and possesses the properties in P.
 Transitive closure of R, denoted R+, is
1) If (a, b) is in R, then (a, b) is in R+.
2) If (a, b) is in R+ and (b, c) is in R, then (a, c) is
in R+.
3) Nothing is in R+ unless it so follows from (1) and
(2).
 Reflexive and transitive closure of R,
denoted R*, is easily seen to be R+ 
{(a,a)| aS}
Example
 R = {(1,2), (2,2), (2,3)} be a relation
on the set {1, 2, 3}. Then
 R+ = {(1,2), (2,2), (2,3), (1,3)}
 R* = {(1,1), (1,2), (1,3), (2,2), (2,3),
(3,3)}
Graphs
 G = (V, E), V is a set of vertices (or
nodes), and E is a set of pairs of
vertices (called edges).
 E.g. G=(V, E), V={1,2,3,4},
E={(1,2), (2,3), (1,3), (2,4), (1,4)}
 A path in a graph is a sequence of
nodes connected by edges
 Directed graph
 Tree
Strings
 Symbol: e.g. 0, a, #, !, …
 String: a finite sequence of symbols





E.g. 011001, abbc
Length of string: |011001| = 6
Empty string: ,
|| = 0
Substring: 100 is a substring of 011001
Concatenation of strings: 011001abbc
Languages
 Alphabet: a finite set of symbols
 E.g.  = {0, 1}
 Language: a set of strings from some
one alphabet
 Empty set: 
 Set consisting of the empty string: {}
 E.g. L = {, 01, 0011, 000111, …},
English
Proofs
 Proof by construction
 Proof by contradiction
 Proof by induction
Inductive Proofs
 Prove a statement S(X) about a
family of objects X (e.g., integers,
trees) in two parts:
 Basis: Prove for one or several small
values of X directly.
 Inductive step: Assume S(Y ) for Y
“smaller than” X; prove S(X) using that
assumption.
Example
 A binary tree with n leaves has 2n-1
nodes.
If-And-Only-If Proofs
 Often, a statement we need to prove
is of the form “X if and only if Y .” We
are then required to do two things:
 Prove the if-part: Assume Y and prove X.
 Prove the only-if-part: Assume X, prove
Y.
Example
 Equivalence of two sets
 To prove two sets S=T, prove that x is in
S if and only if x is in T