Chapter 0: Introduction
slides © 2017, David Doty
ECS 120: Theory of Computation
based on “Introduction to the Theory of Computation” by Sipser
-
Multivariate polynomials
16
15
14
13 2 2
4 –3 =7
12
11
x2 – y2 ≠ 2
10
9
for
any
integers
8
7
32 – 22 = 5
6
5
4
3 22 – 12 = 3
2
1
12 – 02 = 1
0
x2 – y2 = 2
Does this have a solution?
YES: x = √2, y = 0
x,y
Does this have an integer solution?
NO
x2 + 2xy – y3 = 13
Does this have an integer solution?
YES: x = 3, y = 2
32 + 2∙3∙2 – 23 = 9 + 12 – 8 = 13
Diophantine equations: multivariate polynomials with integer roots
0.1: Automata, Computability, and Complexity
2
Root-finding algorithms
Task A: write a program that indicates whether a multivariate
polynomial has a real root
fun fact:
this is possible
real_root("x*x – y*y – 2")
real_root("x*x + 2*x*y – y*y*y – 13")
real_root("x*x + 1")
== true
== true
== false
Task B: write a program that indicates whether a multivariate
polynomial has an integer root
depressing fact:
this is impossible
integer_root("x*x – y*y – 2")
== false
integer_root("x*x + 2*x*y – y*y*y – 13") == true
integer_root("x*x + 1")
== false
0.1: Automata, Computability, and Complexity
3
Euler vs. Hamilton
Eulerian cycle: visits each edge exactly once
Hamiltonian cycle: visits each node exactly once
exists if and only if every node has an even
number of neighbors
exists if and only if… ?
a
G1
b
G2
e
d
f
c
a
b
c
d
0.1: Automata, Computability, and Complexity
4
Efficient cycle-finding algorithms
Task A: write an efficient program that indicates whether a graph has
a cycle visiting each edge exactly once (Eulerian cycle)
easy (check degree of each node)
Task B: write an efficient program that indicates whether a graph has
a cycle visiting each node exactly once (Hamiltonian cycle)
impossible (assuming P ≠ NP)
0.1: Automata, Computability, and Complexity
5
Regular expressions
• Regular expression (regex) example
0 (0|1|2)* | (0|2)* 1
matches ternary strings (consisting of the symbols 0, 1, 2) that either
• start with a 0, followed by any number of 0’s, 1’s, or 2’s, e.g.,
010201 0210021 0000 0111 0222 0
• or starts with any number of 0’s or 2’s, followed by a 1, preceded by, e.g.,
202201 00000001 2222221 1
• does not match
11 21111 101
(ends in 1, but other 1’s appear earlier)
12 222222 202020 (does not start with 0 or end with 1)
0.1: Automata, Computability, and Complexity
6
Digit-counting regular expressions
Definition: given string x and symbol b, let #b(x) = number of b’s in x
• #0(0001200121) = 5
• #1(0001200121) = 3
Task A: write a regex matching ternary strings x where #0(x) + #1(x) = 3
2* (0|1) 2* (0|1) 2* (0|1) 2*
Task B: write a regex matching ternary strings x where #0(x) – #1(x) = 3
impossible
0.1: Automata, Computability, and Complexity
7
What this course is about
• What are the fundamental abilities and limitations of computers?
• In ECS 30/40/60, you programmed computers, but did not use formal
mathematical models of computers.
• In ECS 20, you studied formal mathematical models, but not models
of computers.
• In ECS 120, you will use the tools of ECS 20 to study formal
mathematical models of computers.
0.1: Automata, Computability, and Complexity
8
What lies ahead
• Computability theory (unit 2)
What problems can computers solve?
• Complexity theory (unit 3)
What problems can computers solve
efficiently? [ polynomial time: for n-bit inputs,
• Order by power of algorithms: 2,3,1
• Historical order: 2,1,3
• Traditional order to teach: 1,2,3
• Order we will use: 1,2,3,2
# steps = O(n), or O(n2), or O(n3), or O(n4), … ]
• Automata theory (unit 1)
What problems can computers solve
with optimal efficiency? [ constant memory
and “real time”: for n-bit inputs, # steps = n ]
0.1: Automata, Computability, and Complexity
9
Some course logistics (not all!)
Syllabus on Canvas has full details
Course Logistics
10
Websites, questions, discussion
• Three different websites: Canvas, Piazza, Gradescope
Canvas is run by UC-Davis, but you need to sign up for Piazza and Gradescope.
• Announcements on Piazza, homework/reading quizzes posted to Canvas.
• Please post homework/logistical questions on Piazza publicly.
• Please use email only for questions of a personal nature.
• Please do not ask logistical questions in lecture (e.g., what problems are on the
exam); if you think of these in lecture, write them down and ask on Piazza.
• Discussion is interactive: show up with questions
If you find the thought repeating in your head, “I wish the TA would do X,” then
raise your hand and ask the TA to do X!
Course Logistics
11
Grading
• Reading quizzes (Canvas): 5% • Two types of homework questions:
• auto-graded
• Midterm exam: 35%
• written
• Both turned in on Gradescope (PDF for written)
• Final exam: 35%
• Auto-graded is assigned a score by Gradescope
• Homework/quizzes: 25%
These are graded a bit strangely,
so pay attention!
• Written is checked, but not assigned a direct score
• Score on written portion comes from in-lecture quiz
• If homework not submitted, quiz is scored 0.
• If homework is late, quiz score receives late penalty.
All homework, written and auto-graded, must be your own work.
You are free to discuss it with anyone, and to consult whatever sources you want,
as long as these discussions/sources are fully cited in your PDF submission.
Course Logistics
12
The internet: how it helps us, and how it doesn’t
Not the point of college: solve problems
The point of college: learn how to solve problems
Best way to learn how to solve a problem:
solve it
2nd best way to learn how to solve a problem:
try hard but fail to solve it, then see someone
else’s solution
not a way to learn how to solve a problem:
see someone else’s solution before trying
Course Logistics
13
Late penalty
• Turn-in allowed up to 24 hours
after homework deadline.
• Original score is multiplied by
p(h)%, where h = # hours late.
• examples:
•
•
•
•
1.2 hours late: 1% deducted
12 hours late: 10% deducted
18 hours late: 55% deducted
23 hours late: 92.5% deducted
Course Logistics
14
More details in syllabus on Canvas
https://canvas.ucdavis.edu/courses/113660/assignments/syllabus
Course Logistics
15
Mathematical notation and
terminology
Review of discrete math (ECS 20)
0.2: Mathematical Notions and Terminology
16
Sets
• Set: group of objects called elements or members of set
•
•
•
•
no duplicates
order doesn’t matter
{7, 21} = {21, 7} = {7, 7, 21}
could be finite or infinite
• programming language implementations:
std::set
java.util.Set
set
in C++
in Java
in Python
Note: mathematical sets are “immutable”
• not dynamic data structures: they don’t
grow or shrink
• to “add to a set”, define new sets: define
S0 = {stuff} and let Sn = Sn-1 ∪ {more stuff}
0.2: Mathematical Notions and Terminology
17
Notation for sets
• If the set is finite, we can list the elements explicitly:
• S = {7, 21, 57} contains three elements: 7, 21, and 57
• |S| = 3 is the cardinality of S: the number of elements it contains
• 7 ∈ S denotes that 7 is an element of S.
• 8 ∉ S denotes that 8 is not an element of S.
• A ⊆ B denotes that A is a subset of B:
• for every x ∈ A, we also have x ∈ B
• {7, 57} ⊆ {7, 21, 57} and {7, 57} ⊆ {7, 57}
• A ⊊ B denotes that A is a proper subset of B: A ⊆ B but A ≠ B
• {7, 57} ⊊ {7, 21, 57}
• ∅ = {} is the empty set: |∅| = 0, and x ∉ ∅ for all x
• {7} is a singleton (one-element set), and {7, 21} is an unordered pair
0.2: Mathematical Notions and Terminology
18
Common sets with standard names
• ℕ = nonnegative integers (“natural numbers”) = { 0, 1, 2, 3, … }
some authors use ℕ = { 1, 2, 3, … }
• ℤ = integers = { …, –2, –1, 0, 1, 2, … }
• ℝ = real numbers
• ℚ = rational numbers = { n/d | n ∈ ℤ, d ∈ ℕ, d ≠ 0 }
• last is example of set-builder notation: many programming languages
have related feature: list/set comprehension, e.g., Python:
set(n/d for n in [-1,0,1] for d in [0,1,2,3] if d != 0)
creates the set {–1.0, –0.5, –0.333, 0.0, 0.333, 0.5, 1.0}
0.2: Mathematical Notions and Terminology
19
union of A and B:
Set operations
Venn diagram of sets A = {7, 21} and B = {21, 57}:
(within “universe” ℕ)
A ∪ B = { x | x ∈ A or x ∈ B }
{7, 21} ∪ {21, 57} = {7, 21, 57}
ℕ
intersection of A and B:
A ∩ B = { x | x ∈ A and x ∈ B }
{7, 21} ∩ {21, 57} = {21}
difference between A and B:
A \ B = { x | x ∈ A and x ∉ B }
{7, 21} \ {21, 57} = {7}
A
7
B
21
57
complement of A:
A = { x | x ∉ A } = “universe” \ A
{7, 21} = ℕ \ A
0.2: Mathematical Notions and Terminology
20
Sets can contain other sets
• P(A) = power set of A = set of all subsets of A
sometimes written 2A
• P({a,b,c}) = { ∅, {a}, {b}, {c}, {a,b}, {a,c}, {b,c}, {a,b,c} }
• If |A| = n, then |P(A)| = ??
2n
0 0 0
0 0 1
0 1 0
0 1 1
1 0 0
1 0 1
1 1 0
1 1 1
{a,b,c} {a,b,c} {a,b,c} {a,b,c} {a,b,c} {a,b,c} {a,b,c} {a,b,c}
{
} {
c} { b } { b,c} {a
} {a, c} {a,b } {a,b,c}
0.2: Mathematical Notions and Terminology
21
Sequences and tuples
• Sequence: list of objects
•
•
•
•
duplicates allowed
order matters
(7, 21, 57) ≠ (21, 7, 57) ≠ (21, 7, 57 , 7)
could be finite or infinite
• finite sequence is called a tuple
• tuple with k elements is a k-tuple
• programming language
implementations:
•
•
•
•
std::vector
in C++
java.util.List
in Java
list/tuple
in Python
arrays in any language
• (ordered) pair = 2-tuple
• triple = 3-tuple
0.2: Mathematical Notions and Terminology
22
Cross product (a.k.a., Cartesian product)
• A × B = { (x,y) | x ∈ A and y ∈ B }
• set of all pairs where first element is from A and second element is from B
• example: {soup, salad} × {steak, pasta, ribs} =
{ (soup, steak), (soup, pasta), (soup, ribs), (salad, steak), (salad, pasta), (salad, ribs) }
• If |A| = n and |B| = k, then |A × B| = ??
n∙k
soup
salad
• Python equivalents:
steak
set((x,y) for x in A for y in B)
pasta
• A2 = A × A
ribs
example: {7,21}2 = { (7,7), (7,21), (21,7), (7,21) }
(soup, steak)
(soup, pasta)
(soup, ribs)
• A × B × C is a set of triples, A × B × C × D is a set of 4-tuples, etc.
• Note: A × B is not the same as A ∩ B = { x | x ∈ A and x ∈ B }
{7, 21} ∩ {21, 57} = {21}
{7, 21} × {21, 57} = { (7,21), (7,57), (21,21), (21,57) }
0.2: Mathematical Notions and Terminology
(salad, steak)
(salad, pasta)
(salad, ribs)
23
Functions and relations
• function: an input/output relationship (mapping)
• f(a) = b expressed that given input a, f’s output is b
• a function in a programming language is different: it is an algorithm telling how
to compute the output, given the input
• a mathematical function is just the abstract relationship itself; there may be no
algorithm for computing it!
• input comes from domain D (a set) and output is in range R (also a set)
• f: D  R
• similar to programming language type signature for function:
f: ℕ  {true,false} is expressed in C++ as
bool f(int a);
0.2: Mathematical Notions and Terminology
24
Examples of function definitions
abs: take absolute value of integer
type signature? abs: ℤ  ℤ or abs: ℤ  ℕ
definition: for all x ∈ ℤ,
abs(x) =
C++ version:
int abs(int x) {
if (x < 0)
return –x;
else
return x;
}
–x if x < 0;
add: sum of two integers
type signature? add: ℤ × ℤ  ℤ
definition: for all x,y ∈ ℤ,
add(x,y) = x+y
x otherwise
C++ version:
int add(int x, int y) {
return x+y;
}
0.2: Mathematical Notions and Terminology
25
1-1 and onto functions
• f: D  R is 1-1 (a.k.a., injective) if,
for all x1,x2 ∈ D, if f(x1) = f(x2), then
x 1 = x2
• f: D  R is onto (a.k.a., surjective)
if, for all y ∈ R, there is x ∈ D such
that f(x) = y
• f is bijective if it is 1-1 and onto
no two arrows “collide”:
1-1,
not onto:
D
f
R
all elements of R are “covered”:
onto,
not 1-1:
1-1 and
onto:
0.2: Mathematical Notions and Terminology
26
Predicate: function with Boolean range
• even: ℤ  {true,false} defined by even(n) = true if n is even and
false otherwise
• Any predicate is identified by the subset of inputs that make it true,
e.g., even is identified with the set of even integers.
• Relation: predicate with domain A2 = A × A; equivalently, subset of A2
e.g., “less than” relation on ℤ is the set of pairs (a,b) ∈ ℤ2 such that a < b
0.2: Mathematical Notions and Terminology
27
Graphs
directed graph
undirected graph
G = (V,E)
a
c
b
d
nodes/vertices
V = {a, b, c, d}
w
x
edges
E = { {a,b}, {a,d},
{b,d}, {c,d} }
z
y
degree of node =
# of edges connected to it
degree(d) = 3
edges
E = { (x,w), (w,y),
(y,x), (y,z) }
outdegree of node =
# of edges leaving it
indegree of node =
# of edges entering it
0.2: Mathematical Notions and Terminology
outdegree(y) = 2
indegree(y) = 1
28
Paths in graphs
• path = sequence of nodes connected by edges
(c, d, b, d, a)
• simple path = path with no repeated nodes
(c, d, b, a)
• graph is connected if all pairs of nodes have a path
between them
connected
disconnected
a
b
1
c
d
4
2
3
5
• directed graph is strongly connected if all pairs of nodes
have a directed path between them in each direction connected, but not strongly connected
• cycle = path starting and ending with same node
1
2
(c, d, a, d, c)
• simple cycle = cycle repeating only first/last node
4
3
(a, d, b, a)
0.2: Mathematical Notions and Terminology
29
Trees
• tree = connected graph with no
simple cycles
• rooted tree = tree with one special
node designated as the root
• leaf = node of degree 1 that is not
the root
b
c
d
r
a
b
f
0.2: Mathematical Notions and Terminology
a
c
e
d
g
h
30
Boolean logic
• two Boolean values: {0,1}, a.k.a., {false,true}, a.k.a., {no,yes}
• Boolean operations given by truth table
negation/NOT
¬0 = 1
¬1 = 0
conjunction/AND
0∧0=0
0∧1=0
1∧0=0
1∧1=1
disjunction/OR
0∨0=0
0∨1=1
1∨0=1
1∨1=1
0.2: Mathematical Notions and Terminology
31
Other Boolean operations
exclusive or/XOR
0⊕0=0
0⊕1=1
1⊕0=1
1⊕1=0
implication
0⇒0=1
0⇒1=1
1⇒0=0
1⇒1=1
p ⇒ q is pronounced
“if p then q” or “p implies q”;
equivalent to (¬p ∨ q)
equality/iff
0⇔0=1
0⇔1=0
1⇔0=0
1⇔1=1
p ⇔ q is pronounced
“p if and only if q”;
equivalent to (p ⇒ q) ∧ (q ⇒ p)
Q: Why is p ⇒ q automatically true if p is false?
A: The same reason the bartender doesn’t check your ID when you order water.
Bartender must ensure “if you order alcohol, then you are 21”
0.2: Mathematical Notions and Terminology
32
Strings and languages
• alphabet = any nonempty finite set
• symbol = element of alphabet
Σ1 = {0,1}
Σ2 = {a,b,c,d,e,f,g,h,I,j,k,l,m,n,o,p,q,r,s,t,u,v,w,x,y,z}
Γ = {0,1,x,y,z}
• string (over an alphabet Σ) = finite sequence of symbols from Σ
written without commas or parentheses: x = 01001 is a string over Σ
• length of string x = |x| = # of symbols in x
• if |x| = n, we write x = x1x2…xn or x = x[1]x[2]…x[n]
• empty string = ε = string of length 0; "" in most programming languages
0.2: Mathematical Notions and Terminology
33
The emptiness inside
thing
symbol Python C++
empty set
∅
empty string
ε
set containing {ε}
empty string
{}
""
{""}
cardinality
length
doesn’t have
a length
std::string eps("");
doesn’t have 0
a cardinality
std::set set_with_eps;
1
doesn’t have
set_with_eps.insert(eps);
a length
std::set empty_set;
0.2: Mathematical Notions and Terminology
0
34
String operations
• xR = reverse of x = xnxn-1…x1
010001R = 100010
• z is a substring of x if z appears consecutively within x, i.e., if z = xixi+1…xk
for 1 ≤ i ≤ k ≤ n = |x|
cad is a substring of abracadabra
another common notation is x[i..k] = xixi+1…xk
• if x and y are strings with |x| = m and |y| = n, their concatenation is the
string xy = x1…xmy1…yn
k
• for k ∈ ℕ, xk denotes the string xx…x
0.2: Mathematical Notions and Terminology
35
More string notation
• The set of all strings over Σ is denoted Σ*.
{0,1}* = { ε, 0, 1, 00, 01, 10, 11, 000, 001, 010, 011, 100, 101, 110, 111, 0000, …}
• Given any set of strings A, the length-lexicographical order of A orders
them first by length, then within length by “alphabetical” ordering.
i.e., alphabet has some “natural” order (e.g., 0 < 1, and a < b < c < … < z),
for strings x,y: x < y if for some i, x1x2…xi = y1y2…yi, but xi+1 < yi+1
textbook calls this “shortlex order” or “string order”; it is common to call it simply
“lexicographical”, but some authors use this to mean simply “alphabetical”
• x is a prefix of y, written x ⊑ y, if there exists a string z such that xz = y
abb ⊑ abbcd
(x=abb, y=abbcd, z=cd)
• x is a proper prefix of y if x ⊑ y and x ≠ y.
• A language is a set of strings, a.k.a., decision problem.
0.2: Mathematical Notions and Terminology
36
Even more string notation
• If A,B are languages, AB = { xy | x ∈ A, y ∈ B }
• example: A = {0, 22} and B = {000, 11, 2}
• AB = {0000, 011, 02, 22000, 2211, 222}
• If A is a language and k ∈ ℕ, Ak = AA…A.
•
•
•
•
{0,22}2 = {00, 022, 220, 2222}
k
{0,22} × {0,22} = { (0,0), (0,22), (22,0), (22,22) }
{0,1}3 = {000, 001, 010, 011, 100, 101, 110, 111}
{0,1}0 = {ε}… actually A0 = {ε} for any language A
Ambiguity! When A is a language,
A2 usually means AA, not A × A.
|A2| = |A|2?
{1,11}2 = {11, 111, 1111}
{1,11} × {1,11} =
{(1,1), (1,11), (11,1), (11,11)}
• If A is a language, A* = A0 ∪ A1 ∪ A2 ∪ …
• If Σ is an alphabet (i.e., a finite language of one-symbol strings), then
Σ* = all finite strings over Σ
Σk = strings of length k
0.2: Mathematical Notions and Terminology
|Σk| = |Σ|
?? k
37
0.3 Definitions, Theorems, and Proofs
0.4 Types of Proof
Review on your own.
0.3: Definitions, Theorems, and Proofs, and 0.4 Types of Proof
38