CPSC 121: Models of Computation
Module 11: Sets, Functions and DFAs
By the start of class, you should be able to:
Define the set operations union, intersection,
complement and difference, and the logical
operations subset and set equality in terms of
predicate logic and set membership.
Module 11: Sets, Functions and DFAs.
Translate between sets represented explicitly
(possibly using ellipses, e.g., { 4, 6, 8, … }) and using
"set builder" notation (e.g., { x in Z+ | x2 > 10 and x is
even }).
Execute set operations on sets expressed explicitly,
using set builder notation, or a combination of these.
CPSC 121 – 2016W T1
1
Module 11: Sets, Functions and DFAs
2
Module 11: Sets, Functions and DFAs.
By the start of class, you should be able to:
?
?
CPSC 121: the BIG questions:
Interpret the empty set symbol, including the fact that
the empty set has no members and that it is a subset
of any set.
?
?
?
1. How does the computer (e.g. Dr. Racket) decide if
the characters of your program represent a name,
a number, or something else? How does it figure
out if you have mismatched " " or ( )?
Define the terms domain, co-domain, range, image,
and pre-image
Use appropriate function syntax to relate these terms
(e.g., f : A → B indicates that f is a function mapping
domain A to co-domain B).
Determine whether f : A → B is a function given a
definition for f as an equation or arrow diagram.
CPSC 121 – 2016W T1
CPSC 121 – 2016W T1
3
?
?
More generally:
?
?
What can we compute?
? not solve?
Are there problems we can
?
?
?
?
CPSC 121 – 2016W T1
?
?
4
Module 11: Sets, Functions and DFAs
Final Exam information
By the end of this module, you should be able
to:
Announcements:
Final exam: Monday December 19th, 8:30 in OSBO.
Define the terms injective (one-to-one), surjective
(onto), bijective (one-to-one correspondence), and
inverse.
Questions & Answers Sessions (locations TBD):
Alice: Wednesday December 14th, 14:00 to 16:00
Patrice: Thursday December 15th, 14:00 to 16:00
Determine whether a given function is injective,
surjective, and/or bijective.
There will be additional office hours; they will be
posted on the course web site.
Relate DFAs to sets and functions.
Better understand the limitations of DFAs, that is,
what they can and can not do.
CPSC 121 – 2016W T1
5
Final Exam Information
CPSC 121 – 2016W T1
6
Teaching/TA Evaluations
Teaching assistants:
Final exam details:
Adam Jozefiak; Angad Kalra; Arshdeep Ganda; Cindy Tu
Christine Liu; Elad Michaeli; Hannah Lin; Harsh Patel;
Jastine Irwan; Zara Lim; Jessica Que; Jingwei Zhang
Jonathan Budiardjo; Leo Sporn; Linh Phan; Luke Schuster
Minnie Liu; Pinia Chandra; Sherry Yuan; Willis Peng; Xuejie Tang
Yi Ren; Yves Chan
2.5 hours
Covers everything discussed in the course
includes labs, although you don't need to memorize
all of the solutions.
Slightly more emphasis on later topics.
The following will be provided if necessary
Logical equivalences, rules of inference, powers of 2,
regular expression syntax
You can bring one 8.5 x 11in 2-sided sheet of paper.
CPSC 121 – 2016W T1
7
CPSC 121 – 2016W T1
8
Teaching/TA Evaluations
Module 11: Sets, Functions and DFAs
Labs:
Module Summary
Sets.
Cardinality, Power sets and Cartesian Products.
Injections, surjections, bijections and inverses.
DFAs and regular expressions.
Computations that we are unable to perform.
CPSC 121 – 2016W T1
9
CPSC 121 – 2016W T1
Module 11.1: Sets
10
Module 11.1: Sets
Confusion: versus
How does Set Theory fit in with what we have
been doing so far in the course?
Every set operation is defined in terms of predicate
logic and the basic predicate ∈.
So we prove theorems about sets by
figuring out which predicate logic statement
corresponds to the terms and operations used in the
theorem.
applying already known proof techniques.
The predicate takes in an element x and a set S,
and returns true if the element x belongs to S.
The predicate takes in two sets S and T, and
returns true if every element of S is also an
element of T. That is, if xD, xS xT
No new proof techniques are needed!
CPSC 121 – 2016W T1
11
CPSC 121 – 2016W T1
12
Module 11.1: Sets
Module 11.1: Sets
Historical Notes:
Russel's paradox (part 1).
Mathematicians formalized set theory to create a
foundation for all of mathematics. Essentially all
mathematical constructs can be defined in terms of
sets.
Let S be the set of all sets that contain themselves.
Does S contain itself?
a) Yes, definitely.
b) No, certainly not.
c) Maybe (either way is fine).
d) None of the above.
Hence sets are a powerful means of formalizing
new ideas.
But we have to be careful!
▷
CPSC 121 – 2016W T1
13
CPSC 121 – 2016W T1
Module 11.1: Sets
Module 11.1: Sets
Russel's paradox (part 2).
Despite this, sets (and functions) are incredibly
useful.
Let S be the set of all sets that do not contain
themselves (that is, S S).
E.g. We can definite valid DFAs formally: a DFA
is a 5-tuple (Σ, S, s0, F, δ) where
Does S contain itself?
a) Yes, definitely.
b) No, certainly not.
c) Maybe (either way is fine).
d) None of the above.
Σ is a finite set of characters (input alphabet).
S is a finite set of states.
s0 ∈ S is the initial state.
F ⊆ S is the set of accepting states.
▷
CPSC 121 – 2016W T1
14
16
δ: S x Σ → S is the transition function.
CPSC 121 – 2016W T1
18
Module 11: Sets, Functions and DFAs
Module 11.2: Cardinality, Power sets, etc
Cardinality: the number of elements of a set S,
denoted by |S|.
Module Summary
Sets.
Cardinality, Power sets and Cartesian Products.
What is the cardinality of the following set:
{ 1, 2, 3, { a, b, c }, snow, rain } ?
Injections, surjections, bijections and inverses.
DFAs and regular expressions.
a) 3
Computations that we are unable to perform.
b) 6
c) 8
d) Some other integer
e) The cardinality of the set is undefined.
CPSC 121 – 2016W T1
19
Module 11.2: Cardinality, Power sets, etc
This gets trickier with infinite sets.
OPTIONAL
Consider A = Z+, B = Z, C = Q (the rationals).
Which of these sets have the smaller and larger
cardinality?
a) Smaller: A, larger: C
CPSC 121 – 2016W T1
▷
20
Module 11.2: Cardinality, Power sets, etc
Suppose we have a set S with n students, and
we want to form a group to go watch a movie.
What are the different groups that can we form?
OR: we have a circuit with n inputs (so S is the
set of inputs).
What combinations of inputs might be true?
b) Smaller: A and B, larger: C
In both cases, we are looking for all the subsets
of S. That is, we want to find { T | T  S }
c) All three sets have the same cardinality
d) None of the above.
▷
CPSC 121 – 2016W T1
22
CPSC 121 – 2016W T1
24
Module 11.2: Cardinality, Power sets, etc
Module 11.2: Cardinality, Power sets, etc
The power set of a set S, denoted P (S), is the
set whose elements are all subsets of S.
The Cartesian product of two sets S and T,
denoted S x T, is the set { (s, t) | s ∈ S  t ∈ T }.
Given the definitions A = { a, b, f }, B = { b, c },
what are:
Each element of S x T is called an ordered tuple, or
simply a tuple (or, in this case, a 2-tuple or a pair).
a)
b)
c)
d)
e)
P (A)
P (A - B)
|P (A  B)|
|P (A B)|
Ac
Two ordered tuples are equal if the elements in
corresponding positions are the same.
Examples:
(3, 7) = (4 – 1, 14 / 2)
(3, 7) ≠ (7, 3)
CPSC 121 – 2016W T1
25
Module 11.2: Cardinality, Power sets, etc
We can visualize Z+ x Z+ as follows:
CPSC 121 – 2016W T1
26
Module 11: Sets, Functions and DFAs
Module Summary
Sets.
5
4
Cardinality, Power sets and Cartesian Products.
3
Injections, surjections, bijections and inverses.
2
DFAs and regular expressions.
1
Computations that we are unable to perform.
1
2
3
4
5
6
7
8
What is { a, b } x { 8, 5, 1 } ?
CPSC 121 – 2016W T1
27
CPSC 121 – 2016W T1
28
Module 11.3: Special types of functions
You are already familiar with functions:
Module 11.3: Special types of functions
You are already familiar with functions:
Every combinational circuit implements a function.
Every combinational circuit implements a function.
booleans
e.g. m : B x B x B → B
m(a, b, c) = a if c is false, b if c is true
booleans
e.g. m : B x B x B → B
m(a, b, c) = a if c is false, b if c is true
Which function is this?
Which function is this?
Sequential circuits and Java methods are not
always functions. Why not?
CPSC 121 – 2016W T1
Sequential circuits and Java methods are not
always functions. Why not?
29
Module 11.3: Special types of functions
Some special types of functions:
CPSC 121 – 2016W T1
30
Module 11.3: Special types of functions
Some special types of functions (continued)
A function f : A → B is injective (one-to-one) if
∀x ∈ A, ∀y ∈ A, x ≠ y → f(x) ≠ f(y).
A function f : A → B is surjective (onto) if
∀y ∈ B, ∃x ∈ A, f(x) = y.
In the arrow diagram: at most one arrow points to each
element of B.
In the arrow diagram: at least one arrow points to each
element of B.
Injective
CPSC 110/201
Not injective: f(George) = f(Patrice)
Not Surjective
CPSC 110/201
Surjective
George
CPSC 121/202
George
CPSC 110
George
CPSC 121/202
George
Patrice
CPSC 121/203
Patrice
CPSC 121
Patrice
CPSC 121/203
Patrice
CPSC 121
Gail
CPSC 210
Gail
CPSC 210
Kimberly
CPSC 310
Kimberly
CPSC 310
Neil
CPSC 319
Neil
CPSC 319
Neil
CPSC 210/BCS
Kimberly
CPSC 310
Gail
CPSC 319
CPSC 121 – 2016W T1
31
Neil
CPSC 210/BCS
Kimberly
CPSC 310
Gail
CPSC 319
CPSC 121 – 2016W T1
32
Module 11.3: Special types of functions
Some special types of functions (continued)
Some special types of functions (continued)
A function f : A → B is bijective if it is both injective
and surjective
Bijective
In the arrow diagram: exactly one arrow points to each
element of B.
Not Bijective
CPSC 110/201
Not Bijective either
George
CPSC 121/202
George
Patrice
CPSC 121/203
Patrice
CPSC 121
Gail
CPSC 210
Kimberly
CPSC 310
Neil
CPSC 319
Neil
CPSC 210/BCS
Kimberly
CPSC 310
Gail
CPSC 319
CPSC 121 – 2016W T1
Module 11.3: Special types of functions
33
Module 11.3: Special types of functions
The inverse of a function f: A → B, denoted f -1, is
the set { (y, x) ∈ B x A | y = f(x) }.
In other words:
Patrice
CPSC 121
Gail
CPSC 210
Kimberly
CPSC 310
Neil
CPSC 319
CPSC 121 – 2016W T1
34
Module 11.3: Special types of functions
Is f -1 a function?
a) Yes, always.
b) No, never.
c) Yes, but only if f is injective.
If we think of a function as a list of pairs.
d) Yes, but only if f is surjective.
E.g. f(x) = x2: { (1, 1), (2, 4), (3, 9), (4, 16), ... }
Then f -1 is obtained by swapping the elements of
each pair:
e) None of the above.
f-1 = { (1, 1), (4, 2), (9, 3), (16, 4), ... }
▷
CPSC 121 – 2016W T1
35
CPSC 121 – 2016W T1
36
Module 11.4: DFAs and regular
expressions
Module 11: Sets, Functions and DFAs
Module Summary
DFAs and Regular expressions.
Sets.
In lab 8, you wrote regular expressions matching
the patterns we gave you.
Cardinality, Power sets and Cartesian Products.
Regular expressions are very useful when you need
to read and validate input.
Injections, surjections, bijections and inverses.
DFAs and regular expressions.
Many modern programming languages provide
functions that allow you to manipulate regular
expressions:
Computations that we are unable to perform.
Dr. Racket, Java, Python, Perl, etc.
CPSC 121 – 2016W T1
38
Module 11.4: DFAs and regular
expressions
CPSC 121 – 2016W T1
39
Module 11.4: DFAs and regular
expressions
How does a program determine if an input string
matches a regular expression?
Theorem: every set of strings matched by a regular
expression can be recognized by a DFA.
Hence the functions build a DFA corresponding to
the regular expression.
This result goes both ways:
if a set of strings is accepted by a DFA, then there is
a regular expression for this set of strings.
In this sense, DFAs are exactly as “powerful” as
regular expressions (no more, no less).
How do we build the DFA given a regular
expression?
And then this DFA is given the input string one
character at a time.
First we build a NFA (non-deterministic finite-state
automaton) for the regular expression.
Then we convert the NFA into a DFA.
CPSC 121 – 2016W T1
40
CPSC 121 – 2016W T1
41
Module 11.4: DFAs and regular
expressions
Module 11.4: DFAs and regular
expressions
What is a NFA?
Example:
It is like a DFA but
There can be multiple arrows with the same label
leaving from a state
There can be arrows labelled ε that we can take
without reading the next input character.
So we can sometimes choose which state to go to.
ε
A NFA accepts a string if at least one sequence of
choices leads to an accepting state.
CPSC 121 – 2016W T1
ε
42
Module 11.4: DFAs and regular
expressions
43
Module 11.4: DFAs and regular
expressions
Theorem: we can transform every regular
expression into a NFA with
What regular expression corresponds to the
strings that this NFA accepts?
a) ε | ab
CPSC 121 – 2016W T1
Exactly one accepting state
empty string
No arcs pointing to the initial state
b) ε | abaa
No arcs leaving the accepting state
c) ε | ab | abaa
Fact: every regular expression can be rewritten
to use only the following:
d) ε | ab | (aba)+a
e) None of the above.
The empty string ε.
Individual characters
▷
CPSC 121 – 2016W T1
44
The operators |, *, and string concatenation.
CPSC 121 – 2016W T1
46
Module 11.4: DFAs and regular
expressions
Module 11.4: DFAs and regular
expressions
Exercise: rewrite the following expressions to
use only the options listed on the previous slide:
a?
Proof: by induction on the structure of the
regular expression.
Base cases:
a+
The expression that matches the empty string:
a{3,5}
ε
\d
The expression that matches no string:
CPSC 121 – 2016W T1
47
Module 11.4: DFAs and regular
expressions
CPSC 121 – 2016W T1
48
Module 11.4: DFAs and regular
expressions
Base cases (continued)
Induction step (continued and finished):
The expression that matches a single character a:
The expression E1E2 where E1, E2 are regular
expressions:
ε
Induction step:
The expression E1|E2 where E1, E2 are regular
expressions:
ε
ε
CPSC 121 – 2016W T1
The expression E* where E is a regular expression:
ε
ε
ε
ε
ε
49
CPSC 121 – 2016W T1
ε
50
Module 11.4: DFAs and regular
expressions
Module 11.4: DFAs and regular
expressions
Example: (a|b)*c
How do we transform a NFA into a DFA?
A DFA that reads a string with n characters ends up
in exactly one state.
a
A NFA that reads a string with n characters may end
up in many different stages.
b
Can we figure out which states?
a|b
(a | b)*
CPSC 121 – 2016W T1
51
Module 11.4: DFAs and regular
expressions
52
Module 11.4: DFAs and regular
expressions
Which state(s) will the following NFA end up in
after reading the string ab?
So we build the DFA as follows:
Each state of the DFA is a subset of the states of
the NFA.
a) S1 only
If the DFA is in state { Si1, Si2, ..., Sik }, and it sees a
character x, then the new state is the state that
contains every NFA state that we can get to from
one of Si1, Si2, ..., Sik upon reading x.
b) S6 only
c) S3 or S7
d) S4 or S6
e) None of the above
CPSC 121 – 2016W T1
ε
A state of the DFA is accepting if it contains the
accepting state of the NFA.
ε
▷
CPSC 121 – 2016W T1
53
CPSC 121 – 2016W T1
55
Module 11.5: Computations we are unable
to perform
Module 11: Sets, Functions and DFAs
We have discussed several models of computation
in the course:
Module Summary
Sets.
Cardinality, Power sets and Cartesian Products.
Combinational circuits
Injections, surjections, bijections and inverses.
Sequential circuits (the working computer).
DFAs and regular expressions.
DFAs
One thing computer scientists (we) like to know
about their (our) computational models is:
Computations that we are unable to perform.
What can they do?
What can they not do?
CPSC 121 – 2016W T1
56
Module 11.5: Computations we are unable
to perform
CPSC 121 – 2016W T1
57
Module 11.5: Computations we are unable
to perform
Definition: we denote by anbn a string that
consists of
Example: DFAs
Intuition:
n copies of the letter a, followed by
DFAs have no memory apart from the current state.
So a DFA with n states can only “count” up to n
before it gets confused.
n copies of the letter b.
How do we formalize this?
We need to define a set L of strings (language).
And show that no DFA can accept exactly the strings
in L.
CPSC 121 – 2016W T1
58
Theorem: no DFA can recognize the language
{ anbn | n ∈ Z+ }.
CPSC 121 – 2016W T1
59
Module 11.5: Computations we are unable
to perform
Proof: we use a proof by contradiction.
Suppose there is such a DFA. This DFA has k
states for some positive integer k.
Now look at what happens when the DFA is looking
at the input akbk.
While it's reading the input string, it goes through a
number of states:
CPSC 121 – 2016W T1
Module 11.5: Computations we are unable
to perform
q0 (initial state)
q1 (after reading the string a)
q2 (after reading the string aa)
...
qk (after reading the string ak).
The DFA only has k different states, so two of q0, q1,
..., qk must be the same state. Let us call these two
qi and qj, with i < j.
60
Module 11.5: Computations we are unable
to perform
Observation 1: If the DFA is in state qi and it sees
j – i copies of a then it ends up in state
CPSC 121 – 2016W T1
Module 11.5: Computations we are unable
to perform
Observation 2: If the DFA is in state qi and it sees
k – j copies of a then it ends up in state .
a) q0
a) q0
b) qi
b) qi
c) qk
c) qk
d) Some other state.
d) Some other state.
▷
CPSC 121 – 2016W T1
61
62
▷
CPSC 121 – 2016W T1
64
Module 11.5: Computations we are unable
to perform
What happens if we give the DFA the string ak+(j-i)bk?
While reading the first i copies of a, it goes through
states q0 .. qi.
Module 11.5: Computations we are unable
to perform
But akbk should be accepted, and ak+(j-i)bk should be
rejected!
Then it reads the next j – i copies of a, and ends up
in state qi again.
So the DFA will make a mistake on one of these two
strings, which means it's not recognizing { anbn | n ∈
Z+ } correctly.
From qi it reads the next k – j copies of a, and ends
up in state qk.
This contradictions our initial assumption, and
hence no DFA can recognize this language. QED
Then it reads the k copies of b, and terminates in the
same state it terminated when it was reading akbk.
CPSC 121 – 2016W T1
66
Module 11.5: Computations we are unable
to perform
But what about sequential circuits/Java/Racket?
They are more powerful than DFAs.
For instance, you can easily write a Racket function
or a Java method that will recognize { anbn | n ∈ Z+ }.
CPSC 121 – 2016W T1
67
Module 11.5: Computations we are unable
to perform
Theorem: It is not possible to write a program
that solves the halting problem.
Aside: remember Russel's paradox? We will use a
similar trick to prove this theorem.
Can they solve every problem?
Proof 1: proof by video:
No: there are problems that can not be solved.
https://www.youtube.com/watch?v=92WHN-pAFCs
Halting Problem: given a program P and an
input I, will P halt if we run it on input I?
CPSC 121 – 2016W T1
68
CPSC 121 – 2016W T1
69
Module 11.5: Computations we are unable
to perform
Module 11.5: Computations we are unable
to perform
Racket version:
Proof 2: we use a proof by contradiction.
(define (paradox input)
Suppose this program exists.
(if (will-halt input input)
Let us call it will-halt (Racket) or willHalt (Java).
(paradox input) ; go into an infinite recursion
We use this function or method to write the following
function or method:
true))
Java version:
public static void main(String[ ] args) {
if (willHalt(args[0], args[0]))
while(true) ;
else
return;
}
CPSC 121 – 2016W T1
70
Module 11.5: Computations we are unable
to perform
CPSC 121 – 2016W T1
71
Module 11: Sets, Functions and DFAs
What happens when we call this program with itself
as input?
If it halts, then will-halt/willHalt returns true, and so it
won't halt.
But if it doesn't halt, then will-halt/willHalt returns
false, and so it will halt.
 The End 
So whether the program halts or not, we end up with
a contradiction.
Therefore this program does not exist. QED
CPSC 121 – 2016W T1
72
CPSC 121 – 2016W T1
73