Computer Sciences Department
1
Reference
Book: INTRODUCTION TO THE THEORY OF
COMPUTATION, SECOND EDITION, by:
MICHAEL SIPSER
Computer Sciences Department
3
ADVANCED TOPICS IN
COMPUTABILITY
THEORY
The Recursion Theorem
Pages 217– 226
4
Objectives
 Explanation: The possibility of making machines that
can construct replicas of themselves
 SELF-REFERENCE (algorithm)
Computer Sciences Department
5
Recursion
 It concerns the possibility of making machines that can
construct replicas of themselves.
1. Living things are machines (operate in a mechanistic way).
2. Living things can self-reproduce (essential characteristic).
3. Machines cannot self-reproduce.
6
a machine A that constructs
a machine B
 A must be more complex than B. But a machine
cannot be more complex than itself.
How can we resolve this paradox?
 Making machines that reproduce themselves is
possible. (The recursion theorem demonstrates how.)
7
SELF-REFERENCE
 Let's begin by making a Turing machine that ignores its
input and prints out a copy of its own description.
 We call this machine SELF.
 To help describe SELF, we need the following lemma.
 LEMMA 6.1
8
SELF-REFERENCE (algorithm)
9
Machines A and B
 The job of A is to print out a description of B, and
conversely the job of B is to print out a description of
A.
 The result is the desired description of SELF.
 The jobs are similar, but they are carried out
differently.
 Our description of A depends on having a description
of B. So we can't complete the description of A until
we construct B.
10
Machines A and B
(cont)
For A we use the machine P<B>, described by q(<B>).
q(<B>) means applying the function q to <B>.
 If B can obtain <B>, it can apply q to that and obtain
<A>.
 B only needs to look at the tape to obtain <B>.
 Then after B computes q(<B>) = <A>, it combines A
and B into a single machine and writes its description
<AB> = <SELF> on the tape.
11
Machines A and B (algorithm)
12
TM that prints its own description
Suppose that we want to give an English sentence that
commands the reader to print a copy of the same sentence.
One way to do so is to say: Print out this sentence.
13
Example 2
14
TERMINOLOGY FOR THE RECURSION
THEOREM
 recursion theorem in TM - (If you are designing a
machine M, you can include the phrase "obtain own
description <M>" in the informal description of M's
algorithm.)
 Two ways:
1. use any other computed value <SELF>
2. simulate <M>.
15
Algorithms
1. use any other computed value <SELF>
2. simulate <M>.
APPLICATIONS - computer virus
16
THEOREM 6.5
THEOREM 6.6
THEOREM 6.8
17
Decidability of logical Theories





What is a theorem?
What is a proof?
What is truth?
Can an algorithm decide which statements are true?
Are all true statements provable?
18
Decidability of logical Theories (cont.)
We focus on the problem of:
 determining whether mathematical statements are
true or false and,
 investigate the decidability of this problem.
 Can be done - an algorithm to decide truth and
another for which this problem is undecidable.
19
Decidability of logical Theories
(cont)
 Statement 1 - infinitely many prime numbers exist - solved.
 Statement 2 is Fermat' last theorem - solved, and
 Statement 3 - infinitely many prime pairs1 exist - unsolved.
1. differ by 2
20
Decidability of logical Theories (cont.)
let's describe the form of the alphabet of this language:
- A formula is a well-formed string over this alphabet
- All quantifiers appear in the front of the formula.
- A variable that isn't bound within the scope of a
quantifier is called a free variable
21
COMPLEXITY
THE0RY
22
TIME COMPLEXITY
Pages 247 - 256
23
Objectives

-
investigation of the:
time,
memory, or
Other resources required for solving computational
problems.
 to present the basics of time complexity theory.
24
Objectives (cont.)
 First - introduce a way of measuring the time used to
solve a problem.
 Then - show how to classify problems according to
the amount of time required.
 After - discuss the possibility that certain decidable
problems require enormous amounts of time and
how to determine when you are faced with such a
problem.
25
Introduction
 Even when a problem:
- is decidable and
- computationally solvable
 it may not be solvable in practice if the solution
requires an inordinate amount of time or memory.
26
MEASURING COMPLEXITY
The language
27
MEASURING COMPLEXITY (cont.)
 How much time does a single-tape Turing machine need to
decide A?
28
MEASURING COMPLEXITY (cont.)
 The number of steps that an algorithm uses on a
particular input may depend on several parameters:
(if the input is a graph)
- the number of steps may depend on :
- the number of nodes,
- the number of edges, and
- the maximum degree of the graph, or
- some combination of these and/or other factors.
29
Analysis
 worst-case analysis - consider the longest running time
of all inputs of a particular length.
 average-case analysis - consider the average of all the
running times of inputs of a particular length.
30
31
BIG-O AND SMALL-O NOTATION
 Exact running time of an algorithm often is a complex
expression. (estimation)
 Asymptotic analysis - seek to understand the running time of
the algorithm when it is run on large inputs.
The asymptotic notation or big-O notation for describing this
relationship is
32
BIG-O AND SMALL-O NOTATION
(cont.)
In stage 1
 Performing this scan uses n steps.
 Typically use n to represent the length of the input.
 Repositioning the head at the left-hand end of the tape uses
another n steps.
 The total used in this stage is 2n steps.
33
BIG-O AND SMALL-O NOTATION
(cont.)
In stage 4 the machine makes a single scan to
decide whether to accept or reject.
The time taken in this stage is at most O(n).
34
BIG-O AND SMALL-O NOTATION
(cont.)
 Thus the total time of M1 on an input of length n is
O(n) + O(n2 ) + O(n) or O(n2 ).
In other words, it's running time is O(n2 ), which
completes the time analysis of this machine.
35
Is there a machine that decides A
asymptotically more quickly?
36
Executed Time
 Stages 1 and 5 are executed once, taking a total of
O(n) time.
 Stage 4 crosses off at least half the 0s and 1s is each
time it is executed, so at most 1+log2 n.
 the total time of stages 2, 3, and 4 is (1 + log2 n)O(n),
or O(n log n).
 The running time of M2 is O(n) + O (n logn) = O(n log n).
37
COMPLEXITY RELATIONSHIPS
AMONG MODELS

-
We consider three models:
the single-tape Turing machine;
the multi-tape Turing machine; and
the nondeterministic Turing machine
38
COMPLEXITY RELATIONSHIPS
AMONG MODELS (cont.)
 convert any multi-tape TM into a single-tape TM that
simulates it.
 Analyze that simulation to determine how much additional
time it requires.
 simulating each step of the multi-tape machine uses at most
O(t(n)) steps on the single-tape machine.
 the total time used is O(t2 (n)) steps.
O(n) + O(t2 (n)) running time O(t2 (n))
39
SPACE COMPLEXITY
INTRACTABILITY
Pages 303 – 308
40
Objective
 Consider the complexity of computational problems
in terms of the amount of space, or memory, that
they require.
 Time and space are two of the most important
considerations when we seek practical solutions to
many computational problems.
 Space complexity shares many of the features of time
complexity and serves as a further way of classifying
problems according to their computational difficulty.
41
Introduction
 select a model for measuring the space used by an
algorithm.
 Turing machines are mathematically simple and close
enough to real computers to give meaningful results.
42
43
Estimation the space complexity
 We typically estimate the space complexity of Turing
machines by using asymptotic notation.
44
EXAMPLE 8.4
45
SAVITCH'S THEOREM read only
46
INTRACTABILITY
Pages 335 - 338
47
 Certain computational problems are solvable in principle, but
the solutions require so much time or space that they can't be
used in practice. Such problems are called intractable.
 Turing machines should be able to decide more languages in
time n3 than they can in time n2. The hierarchy theorems prove
that .
48
ADVANCED TOPICS IN COMPLEXITY THEORY
Approximation Algorithms
Pages 365 - 367
49
Optimization problems
 Optimization problems - seek the best solution among
a collection of possible solutions.
 Example: shortest path connecting two nodes
 Approximation algorithm is designed to find such
approximately optimal solutions.
 A solution that is nearly optimal may be good
enough and may be much easier to find.
50
Polynomial & Exponential
 Decision problem-one that has a yes/no answer.
 POLYNOMIAL TIME - polynomial differences in running time
are considered to be small and fast, whereas exponential
differences are considered to be large.
 Polynomial time algorithm - n3.
 Exponential time algorithm - 3n .
 MIN-VERTEX-COVER is an example of a minimization problem
because we aim to find the smallest among the collection of
possible solutions.
51
Exponential time algorithm
 Exponential time algorithms typically arise when we solve
problems by exhaustively searching through a space of
solutions, called brute-force search.
52
Exponential time algorithm
 factor a number into its constituent primes is to search
through all potential divisors.
53
 minimization problem - find the smallest among the
collection of possible solutions.
 maximization problem - seek the largest solution.
 Decision problem and NP- Decision.
 Optimization problem and NP-Optimization.
 Approximation problem and approximation.
Note: (NP) Nondeterministic polynomial.
54
Explanation
55
Explanation
56
PROBABILISTIC ALGORITHMS
Pages 368 - 375
57
PROBABILISTIC ALGORITHMS
 A probabilistic algorithm is an algorithm designed to use the
outcome of a random process.
 Example: flip a coin.
 How can making a decision by flipping a coin ever be better
than actually calculating, or even estimating, the best choice
in a particular situation?
58
THE CLASS BPP
 We begin our formal discussion of probabilistic
computation by defining a model of a probabilistic
Turing machine.
59
60
Definition 10.3
(cont.)
 When a probabilistic Turing machine recognizes a
language = it must accept all strings in the language
and reject all strings out of the language as usual.
 Except that now we allow the machine a small
probability of error. For
say
that M recognizes language A with error probability
61
Definition 10.3
(cont.)
 We also consider error probability bounds that depend on
the input length n. For example, error probability = 2-n
indicates an exponentially small probability of error.
worst case computation branch on each input
62
= amplification lemma.
 Amplification lemma gives a simple way of making the error
probability exponentially small.
 LEMMA 10.5 and proof IDEA. (self study) PROOF.
63
PRIMALITY
 A prime number is an integer greater than 1 that is not
divisible by positive integers other than 1 and itself.
 A nonprime number greater than 1 is called composite.
 One way to determine whether a number is prime is to try
all possible integers less than that number and see whether
any are divisors, also called factors.
 exponential time complexity
64
Fermat's little theorem
 For example, if p = 7 and a = 2, the theorem says that 2 (7-1)
mod 7 should be 1 because 7 is prime.
 The simple calculation 2(7-1) = 26 = 64 and 64 mod 7 = 1
confirms this result.
 Suppose that we try p = 6 instead.
Then 2(6-1) = 25 =
32 and 32 mod 6 = 2
65
Algorithm Fermat test
66
67
Note
The probabilistic primality algorithm has one-sided error.
When the algorithm outputs reject, we know that the input
must be composite.
When the output is accept, we know only that the input could
be prime or composite.
Thus an incorrect answer can only occur when the input is a
composite number.
The one-sided error feature is common to many probabilistic
algorithms, so the special complexity class RP is
designated for it.
68