CSCI 2670
Introduction to Theory of
Computing
November 15, 2005
Agenda
• Today
– Comparing complexity of different
computing models
• This week
– Finish 7.1 and do 7.2
November 15, 2005
Announcement
• Office hours changed tomorrow
– 11:30 to 1:00 instead of 11:00 – 12:30
November 15, 2005
Analyzing algorithms
• We can examine an algorithm to
determine how long it will take to halt
on an input of length n
• The amount of time to complete is
called the algorithms complexity class
Definition: Let t:N→N be a function.
The time complexity class,
TIME(t(n)), is defined as follows.
TIME(t(n))={L|L is a language decided
by an O(t(n)) time
November TM}
15, 2005
Another example
• Finding minimum element in a set
• Amount of time depends on the
structure of the input
• If set is a sorted array?
– O(1)
• If set is an unsorted array?
– O(n)
• If set is a balanced sorted tree?
– O(log n)
November 15, 2005
Importance of model
• The complexity of our algorithms
depends on assumptions about our
data & other model assumptions
• The complexity of an algorithm can
vary depending on the machine we use
• Recall we assume reasonable encoding
of all numbers
– Any integer of value v should take
O(log v) space
November 15, 2005
Machine-dependent complexity
• Example, let L={w | w is a palindrome}
• How long will it take us to decide L on a
standard TM?
– Go back and forth crossing off matching
symbols at beginning and end
– O(n2)
• How long will it take us to decide L on a 2tape TM?
– Copy string
– Compare symbols reading forward on tape 1 and
backward on tape 2
– O(n)
November 15, 2005
Complexity relationships
Theorem: Let t(n) be a function, where
t(n)  n. Then every t(n) time
multitape TM has an equivalent
O(t2(n)) time single-tape TM
Proof idea: Consider structure of
equivalent single-tape TM. Analyzing
behavior shows each step on multitape machine takes O(t(n)) on single
tape machine
November 15, 2005
Equivalent machines
0 1 ~ ~ ~ ~~ ~
M
a a a ~ ~ ~ ~ ~
a b ~ ~ ~~ ~ ~
S
# 0 1 # a a a # a b# ~ ~
November 15, 2005
Simulating k-tape behavior
• Single tape start string is
#w#~#...#~#
• Each move proceeds as follows:
– Start at leftmost slot
– Scan right to (k+1)st # to find symbol at
each virtual tape head
– Make second pass making updates
indicated by k-tape transition function
– When a virtual head moves onto a #,
shift string to right
November 15, 2005
Proof of theorem
• Analyzing simulation of k-tape
machine
• Each step on single-tape machine has
two phases
– Scan tape
– Perform operations
• How long does first phase take?
– Length of string on tape
– Each portion has O(t(n)) length (this
occurs if tape November
heads
only move right)
15, 2005
Proof of theorem (cont.)
• How long does second phase take?
– Perform k steps
• Each step may require a right shift
– Each step takes O(t(n)) time
– Total of k steps is O(t(n)) because k is a
constant
• What’s the total time?
– O(t(n)) steps each take O(t(n)) time
– Total time is O(t2(n))
November 15, 2005
Determinism vs. non-determinism
Definition: Let P be a nondeterministic Turing machine. The
running time of P is the function
f:N→N, where f(n) is the maximum
number of steps that P uses on any
branch of its computation in any input
of length n.
November 15, 2005
Non-deterministic tree
f(n)
…
…
accept
accept/
reject
Deterministic
reject
November 15, 2005
Non-deterministic
Complexity relationship
Theorem: Let t(n) be a function where
t(n)n. Then every t(n) time nondeterministic single-tape Turing
machine has an equivalent 2O(t(n)) time
deterministic single-tape Turing
machine.
November 15, 2005
Complexity relationship proof
Proof: Given a non-deterministic TM, P,
running in t(n) time, construct a 3-tape
deterministic TM that simulates P.
The height of the tree is at most t(n).
Assume the maximum number of
branches in the tree is b. Therefore,
the number of leaves in the tree is
O(bt(n)).
November 15, 2005
How many nodes in the tree?
Claim: The total number of nodes is less
than twice the number of leaves – i.e.
O(bt(n)).
(Σ0≤k≤n bk)  bn = (1 – bn+1)/(1-b)  bn
< – bn+1/(1-b)  bn
= - b / (1-b)
≤2
November 15, 2005
Complexity relationship proof (cont.)
Deterministic TM does a breadthfirst search of the non-deterministic
TM’s tree.
Total time to search tree is O(t(n))
to travel from the root to a leaf ×
O(bt(n)), the number of leaves.
O(t(n)bt(n)) = O(2log_2 t(n) 2(log_2 b)t(n)) =
O(2O(t(n)))
November 15, 2005
Complexity relationship proof (cont.)
Are we done?
No! We constructed a 3-tape TM
with running time O(2O(t(n)))
Single-tape TM will take
O((2O(t(n)))2)=O(22O(t(n)))=O(2O(t(n)))
Are we done?
Yes!
November 15, 2005