CS 461 – Nov. 21
Sections 7.1 – 7.2
• Measuring complexity
• Dividing decidable languages into complexity classes.
• Algorithm complexity depends on what kind of TM you
use.
• Formal definition of “P”
• Next: Concepts of NP and NP-complete
Examples
• Some graph problems
–
–
–
–
Finding the shortest path
Finding the cheapest network (spanning tree)
Hamiltonian and Euler cycles
Traveling salesman problem
• Why do similar sounding problems have vastly different
complexity?
“O” review
• Order of magnitude upper bound
• Rationale: 1000 n2 is fundamentally less than n3, and is
in the same family as n2.
• Definition: f(n) is O(g(n)) if there exist integer constants
c and n0 such that f(n)  c g(n) for all n  n0.
– In other words: in the long run, f(n) can be bounded by g(n)
times a constant.
– e.g. 7n2 + 1 is O(n2) and also O(n3) but not O(n). O(n2) would
be a tight or more useful upper bound.
– e.g. Technically, 2n is O(3n) but 3n is not O(2n).
– log(n) is between 1 and n.
• Ordering of common complexities:
O(1), O(log n), O(n), O(n log n), O(n2), O(n3), O(2n), O(n!)
Measuring complexity
• Complexity can be defined:
– as a function of n, the input length
– It’s the number of Turing machine moves needed.
– We’re interested in order analysis, not exact count.
• E.g. About how many moves would we need to
recognize { 0n1n } ?
– Repeatedly cross of outermost 0 and 1.
– Traverse n 0’s, n 1’s twice, (n-1) 0’s twice, (n-1) 1’s twice, etc.
– The total number of moves is approximately:
3n + 4((n-1)+(n-2)+(n-3)+…+1) = 3n + 2n(n-1) = 2n2 + n ~ 2n2
– 2n2 steps for input size 2n  O(n2).
Complexity class
• We can classify the decidable languages.
• TIME(t(n)) = set of all languages that can be decided by
a TM with running time O(t(n)).
– { 0n 1n }  TIME(n2).
– 1*00*1(0+1)*  TIME(n).
– { 1n 2n 3n }  TIME(n2).
– CYK algorithm  TIME(n3).
– { ε, 0, 1101 }  TIME(1).
• Technically, you can also belong to a “worse” time
complexity class. L  TIME(n)  L  TIME(n2) .
•  It turns out that { 0n 1n }  TIME(n log n). (p. 252)
TM variants
• It turns out that the complexity depends on the variant!
– Ordinary one-tape TM.
– Multi-tape TM.
– Non-deterministic TM.
(for definition of TIME(t(n))
(useful for programming)
• E.g. { 0n 1n } can be decided in O(n) on a multi-tape TM.
– See page 253.
• In general, a multi-tape TM that runs in O(t(n)) can be
converted to a single-tape TM that runs in O((t(n)2).
– See page 254.
• In general, a non-deterministic TM that runs in O(t(n))
can be converted to an ordinary TM running in 2 O(t(n)).
– See page 256.
– This is why we say that NP problems are exponential. 
Multi-tape { 0n 1n }
• We have seen an O(n2) algorithm on a single-tape TM.
• And a better algorithm that runs in O(n log n), also with
single tape.
• With multiple tapes, we can do it in O(n).
–
–
–
–
–
Scan input to make sure of the form 0*1*.
Return to beginning.
Read 0’s and copy them to 2nd tape.
When you reach the first 1, start the 2nd tape’s head at the first 0.
Now we match 0’s and 1’s simultaneously.
• So, the problem is O(n), although technically it’s still in
TIME(n log n). Not a big deal in the long run.
Single vs. multi tape
O(t(n)) multi-tape  O(t(n)2) single tape. Proof:
• We simulate the multi-tape operation on single tape TM.
– Each of the k tapes is performing a computation that runs in t(n)
time. In other words, we take up to t(n) steps.
• Scan the tape, taking note of where virtual heads are.
• How long is the tape contents?
– Each virtual section of the tape could be t(n) symbols long,
because each step could write a symbol on the tape.
• Final calculation:
– Scanning the entire tape takes O(t(n)) time.
– And we have to scan the tape t(n) times!
– Multiply together to obtain O(t(n)2).
DTM vs. NTM
O(t(n)) NTM  2 O(t(n)) DTM. Proof:
• The NTM has a tree of possible computational futures.
• For input length n, the depth of tree is up to t(n).
• Each node of the tree can have up to c children.
– Total number of nodes in the tree is on the order of O(c t(n)).
• The time it takes to reach the appropriate node in the
tree is t(n)
– Multiply this by the number of nodes to obtain the upper bound
on the complexity: O(t(n) c t(n)).
– Claim: this is equivalent to 2 O(t(n)). Actually it’s greater but ok.
• Finally, need to convert multi-tape to single tape. But
squaring an exponential is still just an exponential.
Definition of P
• The union of all language classes of the form:
TIME(1)  TIME(n)  TIME(n2)  TIME(n3) …
• Thus, it’s all algorithms that can be decided by an
ordinary (one-tape, deterministic) TM in polynomial time.
– Even if we allowed multiple tapes, the time complexity would still
by polynomial, by our earlier theorem.
– We also generalize “TM” to computer program. When we speak
of a O(f(n)) algorithm, this is generally assumed. But it’s still
deterministic! So, it’s ok to say { 0n 1n } is O(n) problem.
• Many algorithms are in P. This is where we like to be.
– No recursion
– Nested loops ok.