Theory Of Computation
What problems cannot be solved on a computer?
What problems cannot be solved on a computer in a
“reasonable” amount of time?
These aren’t just philosophical questions; their answers
will determine how practical it is to pursue computerbased solutions to real-world problems like hurricane
prediction, disease control, and economic forecasting.
CS 111.01
Chapter 12 – Theory Of Computation
Page 157
Computability
To examine the limits of what it is possible to do with a
computer, Alan Turing (1912-1954) developed a
simplified mathematical model, called a Turing machine.
A Turing machine consists of three parts:
1) A tape of cells from which symbols can be read
and into which symbols can be written,
2) A read/write head that moves back and forth
across the tape, reading the symbol inside the
current cell and/or writing a new symbol into
the current cell, and
Control
Unit
3) A control unit that keeps track of what “state”
the machine is in, and uses that state and the
current symbol under the read/write head to:
Read/Write
Head
Tape
a) Determine which symbol to place in the current cell,
b) Determine whether to move the read/write head one cell to the
left or right, and
c) Determine the next state of the machine.
CS 111.01
Chapter 12 – Theory Of Computation
Page 158
State Transition Diagram
A state transition diagram may be used to
define a Turing machine.
Each // transition signifies reading  on
the tape, replacing it with , and then moving
the read/write head in the  direction.
START
*/*/L
1/0/L
ADD
0/1/L
*/*/R
CARRY
0/1/L
NO CARRY
*/1/L
0/0/L,
1/1/L
*/*/R
HALT
*/*/-
1/0/L
OVERFLOW
*/*/R
RETURN
0/0/R,
1/1/R
The state transition diagram above defines a
Turing machine that increments a binary
number on the tape by one.
CS 111.01
State:
START
*
1
0
1
*
State:
ADD
*
1
0
1
*
State:
CARRY
*
1
0
0
*
State:
NO
CARRY
*
1
1
0
*
State:
NO
CARRY
*
1
1
0
*
State:
RETURN
*
1
1
0
*
State:
RETURN
*
1
1
0
*
State:
RETURN
*
1
1
0
*
State:
RETURN
*
1
1
0
*
State:
HALT
*
1
1
0
*
Chapter 12 – Theory Of Computation
Page 159
The Church-Turing Thesis
Computer scientists commonly accept the Church-Turing Thesis,
which states that the set of functions that can be calculated on a
computer is exactly the same as the set of functions for which a
Turing machine can be devised.
There are problems that have been proven to be non-computable
(i.e., no Turing machine can be devised to calculate their solutions).
One classical example:
The Halting Problem
Given a program with a set of input values,
does the program halt on that input, or does
it get stuck in an infinite loop?
CS 111.01
Chapter 12 – Theory Of Computation
Page 160
Complexity
The time complexity of an algorithm is a measure of how many steps
are executed when the associated program is run.
void printA()
{
cout << 0 <<
cout << 0 <<
cout << 0 <<
cout << 0 <<
}
endl;
endl;
endl;
endl;
Number of Output
Statements Executed: 4
Time Complexity: O(1)
void printB()
{
int i;
for (i = 1; i <= 100; i++)
cout << 0 << endl;
}
void printC(int n)
{
int i;
for (i = 1; i <= n; i++)
cout << 0 << endl;
}
void printD(int n)
{
int i,j;
for (i = 1; i <= n; i++)
for (j = 1; j <= n; j++)
cout << 0 << endl;
}
CS 111.01
Number of Output
Statements Executed: 100
Time Complexity: O(1)
Number of Output
Statements Executed: n
Time Complexity: O(n)
Number of Output
Statements Executed: n2
Time Complexity: O(n2)
Chapter 12 – Theory Of Computation
The “big-O” notation
provides information
regarding the program’s
“order of complexity”.
O(1) indicates that the
execution time doesn’t
relate to the size of the
number n.
O(n) indicates that the
execution time
increases linearly as n
increases.
O(n2) indicates that the
execution time
increases quadratically
as n increases.
Page 161
Logarithmic vs. Polynomial vs. Exponential
An algorithm is said to have logarithmic time complexity if the number of
steps in its execution is bounded by some logarithmic function:
k log2(n)
Essentially, this means that doubling the size of the problem (n) would only
increase the execution time by a constant amount (k).
An algorithm is said to have polynomial time complexity if the number of
steps in its execution is bounded by some polynomial function:
aknk+ak-1nk-1+…+a2n2+a1n+a0
An algorithm is said to have exponential time complexity if the number of
steps in its execution is bounded by some exponential function:
k(2n)
Essentially, this means that increasing the size of the problem (n) by one
would double the execution time.
CS 111.01
log2(n)
n
n2
n3
2n
2
5
25
125
31
3
10
100
1000
1024
4
20
400
8000
1048576
Chapter 12 – Theory Of Computation
Page 162
Big-O
An algorithm’s time complexity is dominated by its
most significant term.
For example, an algorithm that executes in time
n2+10n is considered to be O(n2) because, as n
increases, the n2 term ultimately dominates the n term.
Additional examples:
5n+n2+0.125n3 is O(n3)
log2(n)+n2+2n is O(2n)
100n+max(n2,1000-n3) is O(n2)
n3 -⅞(n3-80n2-800n) is O(n3)
1000000+0.000001log2(n) is O(log2(n))
CS 111.01
Chapter 12 – Theory Of Computation
Page 163
P and NP Problems
A problem is said to be a P problem if it can be solved with a
deterministic, polynomial-time algorithm.
(Deterministic algorithms have each step clearly specified.)
A problem is said to be an NP problem if it can be solved with a
nondeterministic, polynomial-time algorithm.
In essence, at a critical point in the NP problem’s algorithm, a
decision must be made, and it is assumed that some magical “choice”
function (also called an oracle) always chooses correctly.
For example, take the Satisfiability Problem:
Given a set of n boolean variables b1, b2, … bn, and a boolean function
f (b1, b2, …, bn), are there any values that can be assigned to the
variables so the function value will evaluate to TRUE?
To try every combination of boolean
values would take exponential time,
but the nondeterministic solution at
right has polynomial time
complexity.
CS 111.01
for (i = 1; i <= n; i++)
bi = choice(true, false);
if (f(b1, b2,…, bn) == true)
satisfiable = true;
else satisfiable = false;
Chapter 12 – Theory Of Computation
Page 164
The Knapsack Problem
The Knapsack Problem involves
taking n valuable jewels J1, J2,…,Jn,
with respective weights w1, w2,…, wn,
and prices p1, p2,…, pn, and placing
some of them in a knapsack that is
capable of supporting a combined
weight of M.
The problem is to pack the maximum
worth of gems without exceeding the
capacity of the knapsack.
(It’s not as easy as it sounds; three
lightweight $1000 gems might be
preferable to one heavy $2500 gem,
and one 20-pound gem worth a lot of
money might be preferable to twelve
1-pound gems that are practically
worthless.)
CS 111.01
A Nondeterministic
Polynomial Solution:
TotalWorth = 0;
TotalWeight = 0;
for (i = 1; i <= n; i++)
{
bi = choice(true, false);
if (b1 == true)
{
TotalWorth+= pi;
TotalWeight += wi;
}
}
if (TotalWeight <= M)
cout << “Woo-hoo!” << endl;
else
cout << “Doh!” << endl;
Chapter 12 – Theory Of Computation
Page 165
Cryptography
Networks are set up to send messages right past
stations that aren’t authorized to read them, but
what’s to prevent such unauthorized viewing?
message
from
to
message
from
to
essage
to
The most common solution to this problem is
encryption, where the message is coded in such a
way that only the receiving station can decode it.
CS 111.01
Chapter 12 – Theory Of Computation
Page 166
Public-Key Encryption
I have affixed
to me the dirt
and dust of
countless ages!
Chuck
Linus
Lucy
Patty
1.
Create
Message
mdbriugndlwg
mamnsgfyddkd
qhgwdnchsgsh
ahwbsgcydhzx
2.
Look Up
Recipient’s
Public Key
xsjb2dhdkWb$
xzduYdm!dj5slL
ssghd8nd&hsnq
abi?dsjsg%
3.
Encrypt Message
With Recipient’s
Public Key
4.
Transmit Encrypted
Message
xsjb2dhdkWb$
xzduYdm!dj5slL
ssghd8nd&hsnq
abi?dsjsg%
CS 111.01
I have affixed
to me the dirt
and dust of
countless ages!
5.
Decrypt Message With
Recipient’s Private Key
Chapter 12 – Theory Of Computation
Page 167
Key-Based Authentication
Ma3ndhvyr#bcjaqwp
fQkguiorkfohskxi8vc
e%fpgkjfhikfvdamxx
yemfideychssfhsgdh
ahdm$dlglyn7buchso
I’m going to recruit
that funny-looking kid
who plays shortstop on
Chuck’s team!
2.
Encrypt Message With
Sender’s Private Key
1.
Create
Message
3.
Transmit
Encrypted
Message
Ma3ndhvyr#bcjaqwp
fQkguiorkfohskxi8vc
e%fpgkjfhikfvdamxx
yemfideychssfhsgdh
ahdm$dlglyn7buchso
CS 111.01
I’m going to recruit
that funny-looking kid
who plays shortstop
on Chuck’s team!
4.
Decrypt Message
With Sender’s
Public Key
Chapter 12 – Theory Of Computation
Page 168