The Complexity of Theorem-Proving Procedures
Stephen A. Cook
University of Toronto
Summary
certain recursive set of strings on
this alphabet, and we are interested
in the problem of finding a good
lower bound on its possible recognition times.
We provide no such
lower bound here, but theorem 1 will
give evidence that {tautologies} is
a difficult set to recognize, since
many apparently difficult problems
can be reduced to determining tautologyhood.
By reduced we mean,
roughly speaking, that if tautologyhood could be decided instantly
(by an "oracle") then these problems
could be decided in polynomial time.
In order to make this notion precise,
we introduce query machines, which
are like Turing machines with oracles
in [I].
It is shown that any recognition
problem solved by a polynomial timebounded nondeterministic Turing
machine can be "reduced" to the problem of determining whether a given
propositional formula is a tautology.
Here "reduced" means, roughly speaking, that the first problem can be
solved deterministically in polynomial time provided an oracle is
available for solving the second.
From this notion of reducible,
polynomial degrees of difficulty are
defined, and it is shown that the
problem of determining tautologyhood
has the same polynomial degree as the
problem of determining whether the
first of two given graphs is isomorphic to a subgraph of the second.
Other examples are discussed.
A
method of measuring the complexity of
proof procedures for the predicate
calculus is introduced and discussed.
Throughout this paper, a set of
strings means a set of strings o n
some fixed, large, finite alphabet Z.
This alphabet is large enough to include symbols for all sets described
here.
All Turing machines are deterministic recognition devices, unless
the contrary is explicitly stated.
i.
Tautologies and Polynomial ReReducibility.
Let us fix a formalism for
the propositional calculus in
which formulas are written as
strings on I. Since we will require infinitely many proposition
symbols (atoms), each such symbol
will consist of a member of Z
followed by a number in binary
notation to distinguish that
symbol.
Thus a formula of length
n can only have about
n/logn
distinct function and predicate
symbols.
The logical connectives
are
&
(and), v (or), and ~(not).
The set of tautologies
(denoted by {tautologies}) is a
A query machine is a multitape
Turing machine with a distinguished
tape called the query tape, and
three distinguished states called
the query state, yes state, and n._o_
state, respectively.
If M
is a
query machine and T
is a set of
strings, then a T-computation of M
is a computation of M
in which
initially M
is in the initial
state and has an input string w on
its input tape, and each time M
assumes the query state there is a
string u on the query tape, and
the next state M
assumes is the
yes state if uET
and the no state
if u~T.
We think of an "oracle",
which knows T, placing M
in the
yes state or no state.
Definition
A set S of strings is P-reducible (P for polynomial) to a set
T of strings iff there is some
query machine
M and a polynomial
Q(n)
such that for each input string
w, the T-computation of M with input w halts within
Q(Iwl)
steps
(lwl is the length of w ~ and ends
in an accepting state i f f wcS.
It is not hard to see that
P-reducibility is a transitive rel a t i o n . Thus the relation E on
-151-
machine.
sets of strings, given by (S,T)eE
iff
each of S and T
is P-reducible to
the other, is an equivalence relation.
The equivalence class containing a set
S will be denoted by deg (S) (the polynomial degree of difficulty of S).
Definition:
~,,
where
tion.
We will denote deg ({0}) by
0 denotes the zero func-
Thus ~ ,
is the class of sets recognizable in polynomial time. ~ ,
was discussed in [2], p. 5, and is the
string analog of Cabham's class
of
functions [3].
We now define
sets of strings.
the following
special
i) The subgraph problem is the
problem given two finite undirected
graphs, determine whether the first is isomorphic to a subgraph of the second.
A
graph
G can be represented by a string
on the alphabet
{0,i,*}
by listing
the successive rows of its adjacency
matrix, separated by *s. We let {subgraph pairs} denote the set of strings
Gi**G 2
such that
a subgraph
of
G
G1
Proof of the theorem:
Suppose a nondeterministic Turfng machine
M
accepts
a set
S of strings within time
Q(n),
where
Q(n)
is a polynomial.
Given an
input w for M, we will construct a
proposition formula
A(w)
in conjunctive normal form such that A(w)
is
satisfiable iff M accepts
w . Thus
A(w)
is easily put in disjunctive
normal form (using De Morgan's laws),
and A(w)
is a tautology if and only
if w~S.
Since the whole construction
can be carried out in time bounded by
a polynomial in
lwl (the length of w),
the theorem will be proved.
is isomorphic
to
We may as well assume the Turing
machine
M has only one tape, which is
infinite to the right but has a leftmost square.
Let us number the squares
from left to right I, 2,
Let us
fix an input
w
to M of length
n,
and suppose
wES.
Then there is a
computation of M with input
w
that
ends in an accepting state within
T = Q(n)
steps.
The formula
A(w) will
be built from many different proposition symbols, whose intended meanings,
listed below, refer to such a computation.
2"
Suppose the tape alphabet for
{oi, ..., o£}, and the set of
2) The graph isomorphism problem
will be represented by the set, denoted
by {isomorphic graphpairs}, of all
states
strings
since the computation
morphic
Gi**G 2
to
such that
G1
is
is iso-
T = Q(n)
G 2.
3) The set {Primes}
is the set of
all binary notations for prime numbers.
number
The set
D3
consists
{ql'
steps,
T
pi
s,t
pi
s,t
of those
tautologies in disjunctive normal form
in which each disjunct has at most three
conjuncts (each of which is an atom or
negation of an atom).
Theorem I:
If a set
S of strings is
accepted by some nondeterministic Turing
machine within polynomial time, then
S
is P-reducible to {DNF tautologies}.
"''' qs }"
has at most
no tape square beyond
symbols:
fo=
i~'i~£,
t
i
Qt
contains
l~s,t~T.
for
l~i~r,
true iff at step
state
the symbol
t
l~t~T.
i
Qt
the machine
s
°i
is
is in
qi"
S
.s,t
t
for
l~s,t~T
square number
Corollary:
Each of the sets in definitions 1)-5)
is P-reducible to {DNF
tautologies}.
by the tape head.
This is because each set, or its
complement, is accepted in polynomial
time by some nondeterministic Turing
The formula
A(w)
tion
B&C&D&E&F&G&H&I
follows.
Notice
A(w)
junctive normal form.
-152-
that
is true iff tape square number
at step
time
Notice
is scanned.
Proposition
4) The set {DNF tautologies}
is
the set of strings representing tautologies in disjunctive normal form.
5)
is
M
is true
s
iff at
is scanned
is a conjuncformed as
is in con-
proof.
B will assert that at each step
t, one
and only one square is scanned.
B is a
conjunction
B 1 & B 2 & ... & BT,
where
Bt
asserts
that at time
only one square
t
one and
is scanned:
B t = (Si, t v $2, t v ... v ST,t)
[
&
l_<i< j_<T
For
T h e o r e m 2: The f o l l o w i n g sets are
P - r e d u c i b l e to each other in pairs (and
hence each has the same p o l y n o m i a l degree
of difficulty):
{ t a u t o l o g i e s } , {DNF
tautologies},
D3, { s u b g r a p h pairs}.
C~
v ~S
Si't
l_<ssT
and
J't)
&
Remark:
We have not been able to add
~ h ~
{ p r i m e s } or { i s o m o r p h i c graph
pairs} to the above list.
To show
( t a u t o l o g i e s } is
P - - r e d u c i b l e to
(primes} w o u l d seem to require some deep
results in number theory, while showing
{tautologies}
is
P - r e d u c i b l e to (isom o r p h i c graph pairs}
would probably
upset a c o n j e c t u r e of C o r n e i l ' s [4] from
w h i c h he deduces that the ~ r a p h isomor- .
p h i s m p r o b l e m can be s o l v e d in p o ± y n o m i a ±
time.
]
l_<t_<T•
j
C
s,t
asserts that at square
s and time
t
there is one and only one symbol.
C
is
the c o n j u n c t i o n of all the
C
s,t"
D asserts that for each
is one and only one state.
E asserts
are satisfied:
o
E = Q1
& Sl,l
the initial
iI
& Pi,i
& pl
i2
& P2,1
t
there
Incidently, it not hard to see from
the D a v i s - P u t n a m p r o c e d u r e [5] that the
set
D~
c o n s i s t i n g of all DNF tautologies wlth at most two c o n j u n c t s per disjunct, is in ~ * .
Hence
D 2 cannot be
conditions
p in
& "'" &
added to the list in theorem
all sets in the list are in
n,l
1
Proof
n+l,1 &''" & PT,1
where
the initial
w
=
state
Oil
and
°in'
oI
qo
of theorem
2:
2 ~unless
~-*).
By the c o r o l l a r y
to
theorem i, each of the sets is
P-reducible to {DNF tautologies}.
Since
o b v i o u s l y { D N F t a u t o l o g i e s } is P-reducible to { t a u t o l o g i e s } , it remains to show
{DNF t a u t o l o g i e s } is P - r e d u c i b l e to D 3
is
is the blank
and
symbol.
D3
is P - r e d u c i b l e
to { s u b g r a p h
pairs}.
F, G, and
H
assert that for each
time
t the values of the
P's,
Q's
and S's
are u p d a t e d properly.
For example,
G
is the c o n j u c t i o n over all
t, i ' j
that
G.tl,j' where
G.tl,j
if at time
t the m a c h i n e
state
of
qi
time
t + 1
where
qk
transition
t
Gi,j
=
scanning
symbol
the m a c h i n e
oj,
To show
{DNF tautologies} is
P - r e d u c i b l e to D3, let A be a proposition formula in d i s j u n c t i v e normal
form.
Say A = B 1 v B 2 v ... v Bk,
asserts
is in
where
then at
is in state
Ri
and
qk'
for
R 1 & ... &
R s,
is an atom or n e g a t i o n
s > 3.
and only
is the state g i v e n by the
function
B1 =
if
Then
A'
A
and each
of an atom,
is a t a u t o l o g y
is a tautology
if
where
M.
T
i
"
k
&
( 7 Qt v 7 S
v 7P~
v
s= 1
s,t
,t
Qt+l )
A' = P & R 3 & . . . & R s v ~ P & R i & R 2 v B 2 v ... v Bk,
where
P
is a new atom.
Since we have
r e d u c e d the number of conjuncts in Bi,
Finally, the formula
I asserts
that the m a c h i n e reaches an a c c e p t i n g
state at some time.
The m a c h i n e
M
should be m o d i f i e d so that it continues
to compute in some trival f a s h i o n after
r e a c h i n g an a c c e p t i n g state, so that
A(w)
will be satisfied.
this process may be repeated until
e v e n t u a l l y a formula is found with at
most three conjuncts per disjunct.
Clearly the entire process is b o u n d e d in
time by a p o l y n o m i a l in the length of A.
It remains to show
D 3 is P-reducible to { s u b g r a p h pairs}.
Suppose
A
is a formula in d i s j u n c t i v e normal form
with three conjuncts per disjunct.
Thus
A = C 1 v ... v Ck,
where
It is now s t r a i g h t f o r w a r d to v e r i f y
that
A(w)
has all the p r o p e r t i e s asserted in the first p a r a g r a p h of the
-153-
C i = Ril & Ri2 & Ri3,
and each
an atom or a negation
of an atom.
let
G1
tices
be the complete
{v I, v 2,
Rij
a construction
is
lynomial
Now
graph with ver-
... , Vk},
and let
be the graph with vertices
{uij},
bedded
1 ~ j ~ 3, such that
is connected
by an edge to
and only if
i ~ r
(Rij,
Rrs)
4ij
.
if
Urs
and the two literals
do not form an opposite
pair
(that is they are neither
of the form
(P, ~P)
(~P,P~.
there
nor of the form
is a falsifying
to the formula
homomorphism
each
A
i, ~(vi)
iff there is a graph
= uij
(The homomorphism
which
of
sified,
in G 2
Ril,
such that for
for some
tells
that
i
should be fal-
and the selective
guarantees
j.
for each
Ri2 , Ri3
truth assignment
Thus
truth assignment
$ : G1 + G 2
lack of edges
the resulting
is consistently
spe-
cified).
In order to guarantee
homomorphism
perty
~ : G1 + G2
that for each i,
some j, we modify
lows.
G1
which
are sufficiently
other
that if
G½
Hi
is formed
has the pro-
and
We select graphs
attaching
that a one-one
¢(v i) = uij
G2
Gi
is formed
to
vi,
from
from each
G1
by
1 ~ i ~ k, and
G2
by attaching
Uil
1 ~ i ~ k,
then every one-one
phism
G~ ÷ G~
just stated.
as fol-
from
to each of
~ :
and
for
H I , H 2, ..., H k
distinct
Ui2
and
in
G~
This completes
G2
2.
1 ~ i ~ k,
time.
Hi
ui3,
homomor-
can be carried
Then
if
G~
out in po-
can be em-
and only if
A ~ D 3.
the proof of theorem
2.
Discussion
Theorem 1 and its corollary give
strong evidence that it is not easy to
determine whether a given proposition
formula is a tautology, even if the
formula is in normal disjunctive form.
Theorems I and 2 together suggest that
it is fruitless to search for a polynomial decision procedure for the subgraph problem, since success would bring
polynomial decision procedures to many
other apparently intractible problems.
Of course the same remark applies to any
combinatorial problem to which {tautologies}
is P-reducible.
Furthermore, the theorems suggest
that {tautologies} is a good candidate
for an interesting set not in ~ * ,
and
I feel it is worth spending considerable effort trying to prove this conjecture.
Such a proof would be a major
breakthrough in complexity theory.
In view of the apparent complexity of
{DNF tautologies}, it is interesting to
examine the Davis-Putnam procedure [5].
This procedure was designed to determine
whether a given formula in conjunctive
normal form is satisfiable, but of course
the "dual" procedure determines whether
a given formula in disjunctive normal
form is a tautology.
I have not yet been
able to find a series of examples showing
the procedure (treated sympathetically to
avoid certain pitfalls) must require more
than polynomial time.
Nor have I found
an interesting upper bound for the time
required.
If we let strings represent natural
numbers, (or k-tuples of natural numbers) using m-adic or other suitable
notation, then the notions in the preceeding sections can be made to apply to
sets of numbers (or k-place relations on
numbers).
It is not hard to see that the
set of relations accepted in polynomial
time by some nondeterministic Turing machine is precisely the set ~f+
of relations of the form
(I)
(3y_<gk(i)) R(i,y)
where
gk(x)
has the property
It is not hard to see such
-154-
= 2 (£ (max i))k , £(z)
is the
dyadic
length of z, and
R(x,y)
is an
* relation, ( ~ + is the class of extended positive rudimentary relations
of Bennett [6]).
If we remove the bound
on the quantifier in formula (i), the
class ~ + would become the class of recursively enumerable sets.
Thus if
~
is the analog of the class of r.e.
sets, then determining tautologyhood is
the analog of the halting problem; since,
according to theorem i, {tautologies}
has the complete g~+
degree just as the
halting problem has the complete r.e.
degree.
Unfortunately, the diagonal
argument which shows the halting problem
is not recursive apparently cannot be
adapted to show {tautologies} is not in
3.
The Predicate
Calculus
Formulas in the predicate calculus
are represented by strings in a manner
similar to the propositional calculus.
In addition to the symbols for the latter, we need the quantifier symbols V
and J, and symbols for forming an infinite list of individual variables, and
infinite lists of function and predicate
symbols of each order (of course the
underlying alphabet
Z is still finite).
Definition:
If A
is unsatisfiable,
then
~(A)
is the least
k such that
A 1 & A 2 & ... & A k is truth-functionally inconsistent.
If A
is satisfiable,
then
~(A)
is undefined.
given
Now let Q be the procedure which,
A, computes the sequence
A I, A 2,
... and for each
i, tests whether
A 1 & ... & A i is truth-functionally
consistent.
If the answer is ever no,
the procedure terminates successfully.
Then clearly there is a recursive
T(k)
such that for all
k and all formulas
A,
if the length of A ~ k and
~(A) ~ k,
then Q will terminate within T(k)
steps.
We suggest that the
function
T(k)
is a measure of the efficiency of Q.
For convenience, all procedures in
this section will be realized on single
tape Turing machines, which we shall
call simply machines.
Definition:
recursive
MQ
Given a machine
function
is of type
TQ(k)
Q
provided
its tape,
A
then
~ k
and
halts within
We Jill take the following approach.
Most automatic theorem provers depend on
the Herbrand theorem, which states briefly that a formula
A
is unsatisfiable if
and only if some conjunction of substitution instances of the functional form
fn(A)
of A
is truth functionally inconsistent.
Suppose we order the terms
in the Herbrand universe of fn(A) according to rank, and then order in a
natural way the substitution instances
of fn(A)
from the Herbrand universe.
The ordering should be such that in
general subst&tution instances which use
terms with greater rank follow substitution instances which use terms of lesser
rank.
Let A I, A 2 . . . . be these substitution instances in order.
and runs within
MQ
formula
MQ
A
halts
is unsatisfiable,
~(A)
IAI ~ log 2 k,
steps.
also say that
of type
Here
IAI
time
starts
written
and for all
TQ(k)
say
on
if and only if
case we will
Q.
and
we will
that when
with a predicate
Suppose
Q is a procedure which
operates on the above formulas and which
terminates on a given input formula
A
iff A
is unsatisfiable.
Since there is
no decision procedure for satisfiability
in the predicate calculus, it follows
that there is no recursive function
T
such that if A
is unsatisfiable, then
Q will terminate within
T(n)
steps,
where
n is the length of A.
How then
does one appraise the efficiency of the
procedure?
TQ(k),
MQ
k, if
then
MQ
In this
TQ(k)
is
is the length of
A.
The reason for the condition
IAI ~ log 2 k
instead
of
IAI ~ k,
that with the latter condition,
a lower bound for
ly equivalent
TQ(k)
to finding
for the decision problem
sitional
calculus.
theorem
3A
trivial.
-155-
is
finding
would be neara lower bound
for the p r o p o -
In particular,
would become
obvious
and
Theorem
3:
A)
For any
TQ(k)
of type
Q,
Tq,(k)
(2)
is unbounded.
~f/(log
B)
type
Q
k) 2
There
Proof:
is a
TQ(k)
2n.
~ k 2 k(log k)
~ s2 .
TQ(k)
dification
Thus,
2n
if, contrary
= 0(~fk/log2k),
of
MQ
to
that
M
then a mo-
could verify
halted
(provided
m ~ log s
length of
A(M)).
(see
[8] p. 153)
possible
B)
time
shows
by following
, where
m
argument
has
we can assume
4:
accepted
length
operates
is an honest
in
w, if
T(n)
w ~ S
for which
halt,
put
M2
w
A1 & A2 &
0(k log2k),
since
S
= 2 n, and if
(i.e. real-time
machine
TQ(k)
is
in which case
M2
Thus for all strings
M2
~(A(w))
appears
4.
Further,
steps for all
Thus,
whether
with
input
within
w c S
A(w).
TQ(K8 n)
by
If
steps,
w ~ S.
More Discussion:
functions
is a large gap between
of
v~/(logk) 2
TQ(k)
given
rQ(k)
the
for time
in theorem
and a possible
-156-
a de-
can be con-
w ~ S, and otherwise
lower bound
countable)
w.
then
M
to determine
of
is satis-
accepts
K8 n.
machine
no result
A(w)
4n
=
A(w)
of iA).
computations,
MQ
of length
in a way similar
halts within
presenting
then
w
MI
in the proof
~ K(4n) 2
in-
steps for all
such that
is constructed
to
with
a formula
(A(w)
if
fails
M2
Now given
if and only if
A(M)
w
4n
fiable
to
is a computation
with input
we may construct
There
of strings
MI
w ~ S, then
0(n)
for exactly
unless
halts within
length
in time
MI
then there
and if
structed
of this sec-
S
be a nondeterministic
forever.
possible
the procedure
by a nondeterministic
time
computes
JAJ ~ log k.
If the set
accepts
the input,
terministic
MQ
Note that the formula
Theorem
is the
that this is im-
at the beginning
... & A k
within
steps
A diagonal
The machine
outlined
s
M2
is a nondeterminis-
steps and then halts,
accepts
n,
in only
in general.
TQ
tion.
2
in
Let
computations.
0( ~s ~ / l o g 2 s 2) = 0(s/log2s)
steps
MI
machine which simulates
2
Outline of proof:
A).
Given any machine
M, one can construct a predicate
formula
A(M)
which is satisfiable if
and only if M
never halts when starting on a blank tape.
This is done
along the lines described in Wang [7]
in the proof which reduces the halting
problem to the decision problem for the
predicate calculus.
Further, if M
halts in s
steps, then
~(A(M))
Suppose
tic machine which
of
such that
T~rk~
(2),
function of type
Q, then there is a constant
K so S can be recognized by
a deterministic machine within time
TQ(K8 n) .
= k2 k(l°g k) 2
3A
given in 3B.
However,
for the gap.
For example,
improve the result
TQ(k)
bounded
there are reasons
if we could
then by theorem
in k,
4 we could simulate
nondeterministic
2n
for some polynomial
in time
p.
This
The criterion
TQ(k))
example,
TQ(k)
a
time bounded ma-
chine deterministically
and suggest
tion
in 3B and find a
by a polynomial
tions
p(2 n)
a function
of which
one is
be the minimum
of
contradiction
ministic
of a nondeterminis-
all of
T(n)
time
time bounded machine
k T(n)
in general.
On the other hand,
up the lower bound given
in theorem
3A
and show
and,
too crude.
sures
For
to make
and another might
of substitution
needed
...,
A~(A)
to form a
a basis
finding better
not
are needed.)
may be a crude measure,
but
for discussion,
I hope, will stimulate
toward
(the func-
(note that in general
it does provide
if we could push
~(A),
number
Ai, A2,
TQ(k)
requires
to pursue.
of several variables,
fn(A)
which indicates deter-
tic
here
is probably
trary to experience
simulation
suggested
it might be better
instances
is con-
new goals
progress
complexity
mea-
for theorem provers.
Tq(k)
REFERENCES
2k
is
unbounded,
t h e n we c o u l d
{Tautologies}
~ S~,
general Herbrand
provide
a
since otherwise
proof procedure
TQ(k)
smaller
such an improvement
than
paring
2 k.
appear
the dozens
by computer
is a good criterion,
retical
which will bring
4.
D. G. Corneil and C. C. Gotlieb:
An
Efficient Algorithm for Graph Isomorphism.
J. Assoc Computing Machinery
Vol. 17, No. i, Jan. 1970, pp 51-64.
5.
M. Davis and H. Putnam:
A Computing
Procedure for Quantification Theory.
J. Assoc. Computing Machinery, 1960,
pp. 201-215.
6.
J. H. Bennett:
On Spectra. Doctoral
Dissertation, Princeton University,
1962.
on examples
the procedure
in some practical
complexity
of pro-
in the literature.
of a procedure
useful
Cobham, Alan:
The intrinsic computational difficulty of functions.
Proc. of the 1964 International Congress for Logic, Methodology, and the
Philosophy of Science, North Holland
Publishing Co., Amsterdam, pp. 24-30.
for com-
Performance
(unless
3.
Thus
theorem
a basis
and evaluating
sufficient
S. A. Cook:
Characterizations of
Pushdown Machines in terms of TimeBounded Computers.
J. Assoc. Computing Machinery, Vol. 18, No. i,
Jan. 1971, pp 4-18.
would
in complexity
of mechanical
badly needs
cedures which
2.
the
theory.
proving
D. L. Kreider and R. W. Ritchie:
Predictably Computable Functionals
and Definitions by Recursion.
Zeitschrift fHr math.
Logik und
Grundlagen der Math., Vol. I0,
65-80 (1964).
in 3A would require
a major breakthrough
The field
I.
conclude
way).
criterion
but not
proves
A theo-
is needed
out fundamental
limita-157-
?.
Hao Wang:
Dominoes and the AEA case
of the decision problems.
Proc. of
the Symposium on Mathematical Theory
of Automata, at Polytechnic Institute
of Brooklyn, 1962.
pp. 23-55.
8.
John Hopcroft and Jeffrey Ullman:
Formal Languages and their Relation
to Automata.
Addison-Wesley, 1969.
-158-