§4.4 Mathematical induction

§4.4 Mathematical induction
This section treats three kinds of induction: Ordinary or “weak” mathematical
induction (which you are probably already familiar with), strong
mathematical induction, and structural induction, which we study later.
Consider the inductive definition of a set S of integers, as follows:
◮ Basis: n0 ∈ S.
◮
Induction: If n ∈ S, then so is n + 1.
Then, to prove P(n) for all n ∈ S, by (weak) mathematical induction, it
suffices to prove that
◮ Basis: P(n0 ), and
◮
Induction: for all n ∈ S, if P(n), then P(n + 1).
Often people distinguish between “strong” and “weak” mathematical
induction. . .
An example of weak mathematical induction
I’d like you to follow a fairly rigid format in presenting proofs by weak
mathematical induction. Here’s an example.
Claim: For all n ≥ 4, n! > 2n .
Proof by weak mathematical induction on n (n ≥ n0 = 4).
Basis:
n0 = 4
4! = 24 > 16 = 24 .
Induction:
IH: n! > 2n
NTS: (n + 1)! > 2n+1
(n + 1)!
=
>
>
=
n! · (n + 1)
2n · (n + 1)
2n · 2
2n+1
OK, so what is “essential” about this proof? . . .
(def of !)
(IH)
(n ≥ 4)
Format for weak mathematical induction proof
In this class, please use the format of the previous example for proofs by
weak mathematical induction.
Essential elements:
◮
Say what Claim you are proving (the Claim).
◮
Say that the proof is by weak mathematical induction.
◮
Make it clear what the value of n0 is.
◮
Divide the argument into basis and induction cases, labeled as such.
◮
In the induction case, state the induction hypothesis (IH) and what you
need to show (NTS).
◮
Indicate clearly where and how you use the IH.
More weak mathematical induction examples
n
X
Claim: For all n ∈ N,
i =0
i=
n2 + n
.
2
Proof by (weak) mathematical induction on n (n ≥ n0 = 0).
Basis:
n0 = 0
0
X
i =0=
i =0
02 + 0
.
2
Induction:
IH:
NTS:
Pn
i =0
i=
Pn+1
i =0
n2 +n
2
i=
.
(n+1)2 +(n+1)
2
Pn+1
i =0
i
=
=
=
=
=
.
Pn 2 i =0i + n + 1
n +n
+n+1
2
n2 +n+2n+2
2
(n2 +2n+1)+(n+1)
2
(n+1)2 +(n+1)
2
(IH)
Claim: For all n ∈ N, n(n2 + 5) is a multiple of 6.
Proof by (weak) mathematical induction on n (n ≥ n0 = 0).
Basis: n0 = 0
0(02 + 5) = 0 = 6 · 0.
Induction:
IH:
n(n2 + 5) is a multiple of 6.
NTS:
(n + 1)((n + 1)2 + 5) is a multiple of 6.
(n + 1)((n + 1)2 + 5)
=
=
=
=
=
=
(n + 1)(n2 + 2n + 6)
(n + 1)(n2 + 5) + (n + 1)(2n + 1)
n(n2 + 5) + (n2 + 5) + (n + 1)(2n + 1)
n(n2 + 5) + (n2 + 5) + (2n2 + 3n + 1)
n(n2 + 5) + (3n2 + 3n + 6)
n(n2 + 5) + 3(n2 + n) + 6
Now, to show that the rhs sum is a multiple of 6, we show that all three
summands are.
By IH, n(n2 + 5) is a multiple of 6.
Of course 6 is also.
To show that 3(n2 + n) is a multiple of 6, it is enough to show that n2 + n is
even, which follows easily from the fact that n2 + n = n(n + 1) and so is the
product of an odd and an even number.
A harder example of weak mathematical induction
Claim: For any language L (over alphabet A),
if L2 ⊆ L, then L+ ⊆ L.
Let’s prove this by mathematical induction.
Question: Where is the parameter n?
Recall that
L+ =
[
Ln+1 .
n∈N
So we can establish the claim by showing that
if L2 ⊆ L, then for all n ∈ N, Ln+1 ⊆ L.
We’ll need the following easy lemma.
Lemma: For all L1 , L2 , M ⊆ A∗ ,
if L1 ⊆ L2 , then L1 M ⊆ L2 M.
Proof. Assume L1 ⊆ L2 and x ∈ L1 M. [What is our goal?]
So there exist y ∈ L1 and z ∈ M s.t. x = yz.
Since L1 ⊆ L2 and y ∈ L1 , we have y ∈ L2 .
And since y ∈ L2 and z ∈ M, we have x = yz ∈ L2 M.
Lemma: For all L1 , L2 , M ⊆ A∗ , if L1 ⊆ L2 , then L1 M ⊆ L2 M.
Claim: For any language L, if L2 ⊆ L, then L+ ⊆ L.
Proof. Assume that L2 ⊆ L.
We will prove by weak mathematical induction that it follows that,
for all n ∈ N,
Ln+1 ⊆ L ,
S
from which it follows in turn that L+ ⊆ L, since L+ = n∈N Ln+1 .
Basis:
n0 = 0
L0+1 = L1 = L, so L0+1 ⊆ L.
Induction:
IH:
Ln+1 ⊆ L.
NTS:
L(n+1)+1 ⊆ L.
Ln+2
=
⊆
=
⊆
Ln+1 L
LL
L2
L
(def Ln+2 )
(IH, Lemma)
(def L2 )
(assumption)
“Strong” mathematical induction
Let S = {n0 , n0 + 1, n0 + 2, ...} be the set of intergers ≥ n0 .
Let P be a property of integers.
To prove P(n) for all integers n ≥ n0 , by strong mathematical induction, it
suffices to prove that
◮ Basis: P(n0 ), P(n0 + 1), P(n0 + 2), . . . , P(n1 ) (where n1 ≥ n0 ), and
◮
Induction: for all n > n1 , if P(m), for n0 ≤ m ≤ n, then P(n + 1).
Notice that here you get a “stronger” induction hypothesis:
instead of assuming only P(n) in order to prove P(n + 1),
you can assume P(k) for all k ∈ {n0 , n0 + 1, . . . , n}.
That’s why this is called “strong” mathematical induction.
An example of strong mathematical induction
A typical first example involves Fibonacci numbers, defined recursively as:
◮ Basis: F0 = 0 and F1 = 1 (so n0 = 0 and n1 = 1)
◮ Induction: for n ≥ 0, Fn+2 = Fn + Fn+1
“Weak” induction is usually less convenient for proofs about Fn for all n ∈ N,
since Fn+2 is defined in terms of both Fn and Fn+1 .
P
Claim: For every n ∈ N, ni=0 Fi = Fn+2 − 1 .
Proof by strong mathematical induction on n (n ≥ 0). The base cases are
n0 = 0 and n1 = 1
Basis Case 1: n = 0.
P0
i =0
Basis Case 2: n = 1.
P1
i =0
Fi
Fi
=
=
=
=
F0
(F0 + 1) − 1
(F0 + F1 ) − 1
F2 − 1
=
=
=
=
=
F0 + F1
F2
(1 + F2 ) − 1
(F1 + F2 ) − 1
F3 − 1
(defn F1 )
(defn F2 )
(defn F2 )
(defn F1 )
(defn F3 )
Claim: For every n ∈ N,
Pn
i =0
Fi = Fn+2 − 1 .
Induction: n ≥ 2.
Pk
IH: For all k, 0 ≤ k ≤ n,
i =0 Fi = Fk+2 − 1.
Pn+1
NTS:
F
=
F
−
1.
i
n+3
i =0
Pn+1
i =0
Fi
=
=
=
=
=
Pn−1
i =0 Fi + (Fn + Fn+1 )
(Fn+1 − 1) + (Fn + Fn+1 )
(Fn+1 − 1) + Fn+2
(Fn+1 + Fn+2 ) − 1
Fn+3 − 1
(n ≥ 2)
(IH)
(defn Fn+2 , n ≥ 0)
(defn Fn+3 )
Format for strong mathematical induction proof
In this class, please use the format of the previous example for proofs by
strong mathematical induction.
Essential elements:
◮
Say what Claim you are proving (the Claim).
◮
Say that the proof is by strong mathematical induction.
◮
Make it clear what the values of n0 and n1 are (and if there is only one
basis case that n0 = n1 ).
◮
Divide the argument into basis and induction cases, labeled as such.
◮
In the induction case, state the induction hypothesis (IH) and what you
need to show (NTS).
◮
Indicate clearly where and how you use the IH.
Structural induction
We now introduce structural induction ( called well-founded induction in the
text), a powerful method for proving claims about inductively-defined sets. . .
In structural induction,
the structure of the inductive proof
(that every element of a set S has some property)
follows the structure of the inductive definition (of S).
For instance, to prove that every binary tree T (over A) has a property P,
we would simply argue that
◮
P(hi), and
◮
for all binary trees L and R (and all x ∈ A),
if P(L) and P(R), then P(hL, x, Ri).
(Compare the structure of this argument to the structure of the inductive defn
of the set of binary trees over A.)
NOTE: It isn’t necessary to understand the basis of well-founded induction as
presented in the text — it is enough to understand structural induction as
presented in these notes.
Soundness of structural induction
Let S be an inductively-defined set.
Let P(x) be a property of x.
Claim. To show that, for all x ∈ S, P(x), it suffices to show that:
◮
Basis: For all initial elements x of S, P(x).
◮
Induction: For each construction rule using elements x1 , . . . , xk of S to
construct an element x, show that
if P(x1 , ), . . . , P(xk ),
then P(x).
Let’s verify the soundness of this claim. . .
So assume that we can meet the obligations of the proof by structural
induction described above.
What we’ll show is that the set
SP = { x | x ∈ S, P(x) }
satisfies both the basis and induction parts of the inductive definition of S. It
follows that SP is a superset of S (WHY?), and so that, for every x ∈ S, P(x).
Let S be an inductively-defined set.
To prove P(x) for all x ∈ S, it suffices to show that:
◮
◮
Basis: For all initial elements x of S, P(x).
Induction: For each construction rule using elements x1 , . . . , xk of S to
construct an element x, show that
if P(x1 , ), . . . , P(xk ), then P(x).
Assuming that we can meet the proof obligations given above, we can show
that it follows that the set
SP = { x | x ∈ S, P(x) }
satisfies both the basis and induction parts of the inductive definition of S.
[Hence, S ⊆ Sp . Thus, for all x ∈ S, P(x).]
Notice first that the basis part of the structural induction proof establishes that
SP satisfies the basis part of the inductive definition of S.
(That is, all initial elements of S have property P, and so belong to Sp .)
The inductive part of the structural induction proof establishes that,
for any x1 , . . . , xn ∈ Sp ,
if x can be constructed from x1 , . . . , xn (by the inductive definition of S),
then x ∈ SP .
(So SP also satisfies the induction part of the defn of S.)
Structural induction example 1 (Ex. 10 b. p.275)
Let BT be the set of binary trees over A, and consider the function
f : BT → N defined recursively as follows.
f (hi) = 0
1
, if L = R = hi
f (hL, x, Ri) =
f (L) + f (R) , otherwise
Prove: For all T ∈ BT , f (T ) is the number of leaves in T .
By structural induction on T .
Basis: f (hi) = 0, which is the number of leaves in hi.
Induction: L, R ∈ BT , x ∈ A.
IH: f (L) is the number of leaves in L
and f (R) is the number of leaves in R.
NTS: f (hL, x, Ri) is the number of leaves in hL, x, Ri.
Subcase 1: L = R = hi
In this subcase, the number of leaves in hL, x, Ri is 1 = f (hL, x, Ri).
Subcase 2: Otherwise.
Here, the number of leaves in hL, x, Ri is the number of leaves in L plus the
number of leaves in R, which, by IH, is exactly f (L) + f (R) = f (hL, x, Ri).
Structural induction example 2 (not in text)
Claim: For every binary tree T over A,
2depth(T )+1 ≥ nodes(T ) + 1 .
Proof by structural induction
Basis:
(on the inductive definition of BT over A).
2depth(hi)+1 = 1 = nodes(hi) + 1.
Induction: L, R ∈ BT , and x ∈ A.
IH: 2depth(L)+1 ≥ nodes(L) + 1 and 2depth(R)+1 ≥ nodes(R) + 1.
NTS: 2depth(hL,x,Ri)+1 ≥ nodes(hL, x, Ri) + 1 .
2depth(hL,x,Ri)+1
=
=
=
=
≥
=
≥
=
=
2 · 2depth(hL,x,Ri)
2 · 2max(depth(L),depth(R))+1
2 · 2max(depth(L)+1,depth(R)+1)
2 · max(2depth(L)+1 , 2depth(R)+1 )
2 · max(nodes(L) + 1, nodes(R) + 1)
(2 · max(nodes(L), nodes(R))) + 2
(nodes(L) + nodes(R)) + 2
(nodes(L) + nodes(R) + 1) + 1
nodes(hL, x, Ri) + 1
(defn depth)
(IH)
(defn nodes)
Structural induction example 3 (Ex. 5 a. p.174)
Recall: For any alphabet A, A∗ can be defined inductively as follows:
Λ ∈ A∗ .
◮
Basis:
◮
Induction:
If x ∈ A∗ and a ∈ A, then ax ∈ A∗ .
And here’s the recursively defined “reversing” function we’ve discussed before:
Take r : A∗ → A∗ s.t.
r (Λ)
r (ax)
=
=
Λ
r (x)a
(x ∈ A∗ , a ∈ A)
Claim: For all x, y ∈ A∗ , r (xy ) = r (y )r (x).
Hmmm, should we induct on x or on y ?
You can try it both ways. One way is much nicer. . .
For any alphabet A, A∗ is defined inductively as follows:
◮
◮
Basis: Λ ∈ A∗ .
Induction: If x ∈ A∗ and a ∈ A, then ax ∈ A∗ .
Take r : A∗ → A∗ s.t.
r (Λ)
r (ax)
Claim:
=
=
Λ
r (x)a
(x ∈ A∗ , a ∈ A)
For all x, y ∈ A∗ , r (xy ) = r (y )r (x).
Take an arbitrary y ∈ A∗ . We’ll prove, by structural induction on the inductive
defn of A∗ , that for all x ∈ A∗ , r (xy ) = r (y )r (x) .
Basis:
r (Λy )
=
=
=
r (y )
r (y )Λ
r (y )r (Λ)
=
=
=
r (xy )a
r (y )r (x)a
r (y )r (ax)
(defn r )
∗
Induction: x ∈ A , a ∈ A
IH: r (xy ) = r (y )r (x)
NTS: r (axy ) = r (y )r (ax)
r (axy )
(defn r )
(IH)
(defn r )
Format for structural induction proof
In this class, please use the format of the previous example for proofs by
structural induction.
Essential elements:
◮
Identify the Claim you are proving by structural induction. It should be a
claim about all elements of some inductively-defined set.
◮
Say it is proof by structural induction, and clearly indicate the inductive
definition on which the proof is based.
◮
Clearly label basis and induction cases as such.
◮
In the induction case, state IH and NTS.
◮
Clearly indicate where and how the IH is used.
Proving correctness of the insert function (Ex. 12 b. p275)
Recall the inductive definition of lists(A):
◮
Basis: hi ∈ lists(A).
◮
Induction: If x ∈ A and L ∈ lists(A), then x :: L ∈ lists(A).
Recall also the following recursive definition of a function from N × lists(N) to
lists(N) (2.36 page 163):
insert(x, hi) = hxi
x :: y :: L
, if x ≤ y
insert(x, y :: L) =
y :: insert(x, L) , otherwise
Claim: For all L ∈ lists(N), for all x ∈ N, if L is sorted,
then insert(x, L) is a sorted permutation of x :: L.
We previously discussed the idea for an argument proving this claim. The idea
of that argument can be made precise using structural induction on L.
Consider any x ∈ N.
We’ll show that, for all L ∈ lists(N), if L is sorted,
then insert(x, L) is a sorted permutation of x :: L.
insert(x, hi)
=
insert(x, y :: L)
=
hxi
x :: y :: L
y :: insert(x, L)
, if x ≤ y
, otherwise
We’ll show that, for all L ∈ lists(N), if L is sorted, then insert(x, L) is a sorted
permutation of x :: L (Exercise 12 b. page 275).
Proof by structural induction on L.
Basis: insert(x, hi) = hxi, which is a sorted permutation of x :: hi = hxi.
Induction: L ∈ lists(N), y ∈ N.
IH: If L is sorted, then insert(x, L) is a sorted permutation of x :: L.
NTS: If y :: L is sorted, then insert(x, y :: L) is a sorted permutation of
x :: y :: L.
Assume y :: L is sorted. Consider two cases:
Case 1: x ≤ y .
Then insert(x, y :: L) = x :: y :: L. Since y :: L is sorted and x ≤ y , x :: y :: L
is also sorted. And of course x :: y :: L is a permutation of x :: y :: L.
insert(x, hi)
=
insert(x, y :: L)
=
hxi
x :: y :: L
y :: insert(x, L)
, if x ≤ y
, otherwise
Case 2: x > y .
So insert(x, y :: L) = y :: insert(x, L).
Since y :: L is sorted (by assumption), so is L.
So by IH we can conclude that insert(x, L) is a sorted permutation of x :: L.
It follows easily that y :: insert(x, L) is a permutation of x :: y :: L.
And to see that y :: insert(x, L) is sorted, it is enough to check that every
element of insert(x, L) is at least y , since we already know that insert(x, L) is
sorted. This is easy to check. . .
Notice first that since y :: L is sorted, every element of L is at least y . We also
know that x is at least y .
So every element of x :: L is at least y , and since insert(x, L) is a permutation
of x :: L, we can conclude that every element of insert(x, L) is at least y .
Proving correctness of the sort function (Ex. 12 a. p275)
Recall the following recursively-defined function, from lists(N) to lists(N):
sort(hi)
=
hi
sort(x :: L)
=
insert(x, sort(L))
Given the previous result for insert, we can prove that
for all L ∈ lists(N), sort(L) is a sorted permutation of L.
By structural induction on (the inductive definition of) L.
Basis: sort(hi) = hi, which is a sorted permutation of itself.
Induction: L ∈ lists(N), x ∈ N.
IH: sort(L) is a sorted permutation of L.
NTS: sort(x :: L) is a sorted permutation of x :: L.
Notice that sort(x :: L) = insert(x, sort(L)), and it remains only to show that
insert(x, sort(L)) is a sorted permutation of x :: L. By IH, sort(L) is a sorted
permutation of L. Since sort(L) is sorted, we know by the result from the
previous problem that insert(x, sort(L)) is a sorted permutation of x :: sort(L),
and it remains only to observe that x :: sort(L) is in turn a permutation of
x :: L, which is true since sort(L) is a permutation of L.
Remarks on these last two correctness proofs
The previous two problems are similar to Exercise 12, Section 4.4 in the
textbook. But the textbook’s version of these problems does not give an
adequate account of “correctness” of the sort and insert functions. Let me
elaborate. . .
About insert, the textbook asks you to prove that insert(x, L) is sorted if L is.
If you try to do that in a straightforward fashion by structural induction on L,
you will likely fail. (Or at least you should!)
Why? Because at a key point in the proof, you will need to conclude from the
assumption (via IH) that insert(x, L) is sorted that y :: insert(x, L) is also
sorted. But it doesn’t follow that easily. As it turns out, you need to know
more about insert(x, L) — in particular, you need to know that every element
of insert(x, L) is at least y , and although this is indeed the case, it does not
follow from the IH.
This is one reason why we needed to talk about permutations. . .
Interestingly, the insert function does have the correctness property that the
textbook mentions:
if L is sorted, then insert(x, L) is sorted.
Indeed, this is an immediate consequence of the result we proved above.
There we considered the stronger claim that
if L is sorted,
then insert(x, L) is a sorted permutation of x :: L.
This stronger property is better for proof by induction because we get to make
use of the correspondingly stronger IH.
But the textbook’s version of correctness for the insert function is inappropriate
for another reason.
It is intuitively wrong, because it provides an inadequate characterization of
correctness for an insert function.
Can you see why?
To see the difficulty more starkly, suppose that we define a much simpler
version of the function:
for all x ∈ N and L ∈ lists(N),
insert(x, L) = hi .
This constant function is intuitively wrong; nonetheless, it has the property
that the textbook uses to characterize correctness.
That is, it satisfies the claim of Exercise 12 b, that if L is sorted, then so is
insert(x, L).
By the way, you can see that similarly it would not be sufficient to say that
what we require of sort(L) is that it be sorted, for every L ∈ lists(N).
And yet that is all that is proved in the textbook’s solution to Exercise 12 a!
A general conclusion to draw from this discussion is that it is difficult to give
adequate specifications concerning correctness of functions (or, say, software).
It is generally hard to make our requirements sufficiently explicit. (That is not
to say that it isn’t worth doing!)
Another, more technical conclusion is that in proofs by induction, it is often
the case that stronger claims are easier to prove (because you get a
stronger induction hypothesis).