Reading and Writing
Mathematical Proofs
Spring 2016
Lecture 3: Induction
Previously on:
Reading and Writing
Mathematical Proofs
Basic Example
Theorem
For any teletubby, I do not like it.
Case 1 (Tinky-Winky):
Let’s prove P ⇒ Q
P1 ⇒ Q
…
…
Case 4 (Dispy):
P4 ⇒ Q
…
As any teletubby must fall into these categories
(by definition), I do not like any teletubby.
P ⇒ P1 ⋁ P2 ⋁ P3⋁ P4
Practice 2
Proof
Consider any vertex x ϵ V. We consider the following cases:
Case (1): x is connected to at least three other vertices in G
Let these vertices be u, v, and w. We again consider two cases:
v
v
u
u
x
w
x
w
Case (2): x is connected to at most two other vertices of G
Let u, v, and w be three vertices not connected to x. More cases:
v
u
v
x
w
u
x
w
Basic Example
Theorem
I never leave my house without my Ferrari
Proof
For sake of contradiction,
assume I did leave my house without my Ferrari.
But then I would not look cool (by Lemma X).
I am very cool
(by Axiom Y).
Contradiction, thus the assumption must be false.
Hence, I never leave my house without my Ferrari.
Example
Theorem (pigeonhole principle)
If 𝑛
are put into 𝑚 containers and 𝑛 > 𝑚, then there must
exist a container that contains more than one
.
Tips and Tricks
Finding a Loop Invariant
What do you want to know at the end?
Loop Invariant (generally) proves something that is growing
I.e., 𝐴[1. . 𝑖 − 1]
Think about a specific iteration.
What do you know.
Which indices do you need at the start and end.
Loop Invariant
At the start of iteration 𝑖, 𝑠𝑢𝑚 contains the sum of 𝐴[1. . 𝑖 − 1].
Today on:
Basic Proving Techniques II
The proving cookbook…
Next time!
“Mono-chromatic horses!”
“Matching match sticks”
IH
“Circles around trees!”
IH
Running time
But First…
Time
Algorithm LargeEven(A)
large = 0
O(1)
for i = 1 to n
O(n)
if A[i] > large and A[i] is even
then large = A[i]
O(1)
O(1)
Total time = O(1) + O(n) + O(1) + O(1) = O(n)
This claims that line 1,3, and 4 are executed O(1) times.
That is not true!
Time
Algorithm LargeEven(A)
large = 0
for i = 1 to n
if A[i] > large and A[i] is even
then large = A[i]
Total time =
O(1) + 𝑛𝑖=1(𝑂 1 + 𝑂(1)) =
O(1) + 𝑛𝑖=1(𝑂 1 ) =
O(1) + O(n) =
O(n)
O(1)
𝑛
𝑖=1
O(1)
O(1)
Loop Invariant
MaxSegSum…
Example
MaxSegSum(A, n)
Invariant?
r = -∞
s = -∞
for h = 1 to n
do if s > 0
then s = s + A[h]
else s = A[h]
r = max(r, s)
return r
4
-6
2
5
-3
7
r=-∞
r=4
r=4
r=4
r=7
r=7
r=11
s=-∞
s=4
s=-2
s=2
s=7
s=4
s=11
Theorem
The above algorithm computes the maximum over 1 ≤ 𝑖 ≤ 𝑗 ≤ 𝑛
of the sum of elements in 𝐴[𝑖 … 𝑗]
Example
MaxSegSum(A, n)
Invariant?
r = -∞
s = -∞
for h = 1 to n
do if s > 0
then s = s + A[h]
else s = A[h]
r = max(r, s)
return r
4
-6
2
5
-3
7
r=-∞
r=4
r=4
r=4
r=7
r=7
r=11
s=-∞
s=4
s=-2
s=2
s=7
s=4
s=11
Theorem
The above algorithm computes the maxsum of 𝐴[𝑖 … 𝑗] over
1 ≤ 𝑖 ≤ 𝑗 ≤ 𝑛.
Example
MaxSegSum(A, n)
Invariant?
r = -∞
s = -∞
for h = 1 to n
do if s > 0
then s = s + A[h]
else s = A[h]
r = max(r, s)
return r
4
-6
2
5
-3
7
r=-∞
r=4
r=4
r=4
r=7
r=7
r=11
s=-∞
s=4
s=-2
s=2
s=7
s=4
s=11
Invariant
At the start of iteration ℎ, 𝑟 contains the maxsum of 𝐴[𝑖 … 𝑗] for 1 ≤ 𝑖 ≤ 𝑗 ≤ ℎ − 1
and 𝑠 contains the maxsum of 𝐴[𝑘 … ℎ − 1] for 1 ≤ 𝑘 ≤ ℎ − 1.
Example
Invariant
At the start of iteration ℎ,
𝑟 contains the maxsum of 𝐴[𝑖 … 𝑗] for 1 ≤ 𝑖 ≤ 𝑗 ≤ ℎ − 1, and
𝑠 contains the maxsum of 𝐴[𝑘 … ℎ − 1] for 1 ≤ 𝑘 ≤ ℎ − 1.
Initialization
At the start of the loop, ℎ = 1. Hence, both 1 ≤ 𝑖 ≤ 𝑗 ≤ ℎ − 1 and
1 ≤ 𝑖 ≤ ℎ − 1 are empty domains. The maximum over an empty domain
is −∞, so the invariant is correctly initialized.
Example
Invariant
At the start of iteration ℎ,
𝑟 contains the maxsum of 𝐴[𝑖 … 𝑗] for 1 ≤ 𝑖 ≤ 𝑗 ≤ ℎ − 1, and
𝑠 contains the maxsum of 𝐴[𝑘 … ℎ − 1] for 1 ≤ 𝑘 ≤ ℎ − 1.
Maintenance
Before each iteration the invariant holds. We need to show that after
iteration ℎ, 𝑟 contains the maxsum of 𝐴[𝑖 … 𝑗] over 1 ≤ 𝑖 ≤ 𝑗 ≤ ℎ, and 𝑠
contains the maxsum of 𝐴[𝑖 … ℎ] over 1 ≤ 𝑖 ≤ ℎ. We consider two cases:
Case (1): 𝑠 > 0
Case (2): 𝑠 ≤ 0
Example
Invariant
At the start of iteration ℎ,
𝑟 contains the maxsum of 𝐴[𝑖 … 𝑗] for 1 ≤ 𝑖 ≤ 𝑗 ≤ ℎ − 1, and
𝑠 contains the maxsum of 𝐴[𝑘 … ℎ − 1] for 1 ≤ 𝑘 ≤ ℎ − 1.
Maintenance
Before each iteration the invariant holds. We need to show that after
iteration ℎ, 𝑟 contains the maxsum of 𝐴[𝑖 … 𝑗] over 1 ≤ 𝑖 ≤ 𝑗 ≤ ℎ, and 𝑠
contains the maxsum of 𝐴[𝑖 … ℎ] over 1 ≤ 𝑖 ≤ ℎ. We consider two cases:
Case (1): 𝑠 > 0
Let the maxsum of 𝐴[𝑖 … ℎ] be obtained for 𝑖 = 𝑖 ∗ .
Note that 𝑖 ∗ ≠ ℎ, as the maxsum of 𝐴[𝑖 … ℎ − 1], stored in 𝑠 by invariant
and positive, can be added to 𝐴[ℎ … ℎ] to get a larger sum.
So, 𝑖 ∗ < ℎ and the maxsum of 𝐴[𝑖 … ℎ] is 𝐴[ℎ] plus the maxsum of
𝐴[𝑖 … ℎ − 1]. By invariant, 𝑠 contains the maxsum of 𝐴[𝑖 … ℎ − 1], so
𝐴[ℎ] should indeed be added to 𝑠.
Example
Invariant
At the start of iteration ℎ,
𝑟 contains the maxsum of 𝐴[𝑖 … 𝑗] for 1 ≤ 𝑖 ≤ 𝑗 ≤ ℎ − 1, and
𝑠 contains the maxsum of 𝐴[𝑘 … ℎ − 1] for 1 ≤ 𝑘 ≤ ℎ − 1.
Maintenance
Before each iteration the invariant holds. We need to show that after
iteration ℎ, 𝑟 contains the maxsum of 𝐴[𝑖 … 𝑗] over 1 ≤ 𝑖 ≤ 𝑗 ≤ ℎ, and 𝑠
contains the maxsum of 𝐴[𝑖 … ℎ] over 1 ≤ 𝑖 ≤ ℎ. We consider two cases:
Case (2): 𝑠 ≤ 0
Again, let the maxsum of 𝐴[𝑖 … ℎ] be obtained for 𝑖 = 𝑖 ∗ .
If 𝑖 ∗ < ℎ, then the sum of 𝐴[𝑖 ∗ … ℎ − 1] is at most 𝑠 by invariant, and
thus at most zero.
As a result, 𝐴[𝑖 ∗ … ℎ] is at most 𝐴[ℎ], which can be achieved by 𝑖 ∗ = ℎ.
Thus 𝑠 should indeed be set to 𝐴[ℎ].
Example
Invariant
At the start of iteration ℎ,
𝑟 contains the maxsum of 𝐴[𝑖 … 𝑗] for 1 ≤ 𝑖 ≤ 𝑗 ≤ ℎ − 1, and
𝑠 contains the maxsum of 𝐴[𝑘 … ℎ − 1] for 1 ≤ 𝑘 ≤ ℎ − 1.
Maintenance
Before each iteration the invariant holds. We need to show that after
iteration ℎ, 𝑟 contains the maxsum of 𝐴[𝑖 … 𝑗] over 1 ≤ 𝑖 ≤ 𝑗 ≤ ℎ, and 𝑠
contains the maxsum of 𝐴[𝑖 … ℎ] over 1 ≤ 𝑖 ≤ ℎ. We consider two cases:
Case (1): 𝑠 > 0 […..]
Case (2): 𝑠 ≤ 0 […..]
After the if-statement, 𝑠 contains the maxsum of 𝐴[𝑖 … ℎ] over
1 ≤ 𝑖 ≤ ℎ. Let the maxsum of 𝐴[𝑖 … 𝑗] be obtained for 𝑗 = 𝑗 ∗ . If 𝑗 ∗ = ℎ,
then s must contain the maxsum of 𝐴[𝑖 … 𝑗]. Otherwise 𝑗 ∗ < ℎ, and 𝑟 must
already contain the maxsum of 𝐴[𝑖 … 𝑗] by invariant. Thus, the maximum of
𝑟 and 𝑠 will be the maxsum of 𝐴[𝑖 … 𝑗] over 1 ≤ 𝑖 ≤ 𝑗 ≤ ℎ.
Example
Invariant
At the start of iteration ℎ,
𝑟 contains the maxsum of 𝐴[𝑖 … 𝑗] for 1 ≤ 𝑖 ≤ 𝑗 ≤ ℎ − 1, and
𝑠 contains the maxsum of 𝐴[𝑘 … ℎ − 1] for 1 ≤ 𝑘 ≤ ℎ − 1.
Termination
After the loop, ℎ = 𝑛 + 1. By invariant, 𝑟 contains the maxsum of 𝐴[𝑖 … 𝑗]
over 1 ≤ 𝑖 ≤ 𝑗 ≤ ℎ − 1 = 𝑛. Thus, 𝑟 contains the correct value.
Nested loops
Loop in loop…
Nested Loops
How to prove nested loops?
 Simply follow the rules
{𝑃𝑟𝑒1}
for i = 1 to n
do {𝐼𝑛𝑣1}
 Outer invariant should establish 𝑃𝑟𝑒2
 Or should initialize 𝐼𝑛𝑣2
…..
{𝑃𝑟𝑒2}
for j = i+1 to n
do {𝐼𝑛𝑣2}
 Termination of inner loop can be used
to prove maintenance outer loop
….
{𝑃𝑜𝑠𝑡2}
{𝑃𝑜𝑠𝑡1}
 Only 𝑃𝑟𝑒1 and 𝑃𝑜𝑠𝑡1 are given
 The rest must be chosen
Practice
Loop Invariants…
Practice
Prove using loop invariant that…
pro is the product of all elements in 𝐴 after the loop.
pro = 1
for i = 1 to A.length
do pro = pro * A[i]
Practice
pro = 1
for i = 1 to A.length
do pro = pro * A[i]
Loop Invariant:
At the start of iteration 𝑖, 𝑝𝑟𝑜 contains the product of all items in 𝐴[1. . 𝑖 − 1].
Initialization:
At the start, 𝑖 = 1. By the loop invariant 𝑝𝑟𝑜 equals the product of all items in
𝐴[1. . 𝑖 − 1] = ∅. As 𝑝𝑟𝑜 = 1, this is (by mathematical definition) correct.
Practice
pro = 1
for i = 1 to A.length
do pro = pro * A[i]
Maintenance:
Assume that the loop invariant holds at the start of iteration 𝑖.
Then it must be that 𝑝𝑟𝑜 = 𝑃𝑟𝑜𝑑𝑢𝑐𝑡(𝐴 1. . 𝑖 − 1 ).
In iteration 𝑖, 𝑝𝑟𝑜 is multiplied with 𝐴[1], so 𝑝𝑟𝑜 = 𝑃𝑟𝑜𝑑𝑢𝑐𝑡(𝐴[1. . 𝑖 −
1]) ∗ 𝐴[𝑖] = 𝑃𝑟𝑜𝑑𝑢𝑐𝑡(𝐴[1. . 𝑖 − 1]).
Thus at the start of iteration 𝑖 + 1, 𝑝𝑟𝑜 = 𝑃𝑟𝑜𝑑𝑢𝑐𝑡(𝐴[1. . 𝑖]) which is
correct.
Termination:
At termination 𝑖 = 𝐴. 𝑙𝑒𝑛𝑔𝑡ℎ + 1. So by LI 𝑝𝑟𝑜 =
𝑃𝑟𝑜𝑑𝑢𝑐𝑡(𝐴[1. . 𝐴. 𝑙𝑒𝑛𝑔𝑡ℎ]) which is exactly what we wanted.
Practice
pro = 1
for i = 1 to A.length
do pro = pro * A[i]
Maintenance:
Assume that the loop invariant holds at the start of iteration 𝑖.
Then it must be that 𝑝𝑟𝑜 = 𝑃𝑟𝑜𝑑𝑢𝑐𝑡(𝐴 1. . 𝑖 − 1 ).
In iteration 𝑖, 𝑝𝑟𝑜 is multiplied with 𝐴[1], so 𝑝𝑟𝑜 = 𝑃𝑟𝑜𝑑𝑢𝑐𝑡(𝐴[1. . 𝑖 −
1]) ∗ 𝐴[𝑖] = 𝑃𝑟𝑜𝑑𝑢𝑐𝑡(𝐴[1. . 𝑖 − 1]).
Thus at the start of iteration 𝑖 + 1, 𝑝𝑟𝑜 = 𝑃𝑟𝑜𝑑𝑢𝑐𝑡(𝐴[1. . 𝑖]) which is
correct.
Termination:
At termination 𝑖 = 𝐴. 𝑙𝑒𝑛𝑔𝑡ℎ + 1. So by LI 𝑝𝑟𝑜 =
𝑃𝑟𝑜𝑑𝑢𝑐𝑡(𝐴[1. . 𝐴. 𝑙𝑒𝑛𝑔𝑡ℎ]) which is exactly what we wanted.
Practice
Prove using loop invariant that… 𝑦 = 𝑐 after the loop.
x=c
y=0
while x > 0
do x--;
y++;
Practice
Loop Invariant:
At the start of iteration, 𝑥 + 𝑦 = 𝑐.
x=c
y=0
while x > 0
Initialization:
At the start, x = c and 𝑦 = 0,
so 𝑥 + 𝑦 = 𝑐 which is correct.
do x--;
y++;
Practice
Maintenance:
Assume that the loop invariance holds at the start
of loop 𝑖. Then 𝑥 + 𝑦 = 𝑐. Let 𝑥’ and 𝑦’ be the
Values of x and y at the end of the loop.
We know 𝑥’ = 𝑥 – 1 and 𝑦’ = 𝑦 + 1.
But then at the end of the loop it holds that
𝑥’ + 𝑦’ = 𝑥 − 1 + 𝑦 + 1 = 𝑥 + 𝑦 = 𝑐.
Thus the loop invariant is maintained.
x=c
y=0
while x > 0
do x--;
y++;
Termination:
At termination, x = 0. By the loop invariant we know 𝑥 + 𝑦 = 𝑐.
Combining both statements gives 𝑦 = 𝑐.
Practice
Prove using loop invariant that ….?
for i = 1 to A.length
do if not parity(i) = parity(A[i])
A[i] = A[i] + 1
Practice
for i = 1 to A.length
do if not parity(i) = parity(A[i])
Loop Invariant:
A[i] = A[i] + 1
At the start of iteration 𝑖,
all values in 𝐴[1. . 𝑖 − 1] at an even index are even
and all values in 𝐴[1. . 𝑖 − 1] at an odd index are odd.
Initialization:
At the start, i = 1. So by loop invariant all values in 𝐴[1. . 0] match the parity of
their index. As 𝐴[1. . 0] = ∅, this is trivially true.
Practice
for i = 1 to A.length
do if not parity(i) = parity(A[i])
Maintenance:
A[i] = A[i] + 1
Assume that the LI holds at the start of iteration 𝑖.
Then all values in 𝐴[1. . 𝑖 − 1] match the parity of their index.
Case 1) The parity of 𝐴[𝑖] equals the parity of 𝑖.
As we do not change the parity of 𝐴[𝑖], trivially after the iteration the
LI holds for 𝐴[1. . 𝑖 − 1] and also for 𝐴[𝑖], so for 𝐴[1. . 𝑖].
Case 2) The parity of 𝐴[𝑖] is not equal to the parity of 𝑖.
As we increase 𝐴[𝑖] by one, it’s parity will change. Thus at the end of
the loop the parity of 𝐴[𝑖] must be equal to the parity of 𝑖.
Hence, the LI holds for 𝐴[1. . 𝑖].
As both cases hold, the LI is maintained.
Termination:
At termination, 𝑖 = 𝐴. 𝑙𝑒𝑛𝑔𝑡ℎ. So according to the loop invariant all values in
𝐴[1. . 𝐴. 𝑙𝑒𝑛𝑔𝑡ℎ] match the parity of their index.
More Induction
Proving trees, recursion, …
Mathematical Induction
The basic induction…
Recurring structures
Many algorithms and (data)structures have recurrences
sum = 0
 Loop
for i = 1 to A.length
do sum = sum + A[i]
 List
 Tree
Induction
Induction Hypothesis:
If 𝑛 dominos are placed in a row and I push the first; they all fall.
Once again
The Base Idea of Induction
P(1)
P(2)
P(3)
Base Case
One (or more) very simple cases that we can trivially proof.
Induction Hypothesis
The statement that we want to prove (for any 𝑛).
Step
Prove that if the statement holds for a small instance, it must also hold for
a larger instance.
Induction issues
Be careful with induction!
Theorem
In every set of 𝑛 ≥ 1 horses, all horses have the same color
Proof
We use induction on 𝑛.
Base case (𝑛 = 1):
There is only one horse, so it must be true. The IH holds.
IH: …
Step (𝑛 ≥ 1):
What happens if 𝑛 = 1?
Suppose that in every set of 𝑛 horses, all horses have the same color (IH).
We need to show that any set of 𝑛 + 1 horses share the same color.
By the IH, the first 𝑛 horses have the same color. Similarly, by the IH,
the last 𝑛 horses have the same color. Thus all horses have the same
color. □
Induction issues
Be careful with induction!
Theorem
In every set of 𝑛 ≥ 1 horses,
all horses have the same color
We are missing P(1) ⇒ P(2)
This breaks everything!
We could additionally prove P(2) as extra base case.
But we can’t…
Step argument must work for all 𝑛 ≥ 1 (or 𝑛 ≥ 𝑐)!
Otherwise these cases must be proved as extra base cases
Induction
Types of induction
Mathematical induction
 P(1)
 P(n) ⇒ P(n+1)
Strong induction
 P(1)
 P(1) ⋀ … ⋀ P(n) ⇒ P(n+1)
Structural induction
 …
Well-founded induction
 …
P(1)
P(2)
P(3)
Recursion
Recursion, see Recursion…
Recursion
Recursion
Recursion
PHP
PHP Hypertext Preprocessor
Wine
Wine is not an emulator
GNU
GNU's Not Unix
Simple recursion
Fibonacci
f(0) = 0
f(1) = 1
f(n) = f(n-2) + f(n-1)
0 1 1 2 3 5 8 13 21 ….
Simple recursion
Fibonacci
algorithm fib(n)
if (n < 2)
f(0) = 0
f(1) = 1
f(n) = f(n-2) + f(n-1)
return n
a=0
b=1
0 1 1 2 3 5 8 13 21 ….
while (n > 1)
tmp = a
a=b
b = b + tmp
n=n-1
return b
Simple recursion
Fibonacci
f(0) = 0
f(1) = 1
f(n) = f(n-2) + f(n-1)
algorithm fib(n)
if (n < 2)
return n
else
return fib(n-2) + fib(n-1)
0 1 1 2 3 5 8 13 21 ….
MergeSort
MergeSort(A)
// divide-and-conquer algorithm that sorts array A[1..n]
1. if A.length == 1
2.
then compute Sol (the solution for A) brute-force
3.
else
4.
split A in 2 non-empty subsets A1 and A2
5.
Sol1 = MergeSort(A1)
6.
Sol2 = MergeSort(A2)
7.
compute Sol (the solution for A) from Sol1 en Sol2
MergeSort
3 14 1 28 17 8 21 7 4 35
1 3 4 7 8 14 17 21 28 35
3
3 14 1 28 17
8 21 7 4 35
1 3 14 17 28
4 7 8 21 35
3 14
1 28 17
3 14
1 17 28
14
Simple recursion
Fibonacci
f(0) = 0
f(1) = 1
f(n) = f(n-2) + f(n-1)
algorithm fib(n)
if (n < 2)
return n
else
return fib(n-2) + fib(n-1)
0 1 1 2 3 5 8 13 21 ….
Simple recursion
Induction Hypothesis
Algorithm fib returns the nth
Fibonacci number.
Base Case(s)
n = 0:
The 0th Fibonacci number is 0.
As n = 0 < 2, the algorithm also
return 0. So the IH holds for n = 0.
n = 1:
…..
algorithm fib(n)
if (n < 2)
return n
else
return fib(n-2) + fib(n-1)
Simple recursion
Induction Hypothesis
Algorithm fib returns the nth
Fibonacci number.
Step (n ≥ 1):
Assume the IH holds for n, we
want to prove it also holds for n+1.
As 𝑛 + 1 ≥ 2, we are in the else.
fib(n+1) = fib((n+1)-2) + fib((n+1)-2)
fib(n+1) = fib(n-1) + fib(n)
Problem! :’-(
We have only assumed the LI
holds for n, not for n-1.
algorithm fib(n)
if (n < 2)
return n
else
return fib(n-2) + fib(n-1)
Strong Induction
Using all smaller building blocks…
Strong Induction
Normally we must prove P(𝑛 + 1) using P(𝑛)
What if we need P(𝑖) (𝑖 < 𝑛) to prove P(𝑛 + 1)?
After all, P(𝑖) should also already hold
Instead of P(𝑛) ⇒ P(𝑛 + 1) we prove P(1) ⋀ … ⋀ P(𝑛) ⇒ P(𝑛 + 1)
More information in the premise!
Still correct: strong induction
Not always useful, but sometimes very convenient…
Mathematical Induction
P(1)
P(n) ⇒ P(n+1)
P(1)
P(2)
P(3)
Strong Induction
P(1)
P(1) ⋀ … ⋀ P(n) ⇒ P(n+1)
P(1)
P(2)
P(n+1)
P(3)
….
P(n)
Strong Induction
Induction Hypothesis
Algorithm fib returns the nth
Fibonacci number.
Base Case(s)
n = 0:
The 0th Fibonacci number is 0.
As n = 0 < 2, the algorithm also
return 0. So the IH holds for n = 0.
n = 1:
…..
algorithm fib(n)
if (n < 2)
return n
else
return fib(n-2) + fib(n-1)
Strong Induction
Induction Hypothesis
Algorithm fib returns the nth
Fibonacci number.
Step (n ≥ 1):
Assume the IH holds for any m ≤
𝑛, we want to prove it also holds
for n+1.
As 𝑛 + 1 ≥ 2, we are in the else.
So fib(n+1) = fib(n-1) + fib(n).
By IH, fib(n-1) is the (n-1)th
Fibonacci number and fib(n) the nth
Fibonacci number. But then adding
fib(n-1) and fib(n) will give the
(n+1)st Fibonacci number. So the
IH is correct.
algorithm fib(n)
if (n < 2)
return n
else
return fib(n-2) + fib(n-1)
Strong Induction
Theorem (Nim)
If the two piles contain the same number of matches at the start of the
game, then the second player can always win.
Strong Induction
Theorem (Nim)
If the two piles contain the same number of matches at the start of the
game, then the second player can always win.
Player 1
Player 2
Strong Induction
Theorem (Nim)
If the two piles contain the same number of matches at the start of the
game, then the second player can always win.
Proof
We use strong induction on 𝑛.
IH: “If the two piles both contain 𝑛 matches at the … always win”
Base Case (𝑛 = 1):
The first player only has one option, emptying one of the piles. The
second player can empty the second pile and, thus, wins.
Step (𝑛 ≥ 1):
Assume the second player can always win if there are two piles with 𝑘
matchsticks each, for 1 ≤ 𝑘 ≤ 𝑛. (IH)
We prove the IH for two piles with 𝑛 + 1 matchsticks each. Assume
w.l.o.g. that player 1 takes 𝑚 ≥ 1 matchsticks from the first pile.
The second player can then always take 𝑚 matchsticks from the other
pile. We are now left with two piles with both 𝑛 + 1 − 𝑚 ≤ 𝑛 matchsticks.
By IH, player two can always win from this setting. □
Strong Induction
Theorem (Nim)
If the two piles contain the same number of matches at the start of the
game, then the second player can always win.
Proof
We use strong induction on 𝑛.
IH: “If the two piles both contain 𝑛 matches at the … always win”
Base Case (𝑛 = 1):
The first player only has one option, emptying one of the piles. The
second player can empty the second pile and, thus, wins.
Step (𝑛 ≥ 1):
Assume the second player can always win if there are two piles with 𝑘
matchsticks each, for 1 ≤ 𝑘 ≤ 𝑛. (IH)
We prove the IH for two piles with 𝑛 + 1 matchsticks each. Assume
w.l.o.g. that player 1 takes 𝑚 ≥ 1 matchsticks from the first pile.
The second player can then always take 𝑚 matchsticks from the other
pile. We are now left with two piles with both 𝑛 + 1 − 𝑚 ≤ 𝑛 matchsticks.
By IH, player two can always win from this setting. □
Problem?
Theorem
With 4-cent and 5-cent coins we can make any amount 𝑛 ≥ 12
Proof
We use strong induction on 𝑛.
Base case (𝑛 = 12):
We can use three 4-cent coins.
IH: …
Step (𝑛 ≥ 12):
Suppose we can make any amount 𝑖 with 4-cent and 5-cent coins for
12 ≤ 𝑖 ≤ 𝑛. We need to show that we can make the amount 𝑛 + 1.
If we use one 4-cent coin, then the remainder is 𝑛 – 3. By the IH we
can make the amount 𝑛 – 3, so that means we can make 𝑛 + 1. □
Can we apply the IH to 𝑛 – 3? For 𝑛 = 12, 𝑛 – 3 = 9 and we haven’t
proved anything for 𝑛 = 9! The step only works for 𝑛 ≥ 15. We
need more base cases!
Problem?
Theorem
With 4-cent and 5-cent coins we can make any amount 𝑛 ≥ 12
Proof
We use strong induction on 𝑛.
Base case (12 ≤ 𝑛 ≤ 15):
(𝑛
(𝑛
(𝑛
(𝑛
=
=
=
=
12): three 4-cent coins.
13): two 4-cent coins and one 5-cent coin.
14): one 4-cent coin and two 5-cent coins.
15): three 5-cent coins.
IH: …
Step (𝑛 ≥ 15):
Suppose we can make any amount 𝑖 with 4-cent and 5-cent coins for
12 ≤ 𝑖 ≤ 𝑛. We need to show that we can make the amount 𝑛 + 1.
If we use one 4-cent coin, then the remainder is 𝑛 – 3. By the IH we
can make the amount 𝑛 – 3, so that means we can make 𝑛 + 1. □
Practice 1
Theorem
It takes 𝑛 – 1 breaks to break a chocolate bar with 𝑛 ≥ 1 squares into
individual squares
Proof:
We use strong induction on 𝑛.
Base case (𝑛 = 1):
It’s just 1 square, so 1 − 1 = 0 breaks is trivially correct.
IH: …
Step (𝑛 ≥ 2):
Consider a chocolate bar with 𝑛 squares.
Suppose the chocolate bar is broken into 2 pieces of 𝑎 and 𝑏 squares,
where 1 ≤ 𝑎, 𝑏 < 𝑛 and 𝑎 + 𝑏 = 𝑛.
By the IH we need 𝑎 – 1 breaks for the first part and 𝑏 – 1 for the second.
Thus, we need 1 + (𝑎 – 1) + (𝑏 – 1) = 𝑎 + 𝑏 – 1 = 𝑛 – 1 breaks. □
Practice 2
Theorem
Every integer 𝑛 ≥ 2, can be expressed as the product of a series of primes.
Base Case
𝑛 = 2, can be expressed as the product of 1 prime (namely 2).
So the IH holds for 𝑛 = 2.
Induction Hypothesis
Every integer 𝑛 ≥ 2, can be expressed as the product of a series of primes.
Practice 2
Induction Hypothesis
Every integer 𝑛 ≥ 2, can be expressed as the product of a series of primes.
Step (n ≥ 2)
Assume that the IH holds for any value m between 2 and n.
We need to show that the IH holds for n+1.
There are 2 cases:
Case 1) n+1 is prime.
Then n+1 can be expressed as the product of 1 prime, namely n+1.
Case 2) n+1 is not prime
Then 𝑛 + 1 = 𝑎 ∗ 𝑏 for some 𝑎, 𝑏 ≠ 1 and 2 ≤ a,b < 𝑛 + 1.
As 𝑎, 𝑏 are between 2 and 𝑛 + 1, by IH both 𝑎 and 𝑏 can be written as the
product of a series of primes. But then so can 𝑛 + 1. So the IH holds.
As 𝑛 + 1 is either prime or not prime, the IH always holds.
Final notes
Final notes on strong induction
When you apply the IH to P(𝑖), make sure that 𝑖 ≤ 𝑛.
Otherwise you may be proving P(𝑛 + 1) with P(𝑛 + 1)…
As with standard induction, make sure that the step works for all 𝑛
If not, you may need more base cases.
Theorem
6𝑛 = 0 for all 𝑛 ≥ 0
What’s wrong with this proof?
Proof:
Base case (𝑛 = 0): 6 ∗ 0 = 0.
IH: …
Step (𝑛 ≥ 0): 𝑛 + 1 = 𝑎 + 𝑏. By the IH, 6𝑎 = 0 and 6𝑏 = 0.
Thus, 6 (𝑛 + 1) = 6 (𝑎 + 𝑏) = 6𝑎 + 6𝑏 = 0 + 0 = 0. □
Different Induction
Even more advanced induction…
Induction
Mathematical Induction and Strong Induction
 Useful for all positive integers: ∀n[n ϵ ℕ: P(n)]
What if we want to use a different set: ∀x[x ϵ S: P(x)]
 Cannot use standard induction: P(x) ⇒ P(x+1)
 What to do?
Examples
 Prove for all strings that …
 Prove for all rooted binary trees that …
 Prove for all graphs that …
 Prove for all polygons that …
What kind of sets?
Defining Infinite Sets
Using properties:
Set of rationals = {x | ∃p,q ϵ ℤ[q ≠ 0 and qx = p]}
Set of primes = {p | ¬∃d ϵ ℤ[1 < d < p and p is multiple of d]}
Set of squares = {x2 | x ϵ ℤ}
Inductive (or recursive) definition:
Natural numbers ℕ:
1) 1 ϵ ℕ
2) If n ϵ ℕ, then n + 1 ϵ ℕ
Full binary trees T:
1)
ϵT
2) If x ϵ T and y ϵ T, then
x
y
ϵT
Examples
1.
Set of positive even numbers E


2.
Set of (non-empty) binary strings B


3.

1ϵQ
If p ϵ Q, then 3p ϵ Q
Set of arithmetic expressions A


5.
0 ϵ B, 1 ϵ B
If X ϵ B, then 0X ϵ B and 1X ϵ B
Set of powers of 3: Q

4.
2ϵE
If n ϵ E, then n + 2 ϵ E
n ϵ A for all n ϵ ℕ
If e1, e2 ϵ A, then –e1, (e1), e1 + e2, e1 – e2, e1 * e2, e1 / e2 ϵ A
Set of prime numbers P

Don’t know…
Structural Induction
Building trees from subtrees…
Structural Induction
Full binary trees T
1)
ϵT
2) If x ϵ T and y ϵ T, then
y
x
ϵT
Base case
 Prove property for a single node
Induction step
 Prove property for
x
y
 Can use induction hypothesis on x and y
Example 1
Theorem
A full binary tree with 𝑛 nodes has (𝑛 + 1)/2 leaves
IH
IH
Example 1
Proof
We use structural induction on the set of full binary trees.
IH:
A full binary tree with 𝑛 nodes has (𝑛 + 1)/2 leaves
Base case:
For a single node 𝑛 = 1 and there is (1 + 1)/2 = 1 leaf.
Step:
Suppose that the trees 𝑥 and 𝑦, with 𝑎 and 𝑏 nodes, have
(𝑎 + 1)/2 and (𝑏 + 1)/2 leaves, respectively (IH).
We need to show that the tree 𝑇 with 𝑛 nodes formed by
adding a root above 𝑥 and 𝑦 has (𝑛 + 1)/2 leaves.
A leaf of 𝑇 is either a leaf of 𝑥 or a leaf of 𝑦. By IH, the
number of leaves of 𝑇 is then (𝑎 + 1)/2 + (𝑏 + 1)/2 =
(𝑎 + 𝑏 + 2)/2. Since 𝑛 = 𝑎 + 𝑏 + 1, we get that (𝑛 +
1)/2 = (𝑎 + 𝑏 + 2)/2, as required. □
IH
IH
Example 2
Theorem
A tree with height 𝑛 ≥ 1 nodes, has 𝑛 − 1 edges.
IH
IH
Proof
IH: A tree with height 𝑛 ≥ 1 nodes, has 𝑛 − 1 edges.
Base Case (n = 1):
A tree with 1 node, trivially has no edges. Thus the IH is true.
Step (n ≥ 1):
Assume the IH holds for a tree with 1 ≤ k ≤ 𝑛 nodes.
To prove the IH holds for a tree with 𝑛 + 1 nodes.
Let T be a tree with 𝑛 + 1 nodes and 𝑥 be the root of 𝑇. Let 𝐿 and 𝑅 be the
left respectively right subtree of 𝑥. Let 𝐿 and 𝑅 have 𝑎, respectively 𝑏,
nodes. As 𝑎 ≤ 𝑛 and b ≤ 𝑛 we can apply the IH. So, 𝐿 has 𝑎 − 1 and 𝑅 has
𝑏 − 1 edges. To connect 𝐿 and 𝑅 to 𝑥, we need 2 more edges. Thus 𝑇 has
𝑎 − 1 + 𝑏 − 1 + 2 = 𝑎 + 𝑏 = 𝑛 edges. □
What if 𝑎 or 𝑏 is zero?
Example 2
IH
IH
?
Ehmm.. Maybe -1 edges???
That is weird. :(
Structural induction on Trees
IH?
IH
IH
IH
IH?
Example 2
Theorem
A tree with height 𝑛 ≥ 1 nodes, has 𝑛 − 1 edges.
IH
Proof
IH: A tree with height 𝑛 ≥ 1 nodes, has 𝑛 − 1 edges.
Base Case (n = 1):
A tree with 1 node, trivially has no edges. Thus the IH is true.
Step (n ≥ 1):
Assume the IH holds for a tree with 𝑛 nodes.
To prove the IH holds for a tree with 𝑛 + 1 nodes.
Let T be a tree with 𝑛 + 1 nodes and 𝑥 be a leaf of 𝑇. Let 𝑇’ be the tree
which is equal to 𝑇, but where 𝑥 is removed. By definition 𝑇’ is also a valid
tree of size n ≥ 1.
By IH 𝑇’ has 𝑛 − 1 edges. If we add 𝑥 back to 𝑇’, we obtain tree 𝑇 again
and we add one edge doing so. Thus 𝑇 must have 𝑛 = 𝑛 + 1 – 1 edges. □
Example 2
Theorem
A tree with height 𝑛 ≥ 1 nodes, has 𝑛 − 1 edges.
IH
Proof
IH: A tree with height 𝑛 ≥ 1 nodes, has 𝑛 − 1 edges.
Base Case (n = 1):
A tree with 1 node, trivially has no edges. Thus the IH is true.
Step (n ≥ 1):
Assume the IH holds for a tree with 𝑛 nodes.
To prove the IH holds for a tree with 𝑛 + 1 nodes.
Let T be a tree with 𝑛 + 1 nodes and 𝑥 be a leaf of 𝑇. Let 𝑇’ be the tree
which is equal to 𝑇, but where 𝑥 is removed. By definition 𝑇’ is also a valid
tree of size n ≥ 1.
By IH 𝑇’ has 𝑛 − 1 edges. If we add 𝑥 back to 𝑇’, we obtain tree 𝑇 again
and we add one edge doing so. Thus 𝑇 must have 𝑛 = 𝑛 + 1 – 1 edges. □
Tips and tricks
 You need not add or remove 1 leaf.
IH
IH
 Always start with the tree you want to prove!
 A constructive proof (let’s add a leaf to some tree)
 Is not precise (some tree)
 Does not match our induction process.
(we find a smaller problem inside the one we have.)
Tips and Tricks
Induction
 Can you easily combine smaller version to prove the current?
 Be careful the induction can reach the base cases.
 There may be more than one base case.
 In case of doubt, better too many than too few!
 Trees often like structural induction.
 Not always though..
 Recursive function usually like well-founded induction.
Well-founded Induction
Even more kinds of induction…
Ordering
𝒏=𝟏
IH
𝒏> 𝟏
IH
How about?
{1,3}
?
{1,5,6,3,2}
Function(n,m)
?
Function(n-1,m+1)
Partial order
Strict partial order relation ≺ on set S
 Binary relation: x ≺ y for certain pairs x, y ϵ S
 Anti-reflexive: x ⊀ x
 Anti-symmetric: if x ≺ y, then y ⊀ x
 Transitive: If x ≺ y and y ≺ z, then x ≺ z
Examples
 For x, y ϵ ℤ: x ≺ y iff x < y
 For sets X, Y: X ≺ Y iff X ⊂ Y
({1,3} ≺ {1,2,3,4})
 For strings S1, S2: S1 ≺ S2 iff S1 is substring of S2 (“ab” ≺ “cab”)
 For trees T1, T2: T1 ≺ T2 iff T1 is substring of T2
Partial order
Does (strong) induction work for any partial order ≺ ?
 Not exactly…
Theorem
For all x ϵ ℤ it holds that x = x + 1
Proof
By induction on x:
We apply IH to x – 1, so that x – 1 = x. (Note that x – 1 < x)
By adding 1 to both sides we obtain that x = x + 1.
We need base cases!
 The partial order ≺ must have minimal elements
 Minimal elements are base cases
Well-founded Relation
Well-founded relation ≺
 Every non-empty subset X ⊆ S must have a minimal element
 Minimal element m ϵ X: for all x ϵ X it holds that x ⊀ m
 S contains no infinite descending chains: a ≻ b ≻ c ≻ ….
Practice
Which of these partial orders are well-founded?
 For x, y ϵ ℤ: x ≺ y iff x < y
 For x, y ϵ ℤ: x ≺ y iff |x| < |y|
 For x, y ϵ ℕ: x ≺ y iff y is a multiple of x and x ≠ y
 For rational numbers x, y ϵ ℚ: x ≺ y iff x < y
½>⅓>¼>⅕>…
 For strings S1, S2: S1 ≺ S2 iff S1 lexicographically before S2
“b” ≻ “ab” ≻ “aab” ≻ “aaab” ≻ …
Well-founded Relation
Well-founded relation ≺
 Every non-empty subset X ⊆ S must have a minimal element
 Minimal element m ϵ X: for all x ϵ X it holds that x ⊀ m
 S contains no infinite descending chains: a ≻ b ≻ c ≻ ….
Well-founded induction
 First need well-founded (partial) order ≺ on S
 Base case(s): Minimal elements of S
 Induction step: If P(x) for all x ≺ y, then P(y)
(for all y ϵ S)
Example
Ackermann(m, n)
1. if m = 0
2.
then return n+1
3. else if n = 0
4.
then return Ackermann(m – 1, 1)
5. else return Ackermann(m – 1, Ackermann(m, n – 1))
A recursive function terminates on all input if and only if there exists a
well-founded order ≺ on the set of inputs such that:
“input of recursive call” ≺ “original input”
Does the Ackermann function terminate on all inputs?
(m, n) ≺ (m’, n’) iff m < m’ or m = m’ and n < n’
Well-founded Induction
Well-founded induction
 Very general type of (strong) induction
 … but also very abstract
Main lesson
 If you can order the elements of a set, you can do induction
 Induction hypothesis may always be applied to “smaller” elements
Recursion
 First argue that it terminates
 Then you can use IH on recursive call to argue correctness
Loop Invariant or Induction??
Differences and similarities…
(Dis)similarities
Loop Invariant
 … is a special kind of Induction
 … used to prove loops (…obviously…)
 … has a termination condition
Induction
 Induction can have multiple base cases
 Trees often like (structural) induction
 Recursions like (well-founded) induction
Always
 Induction Hypothesis is like Loop Invariant
 Maintenance (/Step), assumes LI (/IH) and proves it for next.
Summary
Loop Invariant
At the start:
Sum = 0
After first iteration:
= Sum first 0 elements
Sum = 0 + 𝐴[1]
After second iteration:
= Sum first 1 element
Sum = 0 + 𝐴 1 + 𝐴[2]
= Sum first 2 elements
Mathematical Induction
Strong Induction
Structural Induction
Well-founded Induction
IH
({1,3} ≺ {1,2,3,4})
IH
Next time on:
Reading and Writing
Mathematical Proofs
Next time!
“Poorly designed toilet paper holders”
𝐴1
𝐴2
𝐴3
“Counting letters in boxes”
“A really poor font!”
Example
Theorem
A set of 𝑛 ≥ 1 elements has 2𝑛 subsets including the empty set.
Example
Proof:
We use induction on 𝑛.
Base case (𝑛 = 1):
If a set 𝑆 has only one element 𝑥, then we have 2 subsets {} and {𝑥}.
Since 21 = 2, the IH holds for this base case.
IH: A set of 𝑛 ≥ 1 elements has 2𝑛 subsets including the empty set.
Step (𝑛 ≥ 1):
Assume that every set of 𝑛 elements has 2𝑛 subsets. (IH)
We need to show that any set 𝑆 of 𝑛 + 1 elements has 2𝑛+1 subsets.
Pick some element 𝑥 𝜖 𝑆 (possible since 𝑛 + 1 ≥ 1). Let 𝑆’ = 𝑆\{𝑥}.
By the IH, 𝑆’ has 2𝑛 subsets 𝐴1, … , 𝐴𝑘 (𝑘 = 2𝑛). The subsets of 𝑆’ are
exactly the subsets of 𝑆 that do not contain 𝑥. On the other hand, the
sets 𝐴’𝑖 = 𝐴𝑖 𝑈 {𝑥} are the subsets of 𝑆’ that contain 𝑥. A subset of 𝑆’
must either contain 𝑥 or not. Thus, 𝑆 has 2 ∗ 2𝑛 = 2𝑛+1 subsets. □