Direct Counting, Subsets, Permutations
Counting
Ahto Buldas
November 24, 2016
Ahto Buldas
Counting
November 24, 2016
1 / 40
Direct Counting, Subsets, Permutations
Subsets of a Set
N = {0, 1, 2, . . .}—the set of all natural numbers.
There are 2n subsets of an n-element set.
How many subsets does the set {A, B, C} have? (is it also true for n = 0)
How many subsets does a 100-element set have?
At a dinner, there are four choices for the appetiser, two choices for the
main course, and three choices for the dessert. How many different meals
are possible?
Ahto Buldas
Counting
November 24, 2016
2 / 40
Direct Counting, Subsets, Permutations
Principles of Direct Counting
Multiplication principle: If A can be done in n different ways and B can
(independently) be done in m different ways, then A and B together can
be done in
n·m
different ways.
Addition principle: If we can either do A or B, where A can be done in n
ways and B can be done in m ways, then we have
n+m
different ways to act.
Ahto Buldas
Counting
November 24, 2016
3 / 40
Direct Counting, Subsets, Permutations
Double Counting
Double-Counting Principle: If we count the same things in two different
ways, we must have the same result.
This is used for proving identities of type:
u(n, m) = v(n, m) ,
where u and v are functions that convert natural numbers to natural
numbers.
Ahto Buldas
Counting
November 24, 2016
4 / 40
Direct Counting, Subsets, Permutations
Permutations
P (n, k)—number of k-permutations of an n-element set.
By applying the multiplication principle, we get:
P (n, k) = n(n − 1)(n − 2) . . . (n − k + 1) =
n!
(n − k)!
P (n, n) = n!—a special case.
Alternative notion P (n, k) = nk (k-th falling factorial power)
How many ways may 10 books be arranged on a shelf?
How many ways may four books from a set of 10 books be arranged on a
shelf?
Ahto Buldas
Counting
November 24, 2016
5 / 40
Direct Counting, Subsets, Permutations
Combinations
C(n, k) = nk —the number of k-element subsets of an n-element set.
Prove by using double counting and the two direct counting principles:
n
n
n
+
+ ... +
= 2n ,
0
1
n
n−1
n−1
n
+
=
for n > 0
k
k−1
k
Prove P (n, k) = C(n, k) · k! via double counting. Imply that
n!
n
nk
=
.
=
k!
k!(n − k)!
k
Ahto Buldas
Counting
November 24, 2016
6 / 40
Direct Counting, Subsets, Permutations
Binomial Theorem
Prove by using direct counting that:
n
n
n 2
n n
n
(1 + x) =
+
x+
x + ... +
x .
0
1
2
n
In case n = 2 (assiciating terms with subsets):
1
2
{}
{2}
{1}
{1,2}
z }| { z }| { z}|{ z}|{ z}|{ z}|{
(1 + x) (1 + x) = 1 · 1 + 1 · x + x · 1 + x · x
or ... using x1 and x2 instead ... with assumption that x1 = x2 = x
(1 + x)2 = (1 + x1 )(1 + x2 ) = 1 + x1 + x2 +x1 x2 .
| {z }
(21)
(1 + x)3 = (1 + x1 )(1 + x2 )(1 + x3 )
= 1 + x1 + x2 + x3 + x1 x2 + x1 x3 + x2 x3 +x1 x2 x3
{z
} |
{z
}
|
3
3
(1)
(2)
Ahto Buldas
Counting
November 24, 2016
7 / 40
Binomial Coefficient Identities
Binomial Coefficient Identities
n
n
=
,
k
n−k
as to define a k-people team it is sufficient to name the n − k people who
will not be in the team.
n
n−1
k=n
,
k
k−1
because one may choose a k-people team with a captain by choosing the
captain first (n possible choices) and then the rest of the team (k − 1
people from an n − 1 element set)
n k
n n−j
=
(for 0 ≤ j ≤ k ≤ n) ,
k
j
j
k−j
as for choosing a k-element committee with a j-element subcommittee by
first choosing the subcommittee and then the rest of the k − j members.
Ahto Buldas
Counting
November 24, 2016
8 / 40
Binomial Coefficient Identities
X
k m+n
m
n
=
k
i
k−i
(for m, n, k ≥ 0) ,
i=0
as for choosing a team with k people from m boys and n girls we have, on
one hand, m+n
possibilities. On the other hand, for
k
a fixed number i
(where 0 ≤ i ≤ k)of boys in the team, we have mi possibilities to choose
n
the boys and k−i
possibilities to choose the girls. As a corollary:
n 2
X
n
k=0
because
n
k
=
Ahto Buldas
n
n−k
k
=
2n
n
,
.
Counting
November 24, 2016
9 / 40
Binomial Coefficient Identities
Odd and Even Subsets
To show that for n > 0 there are always the same number of odd and even
subsets, we compute by using the Binomial theorem
X n
n
0 = (1 − 1) =
n
(−1)k
k
k=0
n
n
n
n
n
n
−
−
− ... .
=
+
+
+ ...−
1
3
5
0
2
4
{z
} |
{z
}
|
even subsets
odd subsets
Ahto Buldas
Counting
November 24, 2016
10 / 40
Binomial Coefficient Identities
Using Derivatives
n
X
n
k
= n2n−1 ,
k
k=1
because for choosing a non-empty team with a captain one may choose
the captain first and then an arbitrary subset of an
set.
k
Pn(n −n1)-element
n
We may prove that same thing from (1 + x) = k=0 k x by applying
d
dx to both sides:
n−1
n(1 + x)
=
n X
n
k=1
k
kxn−1 ,
and taking x = 1.
Ahto Buldas
Counting
November 24, 2016
11 / 40
Binomial Coefficient Identities
Generalized Binomial Coefficients
For every real α, define
α
0
= 1 and
αk
α
α(α − 1)(α − 2) · . . . · (α − k + 1)
=
.
=
k!
k!
k
Binomial Theorem:
α
(1 + x) =
∞ X
α
k=0
k
xk
for every | x |< 1.
Ahto Buldas
Counting
November 24, 2016
12 / 40
Binomial Coefficient Identities
Corollaries
−n
k
(−n)(−n − 1)(−n − 2) . . . (−n − k + 1)
k n+k−1
=
= (−1)
k!
k
∞ ∞
X
−n k X
1
−n
k n+k−1
= (1 + x) =
x =
(−1)
xk
(1 + x)n
k
k
k=0
k=0
1
1
=
=
n
(1 − x)
(1 + (−x))n
Ahto Buldas
Counting
∞ X
k=0
n+k−1 k
x
k
November 24, 2016
13 / 40
Binomial Coefficient Identities
Axiomatic Theory of Natural Numbers
In 1889, Giuseppe Peano (1858–1932) presented an axiomatic theory of
natural numbers (0, 1, 2, . . .), using the next natural number function
n 7→ n+ (where n+ = n + 1) as the fundamental relation that satisfies:
Axiom 1: n 7→ n+ is an injective function, i.e. every natural number n has
one and only one successor n+ , and if n+ = m+ , then n = m.
Axiom 2: There is one and only one m, such that n+ 6= m for any n.
Such m is called zero and is denoted by 0.
Axiom 3 (Induction): If 0 has a property A and (for every n) if n has A
then also n+ has A, then all natural numbers have the propery A:
A(0) ∧ ∀n : A(n) → A(n+ )
Ahto Buldas
Counting
⇒
∀n : A(n) .
November 24, 2016
14 / 40
Binomial Coefficient Identities
Addition and its properties
Define addition as follows:
x+0 = x
x + y + = (x + y)+
Addition is commutative (x + y = y + x) because:
Lemma 1: ∀x : 0 + x = x = x + 0.
Proof: As 0 + 0 = 0 and if 0 + x = x, then
0 + x+ = (0 + x)+ = x+ = x+ + 0 .
Lemma 2: If ∀x : y + x = x + y, then ∀x : x + y + = y + + x.
Proof: 0 + y + = y + + 0 and if x + y + = y + + x, then
x+ + y + = (x+ + y)+ = (y + x+ )+ = (y + x)++ = (x + y)++
= (x + y + )+ = (y + + x)+ = y + + x+ .
Ahto Buldas
Counting
November 24, 2016
15 / 40
Binomial Coefficient Identities
Addition is associative because:
1: (x + y) + 0 = x + y = x + (y + 0)
2: If (x + y) + z = x + (y + z), then
(x + y) + z + = ((x + y) + z)+ = (x + (y + z))+ = x + (y + z)+
= x + (y + z + ) .
Exercise: Define multiplication as follows
x·0 = 0
x · y+ = x · y + x
and prove its main properties in a similar way.
Ahto Buldas
Counting
November 24, 2016
16 / 40
Sums and Recurrence Relations
The Tower of Hanoi
(Picture from Wikipedia)
Objective: transfer the entire tower to one of the other pegs,
·moving only one disk at a time and
·never moving a larger one into a smaller.
Ahto Buldas
Counting
November 24, 2016
17 / 40
Sums and Recurrence Relations
The Tower of Hanoi: Recurrence Relation
Tn —the minimum number of moves that transfer n disks to another peg.
T0 = 0.
Tn ≤ 2Tn−1 + 1, because we can transfer n − 1 smaller disk to another
peg with Tn−1 moves, then move the largest disk, and finally, move the
n − 1 smaller disks onto the largest disk with Tn−1 moves.
Tn ≥ 2Tn−1 + 1, because for moving the largest disk, the smaller ones
have to be moved to another peg, which takes at least Tn−1 moves, then
we have to move the largest disk (1 move) and finally move the n − 1
smaller disks onto the largest, which again takes at least Tn−1 moves.
Hence, Tn = 2Tn−1 + 1.
Ahto Buldas
Counting
November 24, 2016
18 / 40
Sums and Recurrence Relations
The Tower of Hanoi: Solution
To solve the recurrence relation:
T0 = 0
Tn = 2Tn−1 + 1 ,
for n > 0,
we first study T (n) with small values of n:
T (1) = 1, T (2) = 3, T (3) = 7, T (4) = 15, etc.
It seems like T (n) = 2n − 1, which can be proved by induction.
How to compute this formula, without guessing it first?
Ahto Buldas
Counting
November 24, 2016
19 / 40
Sums and Recurrence Relations
Sums as Recurrent relations
The sum
Sn =
n
X
ak
k=0
can also be presented as a recurrence relation:
S0 = a0
Sn = Sn−1 + an ,
Ahto Buldas
Counting
for n > 0.
November 24, 2016
20 / 40
Sums and Recurrence Relations
A Special Case: The Repertoire Method
Let us study the recurrence relation:
R0 = α
Rn = Rn−1 + β + γn ,
for n > 0.
We see that R1 = α + β + γ, R2 = α + 2β + 3γ, etc.
It seems reasonable to search for a solutin in the form:
Rn = A(n)α + B(n)β + C(n)γ ,
where A, B, C are certain funcions of n.
The so-called repertoire method tries to plug in simple functions like
Rn = 1, Rn = n, Rn = n2 , etc. in order to find A(n), B(n), C(n), etc.
Ahto Buldas
Counting
November 24, 2016
21 / 40
Sums and Recurrence Relations
Finding A(n), B(n), C(n)
1. If Rn = 1 = Rn−1 + 0 + 0n, then α = 1, β = γ = 0 and hence:
1 = Rn = A(n) · 1
⇒ A(n) = 1 .
2. If Rn = n = (n − 1) + 1 = Rn−1 + 1 + 0n, then α = 0, β = 1, γ = 0
and hence:
n = Rn = A(n) · 0 + B(n) · 1 + C(n) · 0 = B(n)
B(n) = n .
3. If Rn = n2 = (n − 1)2 − 1 + 2n, then α = 0, β = −1, γ = 2 and hence:
n2 = −B(n) + 2C(n) = −n + 2C(n)
Ahto Buldas
Counting
C(n) =
n(n + 1)
.
2
November 24, 2016
22 / 40
Sums and Recurrence Relations
Back to the Tower of Hanoi
To solve the recurrence relation:
T0 = 0
Tn = 2Tn−1 + 1 ,
We convert it first by assigning Sn =
form:
Tn
2n
for n > 0,
to convert the relation to the
S0 = 0
Sn = Sn−1 +
Hence, Sn =
1
2
+
1
4
+ ... +
2Sn = 1 +
and therefore Sn = 1 −
Ahto Buldas
1
2n
1
,
2n
for n > 0,
and
1 1
1
1
+ + . . . + n−1 = 1 + Sn − n ,
2 4
2
2
1
2n ,
which implies Tn = 2n Sn = 2n − 1.
Counting
November 24, 2016
23 / 40
Sums and Recurrence Relations
Exercise
Find a formula for:
Sn = 12 + 22 + 32 + 42 + . . . + n2 .
For that, we first extend the repertoire method for more general recurrence
relations of type:
R0 = α
Rn = Rn−1 + β + γn + δn2 ,
for n > 0.
and search for a general solution
Rn = A(n)α + B(n)β + C(n)γ + D(n)δ .
As we already know that A(n) = 1, B(n) = n, and C(n) =
only need to determine D(n).
Ahto Buldas
Counting
n(n+1)
,
2
November 24, 2016
we
24 / 40
Sums and Recurrence Relations
For finding D(n), we plug in the function Rn = n3 and get 0 = 03 = α
and:
n3 = (n − 1)3 + β + γn + δn2
(for all values of n)
which implies β = 1, γ = −3, and δ = 3. Hence,
n3 = n − 3
n(n + 1)
+ 3D(n) ,
2
which means that:
D(n) =
3
1
1
1 3
n − n + n(n + 1) = n(n + 1)(n + )
3
2
3
2
If we now take Rn = 12 + 22 + . . . + n2 , it satisfies the general recurrence
relations with α = β = γ = 0 and δ = 1. Hence,
1
1
12 + 22 + . . . + n2 = D(n) = n(n + 1)(n + ) .
3
2
Ahto Buldas
Counting
November 24, 2016
25 / 40
Finite Calculus
Difference Operator
Infinite (ordinary) calculus uses differential operator D that converts a
function f (x) to its derivative:
Df (x) = lim
h→0
f (x + h) − f (x)
.
h
Finite calculus uses the difference operator ∆ instead:
∆f (x) = f (x + 1) − f (x) .
Ahto Buldas
Counting
November 24, 2016
26 / 40
Finite Calculus
Derivative and Difference of a Power Function
Ordinary calculus uses the formula:
D(xm ) = mxm−1 .
The difference ∆(xm ) is not that elegant, for example:
∆(x3 ) = (x + 1)3 − x3 = 3x2 + 3x + 1 .
But there is another ”power function” that behaves nicely: the falling
factorial power xm :
∆(xm ) = (x + 1)m − xm
= (x + 1)x(x − 1) . . . (x − m + 2) −
− x(x − 1) . . . (x − m + 2)(x − m + 1)
= x(x − 1) . . . (x − m + 2)[x + 1 − x + m − 1] = mxm−1 .
|
{z
}
xm−1
Ahto Buldas
Counting
November 24, 2016
27 / 40
Finite Calculus
The “Indefinite Sum” Operator
R
The differential operator D has inverse: the indefinite integral operator :
Z
g(x) = Df (x) ⇔
g(x)dx = f (x) + C ,
where C is any constant. This is because D(C) = 0.
P
The difference operator ∆ also has inverse: the indefinite sum operator :
X
g(x) = ∆f (x) ⇔
g(x)δx = f (x) + C ,
where C is any 1-periodic function, i.e. C(x + 1) = C(x), which implies
∆C(x) = C(x + 1) − C(x) = 0.
Ahto Buldas
Counting
November 24, 2016
28 / 40
Finite Calculus
Properties of Σ
We want
P
to satisfy the Newton-Leibniz formula:
Z
b
g(x)dx = f (x) |ba = f (b) − f (a)
a
b
X
g(x)δx = f (x) |ba = f (b) − f (a) .
a
Assuming that g(x) = ∆f (x) = f (x + 1) − f (x) we have
a
X
g(x) = f (a) − f (a) = 0
a
a+1
X
g(x) = f (a + 1) − f (a) = ∆f (a) = g(a) ,
a
Ahto Buldas
Counting
November 24, 2016
29 / 40
Finite Calculus
Formula for Σba g(x)δx
b+1
X
g(x)δx −
a
b
X
g(x)δx = [f (b + 1) − f (a)] − [f (b) − f (a)]
a
= f (b + 1) − f (b) = g(b) .
Hence,
a
X
a
b+1
X
g(x)δx = 0
g(x)δx =
a
b
X
g(x)δx + g(b) ,
a
and hence, by induction:
b
X
g(x)δx =
a
Ahto Buldas
b−1
X
g(k) = g(a) + g(a + 1) + . . . + g(b − 1) .
k=a
Counting
November 24, 2016
30 / 40
Finite Calculus
Summing the Powers via Finite Calculus
n
Z
xm dx =
nm+1
m+1
xm δx =
nm+1
.
m+1
0
n
X
0
For example,
n
X
k=
n+1
X
0
k=0
n+1
x2 (n + 1)2
n(n + 1)
x δx =
=
=
,
2 0
2
2
1
and as x2 = x2 + x1 , we have:
n
X
k=0
2
k =
n+1
X
(x2 + x1 )δx =
0
Ahto Buldas
n(n + 1)(n + 12 )
(n + 1)3 (n + 1)2
+
=
.
3
2
3
Counting
November 24, 2016
31 / 40
Finite Calculus
Negative Powers
x2 = x(x − 1)
x1 = x
x0 = 1
x−1 =
1
x+1
x−2 =
1
(x + 1)(x + 2)
1
1
−
(x + 2)(x + 3) (x + 1)(x + 2)
x + 1 − (x + 3)
=
(x + 1)(x + 2)(x + 3)
= −2x−3 .
∆x−2 =
If m 6= Ahto
−1:Buldas
Counting
November 24, 2016
32 / 40
Finite Calculus
The Case m = −1 and Harmonic Sums
For integration, we have:
Z
a
b
x−1 dx = ln x|ba .
For summation, first recall that:
x−1 =
1
= ∆f (x) = f (x + 1) − f (x) ,
x+1
which gives f (x + 1) = f (x) +
1
x+1 ,
i.e.
f (x) = Hx = 1 +
1 1
1
+ + ... +
2 3
x
for every positive integer x, where Hx is the harmonic sum. Note that
Hx ≈ ln x + 0.577 +
Ahto Buldas
Counting
1
.
2x
November 24, 2016
33 / 40
Finite Calculus
ex and 2x
We know from calculus that D(ex ) = ex , but for which function f (x)
∆f (x) = f (x) .
From
∆f (x) = f (x + 1) − f (x) = f (x)
we get f (x + 1) = 2f (x), which means that f (x) = 2x .
Ahto Buldas
Counting
November 24, 2016
34 / 40
Finite Calculus
Differences of Products
D(uv) = uDv + vDu
∆(uv) = u∆v + v(x + 1)∆u
Because
∆(u(x)v(x)) = u(x + 1)v(x + 1) − u(x)v(x)
= u(x + 1)v(x + 1) − u(x)v(x + 1)
+u(x)v(x + 1) − u(x)v(x)
= u(x)∆v(x) + v(x + 1)∆u(x) .
Ahto Buldas
Counting
November 24, 2016
35 / 40
Finite Calculus
Summation by Parts
Z
Z
u(x)Dv(x)dx = uv −
v(x)Du(x)dx
X
u(x)∆v(x)δx = uv −
v(x + 1)∆u(x)δx
Pn
Example: To find k=0 k2k , we first compute the indefinite sum:
X
X
x2x δx = x2x −
2x+1 δx = x2x − 2x+1 + C ,
X
where u(x) = x and ∆v(x) = 2x and hence ∆u(x) = 1 and v(x) = 2x .
Then,
n
X
k2k =
n+1
X
n+1
x2x δx = x2x − 2x+1 0
0
k=0
=
(n + 1)2n+1 − 2n+2 − 0 · 20 − 21
= (n − 1)2n+1 + 2 .
Ahto Buldas
Counting
November 24, 2016
36 / 40
Finite Calculus
Higher Order Differences
∆2 f (x) = ∆f (x + 1) − ∆f (x) = f (x + 2) − 2f (x + 1) + f (x)
∆3 f (x) = f (x + 3) − 3f (x + 2) + 3f (x + 1) − f (x)
...
n X
n
∆ f (x) =
(−1)n−k f (x + k)
k
n
k=0
As for d > 0 we have ∆
x
d
=
x
d−1
and ∆
x
0
= 0, then
x x
if n ≤ d
d−n
∆
=
0
if n > d
d
n
Ahto Buldas
Counting
November 24, 2016
37 / 40
Finite Calculus
Newton Series
Any polynomial f (x) = ad xd + ad−1 xd−1 + . . . + a1 x + a0 can be
expressed in the form:
f (x) = bd xd + bd−1 xd−1 + . . . + b1 x1 + b0
x
x
x
= cd
+ cd−1
+ . . . + c1
+ c0
d
d−1
1
ck
k! .
As it is easy to check that (∆k f )(0) = ck , hence
x
x
x
x
2
d
f (x) = f (0)
+ ∆f (0)
+ ∆ f (0)
+ . . . + ∆ f (0)
0
1
2
d
0
1
2
d
x
x
x
x
= f (0) + ∆f (0) + ∆2 f (0) + . . . + ∆d f (0)
0!
1!
2!
d!
where bk =
This is analogous to the Taylor series in calculus.
Ahto Buldas
Counting
November 24, 2016
38 / 40
Finite Calculus
Example 1
For f (x) = x3 , we first compute the higher order differences:
f (0) = 0,
f (1) = 1,
f (2) = 8,
f (3) = 27
∆f (0) = 1,
∆f (1) = 7,
∆f (2) = 19
∆2 f (0) = 6,
∆2 f (1) = 12
∆3 f (0) = 6
And using the Newton expansion, we have:
x
x
x
x
3
x = 6
+6
+1
+0
3
2
1
0
x2
x3
= 6 + 6 + x1
3!
2!
3
2
= x + 3x + x1 .
Ahto Buldas
Counting
November 24, 2016
39 / 40
Finite Calculus
Example 2
For finding a closed form for f (n) = 12 + 22 + . . . + n2 , we can also use
the difference scheme:
f (0) = 0,
f (1) = 1,
f (2) = 5,
f (3) = 14
∆f (0) = 1,
∆f (1) = 4,
∆f (2) = 9
∆2 f (0) = 3,
∆2 f (1) = 5
3
∆ f (0) = 2
And using the Newton expansion, we have:
n
n
n
n
n2
n3
f (n) = 2
+3
+1
+0
= 2 + 3 + n1
3
2
1
0
3!
2!
2
3
n
n
n(n − 1)(n − 2)
n(n − 1)
=
+ 3 + n1 =
+3
+n
3
2
3
2
n(n + 1)(n + 21 )
n 2 3
1
=
n + n+
=
.
3
2
2
3
Ahto Buldas
Counting
November 24, 2016
40 / 40