2
Mathematical Preliminaries
In this chapter we’ll review some mathematical concepts that will be used throughout
this course. We’ll also learn some new mathematical notations and techniques that
are important for analysis of algorithms.
2.1
Floors and Ceilings
In doing computer computations, we sometimes have to discard the fractional part of
a real number x, while in other cases we may have to round upward to the first integer
larger than or equal to x. It is convenient to have simple mathematical notations that
indicate these operations. The “floor” and “ceiling” notations were invented for this
purpose.
Notations. We shall use N for the set of cardinal or natural numbers,
N = {0, 1, 2, . . .}, Z for the set of integers, Q for the set of rational numbers,
and R for the set of real numbers.
Definition 2.1.1 The floor of a real number x, denoted by x, is the largest integer
that is less than or equal to x. The ceiling of a real number x, denoted by x, is the
smallest integer that is greater than or equal to x. We can express these definitions
in the following way:
x = max{k : k ∈ Z, k ≤ x};
x = min{k : k ∈ Z, k ≥ x}.
Examples 2.1.2
3.7 = 3; 0.04 = 0; −0.04 = −1; 6 = 6; 22.104 = 22;
3.7 = 4; 0.04 = 1; −0.04 = 0; 6 = 6; 22.104 = 23.
© Springer International Publishing Switzerland 2014
D. Vrajitoru and W. Knight, Practical Analysis of Algorithms,
Undergraduate Topics in Computer Science, DOI 10.1007/978-3-319-09888-3_2
9
10
2 Mathematical Preliminaries
In a C or C++ program, an expression of the form n/m indicates integer
n division
denotes
when n and m are of an integer data type. The mathematical expression
m
the exact value of the C/C++ expression n/m when n and m are positive. Thus, when
reasoning about programs in which integer
division is performed, we will make
n
repeated use of expressions such as
.
m
If you consult the documentation for the C/C++ “math.h” or “cmath” library
header file, you will find that it contains the following two prototypes:
double ceil (double x);
double floor(double x);
Note that the values returned by these two functions are of type double, not
long. This may surprise you until you realize that floating point real numbers can
be far larger than the integer LONG_MAX, and so in order to have expressions such as
floor(3.182e19) return a sensible value, the function must return the integer
31820000000000000 in floating point form.
The following lemma gives us a property of integers that we will use often in this
chapter.
Lemma 2.1.3 Let n and m be any integer numbers. If n < m, then n + 1 ≤ m and
n ≤ m − 1.
Proof Suppose n < m. If n + 1 were greater than m, then m would be an integer
strictly between n and n + 1. That is impossible. Thus n + 1 must be less than or
equal to m. It follows from n + 1 ≤ m that n ≤ m − 1.
The following theorem lists some properties of floor and ceiling that we will use
throughout in this book. It gives us a better idea about how the floor and the ceiling
relate to their argument and to each other.
Theorem 2.1.4 Let x and y be any real numbers. Let n, m, and k be any integers.
(a) x ≤ x < x + 1; equivalently, x − 1 < x ≤ x.
(b) x − 1 < x ≤ x; equivalently, x ≤ x < x + 1.
(c) If n ≤ x < n + 1 (or x − 1 < n ≤ x), then n = x; similarly, if
n − 1 < x ≤ n (or x ≤ n < x + 1), then n = x.
(d) If x ≤ y, then x ≤ y and x ≤ y. Similarly for x ≥ y.
(e) If n ≤ x, then n ≤ x. If n ≥ x then n ≥ x.
(f) If x has an integer value, then x = x = x. If x has a non-integer value, then
x = x + 1.
(g) (−1)x = (−1)x and (−1)x = (−1)x.
Proof (a) Take any real number x. The inequality x ≤ x is true because the floor
of x is an element of a set of integers less than or equal to x. The inequality
x < x + 1 is true because x is the largest of the integers less than or equal
11
2.1 Floors and Ceilings
(b)
(c)
(d)
(e)
(f)
(g)
to x, and x + 1 is an integer larger than x, so x + 1 cannot be in the set of
integers less than or equal to x. Thus x + 1 is greater than x. This completes
the proof that x ≤ x < x + 1. To prove that x − 1 < x ≤ x, we need
only prove that x − 1 < x because we have already proved that x ≤ x. Take
the inequality x < x + 1, which we have already proved, and subtract 1 from
both sides. This gives x − 1 < x.
Take any real number x. The inequality x ≤ x is true because the ceiling of x is
an element of a set of integers greater than or equal to x. To prove the inequality
x−1 < x, first note that x is the smallest of the integers greater than or equal
to x, and x − 1 is smaller than x, so x − 1 cannot be in the set of integers
greater than or equal to x. Thus x − 1 is smaller than x. This completes the
proof that x − 1 < x ≤ x. The inequalities x ≤ x < x + 1 are proved by
taking apart the inequalities above, adding 1 to both sides of x − 1 < x, and
then putting the pieces back together again in a different order.
Let x be any real number, and let n be an integer satisfying n ≤ x < n + 1. Then
n must be the largest of the integers that are less than or equal to x because by
Lemma 2.1.3 every integer greater than n is at least n + 1. By definition of x,
it follows that n = x. The second assertion has a similar proof.
Suppose x ≤ y. We’ll prove that x ≤ y and leave the other assertions
as exercises. First note that either x < y or y ≤ x. If x < y, then
we can combine this with the inequality x ≤ x (see part (a)) to conclude that
x < y. If, instead, y ≤ x, we can combine this with x ≤ y and y < y+1
to obtain y ≤ x < y + 1. Since y is an integer, it follows from part (c)
that x = y.
Suppose n ≤ x. Then by (d), n ≤ x. Since n has an integer value, n = n,
which means that n ≤ x.
The proof of the second part is similar.
Suppose x has an integer value. Then trivially x belongs to the set {k : k ∈
Z, k ≤ x} and is the largest element of the set, so x = x. Similarly, x = x.
Now suppose x is not an integer. We will prove that x = x+1 by showing that
x ≤ x+1 and x ≥ x+1. Since x is not an integer, then x < x < x,
from which we get x < x. Since both x and x are integers, we can use
Lemma 2.1.3 to conclude that x + 1 ≤ x. By part (a): x < x + 1. Since
x + 1 is an integer, we can apply (e) to deduce that x ≤ x + 1. From the
two inequalities that we have proved, we can conclude that x = x + 1.
Let us start with the second inequality from (a): x − 1 < x ≤ x. Multiply
through by −1 to reverses the inequalities:
(−1)(x − 1) > (−1)x ≥ (−1)(x).
This can be rewritten from right to left as
(−1)x ≤ (−1)x < (−1)x + 1.
We can see that the integer (−1)x satisfies the first part of (c) with respect to the real
number (−1)x, and thus it must be the floor of this number: (−1)x = (−1)x.
The second part is proved in a similar way.
12
2 Mathematical Preliminaries
The following theorem shows some properties of the floor and ceiling functions in
relation to arithmetic operations applied to their arguments.
Theorem 2.1.5 Let x and y be any real numbers. Let n, m, and k be any integers.
(a) x ± n = x ± n and x ± n = x ± n; but in general,
nx = nx and nx = nx and x + y = x + y and
x + y = x + y.
n n +
= n.
(b)
2
2 n n+1
n
n−1
(c)
=
+1=
; equivalently,
.
2
2
2
2
y
y
y
y
=
=
and
. In particular, if m = 0 we can
(d) If k > 0, then
k
k
k k
n
n
n n n
m
m
and
.
put y =
to get these identities:
=
=
m
k
mk
k
mk
Proof (a) Take any particular integer n. We’ll prove that x + n = x + n. Since
x + n is an integer, by Theorem 2.1.4 (c) it suffices to prove that (x + n) ≤
x + n < (x + n) + 1. These two inequalities are logically equivalent to
(x + n) − n ≤ x + n − n < (x + n) + 1 − n, which are in turn equivalent
to x ≤ x < x + 1, which are true for all x (see Theorem 2.1.4 (a)). This
completes the proof for floors.
The proof for ceilings is similar. The equation x − n = x − n is proved by
noting that x − n = x + (−n) = x + (−n) = x − n because −n is an
integer.
The assertions involving nx and nx are demonstrated by using examples:
4 · 0.5 = 40.5 and 4 · 0.5 = 40.5.
(b) Take any integer
nreal
number x. If n is even, then n = 2m for some integer
n n and
+
m, and thus
2
2
2m
2m
+
= m + m = m + m = 2m = n.
=
2
2
n n If n is odd, then n = 2m + 1 for some integer m, and thus
+
=
2
2
2m + 1
1
1
2m + 1
+
= m+
+ m+
. By part (a) of this theorem,
2
2
2
2
1
1
this last expression equals m +
+m +
= m +0+m +1 = 2m +1 = n.
2
2
(c) Take
any
integer
n. Ifn is even, then
n = 2m
for
some integer m, so
n+1
2m + 1
1
1
=
= m+
= m +
= m + 0 = m and
2
2
2
2
n n n+1
2m
= m = m also, so
=
=
.
2
2
2
2
13
2.1 Floors and Ceilings
n+1
=
If n is odd, then n = 2m + 1 for some integer m, and thus
2 n 2m + 2
1
2m + 1
= m + 1 = m + 1, and
= m+
=
=
2
2
2
2
1
n
n+1
n−1
m+
= m +1 also, and thus
=
+1 =
. The fact that
2
2
2
2
n+1
follows from part (a) and simple algebra.
2
y
(d) Take any real number y and any integer k > 0. By definition of the floor,
k
y
y
=
it suffices by Theorem 2.1.4 (c) to
is an integer. To prove that
k
k
y
y
y
y
prove that the integer
satisfies the inequalities
≤
+1
<
k
k
k
k
y
y
).
(take x to be
and n to be
k k
y y
y
y
y
≤
≤ , so
k ≤ y.
First, let’s prove
. By Theorem 2.1.4 (a),
k
k
k
k
k
y
k is the product of two integers, it is an integer, and it is less or equal
Since
k y y
y
k ≤ y by Theorem 2.1.4 (e). That proves
≤
to y, so
.
k
k
k
y
y
y
y
+ 1, we note that
+ 1 > by Theorem 2.1.4 (a),
<
To prove that
k
k
k
k
y
y
y
y
+1 >
because y ≥ y by Theorem 2.1.4 (a). Thus
,
and ≥
k
k
k
k
which is what we wanted to prove (in turned-around form). The proof of the
assertion about ceilings is quite similar and will be omitted here.
As an example of where some of the formulas in Theorems 2.1.4 and 2.1.5 are useful,
consider what happens during a binary search of a subarray a[first..last].
Here is a fragment of the code for the search (for the complete function see Fig. 5.3,
p. 176):
while (first <= last && !found) // The value parameters first and last will
{
// be modified during execution of this code
int mid = (first + last)/2;
if (target < a[mid])
.......
A similar calculation of mid = (first + last)/2 occurs during the Merge
Sort algorithm, which splits an array into two pieces, sorts them recursively, and then
merges them back together.
When analyzing such algorithms, it is useful to have mathematical expressions
for the lengths of the following three smaller subarrays
a[first..mid-1]
a[first..mid]
a[mid+1..last]
in terms of the length of the larger subarray a[last].
14
2 Mathematical Preliminaries
←−−−−−−−−−−−−− n −−−−−−−−−−−−−→
first
mid
last
a:
← − − − − − − − − − − −− →
n/2
1 = (n − 1)/2
← − − − − − − − − − − − − −− →← − − − − − − − − − − −− →
n/2
n/2
Fig. 2.1 Theorem 2.1.6
0
first-1 first
last
Fig. 2.2 Theorem 2.1.6 a
Theorem 2.1.6 Consider a subarray a[first..last] of some array a, where
0 ≤ first ≤ last. Let n denote the length of the subarray, and let mid denote
the quantity (first + last) / 2 (this is the average of the subscripts at the
ends of the subarray). Then
(a) n is the value of the expression last - first + 1
n
;
(b) the length of the subarray a[first..mid] is
2 n−1
n
;
−1=
(c) the length of the subarray a[first..mid-1] is
2
n 2
(d) the length of the subarray a[mid+1..last] is
.
2
See Fig. 2.1 for a graphical representation.
Proof Part (a) is proved by considering the full array that starts at location 0
and ends at location last. There are last + 1 cells in that subarray. If we
then remove the cells in the subarray a[0..first-1] we produce the subarray a[first..last]. We are removing (first-1)+1 = first cells
from last + 1 cells, leaving (last+1)-first = last-first+1 cells.
See Fig. 2.2.
For the proof of statement (b), we begin by noting that the subarray a[first..
mid] contains mid-first+1 cells, and mid = (first+last)/2 . Then we
can write
first + last
− first + 1
mid − first + 1 =
2
first + last
=
− first + 1 (by Theorem 2.1.5 (a))
2
15
2.1 Floors and Ceilings
last − first + 2
first + last − 2 first + 2
=
2
2
(last − first + 1) + 1
=
2
n
n+1
=
.
=
2
2
=
To prove part (c), note that the subarray a[first..mid-1] has one less cell
than a[first..mid]. By part (b), the length of a[first..mid] is n/2, so
the length of a[first..mid-1] is
n n−1
n+1
−1=
−1=
2
2
2
by Theorem 2.1.5 (c).
To prove part (d), subtract the number of cells in a[first..mid]
n n to
n from
=
by
obtain the number of cells in a[mid+1..last]. We get n −
2
2
Theorem 2.1.5 (b).
2.1.1
Further Examples
Example 2.1.7 (a)
26
Solution (a)
7
26
7
=?
(b)
26
7
=?
(c)
7
26
=?
(d)
7
26
=?
= 3 + 5/7 = 3
(b) 4
(c) 0
(d) 1
Example 2.1.8 Use part (a) of Theorem 2.1.4 to prove that 0 ≤ x − x < 1 for all
real numbers x.
Solution By Theorem 2.1.4 (a) we know that x ≤ x < x + 1 for all real x.
We can pull this expression apart into the following pair of inequalities: x ≤ x
and x < x + 1 for all real x. From the first of these it follows that 0 ≤ x − x
(subtract x from both sides of the inequality); from the second it follows similarly
that x − x < 1. Thus we have 0 ≤ x − x and x − x < 1 for all x. Putting
these back together gives the desired expression: 0 ≤ x − x < 1.
Example 2.1.9 Verify part (a) of Theorem 2.1.5 in the case where x = 12.46 and
n = −5.
Solution x +n = 12.46+(−5) = 7.46 = 7, and x+n = 12.46+(−5) =
12 − 5 = 7. Thus we have verified in this one particular case that x + n = x + n.
Similarly, x + n = 12.46 + (−5) = 7.46 = 8 and 12.46 + (−5) =
13 − 5 = 8.
16
2 Mathematical Preliminaries
Example 2.1.10 Verify part (c) of Theorem 2.1.5 for some odd integer n and also
for some even integer n.
13
=
, i.e., that 7 = 6.5?
Solution For n = 13, is it true that
2
14 + 1
14
Yes: both are equal to 7. For n = 14, is it true that
=
, i.e., that
2
2
7.5 = 7? Yes: both are equal to 7.
13 + 1
2
Example 2.1.11 For each of the assertions below, cite one or more parts of Theorems
2.1.4 and 2.1.5 to help you justify the assertion.
p+q
p+q
+
= p + q.
2√
√2
(b) For all integers m, 2m + m = 2m
n n+ m. p and q =
, then q =
. More
(c) For all integers n, p, and q, if p =
n p 2
p 2
p 4 1
2
n−1
, p2 =
, p3 =
, . . . , pn =
, then
generally, if p1 =
2
2
2
2
n
pn = n .
2
n n + 1
− 1.
=
(d) For all integers n,
2
2
(a) For all integers p and q,
Solution (a) Use part (b), with
√ n = p + q.
(b) Use part (a), with x = m and n = 2m.
p
=
(c) Use part (d) with m = 2 and k = 2. That is, q =
2
n . The second sentence in (c) is proved by induction.
4
(d) Use part (c), with m = 2.
n
2
2
=
n =
2·2
Example 2.1.12 In the C++ loop given below, assume that the variable n has some
positive integer value. When the loop is executed, how many times will the body of
the loop be executed? (In this loop, the body of the loop consists of just the cout
statement.) Express your answer as a mathematical formula in terms of the variable n.
int k; for (k = 1; (3*k)+1 <= n; ++k)
cout << k << endl;
Solution First note that the initial value computed for (3*k) + 1 is 4, so if the value
of n is 3 or less, the body of the loop will be executed 0 times. So now let’s look at
the cases where the value of n is at least 4. In these cases the body of the loop will
be executed at least once.
Since the variable k starts with the value 1 and is incremented by 1 each time
the body of the loop is executed, k is a loop counter. That is, when its value is 1 in
17
2.1 Floors and Ceilings
the cout statement, the body of the loop is being executed for the first time. When
its value is 2 in the cout statement, the body of the loop is being executed for the
second time. Etc. Let L denote the value of k in the cout statement the last time
the body of the loop is executed. Then that number L is the total number of times
the body of the loop is executed. That is, L is the number we want to compute.
Let n denote the value of the program variable n. Then 3L + 1 ≤ n because
when k has the value L, the loop control condition (3∗k)+1 <= n must be true. (If
that were not so, the body of the loop would not be executed when k gets the value
L.) Also note that when k is then incremented to L + 1 (in the expression k++),
the loop control condition must then be false in order to make the loop halt. Thus
3(L + 1) + 1 > n. We now have the following two inequalities involving L and n :
3L + 1 ≤ n < 3(L + 1) + 1.
Subtracting 1 from each the part of this expression, and then dividing each part
by 3 gives the inequalities
n−1
< L +1
L≤
3
It follows that the integer L must
bethe floor of the fraction (n-1)/3 (see
n−1
. This is the number of times the body of
Theorem 2.1.4 (c)). That is, L =
3
the loop is executed when n ≥ 4. In fact, this formula gives the correct number of
times even in the case where n is 1, 2, or 3.
Example 2.1.13 Prove that for all real numbers x and y we have x + y ≤
x + y + 1.
Solution Take any real numbers x and y. By Theorem 2.1.4 (b), x < x + 1 and
y < y + 1. It follows that x + y < x + y + 2. By Theorem 2.1.4 (b),
x + y ≤ x + y, so x + y + 2 ≤ x + y + 2. Putting together the inequalities in
the two preceding sentences we get x + y < x + y + 2. Note that the left and
right sides of this inequality are integers. If i < j for integers i and j, then we know
that i ≤ j − 1, so x + y ≤ x + y + 1.
2.1.2
Exercises
2.1.14 Let a be the name of an array of length n > 0 in a C or C++ program.
Explain why the expression a[n/2] will be accepted by the compiler (even though
n may be an odd integer) but a[floor(n/2.0)] will not compile.
2.1.15 Use part (b) of Theorem 2.1.4 to prove that 0 ≤ x − x < 1 for all real
numbers x.
2.1.16 Is it always true that x = 1 + x? If so, prove this fact. If not, give a
counterexample, i.e., an example of a real number for which the equation is not true.
18
2 Mathematical Preliminaries
2.1.17 Theorem 2.1.4 (d) says that if x ≤ y then x ≤ y for all real numbers x
and y. Can we conclude that if x < y then x < y?
2.1.18 (a) Give an example of an integer n and a real number x for which
nx = nx. (See part (a) of Theorem 2.1.5.)
(b) Give an example of two real numbers x and y for which x + y = x + y.
(See part (a) of Theorem 2.1.5.)
2.1.19 Verify part (b) of Theorem 2.1.5 for the odd integer n = 11 and also for the
even integer n = 12.
2.1.20 (a) Verify part (d) of Theorem 2.1.5 in the case where n = 37, m = 8, and
k = 3.
(b) Verify part (d) of Theorem 2.1.5 in the case where n = 26, m = 7, and k = 2.
2.1.21 Is it true that x−n = x −n for all integers n and real numbers x? Answer
the same question for the equation n − x = n − x.
2.1.22 For each of the assertions below, cite one or more parts of Theorems 2.1.4
and 2.1.5 to help you justify the assertion.
(a)
(b)
(c)
(d)
(e)
n+1
= n.
+
For all integers n,
2
2
√
√
For all non-negative
x, x − 1 = x − 1.
real numbers
n
n+1
+
= n + 1.
For all integers n,
2
2
For all real numbers x and non-negative integers p, x 2 + 2 p = x 2 + 2 p .
For all real numbers x, x + x = 2x.
n 2.1.23 In the C++ loop given below, assume that the integer variable n has some
positive value. How many times is the body of the loop executed? Express your
answer as a mathematical expression in terms of the value of n. Justify your answer
by an argument similar to the one in Example 2.1.12 on p. 17. Also check your
answer carefully by determining what integers are sent to output when n = 50.
int k;
for (k = 1; k*k <= n; ++k)
cout << k << endl;
2.1.24 In the C++ loop given below, assume that the integer variable n has some
positive value. How many times is the body of the loop executed? Express your
answer as a mathematical expression in terms of the value of n. Justify your answer
by an argument similar to the one in Example 2.1.12 on p. 17. Also check your
answer carefully by determining what integers are sent to output when n = 10 and
also when n = 11.
19
2.1 Floors and Ceilings
int k;
for (k = 1; 2*k < n; ++k)
cout << k << endl;
2.1.25 (a) Prove that for all real numbers x and y we have x + y ≤ x + y ≤
x + y + 1.
(b) Prove that for all real numbers x and y we have x + y − 1 ≤ x + y ≤
x + y.
(c) Prove that for all non-negative real numbers x and y, xy ≤ xy ≤ xy+
(x + y).
2.1.26 Let n and m be two positive integer numbers. Prove that
n n+m−1
n
n
n
≤
≤
and
≤
.
m
m
m
m
m
2.1.27 Let n be an integer number. If n is odd, then
2.2
n 2
=
n−m+1
≤
m
n n + 1
n−1
=
and
.
2
2
2
Logarithms
Definition 2.2.1 For a fixed real number b > 1, the logarithm base b function is
defined on positive real numbers x as follows: logb (x) = “the power on b that
produces x”. In other words, logb (x) = y if and only if by = x.
Examples:
log3 (9) = 2 because 32 = 9.
log5 (125) = 3 because 53 = 125.
log4 (4) = 1 because 41 = 4.
log6 (1) = 0 because 60 = 1.
Two important equations follow instantly from this definition.
Theorem 2.2.2 For any fixed real number b > 1,
(a) blogb (x) = x for all real x > 0;
(b) logb (by ) = y for all real y.
Logarithms were invented nearly 400 years ago because they have several important properties that could be used to simplify certain difficult arithmetic calculations:
they convert products to sums, they convert quotients to differences, and they convert
powers to products. Here is a precise statement of these properties.
Theorem 2.2.3 For any fixed real number b > 1,
20
2 Mathematical Preliminaries
(a) logb (s · t) = logb (s) + logb (t) for all positive real numbers s and t;
(b) logb (s/t) = logb (s) − logb (t) for all positive real numbers s and t;
(c) logb (t p ) = p logb (t) for all real numbers p and all positive real numbers t.
Proof The proofs are given in algebra or pre-calculus textbooks and will not be
repeated here.
Example 2.2.4 Most of us met logarithms base 10 in our high school algebra class.
log10 (1000) = 3; log10 (100) = 2; log10 (10) = 1; log10 (1) = 0 because
100 = 1;
log10 (0.01) = −2 because 0.01 = 10−2 ;
log10 (−100) is undefined because 10x > 0 for all real x;
1 < log10 (25) < 2 because 101 < 25 < 102 ; by calculator, log10 (25) ≈ 1.4;
log10 (1/t) = − log10 (t).
Example 2.2.5 Here is the logarithm you met in calculus; it’s called the natural
logarithm:
x
loge (x) = ln(x) =
1
dt for all real x > 0, where e ≈ 2.71828.
t
1
Of course, eln(x) = x if x > 0, and ln(ey ) = y for all real numbers y.
In computer science, the logarithm that we encounter more often than any other is
the logarithm base 2 function, denoted by “lg(x)” or log2 (x). Here are some easily
computed values of this function:
lg(32) = 5; lg(8) = 3; lg(1024) = 10; lg(1) = 0; lg(2) = 1; lg(1/16) = −4.
What is the numerical value of lg(100)? We can see that it must be between 6 and
7 because 26 < 100 < 27 . If we want a more exact answer we will need to consult a
table of logarithms or use an electronic calculator. Most pocket calculators, however,
don’t have a “lg” button. This is not a problem because, as we will now prove, all
logarithm functions are just constant multiples of each other.
Theorem 2.2.6 For all real numbers a and b greater than 1 and all positive real
numbers x,
loga (x)
logb (x) =
.
loga (b)
Proof Take the logarithm, base a, of both sides of the identity blogb (x) = x.
21
2.2 Logarithms
Example 2.2.7 ln(x) =
Similarly, lg(x) =
log10 (x)
≈ 2.303 log10 (x) by using a calculator.
log10 (e)
ln(x)
ln(x)
log10 (x)
≈
≈ 1.44 ln(x); also, lg(x) =
≈ 3.32 log10 (x).
ln(2)
0.693
log10 (2)
In analysis of algorithms, we seldom have occasion to compute lg(x) when x
is a non-integer real number. Mostly we will encounter lg(n) where n is a positive
integer. Moreover, the “lg” usually appears in connection with the floor or ceiling
function. The following theorem gives a number of useful identities involving the lg
function combined with floors and ceilings.
Theorem 2.2.8 Let n be a positive integer.
n
(a) 2lg(n) ≤ n < 2lg(n)+1 ; equivalently, < 2lg(n) ≤ n.
2
n n
= 1 and lg(n)+1 = 0. Moreover, the only integer k for which
(b)
2
2nlg(n)
= 1 is k = lg(n).
2k
We can interpret this as saying that lg(n) is the number of “integer divisions
by 2” that are required to reduce n down to 1.
(c) lg(n + 1) = 1 + lg(n) for all n ≥ 1.
(d) lg(n/2) = −1 + lg(n) for all n ≥ 2.
(e) lg(n/2) = −1 + lg(n) for all n ≥ 2.
(f) lg(n/2) = −1 + lg(n + 1) for all n ≥ 1.
(g) lg(n/2) = −1 + lg(n − 1) for all n ≥ 3.
Proof To prove part (a), we can start from the fact that
lg(n) ≤ lg(n) < lg(n) + 1,
which is an instance of Theorem 2.1.4 (a), with x = lg(n). From this it follows that
2lg(n) ≤ 2lg(n) < 2lg(n)+1 .
Since 2lg(n) = n, this proves that 2lg(n) ≤ n < 2lg(n)+1 .
To prove the first of the two equations in part (b), divide the first double inequality
in part (a) through by 2lg(n) to obtain
n
1 ≤ lg(n) < 2.
2
By Theorem 2.1.4
this immediately implies that the integer 1 is the floor of
(c),
n
n = 1.
;
that
is
2lg(n)
2lg(n)
To prove the second part we start from the double inequality above and divide
1
1
n
it through by 2 to obtain ≤ lg(n)+1 < 1. Since 0 < , we can deduce that
2
2
2
n
n
0 < lg(n)+1 < 1, which implies that lg(n)+1 = 0.
2
2
22
2 Mathematical Preliminaries
To prove the last assertion in part (b), let’s start with n and divide it repeatedly
by 2, discarding remainders. We’ll give names to the integers we produce along the
way:
n
n 0 = n = 0 (here we are using the fact that n = n and 20 = 1)
n 2 n 0
=
n1 =
2
n 2n1 1
=
(here we are using Theorem 2.1.5 (d); note we’ve divided
n2 =
2
22
by 2 twice)
n n 2
n3 =
=
(again we’re using Theorem 2.1.5 (d); note we’ve divided
2
23
by 2 three
times)
etc.
For
n any positive integer k, after k divisions by 2 we obtain
n
k−1
nk =
= k .
2
2
In particular, when k = lg(n), we see by the first equation in part (b) that we
will arrive at the value 1. That is, by dividing n by 2 (discarding remainders) lg(n)
times, we reduce n to 1.
To prove the equation in part (c), it suffices by Theorem 2.1.4 (c) to prove that the
integer 1 + lg(n) satisfies the inequalities
(1 + lg(n)) − 1 < lg(n + 1) ≤ 1 + lg(n),
which are equivalent to
lg(n) < lg(n + 1) and lg(n + 1) ≤ 1 + lg(n).
The first one is easy because lg(n) ≤ lg(n) < lg(n + 1). For the second, start with
the inequality n < 2lg(n)+1 that we proved in part (a). Both sides of the inequality
are integers, so it must be true that n + 1 ≤ 2lg(n)+1 . Taking logarithms on both
sides gives the desired inequality above.
To prove part (d) we begin with the first inequalities in part (a) and divide by 2 to
obtain
2lg(n)−1 ≤ n/2 < 2lg(n) for all positive integers n.
If n ≥ 2, then 2lg(n)−1 and 2lg(n) are both integers, and thus
2lg(n)−1 ≤ n/2 < 2lg(n) for all integers n ≥ 2.
Taking the logarithm of every term gives
lg(n) − 1 ≤ lg(n/2) < lg(n),
which is exactly what is needed to show that lg(n)−1 = lg(n/2) (see Theorem
2.1.4 (c)).
To prove part (e), note that
n−1
lg (n/2) = lg
+1
(by Theorem 2.1.5 (c))
2
n−1
= 1 + lg
(by part (c) above)
2
= 1 + (−1 + lg(n − 1)) (by part (d) above)
23
2.2 Logarithms
Fig. 2.3 Plot for the function y = lg(x)
= lg(n−1) = −1+lg((n−1)+1) (by part (c) above, with n replaced by n−1)
= −1 + lg(n).
The proofs of parts (f) and (g) are similar, and left as an exercise.
Example 2.2.9 To illustrate part (b) of Theorem 2.2.8, take n = 100. Then
100/2 = 50; 50/2 = 25; 25/2 = 12; 12/2 = 6; 6/2 = 3; 3/2 = 1.
There are 6 integer divisions by 2 in that sequence, and indeed, lg(100) = 6
because 26 < 100 < 27 , so 6 < lg(100) < 7.
Since we encounter many “divide and conquer” algorithms that involve continually splitting a list of length n into two roughly equal pieces, Theorem 2.2.8 (b) gives
us some insight into why these algorithms often have running times that involve
lg(n) in one way or another.
The most important thing to understand about lg(n) is how very slowly it grows
as n increases. Even when n gets extremely large, lg(n) tends to remain relatively
small. The reason is this: doubling the argument in lg(n) just adds 1 to the value of
the logarithm lg(2n) = 1 + lg(n). This is illustrated in the graph of y = lg(x) shown
in Fig. 2.3.
You might wonder whether there is some horizontal asymptote for this graph,
i.e., a horizontal line that the graph approaches as n → ∞. In fact, there is no
such asymptote. If we draw a horizontal line at height h, the graph of y = lg(x) will
intersect that line when x reaches the value 2h , and will continue to rise slowly above
the line thereafter. In mathematical notation,
lim lg(x) = +∞
x→∞
(2.1)
Later in this book we will analyze methods for sorting data into key order. Some
sorting algorithms require an amount of time proportional to n 2 in order to sort n
elements, while certain other algorithms require an amount of time proportional to
24
2 Mathematical Preliminaries
n lg(n) in order to sort n elements. Which of these algorithms would require the
least time when sorting n elements? The answer is that the algorithm requiring time
proportional to n lg(n) would require less time when n is large. The reason is this:
for any fixed positive constants of proportionality A and B,
A n lg(n) B n n = B n 2
for all large values of n (the symbol is read “is much smaller than”).
The following theorem states a curious identity that will prove to be useful.
Theorem 2.2.10 For all positive real numbers x and y and b > 1 we have x logb (y) =
y logb (x) .
Proof Since the logarithm function is inductive (one-to-one) on its domain (the set
of all positive real numbers), we can prove that two positive numbers are equal
by proving that their logarithms (in any base) are equal. Thus the formula in the
theorem is true if and only if logb (x logb (y) ) = logb (y logb (x) ) which is equivalent
to logb (y) · logb (x) = logb (x) · logb (y). The latter is surely true for any positive
x and y.
2.2.1
Further Examples
Example 2.2.11 For all positive integers n,
√
3lg(n) = n lg(3) ≈ n 1.58 ≈ n n.
Example 2.2.12 Do the following multiple choice exercises without using a calculator.
(a) ln(e7x ) = Choose one: (i) 7 ln(x) (ii) 7x (iii) 7e x (iv) ln(7) + ln(x)
Choose one: (i) 7 ln(x) (ii)7x (iii)7e x (iv) ln(7) + ln(x)
(b) ln(x 7 ) =
(c) ln(7x) = Choose one: (i) 7 ln(x) (ii) 7x (iii) 7e x (iv) ln(7) + ln(x)
(d) eln(7x) =
Choose one: (i) 7 ln(x) (ii) 7x (iii) 7ex (iv) 7 + x
x+ln(7)
(e) e
= Choose one: (i) 7 ln(x) (ii) 7x (iii) 7e x (iv) 7 + x
−n
(f) lg(2 ) = Choose one: (i) − n (ii) 1/n (iii) 1 − n (iv) (1/2) lg(n)
(g) lg(8n) =
Choose one: (i) 3n (ii) 8n (iii) 3 + n (iv) 3 + lg(n)
(h) lg(8 + n) = Choose one: (i) 3n (ii) 8n (iii) 3 + n (iv) 3 + lg(n)
(i) lg(n/32) = Choose one: (i) n/5 (ii) n 5 (iii) 5 − n (iv) lg(n) − 5
(j) 2lg(18) =
Choose one: (i) 29 (ii) 18 (iii) 11 (iv) 36
3
lg(3)
(k) 2
= Choose one: (i) 27 (ii) 64 (iii) 18 (iv) 81
2)
−
lg(n
(l) 2
= Choose one: (i) n −2 (ii) n −1/2 (iii) 2−n (iv) 2n 2
18
(m) lg(2
2 lg(18) (iv) 182
√ ) = Choose one: (i) 18 (ii) 36 (iii) (n) lg( 2) = Choose one: (i) 1 (ii) 0.5 (iii) lg(2) (iv) 20.5
2
(o) ln(e x ) = Choose one: (i) 2 ln(x) (ii) e2 ln(x) (iii) x 2 (iv) 2x
(p) eln(n+1) = Choose one: (i) n + e (ii) en (iii) n (iv) n + 1
(q) en ln(x) = Choose one: (i) x n (ii) n x (iii) nx (iv) n ln(x)
25
2.2 Logarithms
(r) eln(n)−ln(m) = Choose one: (i) n − m (ii) n/m (iii) ln(n − m) (iv) eln(n−m)
(s) 24 lg(3) =
Choose one: (i) 12 (ii) 24 (iii) 48 (iv) 81
(t) lg(3220 ) = Choose one: (i) 25 (ii) 52 (iii) 5 lg(20) (iv) 100
Solution (a) (ii) because ln(er ) = r for any real number r
(b) (i) because ln(x p ) = p ln(x) for any real number p
(c) (iv) because ln(st) = ln(s) + ln(t) (d) (ii) because eln(t) = t
(e) (iii) because ea+b = ea · eb by the properties of exponents
(f) (i) because lg(2 p ) = p
(g) (iv) because lg(8n) = lg(8) + lg(n) = 3 + lg(n)
(h) There is no correct answer. It was a trick question.
(i) (iv) because lg(n/32) = lg(n)g(n) − 5
(j) (ii) because 2lg(x) = x
3
(k) (i) because 23 lg(3) = 2lg(3 ) = 2lg(27) = 27
1
1
2
= 2 = n −2
(l) (i) because 2− lg(n ) =
2)
lg(n
n
√2
(m) (i) (n) (ii) because 2 = 20.5 (o) (iii) (p) (iv)
(q) (i) because n ln(x) = ln(x n )
(r) (ii) because ln(n) − ln(m) = ln(n/m)
(s) (iv) because 4 lg(3) = lg(34 ) = lg(81)
(t) (iv) because lg(3220 ) = 20 lg(32)
Example 2.2.13 Show how to compute the value of lg(1500) without using a
calculator.
Solution 210 = 1024 < 1500 < 2048 = 211 , so 10 < lg(1500) < 11, so
lg(1500) = 10.
Example 2.2.14 In the block of C++ code shown below, assume that the variable n
has a positive integer value. When the code is executed, how many times is the body
of the loop executed? Express the answer in terms of the value of n.
int k = 1;
int k, p;
for (k = 1, p = 2; p <= n;
{
cout << k << endl;
++k;
}
p *= 2)
// Double p after body of loop is executed.
Solution First consider the special case where n has the value 1. In that case, the loop
control condition n <= n is immediately false, so the body of the loop is executed 0
times. Now let’s look at the cases where the value of n is at least 2, so that the body
of the loop will be executed at least once. As in Example 2.1.12 on p. 17, the variable
k is a loop counter. Let L denote the value of k in the cout statement the last time
the body of the loop is executed. Then L is the number we want to compute.
26
2 Mathematical Preliminaries
When the body of the loop is executed, the initial value 1 of k is sent to output
and then k is incremented to 2. Next, the value of p is doubled to 4 (= 22 ) and then
the loop control condition p <= n is tested. If it is found to be true, then the value
2 of k will be sent to output, k will be incremented to 3, and p will be doubled to
8 (= 23 ). Again the loop control condition will be tested. When k gets the value L
(which we want to compute), the variable p will then get the value 2 L , and the loop
control condition will be true for the last time. The number L will be sent to output,
the variable k will be incremented to L + 1, and p will be doubled to 2 L+1 . The loop
condition will now be found to be false. If we let n denote the (fixed) value of the
program variable n, then the following inequalities must be true:
2 L ≤ n < 2 L+1 .
Taking the logarithm base 2 of each part gives us
L ≤ lg(n) < L + 1.
As we know from Theorem 2.1.4 (c), this means that L = lg(n). This is the number
of times the body of the loop is executed. Note that this formula is valid even when
n has the value 1, for in that case lg(n) = 0.
In the solution of the preceding example, the key observation was that there exists
a simple mathematical relationship between the values k and p of the program
variables k and p each time the loop control condition is tested. That relationship is
p = 2k , and it is true even when the loop control is tested for the last time and found
to be false (at that point k = L + 1 and p = 2 L+1 ). Any condition that’s true for
the program variables every time a loop control condition is tested is called a loop
invariant for the loop. In the block of code in Example A.2.14, the loop invariant
p = 2k is easy to spot, and seeing it was the key to solving the given problem.
There are other loop invariants for the block of code in Example 2.2.14. Examples
are p > 1 and k < p, but these invariants are not useful in solving the problem that
we were given.
2.2.2
Exercises
2.2.15 Do the following multiple choice exercises without using an electronic
calculator.
(a)
(b)
(c)
(d)
(e)
(f)
25+lg(t) =
lg(25n ) =
lg(32 p) =
lg(n 8 ) =
23 lg(k) =
lg(8−k ) =
Choose one:
Choose one:
Choose one:
Choose one:
Choose one:
Choose one:
(i) 32 + t (ii) 32t (iii) 5 + t (iv) 5t
(i) 5 + n (ii) 32 + n (iii) 5n (iv) 32n
(i) 5 lg( p) (ii) 32 lg( p) (iii) 5 p (iv) 5 + lg( p)
(i) 8 lg(n) (ii) 3 lg(n) (iii) 3n (iv) 3 + lg(n)
(i) 8 lg(k) (ii) 8k (iii) k 3 (iv) 3k
(i) 8/k (ii) − 3k (iii) 1/(3k) (iv) 3 lg(k)
2.2 Logarithms
27
(g) lg(32x) = Choose one: (i) 5 + lg(x) (ii) 5x (iii) 32 + x (iv) 5lg(x)
(h) lg(a b c) = Choose one: (i) a + b + c (ii) 2a b c (iii) 2a+b+c
(iv) lg(a) + lg(b) + lg(c)
(i) lg(8/n 2 ) = Choose one: (i) 6 − lg(n) (ii) 3/ lg(n 2 ) (iii) 3/n 2
(iv) 3 − 2 lg(n)
(j) 2lg(5) =
Choose one: (i) 5 (ii) 32 (iii) 25 (iv) lg(25)
(k) 23 lg(5) =
Choose one: (i) 13 (ii) 15 (iii) 40 (iv) 125
(l) 2− lg(8n) = Choose one: (i) − 8n (ii) 1/(8n) (iii) 8/n (iv) 64n 2
8
(m) lg(2
Choose one: (i) 256 (ii) 3 (iii) 8 (iv)6
√) =
(n) lg( 38) = Choose one: (i) 19 (ii) 21 lg(38) (iii) lg(38) (iv) 238
(t) lg(85 ) =
Choose one: (i) 15 (ii) 40 (iii) 8 lg(5) (iv) 3 + lg(5)
2.2.16 For each of the following, give a single numerical value without using a
calculator.
√
√
(d) lg( 3 32)
(a) lg(36) − lg(9)
(b) lg(56) − lg(7)
(c) lg( 128)
2.2.17 (a) Show how to compute the value of lg(1500) without using a calculator.
(b) Show how to compute the value of lg(1900) without using a calculator.
2.2.18 (a) Express log3 (x) as some constant C times log10 (x). Use a pocket calculator to determine the numerical value of the constant C.
(b) Express log5 (x) as some constant C times ln(x). Use a pocket calculator to
determine the numerical value of the constant C.
2.2.19 (a) Use Theorem 2.2.10 to express 2ln(n) as a power of n. Use a calculator
to find the numerical value of the power.
(b) Use Theorem 2.2.10 to express 5ln(2n) as some constant times a power of n. Use
a calculator to find the numerical value of the power.
2.2.20 We noted that the first of the two equations in Theorem 2.2.8 (b) can be
interpreted as saying that lg(n) is the number of “integer divisions by 2” that are
required to reduce n down to 1. Give a similar interpretation for the second of the
two equations in Theorem 2.2.8 (b).
2.2.21 Consider the following block of C++ code in which n is a positive integer.
int k;
for (k = n; k > 1; k = k/2)
cout << "Hello" << endl;
(a) How many times will the cout statement be executed if the value of n is 1 when
the loop begins?
(b) How many times will the cout statement be executed if the value of n is 2 when
the loop begins?
28
2 Mathematical Preliminaries
(c) How many times will the cout statement be executed if the value of n is 3 when
the loop begins?
(d) Give an expression involving the variable n for the number of times that the
code above will send “Hello” to output. Cite one of the theorems in this section
to justify your answer.
2.2.22 (a) Use (among other things) Theorem 2.1.4 on floors and ceilings to prove
that for all positive integers n, n ≤ 2lg(n) < 2n. Warning: in general,
2lg(n) = 2lg(n) . Hint: look at the proof for Theorem 2.2.8 (a).
(b) Use (among other things) Theorem 2.1.4 on floors and ceilings to prove that
n
for all positive integers n,
< 10log10 (n) ≤ n. Warning: in general,
10
10log10 (n) = 10log10 (n) . Hint: look at the proof for Theorem 2.2.8 (a).
2.2.23 (a) Use Theorem 2.1.5 (c) and part (d) of Theorem 2.2.8 to prove part (f) of
Theorem 2.2.8.
(b) Use Theorem 2.1.5 (c) and part (e) of Theorem 2.2.8 to prove part (g) of Theorem
2.2.8.
2.2.24 (a) How many many times is the body of the loop below executed? (Note:
the block differs from the code in Example 2.2.14 only in the initial value of p.)
Justify your answer by using a loop invariant. Assume that n has a positive
integer value.
int k, p;
for (k=1, p = 1; p <= n; p *= 2) // Double p after body is executed.
{
cout << k << endl;
++k;
}
(b) How many many times is the body of the loop below executed? Assume that n
has a positive integer value. You may base your answer on your answer for part
(a).
int p;
for (p = 1; p <= n; p *= 2) // Double p after body is executed.
cout << p << endl;
(c) How many times is the body of the loop below executed? Assume that n has a
positive integer value.
int p;
for (p = 1; p < n; p *= 2)
cout << p << endl;
// Double p each time body of loop is executed.
2.2 Logarithms
29
Hint. Imagine that a loop-counting variable k with initial value 1 is introduced
into the code. Give a loop invariant involving the values k and p of the program
variables k and p. Let L denote the value of k the last time the body of the loop is
executed. Use your loop invariant to calculate L. You will need to cite Theorem
2.1.4 (c) at some point.
2.3
Sums and Summation Notation
What is the value of the sum
12 + 32 + 52 + 72 + · · · + 192 ?
To answer this question, you first have to fill in the missing terms that are indicated
by the “3 dots” in the sum. In this case that’s not hard to do. The 3 dots must surely
stand for
92 + 112 + 132 + 152 + 172 .
What about the sum
2 + 6 + 12 + 20 + 30 + · · · + 90?
Here we may or may not see that the terms can be written as
1 · 2 + 2 · 3 + 3 · 4 + 4 · 5 + 5 · 6 + · · · + 9 · 10,
in which case the missing terms are 42 + 56 + 72 = 6 · 7 + 7 · 8 + 8 · 9. It is useful
to have a notation that lets us express sums in a more precise way that eliminates the
guesswork associated with the 3 dots notation. The sigma summation notation was
invented for this purpose. The two sums given above can be written this way:
10
(2k − 1) and
k=1
9
k(k + 1)
k=1
More generally, suppose a real function f (k) is defined on integers k in the range
m, . . . , n. The expression
n
f (k)
k=m
where
is the upper case Greek letter “sigma”, is a precise and concise way of
writing the sum
f (m) + f (m + 1) + · · · + f (n).
The numerical value represented by these expressions is exactly the value that
would be produced in the variable sum by executing the following fragment of C
code:
30
2 Mathematical Preliminaries
sum = 0;
for (k = m; k <= n; ++k)
sum += f(k);
In particular, by convention,
n
f (k) has the value 0 if n < m (look at the loop
k=m
above).
Examples 2.3.1 (a)
5
k=2
(b)
7
k
2 3 4 5
183
= + + + =
k+1
3 4 5 6
60
k 2k−1 = 0 · 2−1 + 1 · 20 + 2 · 21 + 3 · 22 + 4 · 23 + 5 · 24 + 6 · 25 + 7 · 26 = 769
k=0
(c)
(d)
4
15/k = 15/4 = 3
k=4
25
3 = 3 + 3 + 3 + · · · + 3 = 48 (because there are 15 + 1 terms here, not
k=10
just 15). More generally,
n
c = c(n − m + 1) for any fixed real number c and
k=m
integers
⎛ m ≤ n. ⎞ 2
3
j
3
1
2
2
2
2
⎝ ( j − k) ⎠ =
(1 − k) +
(2 − k) +
(3 − k)
(e)
j=1
k=1
k=1
k=1
k=1
= (02 ) + (12 + 02 ) + (22 + 12 + 02 ) = 6
The variable k in the expression
n
f (k) is called the summation index. It is
k=m
important to understand that it is a “dummy index” that can be replaced by any other
letter that has no pre-assigned meaning. Thus, for example, all four of the following
expressions have the same value:
7
k(k + 1)
k=2
7
i=2
i(i + 1)
7
t (t + 1)
t=2
7
a(a + 1).
a=2
Another important fact is that summation is a “linear operator”; that is,
n
n
n
n
n
c· f (k) = c
f (k)
and
[ f (k)±g(k)] =
f (k)±
g(k).
k=m
k=m
k=m
k=m
k=m
(2.2)
Occasionally we have to “interchange the order of summation” in a double sum.
That is, we have to use the fact that
⎞
⎛
n
q
q
n
⎝
f ( j, k)⎠ =
f ( j, k) .
k=m
j= p
j= p
k=m
31
2.3 Sums and Summation Notation
This equation simply says that if we sum over each row separately in the following
array, and then add up the sums we get, we obtain the same answer as if we sum over
each column separately and then add up those sums. We can do this as long as the
boundaries p and q for the interior sum are independent of the index variable of the
exterior sum k.
f ( p, m)
f ( p + 1, m)
f ( p + 2, m)
. . . f (q, m)
f ( p, m + 1) f ( p + 1, m + 1) f ( p + 2, m + 1) . . . f (q, m + 1)
f ( p, m + 2) f ( p + 1, m + 2) f ( p + 2, m + 2) . . . f (q, m + 2)
...
...
...
...
f ( p, n)
f ( p + 1, n)
f ( p + 2, n)
. . . f (q, n)
We are now going to look at several types of sums that occur frequently in analysis
of algorithms. Our goal will be to find some shortcuts for computing the values of
these sums.
Our first example is the simple sum 1 + 2 + 3 + · · · + n, where n is a positive
integer. (When n is 1 or 2 or 3, then by 1 + 2 + 3 + · · · + n we mean simply 1 or
1 + 2 or 1 + 2 + 3 respectively.) We can write this sum in the sigma summation
n
k. This is called a triangular sum because if we draw a bar
notation this way:
k=1
graph with bars of heights 1, 2, 3, . . ., n, they form a rough triangle. It turns out that
there is a simple formula for the value of the triangular sum. It can be obtained by
n
k. Then
the following algebraic trick. Let S(n) denote the sum
k=1
S(n) = 1
+2
+3
+ · · · + (n − 1) + n.
In reverse order we have S(n) = n
+ (n − 1) + (n − 2) + · · · + 2 +
1.
Adding them gives
2S(n) = (n + 1) + (n + 1) + (n + 1) + · · · + (n + 1) + (n + 1) = n(n + 1)
n(n + 1)
.
so
S(n) =
2
Theorem 2.3.2 For any positive integer n we have
n(n + 1)
.
2
More generally, for any integers m and n satisfying m ≤ n we have
m+n
m + (m + 1) + (m + 2) + · · · + n = (n − m + 1)
.
2
1 + 2 + 3 + ··· + n =
Proof We have already proved the first of these summation formulas. The second
can be proved in a similar way.
These formulas should be memorized. It will help you to note that in both formulas
above the first factor on the right side is the number of terms on the left side, while
32
2 Mathematical Preliminaries
the second factor on the right side is the average of the first and last numbers on the
left side.
In general, when we are confronted with an expression X involving a sigmasummation symbol or a 3-dots ellipsis we like to have a closed form for the expression, by which we mean a mathematical expression Y whose value is the same as that
of X, but Y is compact and easily computed with a fixed number of operations including exponentiation and formation of logarithms. (We regard 2n as a closed form
because it can be evaluated in two steps: let x = n ln(2); then 2n = exp(x).) The caln(n + 1)
culation in Theorem 2.3.2 shows that
is a closed form for the expressions
2
n
k.
1 + 2 + 3 + · · · + n and
k=1
Our next example of a frequently occurring sum looks like this: 1 + t + t 2 + t 3 +
· · · + t n , where t is any real number and n is any non-negative integer. We can write
n
t k . Note that the lower limit of the summation
the sum in sigma notation this way:
k=0
is k = 0. By convention, in sigma sums of this form, t 0 is understood to stand for
1, even when t = 0. (Normally in mathematics the expression 00 is considered to
be undefined.) Sums of this form are called (finite) geometric sums. We can find a
closed form for the sum by a trick similar to the one used for triangular sums. Let
n
t k . Then
G(t, n) denote
k=0
G(t, n) = 1 + t + t 2 + t 3 + · · · + t n .
Multiplying by t gives t G(t, n) = t + t 2 + t 3 + t 4 + · · · + t n+1 .
Subtracting them gives G(t, n) − t G(t, n) = 1 − t n+1 (other terms cancel).
Factoring gives G(t, n)(1 − t) = 1 − t n+1 . If t = 1, then we obtain G(t, n) =
That is, 1 + t + t 2 + t 3 + · · · + t n =
1 − t n+1
t n+1 − 1
=
. when t = 1.
1−t
t −1
1 − t n+1
.
1−t
Theorem 2.3.3 For all real numbers t and non-negative integers n,
⎧
n
⎨ 1 − t n+1
if t = 1;
k
t =
⎩ 1−t
n+1
if t = 1.
k=0
Proof See paragraphs preceding this theorem.
1 1
1
Our third example of a frequently occurring sum looks like this: 1+ + +· · ·+ ,
2 3
n
n
1
. We call a sum of
which can be written in sigma summation notation this way:
k
k=1
33
2.3 Sums and Summation Notation
this form a harmonic sum and denote it by H(n). (In music, if strings of like material,
diameter, and torsion have lengths 1, 1/2, 1/3, etc., then when set in vibration
they produce harmonic tones.) Sadly, there is no closed form for harmonic sums.
Nevertheless, we can approximate their values very closely by using a technique that
is applicable both here and in other similar situations. The technique is given by the
following theorem.
Theorem 2.3.4 Let f (x) be a strictly decreasing function defined over an interval
[m, n], where m and n are integers such that m < n. Then
n
f (x)dx <
f (n) +
n
n
f (k) < f (m) +
k=m
m
f (x)dx.
m
Similarly, if f (x) is strictly increasing on [m, n], then
n
f (x)dx <
f (m) +
m
n
k=m
n
f (k) < f (n) +
f (x)dx.
m
Proof The proof is easier if we begin by assuming that f (x) ≥ 0 for all x in [m, n] .
At the end of the proof we will see how to take care of the cases where f (x) has
some negative values.
For the proof when f (x) ≥ 0 and decreasing we use a graph of the equation
y = f (x) on [m, n], draw some rectangles, and compare their areas with the area of
the region under the curve y = f (x) shown in Fig. 2.4.
Note that the area of the left-most rectangle in Fig. 2.4 is 1 × f (m) = f (m), the
area of the next rectangle is 1 × f (m + 1) = f (m + 1), and so on to the last one,
which has area 1 × f (n − 1) = f (n − 1). Now consider the region R bounded by
the x-axis, the curve y = f (x), and the vertical lines at x = m and x = n. This
region R lies inside the collection of rectangles shown, and thus the sum of areas of
Fig. 2.4 Theorem 2.3.4
34
2 Mathematical Preliminaries
Fig. 2.5 Theorem 2.3.4
these rectangles is greater than the area of R. This can be expressed as follows:
n
f (x) dx < f (m) + f (m + 1) + · · · + f (n − 1) =
area of region R =
n−1
f (k).
k=m
m
Adding f (n) to both sides of the inequality above gives the first of the inequalities
about decreasing functions.
For the second of the inequalities about decreasing functions, consider the graph
shown in Fig. 2.5. All the rectangles in Fig. 2.5 lie inside the region S bounded by
the x-axis, the curve y = f (x), and the vertical lines at x = m and x = n, so the
sum of the areas of the rectangles is less than the area of S. This can be expressed as
follows:
n
f (x) dx.
f (m + 1) + f (m + 2) + · · · + f (n) < area of region S =
m
Adding f (m) to both sides of the inequality above gives the second of the inequalities
about decreasing functions.
Now let’s take care of the case where some or all of the values of f (x) are negative.
Let C denote the value of f (n). Since f (x) is decreasing on the interval [m, n], it
follows that f (x) ≥ C for all x in that interval. This implies that f (x) − C ≥ 0
on that interval. Now define a new function g(x) by the formula g(x) = f (x) − C.
Then g(x) ≥ 0 everywhere on [m, n]. Moreover, g(x) is strictly decreasing on that
interval. By the first part of this proof,
n
g(x)dx <
g(n) +
m
n
n
g(k) < g(m) +
k=m
g(x)dx.
m
Replacing g(x) by f (x) − C above gives us
n
f (n) − C +
[ f (x) − C] dx <
m
n
k=m
n
[ f (k) − C] < f (m) − C +
[ f (x) − C] dx.
m
35
2.3 Sums and Summation Notation
Now we use properties of summation mentioned in Examples 2.3.1 (d) and in
formula (2.3) on p. 29, together with standard integration formulas to produce the
following inequalities:
n
f (n) − C +
f (x)dx − C(n − m) <
n
f (k) − C(m − n + 1) < f (m) − C
k=m
m
n
+
f (x)dx − C(n − m).
m
Adding C(n − m + 1) to each part of the inequalities above gives the inequalities
for f (x) stated in the theorem.
The proofs for the inequalities involving increasing functions are similar and will
be left as an exercise.
Theorem 2.3.5 (Approximation for Harmonic Sums). For all integers n > 1 we
have
1
+ ln(n) < H(n) < 1 + ln(n).
n
Proof We want to approximate H(n) =
n
1
for n > 1.
k
k=1
1
To do this we apply Theorem 2.3.4 to the strictly decreasing function f (x) = ,
x
with m = 1 and n > 1. We get
1
+
n
n
n
n
k=1
1
1
1
1
dx <
< +
x
k
1
1
1
dx for all n > 1,
x
which immediately yields the inequalities stated in the theorem.
The important thing to remember from Theorem 2.3.5 is that for all large n we
have H(n) ≈ ln(n). In rare cases where we want a more accurate approximation for
H(n) when n is large, we can use the following theorem discovered by Leonhard
Euler.
Theorem 2.3.6 (Euler) lim [H(n) − ln(n)] = 0.5772 approximately and therefore
n→∞
H(n) ≈ 0.5772 + ln(n) for all large n.
Proof See almost any advanced calculus textbook. The number 0.5772 given in this
theorem is an approximation to the actual limit, which is known to be an irrational
number.
36
2 Mathematical Preliminaries
When n ≥ 15, Euler’s approximation is correct to within 1 % of the actual value.
Of course, for small values of n, we can calculate H(n) very accurately with a pocket
calculator.
Closed forms for several other types of sums are occasionally needed in analysis
of algorithms. They do not, however, occur with sufficient frequency to make them
worth memorizing. We give four such sums below.
Theorem 2.3.7 For all positive integers n and real numbers t = 1,
1 + 2t + 3t 2 + · · · + nt n−1 =
1 + nt n+1 − (n + 1)t n
.
(t − 1)2
Proof The left side is just the derivative with respect to t of the finite geometric
sum 1 + t + t 2 + t 3 + · · · + t n . Thus we can obtain the formula above simply
by differentiating both sides of the formula given in Theorem 2.3.3 with respect
to t.
Theorem 2.3.8 For all positive integers n,
n
lg(k) = nlg(n) + (n + 1) − 2lg(n)+1
k=2
and
n
lg(k) = (n + 1)lg(n + 1) + 2 − 2lg(n+1)+1 .
k=2
Proof When k = 2 we have lg(k) = 1. When k = 3 or k = 4 we have lg(k) = 2.
n
When k = 5 or 6 or 7 or 8 we have lg(k) = 3. Etc. Thus, the terms in
lg(k)
k=2
can be grouped as follows
k=2
k=3,4
k=5,6,7,8=23
k=9,10,...,16=24
k=2lg(n)−1 +1,...,2lg(n)
20 term
21 terms
22 terms
23 terms
2lg(n)−1 terms
! "# $ !
"#
$ !
"#
$
"#
$
!
1+ (2 + 2) + (3 + 3 + 3 + 3) + (4 + 4 + · · · + 4) + · · · + (lg(n) · · · + lg(n)) + more
# $! " #
$!
" #
$!
"
$!
"
#
The “more” (above) gives the remaining terms, corresponding to k = 2lg(n) + 1
up to k = n (this set of terms is empty if n is a power of 2). Each of these terms is
equal to lg(n) + 1. Thus we have
n
k=2
lg(n)
lg(k) =
p=1
&
%
2 p−1 p + n − 2lg(n) (lg(n) + 1).
(2.3)
37
2.3 Sums and Summation Notation
The sigma sum on the right side is just the sum given in Theorem 2.3.7, but with k
replaced by p, n replaced by lg(n), and t replaced by 2. Thus we get
lg(n)
2 p−1 p =
p=1
1 + lg(n)2lg(n)+1 − (lg(n) + 1)2lg(n)
.
(2 − 1)2
Inserting this into Eq. (2.3) gives us
n
lg(k) = 1+lg(n)2lg n+1 −(lg(n)+1)2lg n +nlg(n)+n −lg(n)2lg n −2lg n ,
k=2
which reduces to the first of the two sums in the statement of the theorem.
To prove the second identity in this theorem, begin by replacing the symbol n by
the symbol n + 1 everywhere in the first equation stated in the theorem. This gives
n+1
lg(k) = (n +1)lg(n +1)+(n +2)−2lg(n+1)+1 for all positive integers n.
k=2
By Theorem 2.2.8 (c), with n replaced by k − 1, we get lg(k) = 1 + lg(k − 1),
so the sum above is
n+1
(1 + lg(k − 1)) = (n + 1)lg(n + 1) + (n + 2) − 2lg(n+1)+1 . (2.4)
k=2
The left side can be expressed as
n+1
k=2
1+
n+1
lg(k −1) = n +0+lg(2)+lg(3)+· · ·+lg(n) = n +
k=2
n
lg(k).
k=2
Putting this into Eq. (2.4) gives
n+
n
lg(k) = (n + 1)lg(n + 1) + (n + 2) − 2lg(n+1)+1 .
k=2
Subtracting n from both sides of this equation gives the desired equation.
Example 2.3.9 Many computer science textbooks discuss the representation of positive integers using various bases. For example, the integer 602910 (base 10 notation)
can also be written in binary notation (base 2 notation) as 10111100011012 or in
octal notation (base 8) as 136158 or in hexadecimal notation (base 16) as 178D16 .
Of course, it is also possible to write this integer in these bases with one or more
leading zeroes: 00602910 , 0000010111100011012 , 0136158 , 178D16 . Let’s agree to
use the phrase standard representation of n base b to describe the representation of
a positive integer n without leading zeroes in base b notation. Note that the number
of digits required by the standard representation of n depends on the base used as
well as on the size of n. For example, the standard representations of the number
discussed in the second sentence of this paragraph require 4, 13, 5, and 4 digits,
respectively, in base 10, base 2, base 8, and base 16.
38
2 Mathematical Preliminaries
How many digits are there in the standard decimal (i.e., base 10) representation
of the integer 34295 ? How many bits (binary digits) does its standard binary representation contain? More generally, given any positive integers n and b, can we find a
formula for the number of digits that will be required for the standard representation
of n base b?
Solution Let’s solve the general case first. Assume we are given positive integers n
and b. Let k denote the (unknown) number of digits in the standard representation of
n base b. Denote those digits by d0 , d1 , d2 , . . . , dk−1 , where d0 is the least significant
digit and dk−1 is the most significant (and is therefore not zero). Each of these digits
lies in the range from 0 to b − 1. (For example, if b is 10, then each digit lies in the
range from 0 to 9.) Moreover, dk−1 is at least 1. Then
n = dk−1 bk−1 + · · · + d2 b2 + d1 b1 + d0 b0 .
Since dk−1 ≥ 1 and all other di ≥ 0, it follows that
n ≥ 1 bk−1 + · · · + 0 + 0 + 0.
That is, n ≥ bk−1 . Similarly, since every di ≤ b − 1, it follows that
n ≤ (b − 1)bk−1 + · · · + (b − 1)b2 + (b − 1)b1 + (b − 1)b0
n ≤ (b − 1)[bk−1 + · · · + b2 + b + 1]
bk − 1
n ≤ (b − 1)
b−1
n ≤ bk−1 .
From this it follows that n < bk . We have now established the inequalities
bk−1 ≤ n < bk ,
where k is the number of digits in the standard representation of n base b. Taking
logarithms base b of each of the quantities above gives us these inequalities:
k − 1 ≤ logb (n) < k.
Since k − 1 is an integer, it follows from Theorem 2.1.4 (c) that k − 1 = logb (n),
or equivalently, k = 1 + logb (n). By Theorem 2.2.8 (c), k = logb (n + 1). Thus
we have proved the following useful fact:
Theorem 2.3.10 Let n and b be positive integers with b ≥ 2. The number of digits
in the standard representation of n base b is
1 + logb (n) = logb (n + 1).
Proof This formula is derived in Example 2.3.9.
39
2.3 Sums and Summation Notation
Now we can answer the question we asked in Example 2.3.9 about the number
of digits in the standard decimal representation of the integer 34295 . The answer,
obtained with the aid of a calculator, is
1 + log10 (34295 ) = 1 + 295 log10 (34) = 1 + 295 × 1.5315 = 1 + 451.79 = 452.
The number of bits in the standard binary representation of 34295 is
1 + log2 (34295 ) = 1 + 295 log2 (34) = 1 + 295 × 5.087 = 1 + 1500.80 = 1501.
The sum in the following theorem appears here and there in analysis of algorithms.
Unfortunately it has no simple closed form, so it is helpful to have good lower and
upper bounds for it.
lg(n) Theorem 2.3.11 The sum
k=1
n
is asymptotic to n. In particular, it satisfies
2k
the inequalities
lg(n) n − 2 − lg(n) <
k=1
n
≤ n − 1.
2k
(2.5)
Proof Using the fact that x − 1 < x ≤ x for all real numbers x, we can write that
lg(n) %
k=1
& lg(n)
n lg(n)
n
n
≤
−
1
<
.
2k
2k
2k
k=1
k=1
These inequalities are equivalent to
lg(n) n
n lg(n)
n
≤
−
lg(n)
<
.
2k
2k
2k
lg(n)
k=1
k=1
(2.6)
k=1
lg(n)
n
that appears in these inequalities can be computed by factoring
2k
k=1
out the n that occurs in every term and then using the geometric sum formula. Note
that the term for k = 0 is missing from the sum and must therefore be subtracted
from it:
The sum
' (lg(n)+1
lg(n) k
n
1
1 − 21
2lg(n)+1 − 1
=
n
=
n
−
1
=
n
−
1
2
2lg(n)
2k
1 − 21
k=1
k=1
1
= n 2 − lg(n) − 1
2
lg(n)
n
.
2lg(n)
Inserting this last formula into line 2.6 we get
=n−
n−
n
2lg(n)
− lg(n) <
lg(n) k=1
n
n
≤ n − lg(n) .
k
2
2
(2.7)
40
2 Mathematical Preliminaries
To get from inequation (2.7) to the desired inequation (2.5) we must estimate the
n
term − lg(n) . Theorem 2.2.8 (a) gives us these useful inequalities:
2
2lg(n) ≤ n < 2lg(n)+1 .
Dividing all terms by 2lg(n) and then multiplying throughout by −1 gives
n
−1 ≥ − lg(n) > −2.
2
It follows that
n
n
n − 2 − lg(n) < n − lg(n) − lg(n) and n − lg(n) ≤ n − 1.
2
2
Combining this pair of inequalities with inequality (2.7) gives the desired result (2.5).
2.3.1
Further Examples
Example 2.3.12 Evaluate the following sums. In each case, give a single numerical
answer.
⎛
⎞
p
5
3
4
50
1
r +1
p⎠
lg(k)
⎝
(c)
(a)
(d)
(b)
k(k + 1)
2r
q
2k
r =0
k=1
p=1
k=100
q=1
1 1
1
1
1
5
1 2 3 4
13
+ +
+
+
=
(b) + + + =
2 6 12 20 30
6
1 2 4 8
4
1
2
3
4
3 3 3
1
2
3
4
2 2
+
(c)
+
+
+
= 1 +
+
+ +
j
j
j
j
1 2
1 2 3
j=1
j=1
j=1
j=1
107
4 4 4 4
=
+
+ + +
1 2 3 4
6
(d) The value of this sum is 0 because the lower limit of summation (100) is larger
than the upper limit (50).
Solution (a)
Example 2.3.13 Write each of the following sums in sigma summation notation:
2
3
4
5
1 1
1
1
(a)
+
+
+
(b) 1 + + +
+ · · · + 10
(1)2
(2)2
(3)2
(4)2
3 9 27
3
Solution (a)
4
k+1
k=1
k2
or
3
j=0
5
j +2
r
or
( j + 1)2
(r − 1)2
r =2
(b)
10
1
3k
k=0
Example 2.3.14 For each of the following sums, evaluate it numerically or express
it in an algebraic closed form.
(a) 1000 + 999 + 998 + 997 + · · · + 1
(b) 100 + 101 + 102 + · · · + 349 + 350
41
2.3 Sums and Summation Notation
(c) 5 + 10 + 15 + · · · + 745 + 750
(e) 1 + 2 + 3 + · · · + (2n − 1) + 2n
2n
m3
r
(g)
t
(f)
r =1
(d) 5 + 8 + 11 + 14 + · · · + 98 =
32
(3k + 2)
k=1
(Express your answer in terms of n.)
t=1+m 2
1000 · 1001
Solution (a) Triangular Sum: 1 + 2 + 3 + · · · + 1000 =
= 500500.
2
(b) A Generalized Triangular Sum: 100 + 101 + 102 + · · · + 349 + 350 =
= 251 · 225 = 56, 475. Alternatively, the sum is
= (350 − 100 + 1) 100+350
2
350 · 351 99 · 100
−
= 56, 475.
2
2
150 · 151
Factor out 5: 5(1 + 2 + 3 + · · · + 150) = 5
= 56, 625.
2
32
32
32
32
32 · 33
(3k + 2) =
3k +
2=3
k + 2 · 32 = 3
+ 64 = 1648.
2
k=1
k=1
k=1
k=1
2n(2n + 1)
Replace n by 2n in the Triangular Sum formula: 1 + 2 + · · · + 2n =
2
= 2n 2 + n.
2n
2n (2n + 1)
n
r=
Replace n by 2 in the Triangular Sum formula:
.
2
(1+2+3+· · ·+350)−(1+2+3+· · ·+99) =
(c)
(d)
(e)
(f)
r =1
(g) A Generalized Triangular Sum: (m 3 − (1 + m 2 ) + 1)
m 2 (m − 1)
1 + m2
2
+ m3
1 + m2 + m3
2
=
.
Example 2.3.15 Consider the following nested pair of loops:
int j, k;
for (k = 2; k <= 50; ++k)
for (j = 1; j < k; ++j)
cout << "hello" << endl;
How many times, exactly, will the word “hello” be sent to output by the block of
code shown above?
Solution When k is given the value 2 in the outer loop, the inner loop then outputs
“hello” just once because j is initialized to 1 and then the body of the loop is executed
only while j < 2. Next, k is incremented to the value 3 in the outer loop; then the
inner loop outputs “hello” twice. Next, k is incremented to the value 4 in the outer
loop; then the inner loop outputs “hello” three times. Etc. When k is given the value
50, the inner loop outputs “hello” 49 times. Then k is incremented once more, but
42
2 Mathematical Preliminaries
now the outer loop exits. The total number of times that “hello” is sent to output is
49 · 50
1 + 2 + 3 + · · · + 49 =
= 1225.
2
Example 2.3.16 Suppose a block of high-level language computer code has 26 instructions, one after another. The 2-way complexity of the block is the number of
2-way interactions that are possible between different instructions in the block.
Assuming that any two instructions can give rise to an interaction, what is the 2-way
complexity of the block of 26 instructions? Hint: suppose we label the instructions
A through Z. Then A can interact with B, C, D, . . ., Z, which is 25 possible interactions. Moreover, B can interact with C, D, . . ., Z, which is 24 more possible
interactions. Etc. How many are there altogether? More generally, if a block of code
has n instructions in sequence, what is the 2-way complexity of the block?
Solution The total number of possible interactions is 25 + 24 + 23 + · · · + 1 =
25 · 26
(n − 1)n
1
= 325. More generally, there are
≈ n 2 possible 2-way interac2
2
2
tions in a block of n instructions. This is why we modularize our programs, to keep
the complexity from growing as the square of the number of instructions.
Example 2.3.17 For each of the following sums, evaluate it or express it in closed
form:
(a) 1 + 4 + 16 + 64 + · · · + 47
(c) 1 + x + x 2 + · · · + x n−1
1
1
1
1
1
+ +
+ · · · + 10 [Hint: k = (1/3)k ]
3
9
27
3
3
30
2n−1
(d)
tp
(e)
3r
(b) 1 +
r =0
p=7
(f) 1 + x 3 + x 6 + x 9 + · · · + x 3n [Hint:x 3k = (x 3 )k .]
(g)
n
(lg(t))k
k=1
Solution (a) Geometric: 1 + 41 + 42 + · · · + 47 =
using a calculator.
(b) 1 + (1/3)1 + (1/3)2 + · · · + (1/3)10 =
48 − 1
= 21, 845, obtained
4−1
(1/3)11 − 1
−.999994355
=
≈ 1.5,
(1/3) − 1
−2/3
by using a calculator.
xn − 1
x n−1+1 − 1
=
if x = 1. The sum is n if x = 1.
(c)
x −1
x −1
t 24 − 1
t 31 − t 7
(d) Factor out t 7 : t 7 (1 + t + t 2 + · · · + t 23 ) = t 7
=
if t = 1. The
t −1
t −1
sum is 24 if t = 1.
2n+1
32n+2 − 1
(32 )n+1 − 1
9n+1 − 1
3r = 1 + 3 + 32 + · · · + 32n+1 =
(e)
=
=
.
3−1
2
2
r =0
43
2.3 Sums and Summation Notation
(x 3 )n+1 − 1
if x = 1; the sum
x3 − 1
is n + 1 is x = 1.
%
(g) (lg(t))1 +(lg(t))2 +(lg(t))3 +· · ·+(lg(t))n = lg(t) 1+(lg(t))1 +
&
(lg(t))2 +· · ·+(lg(t))n−1 if t = 2; the sum is n if t = 2 because lg(2) = 1.
(lg(t))n − 1
= lg(t)
lg(t) − 1
(f) 1 + x 3 + (x 3 )2 + (x 3 )3 + (x 3 )4 + · · · + (x 3 )n =
Example 2.3.18 (With apologies to Mother Goose.) As I was coming from St. Ives,
I met a man with 7 wives. Each wife had 7 sacks, each sack had 7 cats, each cat had 7
kits, each kit had 7 ticks. Ticks, kits, cats, sacks, wives, man. How many were going
to St. Ives?
Solution 1(man) + 7(wives) + 72 (sacks) + 73 (cats) + 74 (kits) + 75 (ticks) =
19, 608.
76 − 1
=
7−1
Example 2.3.19 For each of the following sums, tell which variable the sum is a
function of.
p
n
100
k2
2n+ p
(c)
jk
(b)
(a)
k+1
k=1
n=1
j=1
n
k2
is a function of the variable n; we can assign any positive
k+1
k=1
integer value to n in the sum. For example, assigning n the value 25 produces
25
k2
. The given sum is NOT a function of k; we are not allowed to assign
k+1
k=1
any value we choose to k; the values of k are determined by the value of n;
whatever value of n we pick, the values of k will then be 1, . . . , n.
p
(b)
2n+ p is a function of the variable p; we can assign any positive integer
Solution (a)
n=1
value to p in the sum. For example, if we assign p the value 4 we get the sum
4
2n+4 = 25 + 26 + 27 + 28 . The given sum is NOT a function of n; we cannot
n=1
assign any values we choose to n; instead, the values of n are determined by the
value of p; whatever value of p we pick, the values of n will be 1, 2, . . . , p.
44
2 Mathematical Preliminaries
Fig. 2.6 A perfect tree with 5
levels
(c)
100
j k is a function of k; we can assign any positive integer value to k. For
j=1
example, if we assign k the value 3 we get
100
j 3 . The given sum is NOT a
j=1
function of j. The values of j must be 1, 2, . . . , 100.
Example 2.3.20 A perfect binary tree is one in which every node has either no
children (i.e., is a leaf) or 2 children, and all leaves of the tree are at the same level.
In other words, every non-empty level in the tree is completely filled. It has 5 levels,
numbered 0 (root level), 1, 2, 3, and 4 (bottom level). See Fig. 2.6 for an example.
(a) How many nodes are there in the level number k of a perfect binary tree?
(b) How many nodes are there altogether in a perfect binary tree with levels
0, 1, 2, . . . , n? Solve this by adding up the numbers in all the various levels.
(c) Do parts (a) and (b) over again, this time with a perfect ternary tree, i.e., a tree
in which every non-leaf node has 3 children, and all leaves are at the same level.
Solution (a) 2k
(c) 3k ,
(b) 20 + 21 + 22 + · · · + 2n =
3n+1 − 1
2
2n+1 − 1
= 2n+1 − 1
2−1
Example 2.3.21 For each of the following sums, evaluate it or express it in a closed
form.
(a) 1 + 2 · 2 + 3 · 22 + 4 · 23 + · · · + 10 · 29
(c) t + 2t 2 + 3t 3 + · · · + nt n
(b)
11
k+1
k=0
Hint:
1
= (1/2)k
2k
(Express your answer in terms of t and n.)
Solution (a) Take t = 2 and n = 10 in Theorem 2.3.7:
9217.
2k
1 + 10 · 211 − 11 · 210
=
(2 − 1)2
45
2.3 Sums and Summation Notation
(b) The sum is 1 + 2(1/2) + 3(1/2)2 + 4(1/2)3 + · · · + 12(1/2)11
1 + 12(1/2)13 − 13(1/2)12
6
13
=
= 4(1 + 12/213 − 13/212 ) = 4 + 10 − 10 =
((1/2) − 1)2
2
2
7
4−
≈ 4 − 0.007 = 3.993.
1024
nt n+2 − (n + 1)t n+1 + t
if t = 1.
(c) Factor out t : t (1 + 2t + 3t 2 + · · · + nt n−1 ) =
(t − 1)2
The sum is n if t = 1.
Example 2.3.22 Use Theorem 2.3.4 to find lower and upper bounds for the sum
n √
k. The bounds will be functions of n.
k=1
√
Solution We apply Theorem 2.3.4 to the function f (x) = x on the interval [1, n].
Since this f (x) is strictly increasing on [1, n] for every positive integer n, Theorem
2.3.4 gives us
n
n
n √
√
√
√
√
1+
x dx <
k < n+
x dx for all integers n > 1 .
1
√
Since
1+
x dx =
1
k=1
x 1/2 dx =
2 3/2
the inequalities above yield
x
3
n √
2 3/2 2
2
2
k < n 1/2 + n 3/2 −
n −
<
3
3
3
3
for all integers n > 1 .
k=1
n
2
1
7
is roughly equal to n ln(2) ≈
n. Also give a
Example 2.3.23 Explain why
k
10
k=1
more precise approximation.
n
n
2
1
1
= H(n) ≈ ln(n) (roughly), so
= H(2n ) ≈ ln(2n ) =
k
k
k=1
k=1
7
n ln(2) ≈ 0.693n ≈
n. A more accurate approximation uses Euler’s constant:
10
n
2n
1
1
= H(n) ≈ ln(n) + 0.5772, so
= H(2n ) ≈ ln(2n ) + 0.5772 =
k
k
k=1
k=1
n ln(2) + 0.5772 ≈ 0.693n + 0.5772.
Solution
Example 2.3.24 Use the inequalities we have derived and a pocket calculator to find
a numerical lower bound and a numerical upper bound for the harmonic sum 1 +
1 1
1
+ +· · ·+
. Also give the best estimate you can using Euler’s approximation.
2 3
100
46
2 Mathematical Preliminaries
1
Solution As shown in Theorem 2.3.5,
+ ln(100) < H(n) < 1 + ln(100). Using
100
a calculator we get 4.615 as a lower bound and 5.605 as an upper bound for H(100).
Euler’s approximation says that a very good estimate for H(100) is 0.5772 + ln(100)
≈ 5.182.
Example 2.3.25 How many bits are there in the standard binary representation of
the integer
725194 × 386011?
Solution As shown in Example 2.3.9, the answer is 1 + log2 (725194 × 386011).
Using the fact that logb (xy) = logb (x) + logb (y) and the fact that log2 (x) =
log10 (x)
, we obtain the answer
log10 (2)
log10 (725194) log10 (386011)
= 1 + 19.468 + 18.588 = 39
1+
+
log10 (2)
log10 (2)
with the aid of a calculator.
2.3.2
Exercises
2.3.26 In each of these two problems, show the computations leading to a single
numerical answer.
7
9
(a) If f (x) = lg(x), then
f (k) =?
(b) If f (x) = lg(x), then
f (k) =?
k=1
k=1
2.3.27 Write the following sums in sigma summation notation:
4
6
1000
8
2
(a) 1 + 4 + 7 + 10 + · · · + 97 + 100
(b) √ + √ + √ + √ + · · · + √
1
3
7
999
5
2.3.28 (a) Write the following sums in “3 dot” notation. Note that despite superficial
n
k
resemblances, they are actually different sums:
k n and
kn .
k=1
n=1
(b) Write the following sums in “3 dot” notation. Note that despite superficial ren
k
k+1
k+1
semblances, they are actually different sums:
and
.
n+1
n+1
k=1
n=1
2.3.29 Suppose a positive integer n is given in binary representation by a sum of the
form
n = bk 2k + bk−1 2k−1 + · · · + b2 22 + b1 21 + b0 20 .
where the numbers b0 , b1 , b2 , . . . , bk denote the binary digits (the bits) in the standard binary representation of n (i.e., no leading zeroes; see Example 2.3.9). Express
47
2.3 Sums and Summation Notation
the value of the number k here in terms of the integer n. Then write the equation
above in sigma summation notation.
2.3.30 There is a well-known algorithm for deriving the binary representation of a
positive integer n from its decimal (base 10) representation. Using the notation in
Exercise 2.3.29 above, the least significant bit b0 is calculated as the remainder of n
after division by 2. That is, b0 = 0 if n is even, but b0 = 1 if n is odd. That is, b0 =
parity(n), so b0 = n − 2n/2. Then we divide n by 2 and discard the remainder to
produce a smaller integer, call it n 1 , which is given by the equation
n = bk 2k−1 + bk−1 2k−2 + · · · + b2 21 + b1 20
n1 =
2
(the b0 term has been discarded). Using the same procedure, we can now calculate
the value of b1 : it’s the remainder after n 1 is divided by 2; that is,
n
n n 1
=
−2 2 .
b1 = parity(n 1 ) = n 1 − 2
2
2
2
Now we divide n 1 by 2 and discard the remainder to produce a smaller integer n 2 :
n 1
= bk 2k−2 + bk−1 2k−3 + · · · + b2 20 .
n2 =
2
(a) Express b2 as a difference of two floor expressions involving n.
(b) Express b3 in a similar way as a difference of two floor expressions involving n.
Generalize.
(c) For all positive integers n let B(n) denote the number of 1 s in the standard
binary representation of n. Then B(1) = 1 , B(2) = 1, B(3) = 2, B(4) = 1,
B(5) = 2, B(6) = 2 , B(7) = 3 , and so on. In the notation we have used in this
exercise and Exercise 2.3.29,
B(n) = b0 + b1 + b2 + · · · + bk−1 + bk
Into this sum, substitute the formulas for b0 , b1 , b2 , b3 , b4 etc. derived
in earlier parts of this exercise, and simplify a little to derive this formula:
lg(n) n B(n) = n −
. You will need to use the fact that for all positive
2p
p=1
n
integers n, lg(n)
= 0 (Theorem 2.2.8 (b)).
2
+1
2.3.31 For each of the following sums, tell what variable the sum is a function
of. In other words, if you computed each of those sums, what would the resulting
expression be a function of? See Example 2.3.19 for a similar exercise.
m
20
v
2p
100
m−k
i
lg( p+t)
(c)
v
(d)
r3
(e)
in
(a)
(b)
m+k
r=p
k=1
p=1
i=1
2.3.32 Evaluate the following sums or express them in closed form.
n=i
48
2 Mathematical Preliminaries
(a) 50 + 49 + 48 + 47 + · · · + 10
(c) 1 + 2 + 3 + · · · + (n − 3) + (n − 2)
(d) n + (n + 1) + (n + 2) + · · · + (3n − 1) + 3n
1000
(2k + 7)
(e)
(b) 100 + 102 + 104 + 106 + 108 + · · · + 200
(Express your answer in terms of n.)
(Express your answer in terms of n.)
n2
(f)
i (Express your answer in terms of n.)
k=1
(g)
i=n
2
3
n
1
+ + + · · · + (in terms of n)
n
n
n
n
(h) n + 2n + 3n + · · · + n 2 (express in terms of n)
2.3.33 Evaluate the following sums or express them in closed form.
(a) 25 + 26 + 27 + . . . + 75
(b) 100 + 99 + 98 + 97 + · · · + 31
(c) 1 + 2 + 3 + · · · + (n + 1) + (n + 2)
(Express your answer in terms of n.)
(d) 2n + (2n + 1) + (2n + 2) + · · · + (5n − 1) + 5n (Express your answer in terms of n.)
(e)
100
√
2 n
(7k + 2)
(f)
k=1
(g) n 2 + 2n 2
+ 3n 2 + · · · + 100n 2
2
3
n
1
+
+
+ ··· +
(h)
n+1
n+1
n+1
n+1
√
j= n
j (Express your answer in terms of n.)
(Express your answer in terms of n.)
(Express your answer interms of n.)
2.3.34 Consider the block of code in Example 2.3.15.
(a) How many times, exactly, will the initialization “j = 1” be executed in that
block of code?
(b) How many times, exactly, will the inequality “j < k” be tested (i.e., evaluated)?
(c) How many times, exactly, will the increment operation “++j” be executed?
2.3.35 Give a closed form for each of the following sums.
(a) 2 p +22 p +23 p +24 p +· · ·+2mp =
Hint: 2xy = (2x )y = (2y )x .
m
2k p , where p is a non-zero real constant.
k=1
(b) (a + b)2 + (a + b)3 + (a + b)4 + · · · + (a + b)n =
n
(a + b)k , where a and
k=2
b are real constants with a + b = 1.
n
n2k , where n is a positive integer.
(c) n + 2n + 4n + 8n + · · · + n2n =
k=0
2.3.36 Give a closed form for each of the following sums.
(a) 3 p+1 + 3 p+2 + 3 p+3 + · · · + 3 p+m =
constant. Hint: 3x+y = 3x · 3y .
m
k=1
3 p+k , where p is a non-zero real
49
2.3 Sums and Summation Notation
(b) (1 + x)2 + (1 + x)3 + (1 + x)4 + · · · + (1 + x)n =
n
(1 + x)k , where x is a
k=2
non-zero real constant.
n
2k
1
2
4
8
2n
(c)
+ + + + ··· +
=
, where n is a positive integer.
n
n
n
n
n
n
k=0
2.3.37 (a) Let t be a real number not equal to −1. Give a closed form for the
following sum:
2 3
n p
n t
t
t
t
t
+
+
+ ··· +
=
1+
t +1
t +1
t +1
t +1
t +1
p=0
Your answer should be one of the following:
(t + 1)n+1 − t n+1
(t + 1)n − t n+1
t n + t n+1
(ii)
(iii)
(t + 1)n
(t + 1)n
(t + 1)n
(b) Let x be a real number not equal to 1. Give a closed form for the following sum:
(i)
1+
√
9
9
√
√
√
√
x + ( 9 x)2 + ( 9 x)3 + · · · + ( 9 x)9 =
( 9 x)r
r =0
2.3.38 A clever man has done the sultan a tremendous favor, and the sultan asks the
man to name his own payment. The man says, “Oh, I don’t really want very much.
Just place a grain of corn on one square of this chessboard, then 2 grains on another
square, and twice 2 grains on another square, and so on, doubling the number of
grains each time, until all the squares of the board are filled.” How many grains of
corn has the man asked to be paid? Express the answer in closed form. (Note: for
those who don’t know, there are 64 squares on a chessboard.) How many decimal
digits will be required to express the answer in base 10?
2.3.39 Prove the part of Theorem 2.3.4 that deals with increasing functions.
2.3.40 (a) Prove that if f (x) is a strictly decreasing function defined over an
interval [m − 1, n + 1], where m and n are integers such that m ≤ n, then
n+1
f (x)dx <
n
n
f (k) <
k=m
m
f (x)dx.
m−1
(b) Prove that if f (x) is a strictly increasing function defined over the interval
[m − 1, n + 1], then
n
f (x)dx <
m−1
n
k=m
n+1
f (k) <
f (x)dx.
m
50
2 Mathematical Preliminaries
2.3.41 Use Theorem 2.3.4 to prove that for all real numbers p > 0,
1
1
kp < np +
n p+1 <
n p+1
p+1
p+1
n
1+
k=1
2.3.42 Evaluate each of the following sums by computing the values of all the
fractions in the sum and adding them all up (use a calculator to do this). In each case,
compare your answer with the Euler approximation given in Theorem 2.3.6 (use a
calculator for this also).
1
1 1 1 1
1
1 1 1 1
(b) + + + + · · · + .
(a) + + + + · · · +
1 2 3 4
17
1 2 3 4
15
2.3.43 (a) Use a calculator to help you compute a fairly exact numerical value for
the sum
1500
1
1
1 1
=
.
1 + + + ··· +
2 3
1500
k
k=1
(b) Use a calculator to help you compute a fairly exact numerical value for the sum
1+
2000
1
1
1 1
+ + ··· +
=
.
2 3
2000
k
k=1
2.3.44 (a) Use a calculator to help you compute a fairly exact numerical value for
the sum
700
1
1
1
1
1
+
+
+ ··· +
=
.
250 251 252
700
k
k=250
(b) Use a calculator to help you compute a fairly exact numerical value for the sum
650
1
1
1
1
1
+
+
+ ··· +
=
.
400 401 402
650
k
k=400
2.3.45 (a) Explain why
mn
m
n
1
1 1
is roughly equal to
+
for all large positive
k
k
k
k=1
k=1
k=1
integers m and n.
100n
n
1
1
(b) Explain why
is approximately equal to 5.18 +
for all large positive
k
k
k=1
k=1
integers n. You may use a calculator.
2.3.46 (a) Show that for all large positive integers n the sum
1
1
+
+
2n + 1 2n + 2
1
1
+ ··· +
is approximately equal to 0.405. You may use a calculator.
2n + 3
3n
51
2.3 Sums and Summation Notation
(b) Show that for all large positive integers n the sum
··· +
1
1
1
+
+
+
n+1
n+2
n+3
1
is approximately equal to 0.693. You may use a calculator.
2n
2.3.47 Find an approximate closed form for the sum
n
n
n
n
n
+ + + + ··· + .
1
2
3
4
n
2.3.48 (Index Shifting in Sigma Sums) In Example 2.3.13, p. 41, the instruction is
3
4
5
2
to write the sum 2 + 2 + 2 + 2 in sigma summation notation. Three answers
1
2
3
4
were given:
4
3
5
k+1
j +2
r
or
or
2
2
k
( j + 1)
(r − 1)2
k=1
r =2
j=0
It is possible to pass from one of these answers to another by a mechanical process
of substitution.
k=4
k+1
For example, starting from the first sum, which we write in the form
k2
k=1
(emphasizing that the upper limit 4 is the last value of k), replace k everywhere
j+1=4
j +1+1
. Simplification reduces this to the answer
by j + 1 to obtain
( j + 1)2
j+1=1
j=3
j=0
r −1=4
r −1+1
j +2
.
Alternatively,
replace
k
everywhere
by
r
−1
to
obtain
,
2
( j + 1)
(r − 1)2
r −1=1
which simplifies to
r =5
r =2
r
.
(r − 1)2
(a) What sum is obtained if you replace k everywhere by k+a in the sum
b
k−a
k=a
(b) The sum
21
lg(m +3)) is equal to the sum
m=2
?
k+a
?
lg( p)) (give the correct limits
p=?
of summation in the second sum).
2.3.49 (a) Calculate the exact number of digits in the standard decimal representation of 77029145 .
(b) Calculate the exact number of bits in the standard binary representation of
102499 .
2.3.50 Use mathematical induction to prove that for every positive integer n,
n
n(n + 1)(2n + 1)
k2 =
.
6
k=1
52
2 Mathematical Preliminaries
2.3.51 (This exercise requires a knowledge of combinatorial counting techniques.)
Let n denote a positive integer. Consider any set S that contains exactly n elements.
Label the elements of S by x1 , x2 , . . . , xn . Now take k to be any positive integer
somewhat less than n.
(a) How many ways can we construct a subset of S of size k?
(b) How many ways can we construct a subset of S of size k that contains the last
element xn ?
(c) How many ways can we construct a subset of S of size k that contains xn−1 but
not xn ?
(d) How many ways can we construct a subset of S of size k that contains xn−2 but
not xn−1 and not xn ?
(e) How many ways can we construct a subset of S of size k that contains xk+1 but
not any of the elements xk+2 . . . xn ?
(f) How many ways can we construct a subset of S of size k that contains xk but
not any of the elements xk+1 . . . xn ?
(g) Use parts (a) through (f) above to help explain why
n
k−1
k
k+1
n−3
n−2
n−1
.
=
+
+
+· · ·+
+
+
k
k−1
k−1
k−1
k−1
k−1
k−1
n
and perform
k
a lot of cancellations you produce the following summation formula:
(h) Show that if you divide both sides of the equation in part (g) by
(n − k)(n − k − 1)(n − k − 2)...(1)k
k (n − k)k (n − k)(n − k − 1)k
+
+
+· · ·+
= 1.
n n(n − 1)
n(n − 1)(n − 2)
n(n − 1)(n − 2)(n − 3)...(k)
(i) Show that by factoring out a common factor on the left side and then performing
a little bit of algebra, you can produce the following summation formula:
1+
(n − k)(n − k − 1)
(n − k)(n − k − 1)(n − k − 2)...(1)
n
n−k
+
+ ··· +
= .
n−1
(n − 1)(n − 2)
(n − 1)(n − 2)(n − 3)...(k)
k
(j) Let b denote the integer n − 1, and let a denote the integer n − k. (Note that
a ≤ b.) Then
1+
a!
a a(a − 1) a(a − 1)(a − 2)
+
+
+· · ·+
=
b b(b − 1) b(b − 1)(b − 2)
b(b − 1)(b − 2)...(b − a + 1)
.
(fill in the blank on the right side of the equation)
2.4
Factorials
Definition 2.4.1 The factorial function is defined on the non-negative integers as
follows:
0! = 1, 1! = 1, and n! = 1 · 2 . . . (n − 1) · n = n · (n − 1) . . . 2 · 1 for all n ≥ 2,
53
2.4 Factorials
where “n!” is read “n factorial”. Alternatively, one can define factorials recursively
as follows:
0! = 1, and n! = (n − 1)! · n for all n ≥ 1.
Examples 2.4.2 (a) Is (2n)! = 2(n!) in general?
No. The left side is (2n)! = (2n)(2n − 1)(2n − 2) . . . 2 · 1. The right side is
2[n(n − 1) . . . 2 · 1].
(b) Express the following product in terms of factorials: 2 · 4 · 6 · 8 . . . (2n).
The solution involves noticing that each factor is divisible by 2, so we could
factor out all the 2 s and leave 1 · 2 · 3 · 4 . . . n. There are n 2s, so we find that
2 · 4 · 6 · 8 . . . (2n) = 2n · n!.
(c) Express the following product in terms of factorials: 1 · 3 · 5 · 7 . . . (2n − 1).
The solution for this involves introducing “missing” factors that could give us
a factorial, and then compensating by dividing the expression by the introduced
factors. That is,
1 · 3 · 5 · 7 . . . (2n − 1) =
1 · 2 · 3 · 4 · 5 · 6 · 7 · 8 . . . (2n − 1)(2n)
(2n)!
= n
.
2 · 4 · 6 · 8 . . . (2n)
2 · n!
The factorial function n! grows extremely rapidly as n increases; e.g. 10! ≈
3, 000, 000. In fact, with most C/C++ compilers, the value of 13! overflows a long
int variable. Thus, when computing with factorials in a C/C++ program, you’ll
almost always need to use double precision floating point variables instead of integer
or long integer variables.
We can get some idea of how fast the factorial function grows by using our theorem
on approximation of sums by integrals. We start by deriving a valuable approximation
for ln(n!), which turns up from time to time in the analysis of algorithms.
Theorem 2.4.3 For all integers n ≥ 2,
n ln(n) − n + 1 < ln(n!) < (n + 1) ln(n) − n + 1.
(2.8)
Proof Use Theorem 2.2.3 (a) to write
ln(n!) = ln(1) + ln(2) + ln(3) + · · · + ln(n) =
n
ln(k).
k=1
This identity allows us to use Theorem 2.3.4 to obtain lower and upper bounds for
ln(n!). The function ln(x) is strictly increasing on the open interval (0, +∞), so
Theorem 2.3.4 gives us
n
ln(x)dx <
ln(1) +
1
n
k=1
n
ln(k) < ln(n) +
ln(x)dx for all n > 1.
1
54
2 Mathematical Preliminaries
)
From an integral table or integration by parts we find that ln(x)dx = x ln(x) − x,
so
n
*
+x=n
*
+x=n
0 + x ln(x) − x
<
ln(k) < ln(n) + x ln(x) − x
for all n > 1,
x=1
x=1
k=1
and thus
[n ln(n) − n] − [0 − 1] < ln(n!) < ln(n) + [n ln(n) − n] − [0 − 1].
This simplifies to the inequalities stated in the theorem.
Corollary 2.4.4 For all integers n > 1,
% n &n
% n &n
e
< n! < ne
.
e
e
Proof Starting from the inequality (2.8) in Theorem 2.4.3, we find that
ln(n n ) − n + 1 < ln(n!) < ln(n n ) + ln(n) − n + 1.
Exponentiating gives
eln(n
n )−n+1
< eln(n!) < eln(n
n )−n+ln(n)+1
,
which in turn gives
n n e−n e1 < n! < n n e−n ne1 .
If you think of n! as growing roughly at the same rate as n n you’ll be in the
right ballpark. When we need an even more accurate estimate for the growth of n!
we can use an approximation discovered several hundred years ago by the English
mathematician John Stirling.
% n &n √
2π n for all large values of n. More
Theorem 2.4.5 Stirling’s Formula: n! ≈
e
precisely,
n!
lim
= 1.
√
n
n→∞ (n/e)
2π n
Proof This can be found in most books on advanced calculus. The proof is somewhat
lengthy.
In Stirling’s approximation, the percent error approaches zero as n → ∞. This is
the meaning of the formulation involving the limit of 1. By contrast, the absolute error, i.e., the difference between the left and right sides of the approximation formula,
grows without bound as n → ∞.
55
2.4 Factorials
2.4.1
Further Examples
13!
20!
Example 2.4.6 Compute
and
as efficiently as possible without using a
4! 9!
15! 5!
calculator.
10 · 11 · 12 · 13
13!
=
= 5 · 11 · 13 = 715;
4! 9!
2·3·4
20!
16 · 17 · 18 · 19 · 20
=
= 8 · 17 · 6 · 19 = 15504.
15! 5!
2·3·4·5
Solution
Example 2.4.7 In each case below, write out the three largest factors of the product
indicated. Assume that n is a positive integer of size at least 6.
(b) (n/2)!
(c) (2n )!
(a) (n 2 )!
Solution (a) (n 2 )(n 2 − 1)(n 2 − 2) . . . (1)
(b) n/2(n/2 − 1)(n/2 − 2) . . . (1)
(c) 2n (2n − 1)(2n − 2) . . . (1)
Example 2.4.8 Simplify the expressions
n is a positive integer.
(n + 1)!
n!
(n − 1)!
,
, and
, where
n!
(n + 2)!
(n + 1)!
(n + 1)!
= n + 1;
n!
n!
1
n!
=
=
(n + 2)!
(n + 2)(n + 1) · n!
(n + 2)(n + 1)
(n − 1)!
(n − 1)!
1
=
=
(n + 1)!
(n − 1)!n(n + 1)
n(n + 1)
Solution
Example 2.4.9 Calculate 12! exactly, and then compare with the value you get using
Stirling’s approximation and a calculator.
12
Solution 12! = 479, 001, 600;
e
12
√
2π · 12 ≈ 475, 687, 500.
479001600 − 475687500
The percent error in the approximation is
·100 ≈ 0.7 %,
479001600
i.e., the error is less than 1 % (even though the magnitude of the error is more than
3 million).
2.4.2
Exercises
2.4.10 Without using a calculator, find the exact value of
82!
42!
(a)
(b)
.
3! 79!
3! 39!
56
2 Mathematical Preliminaries
2.4.11 In each case below, write out the four largest factors of the product indicated.
Assume that n is a positive integer of size at least 16.
(a) (2n + 3)!
(b) (lg(n))!
2.4.12 In each case below, write out the four largest factors of the product indicated.
Assume that n is a positive integer of size at least 3.
(a) (5n + 2)!
(b) (n!)!
2.4.13 Simplify the following expressions in which n is a positive integer:
(3n + 1)!
(2n − 1)!
(a)
(b)
(3n − 1)!
(2n + 1)!
2.4.14 Use the multiplication key on your calculator to compute 15! (don’t use
the ! key if your calculator has one). Then use the calculator to compute Stirling’s
approximation for 15!. Calculate the percent error in the approximation.
2.5
Remainders in Integer Division
In this section we’ll explore an arithmetic concept that plays an important role in
a number of interesting algorithms. We begin with the mathematical definition of a
“remainder” in integer division.
Definition 2.5.1 Let n be any integer and let m be a strictly positive integer. Then
the remainder of n divided by m (also called the residue of n modulo m) is defined to
be the integer
n
.
n−m
m
We denote this integer by the expression n mod m.
Examples 2.5.2 27 mod 12 = 3 because 27 − 12
27 − 12(2) = 3.
27
12
= 27 − 122.25 =
33
= 33 − 74.714 = 33 − 7(4) = 5.
7 72
72 mod 9 = 0 because 72 − 9
= 72 − 98 = 72 − 9(8) = 0.
9
6
6 mod 10 = 6 because 6 − 10
= 6 − 100.6 = 6 − 10(0) = 6.
10
−27
−27 mod 12 = 9 because −27−12
= −27−12−2.25=−27−12(−3)=9.
12
−33
−33 mod 7 = 2 because −33−7
= −33−7−4.714 = −33−7(−5) = 2.
7
33 mod 7 = 5 because 33 − 7
57
2.5 Remainders in Integer Division
−72
= −72 − 9−8 = −72 − 9(−8) = 0.
9
−6
−6 mod 10 = 4 because −6 − 10
= −6 − 10−0.6 = −6 − 10(−1) = 4.
10
−72 mod 9 = 0 because −72 − 9
Integers of the form n mod 12 can be visualized in a simple way involving a clock.
Imagine that the hours on the face of a clock are labeled 1, 2, 3, . . . , 11, and 0 (instead
of the usual 12). Then for any positive integer n the integer n mod 12 will turn out
to be the label we stop at if we advance the hour hand clockwise by n hours, starting
at label 0. In Examples 2.5.2 we saw that 27 mod 12 = 3 and indeed, advancing the
hour hand by 27 h, starting at 0, takes the hand past 0 twice (24 h go by) and brings it
to the label 3. As a second example, 60 mod 12 = 0 because advancing the hour hand
by 60 h moves it full circle 5 times. For a negative integer n we can obtain n mod
12 by turning the hour hand backward (counter-clockwise) by |n| hours, starting at
label 0. in Examples 2.5.2 we saw that −27 mod 12 = 9 and now we see that turning
the hour hand backward by 27 h brings it to the label 9. As still another example, −5
mod 12 = 7 because turning the hour hand backward by 5 h brings it to the label 7.
And of course −36 mod 12 = 0.
It is easy to generalize this clock interpretation to expressions of the form
n mod m, where m is any positive integer. Just replace the standard 12 h clock
with a clock whose circular face is divided into m periods of time with labels
{0, 1, 2, . . . , m − 1}.
You might ask whether the integer n mod m, as we’ve specified it in Definition
2.5.1, always turns out to belong to the set {0, 1, 2, . . . , m − 1}. The following
theorem says that it does.
Theorem 2.5.3 For every integer n and positive integer m the remainder n mod m
satisfies the inequalities 0 ≤ n mod m < m. Furthermore, if 0 ≤ n < m, then
n mod m = n.
n
n
n
n
≤ .
Proof By Theorem 2.1.4 (a), with
in place of x, we have
−1<
m
m
mn m
≤ n.
Multiplying throughout by the positive integer m yields n − m < m
m
n
Subtracting n from each part yields −m < −n + m
≤ 0. Multiplying all parts
m
n
by −1 reverses the directions of the inequalities to produce m > n − m
≥ 0.
m
This is equivalent to the double inequality stated in the theorem.
n
n
= 0. It follows
For the second part, 0 ≤ n < m implies 0 ≤
< 1, and so
m
m
n
that n mod m = n − m
= n.
m
Algorithms that make use of “modular arithmetic” typically involve addition and
multiplication of integers followed by a “mod” operation. For example, most encryption algorithms involve computing many remainders of the form n p mod m for
58
2 Mathematical Preliminaries
positive integers n, p, and m that are several hundred digits long. If the only way this
computation could be performed was to first raise n to the power p and afterward
carry out the “mod” operation, the computation of n p would generate integers having
trillions of trillions of digits—far too many to be stored in a computer’s memory.
Fortunately, the computation can be accomplished using the following useful facts
about modular arithmetic.
Theorem 2.5.4 For all integers n and q and positive integers m,
(n + m q) mod m = n mod m.
Proof By the formulas in Definition 2.5.1,
n
n+mq
=n+mq −m
(n + m q) mod m = n + m q − m
+q .
m
m
Since q is an integer, it can be extracted from the floor by Theorem 2.1.5 (a), which
results in
n
n
=n−m
.
n+mq −mq −m
m
m
We recognize this last expression as n mod m by Definition 2.5.1.
Theorem 2.5.5 Division Theorem. For every integer n and and positive integer m,
there exist unique integers q and r such that n = q m + r and 0 ≤ r < m.
n
by the letter q and n mod m by the letter r , then by
m
Definition 2.5.1 and Theorem 2.5.3 the integers q and r satisfy the properties in the
theorem. We still must prove that they are unique.
Suppose that we have q1 , q2 , r1 , r2 , 0 ≤ r1 , r2 < m such that n = q1 m + r1 and
n = q2 m + r2 . Then
q1 m + r 1 = q2 m + r 2
Proof If we denote
which can be rewritten as
(q1 − q2 )m = r2 − r1
which yields
q1 − q2 =
r1 − r2
.
m
r1 − r2
This shows that
is an integer.
m
We know that 0 ≤ r2 < m. From 0 ≤ r1 < m we deduce that −m < −r1 ≤ 0.
By combining these inequalities we get −m < r2 − r1 < m, which yields
r1 − r2
r1 − r2
−1 <
< 1. Since we have shown that
is an integer, and we now
m
m
r1 − r2
is 0. This
see that is lies strictly between −1 and 1, we must conclude that
m
implies that r1 = r2 , which then yields q1 = q2 .
59
2.5 Remainders in Integer Division
Example 2.5.6 As an example of the usefulness of the Division Theorem 2.5.5, let’s
use it to prove the following: for all integers n and positive integers m,
n n + 1
− 1.
=
m
m
The proof goes as follows: let q and r denote the unique integers forwhich n= qm+r
n
n+1
and 0 ≤ r < m. Then
−1 = q.
= q. We’ll now proceed to prove that
m
m
We note that
r +1
r +1
n+1
−1= q +
−1=q +
− 1.
m
m
m
1
r +1
r +1
= 1.
Now 1 ≤ r + 1 ≤ m, so
≤
≤ 1, from which it follows that
m
m
m
Thus,
r +1
q+
−1=q +1−1=q
m
as desired. This completes the proof.
Theorem 2.5.7 Let a and b be any integers, and let m be a positive integer. The remainder (a + b) mod m can be computed by first computing the separate remainders
of a and b, adding those remainders together, and then computing the remainder of
their sum. In symbols,
(a + b) mod m = ((a mod m) + (b mod m)) mod m.
Similarly for the difference a − b and the product ab :
(a − b) mod m = ((a mod m) − (b mod m)) mod m,
(ab) mod m = ((a mod m)(b mod m)) mod m.
Proof We start by using Theorem 2.5.5 to express a as q1 m + r1 and b as q2 m + r2 ,
where 0 ≤ r1 , r2 < m. In particular, the proof of this theorem tells us that a mod m =
r1 and b mod m = r2 . Then
(a + b) mod m = (q1 m + r1 + q2 m + r2 ) mod m.
We can use Theorem 2.5.4 to reduce the above expression to
(r1 + r2 ) mod m.
The right hand side of the equation in the theorem is equal to
((a mod m) + (b mod m)) mod m = (r1 + r2 ) mod m
The proofs of the equations for the difference a − b and the product ab are very
similar and left as an exercise.
60
2 Mathematical Preliminaries
As an example, let’s use Theorem 2.5.7 to compute the remainder 756 mod 11
without first computing 756 . We begin by computing the remainder 75 mod 11 = 9.
Then by Theorem 2.5.7,
752 mod 11 = ((75 mod 11)(75 mod 11)) mod 11 = (9)(9) mod 11 = 81 mod 11 = 4.
Again by Theorem 2.5.7 and the values we’ve already computed, we find that
753 mod 11 = ((75 mod 11)(752 mod 11)) mod 11 = (9)(4) mod 11 = 36 mod 11 = 3.
Finally,
756 mod 11 = ((753 mod 11)(753 mod 11)) mod 11 = (3)(3) mod 11 = 9 mod 11 = 9.
You may wonder how n mod m is defined when m < 0. The answer is that we
seldom find it useful to have that operation defined, so we will not give a definition
in this text.
2.5.1
Implementation of the Mod Operator in C and C++
We now consider the problem of using C or C++ to compute remainders of pairs
of integers. Suppose that n and m are program variables of some integer data type,
and let n and m denote the mathematical values of those variables. Assume in what
follows that m > 0. How can we use the program variables n and m to compute
n mod m, the remainder of n divided by m?
n
. As we noted in
By our mathematical definition, n mod m = n − m
m
n
Sect. 2.1, if n is positive, the value of the C/C++ expression n/m is
, so the
m
C/C++ expression
n − m ∗ (n/m)
(2.9)
can be used as the remainder n mod m when n > 0. Trivially, this expression also
gives the correct value when n is 0.
The C and C++ languages have a “modulus” operator, which is denoted by the
symbol %. The standards for the C and C++ languages require that the expression
n % m have exactly the same value as the expression in formula 2.9 for all non-zero
values of m. Thus n % m returns n mod m when n ≥ 0. (Keep in mind that we are
assuming m > 0.)
When n < 0 the expression n % m will return n mod m if and only if the integer
n
. Some C and C++ compilers implement n/m
division expression n/m returns
m
that way. However, the majority of C and C++ compilers implement it as the result
n
of the function truncate applied to the real number . This function eliminates
m
n
when
the fractional part of the real number. The result is that n/m will return
m
n
n ≥ 0 and
when n < 0. For these compilers the expression n % m does not
m
2.5 Remainders in Integer Division
61
return n mod m when n < 0, except in those cases where the fractional part of the
n
real number
is 0.
m
It would be legal for a compiler to implement the integer division expression n/m
n
as the value obtained by applying the function round to the real number
when
m n
if
n < 0. With this function, the integer division expression n/m will return
m
n n n
n
n
>
− , and will return
otherwise.
−
m
m
m
m
m
The possibilities for n/m described above are the only sensible choices when
n < 0, because they are the only ones that produce values for n % m that lie strictly
between −m and m, which is where we expect
n to fall. Note that for
n“remainders”
or
.
each of these possibilities, n/m returns either
m
m
n
If the value returned is
then, as we noted earlier, n % m will return n mod
m
m, which by Theorem A.5.3 is non-negative.
n
n
If the value returned by n/m is not
, then it must be returning
, in which
m
m
n
case Theorem 2.1.4 (f) tells us that
is not an integer and
n
nm
=1+
.
m
m
It follows that
% thevalue
n
nn % m returns
n &
=n−m 1+
=n−m−m
= n mod m − m.
n−m
m
m
m
From this we can deduce that n%m + m will return n mod m. By Theorem 2.5.3 we
can also see that the value returned by n%m is negative.
The preceding paragraph implies the following:
n if n%m returns a non-negative
value, then the value returned by n/m must be
, so by the paragraph before that,
m
n%m returns
n if
n%m returns a negative value, the value of n/m
nnmod m. Similarly,
, so it must be
, and thus n % m + m returns n mod m. From
cannot be
m
m
these observations we can write the following portable function implementing the
mathematical mod function.
// This function assumes that the parameter m has a strictly positive value.
// It returns n mod m, i.e., the remainder (or residue) of n divided by m,
// which is defined mathematically as n - m * floor(n/m).
// The function has been written so as to be portable among
// C/C++ compilers.
int mod(int n, int m)
{
if (m <= 0)
// We do not define n mod m for m <= 0.
"error; throw an exception"
int n_percent_m = n % m; // Store it for multiple use
if (n_percent_m >= 0) // The % operator correctly implements the
return n_percent_m;
// mathematical mod operator in this case.
else
return n_percent_m + m;
// This is when n % m < 0.
}
62
2.5.2
2 Mathematical Preliminaries
Exercises
2.5.8 Evaluate the following: 3 mod 2; 2 mod 3; 17 mod 5; 34 mod 6;
63 mod 7; −3 mod 2; −2 mod 3; −17 mod 5; −34 mod 6; −63 mod 7.
2.5.9 Evaluate n mod 1 for all integers n. Evaluate n mod 2 for all integers n. What
are the possible values of n mod 100, where n can be any integer (positive, negative,
or zero)?
2.5.10 Prove the identity (a b) mod m = ((a mod m)(b mod m)) mod m stated in
Theorem 2.5.7.
2.5.11 Prove the following variants of the identities in Theorem 2.5.7:
(a) (a + b) mod m = (a + (b mod m)) mod m.
(b) (a b) mod m = ((a mod m)b) mod m.
These identities are examples of the rule that says when we want to calculate
the remainder modulo m of any expression involving addition, subtraction, and
multiplication, we can apply the “mod” operator at any or all stages of the
evaluation process. For example
(a − bc + d) mod m = ((a mod m) − b(c mod m) + d) mod m.
2.5.12 Without using an electronic calculator, compute 214365978 mod 11. Show
the steps in your computation. Hint: n 8 = n 4 × n 4 ; n 4 = n 2 × n 2 .
2.5.13 Use a similar idea to Example 2.5.6 to prove that for all integers n and m and
all positive integers k,
m n − 1 − m n ≥
+
− 1.
k
k
k
[Hint: express m in terms of its integer quotient and remainder when divided by k,
and substitute that expression for m into the left side of the inequality above. Make
use of Theorem 2.1.5 (a).]
2.5.14 Prove that for all integers n, (−n) mod 2 = n mod 2. Give an example to
show that it is not always true that (−n) mod 3 = n mod 3.
http://www.springer.com/978-3-319-09887-6