A Lower Bound for Comparison Sort
Pedro Ribeiro
DCC/FCUP
2014/2015
Pedro Ribeiro (DCC/FCUP)
A Lower Bound for Comparison Sort
2014/2015
1/9
On this lecture
Upper and lower bound problems
Notion of comparison-based sort
An Ω(n log n) worst-case bound for a deterministic comparison
2 player game view of algorithm analysis
Pedro Ribeiro (DCC/FCUP)
A Lower Bound for Comparison Sort
2014/2015
2/9
Upper Bound Problem
One typical question when designing algorithms is:
”given some problem X , can we construct an algorithm that
runs in time O(f (n)) on inputs of size n?”
This can be seen as an upper bound problem, and our goal is to
make g (n) as low as possible.
Pedro Ribeiro (DCC/FCUP)
A Lower Bound for Comparison Sort
2014/2015
3/9
Lower Bound Problem
In this lecture, the question is different:
”given some problem X , what is g (n) such that any algorithm
must take time Ω(g (n)) on inputs of size n?”
This can be seen as an lower bound problem, and our goal is to
make g (n) as high as possible.
Lower bound help understand the intrinsic difficulty of the problem
and how close we are to the best possible solution.
Pedro Ribeiro (DCC/FCUP)
A Lower Bound for Comparison Sort
2014/2015
4/9
Lower Bound Problem
For today we will consider the class of comparison-based sorting
algorithms.
These are sorting algorithms that operate only by comparing elements
and moving them around based on the result of these comparisons
Comparison-Based Sorting Algorithm
A comparison-based sorting algorithm takes as input an array
[a1 , a2 , . . . , an ] with n items, and can only gain information about the
items by comparing pairs of them. Each comparison (”is ai < aj ?”)
returns YES or NO. In the end, the algorithm must output a permutation of
the input in which all items are in sorted order.
Ex: QuickSort, MergeSort, HeapSort, InsertionSort, SelectionSort or
BubbleSort are all comparison-based sorting algorithms.
Pedro Ribeiro (DCC/FCUP)
A Lower Bound for Comparison Sort
2014/2015
5/9
Lower Bound for the Deterministic Case
Theorem
Any deterministic comparison-based sorting algorithm must perform
Ω(n log n) comparisons to sort n elements in the worst case.
We will now prove this theorem, by showing that given any
deterministic comparison-based sorting algorithm A, for all n ≥ 2
there exists an input I of size n such that A makes at least
log2 (n!) = Ω(n log n) comparisons to sort I .
Note that we need to prove for any possible deterministic algorithm,
and not for a specific choice of pivot on QuickSort, or for a specific
merge operation on MergeSort.
Pedro Ribeiro (DCC/FCUP)
A Lower Bound for Comparison Sort
2014/2015
6/9
”20 questions” games
Your goal is to determine the correct ordering of n elements
(that you do not know in advance)
Imagine you can ask me questions about the result of comparing a
pair of elements
I’m answering this questions with the goal of delaying as much as
possible your final answer
How many questions do you need to ask?
Pedro Ribeiro (DCC/FCUP)
A Lower Bound for Comparison Sort
2014/2015
7/9
Proving the Ω(n log n) bound
There are n! permutations that could be the output of the algorithm
For each of these permutations, there is an input for which that
permutation is the only correct answer
Let S be set of permutations that are consistent with the questions
already made
Initially, |S| = n!
What happens when you start asking questions?
Pedro Ribeiro (DCC/FCUP)
A Lower Bound for Comparison Sort
2014/2015
8/9
Proving the Ω(n log n) bound
A comparison will answer YES or NO
This answer will split the permutations still left in S in two groups
Now suppose the adversary answering the question will always choose
the answer leading to the largest group
This means at each time we can only cut |S| by a factor of 2
By design, the algorithm can only stop when |S| = 1, so that it knows
what permutation to output
This means the algorithm must make log2 (n!) questions!
(i.e., comparisons)
log2 (n!) = log2 (n)+log2 (n−1)+log2 (n−2)+. . .+log2 (2) = Ω(n log n)
(we could use Stirling’s Approximation to prove this last step)
Pedro Ribeiro (DCC/FCUP)
A Lower Bound for Comparison Sort
2014/2015
9/9