CHOROCHRONOS Midter Review
Lower bounds
„
„
Finding the minimum
Lower bounds are hard because they make
claims about all possible algorithms in the
set of algorithms being studied.
Let’s start with a simple example
6/1/2004
COSC 3101 - S.Datta
What are we counting?
Running time? Memory? Number of times a
specific operation is used?
„ What (if any) are the assumptions?
„ Is the model general enough?
„
1
6/1/2004
Comparison-based algorithms
„
„
„
„
The algorithm only uses the results of
comparisons, not values of elements.
Very general – does not assume much
about what type of data is being sorted
However, other kinds of algorithms are
possible!
In this model, it is reasonable to count
#comparisons
6/1/2004
COSC 3101 - S.Datta
Claim: Any comparison-based algorithm for
finding the minimum of n keys must use at
least n-1 comparisons
„ Proof: Every other key must have won at
least one comparison.
Each comparison produces at most one
winner.
⇒at least n-1 comparisons have to be made.
„
3
Claim: Every comparison-based algorithm
for finding both the minimum and the
maximum of n elements requires at least
(3n/2)-2 comparisons.
„ Idea: Use adversary argument
Max = maximum and Min=minimum only if:
Every element other than min has won at
least 1
Every element other than max has lost at
least 1
„
Timos Sellis
COSC 3101 - S.Datta
2
Lower bounds for the minimum
6/1/2004
Lower bounds for MIN & MAX
6/1/2004
COSC 3101 - S.Datta
5
COSC 3101 - S.Datta
4
Lower bounds for comparison-based sorting
„
„
„
Trivial: Ω(n) – every element must take
part in a comparison.
Best possible result – Ω(n log n)
comparisons, since we already know
several O(n log n) sorting algorithms.
Proof is non-trivial: how do we reason
about all possible comparison-based
sorting algorithms?
6/1/2004
COSC 3101 - S.Datta
6
1
CHOROCHRONOS Midter Review
Decision tree model
The Decision Tree Model
„
„
„
„
All numbers are distinct (so no use for ai = aj )
All comparisons have form ai ≤ aj (since ai ≤ aj, ai ≥ aj,
ai < aj, ai > aj are equivalent).
„
„
COSC 3101 - S.Datta
‰
‰
‰
‰
7
„
„
„
COSC 3101 - S.Datta
Timos Sellis
>
<3,1,2>
2:3
≤
<2,3,1>
>
<3,2,1>
COSC 3101 - S.Datta
8
S is a set of permutations
S
x[i] : x[j]
≤
COSC 3101 - S.Datta
>
Subset S2 of S
s.t. x[i] > x[j]
Subset S1 of S
s.t. x[i] ≤ x[j]
9
6/1/2004
COSC 3101 - S.Datta
10
Example: insertion sort (n=3)
Claim: worst case number of comparisons=
the height of the decision tree.
Claim: Any comparison sort in the
worst case needs Ω(n log n)
comparisons.
Suppose height of a decision tree is h,
number of paths (i,e,, permutations) is n!.
Since a binary tree of height h has at most
2h leaves,
n! ≤ 2h , so h ≥ lg (n!) ≥ Ω(n lg n)
6/1/2004
≤
>
Intuitive idea
Lower bound for the worst case
„
1:3
6/1/2004
Only consider comparisons
Each internal node = 1 comparison
Start at root, make the first comparison
- if the outcome is ≤ take the LEFT branch
- if the outcome is > - take the RIGHT br
Repeat at each internal node
Each LEAF represents ONE correct ordering
6/1/2004
1:3
≤
<2,1,3>
Internal node i:j indicates comparison between ai and aj.
Leaf node <π(1), π(2), π(3)> indicates ordering aπ(1)≤ aπ(2)≤ aπ(3).
Path of bold lines indicates sorting path for <6,8,5>.
There are total 3!=6 possible permutations (paths).
Summary
‰
>
>
<1,3,2>
Full binary tree
Ignore control, movement, and all other operations,
just see comparison
suppose three elements < a1, a2, a3> with instance
<6,8,5>
6/1/2004
2:3
≤
<1,2,3>
Decision tree model
„
1:2
≤
Assumptions:
11
A[1]: A[2]
≤
>
A[2]: A[3]
A[1]: A[3]
>
≤
A[1]A[2]A[3]
≤
A[1]A[3]A[2]
6/1/2004
A[1]: A[3]
>
≤
A[2]A[1]A[3]
>
A[3]A[1]A[2]
A[2]: A[3]
≤
A[2]A[3]A[1]
COSC 3101 - S.Datta
>
A[3]A[2]A[1]
12
2
CHOROCHRONOS Midter Review
Non-Comparison Sort – Bucket Sort
„
„
„
„
Example of BUCKET-SORT
Assumption: uniform distribution
Input numbers are uniformly distributed in [0,1).
Suppose input size is n.
Idea:
Divide [0,1) into n equal-sized subintervals (buckets).
Distribute n numbers into buckets
„ Expect that each bucket contains few numbers.
„ Sort numbers in each bucket (insertion sort as
default).
„ Then go through buckets in order, listing elements
Can be shown to run in linear-time on average
„
„
6/1/2004
COSC 3101 - S.Datta
13
6/1/2004
„
Assumption: n input numbers are
integers in range [0,k], k=O(n).
Idea:
„
„
1.
2.
3.
4.
5.
Determine the number of elements less
than x, for each input x.
Place x directly in its position.
6.
7.
8.
9.
10.
11.
6/1/2004
COSC 3101 - S.Datta
14
COUNTING-SORT(A,B,k)
Non-Comparison Sort – Counting Sort
„
COSC 3101 - S.Datta
15
for i←0 to k
do C[i] ←0
for j ←1 to length[A]
do C[A[j]] ←C[A[j]]+1
// C[i] contains number of elements equal to i.
for i ←1 to k
do C[i]=C[i]+C[i-1]
// C[i] contains number of elements ≤ i.
for j ←length[A] downto 1
do B[C[A[j]]] ←A[j]
C[A[j]] ←C[A[j]]-1
6/1/2004
COSC 3101 - S.Datta
16
Analysis of COUNTING-SORT(A,B,k)
Example of Counting Sort
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
for i←0 to k
Θ(k)
do C[i] ←0
Θ(1)
for j ←1 to length[A]
Θ(n)
do C[A[j]] ←C[A[j]]+1
Θ(1) (Θ(1) Θ(n)= Θ(n))
// C[i] contains number of elements equal to i. Θ(0)
for i ←1 to k
Θ(k)
do C[i]=C[i]+C[i-1]
Θ(1) (Θ(1) Θ(n)= Θ(n))
// C[i] contains number of elements ≤ i.
Θ(0)
for j ←length[A] downto 1
Θ(n)
do B[C[A[j]]] ←A[j]
Θ(1) (Θ(1) Θ(n)= Θ(n))
C[A[j]] ←C[A[j]]-1
Θ(1) (Θ(1) Θ(n)= Θ(n))
Total cost is Θ(k+n), suppose k=O(n), then total cost is Θ(n).
So, it beats the Ω(n log n) lower bound!
6/1/2004
Timos Sellis
COSC 3101 - S.Datta
17
6/1/2004
COSC 3101 - S.Datta
18
3
CHOROCHRONOS Midter Review
Stable sort
„
„
Radix sort
Preserves order of elements with the same
key
Counting sort is stable
Radix-Sort(A,d)
•
for i←1 to d
•
do use a stable sort to sort A on digit I
Analysis
Given n d-digit numbers where each digit takes on
up to
k values, Radix-Sort sorts these numbers correctly
in Θ(d(n+k)) time.
6/1/2004
COSC 3101 - S.Datta
19
6/1/2004
COSC 3101 - S.Datta
20
Radix sort - example
6/1/2004
Timos Sellis
COSC 3101 - S.Datta
21
4