Multi-Pivot Quicksort
Comparison-Optimal Algorithms and Beyond
Martin Aumüller
IT University of Copenhagen
April 4, 2016
ARCO Workshop
M. Aumüller
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
1/23
Dual-Pivot Quicksort
p
3
2
8
5
1
4
7
6
Input: Permutation of {1, . . . , n}.
M. Aumüller
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
2/23
Dual-Pivot Quicksort
p
3
2
8
5
1
4
7
6
Input: Permutation of {1, . . . , n}.
1. Choose two pivots p, q with p < q.
M. Aumüller
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
2/23
Dual-Pivot Quicksort
q
3
=
=
p
2
8
5
1
4
7
6
Input: Permutation of {1, . . . , n}.
1. Choose two pivots p, q with p < q.
M. Aumüller
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
2/23
Dual-Pivot Quicksort
q
3
=
=
p
2
8
5
1
4
7
6
Input: Permutation of {1, . . . , n}.
1. Choose two pivots p, q with p < q.
2. Partition, i.e., re-arrange elements.
Partition:
M. Aumüller
<p
p
p < ··· < q
q
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
>q
2/23
Dual-Pivot Quicksort
q
2
1
3
=
=
p
5
4
6
7
8
Input: Permutation of {1, . . . , n}.
1. Choose two pivots p, q with p < q.
2. Partition, i.e., re-arrange elements.
Partition:
M. Aumüller
<p
p
p < ··· < q
q
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
>q
2/23
Dual-Pivot Quicksort
q
2
1
3
=
=
p
5
4
6
7
8
Input: Permutation of {1, . . . , n}.
1. Choose two pivots p, q with p < q.
2. Partition, i.e., re-arrange elements.
Partition:
<p
p
p < ··· < q
q
>q
3. Use recursion to sort the three subarrays.
M. Aumüller
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
2/23
Dual-Pivot Quicksort
q
1
2
3
=
=
p
4
5
6
7
8
Input: Permutation of {1, . . . , n}.
1. Choose two pivots p, q with p < q.
2. Partition, i.e., re-arrange elements.
Partition:
<p
p
p < ··· < q
q
>q
3. Use recursion to sort the three subarrays.
M. Aumüller
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
2/23
Dual-Pivot Quicksort
p
1
2
3
4
5
6
7
8
Input: Permutation of {1, . . . , n}.
1. Choose two pivots p, q with p < q.
2. Partition, i.e., re-arrange elements.
Partition:
<p
p
p < ··· < q
q
>q
3. Use recursion to sort the three subarrays.
M. Aumüller
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
2/23
Why Consider It?
(Talk “Sorting in JDK”, V. Yaroslavskiy, 2015.)
M. Aumüller
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
3/23
Part I
The Exact Average Comparison Count of
Comparison-Optimal Dual-Pivot Quicksort
http://arxiv.org/abs/1602.04031
Joint work with Martin Dietzfelbinger (TU Ilmenau), Clemens Heuberger,
Daniel Krenn (AAU Klagenfurt), and Helmut Prodinger (Stellenbosch
University)
M. Aumüller
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
4/23
Goal
Analyze comparison-optimal dual-pivot quicksort algorithms exactly.
Pn : cost for partitioning the input.
Cn : cost for sorting the input.
Recurrence:
n−2
3 X
(n − 1 − k)E(Ck )
E(Cn ) = E(Pn ) + n
2
M. Aumüller
(1)
k=1
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
5/23
Goal
Analyze comparison-optimal dual-pivot quicksort algorithms exactly.
Pn : cost for partitioning the input.
Cn : cost for sorting the input.
Recurrence:
n−2
3 X
(n − 1 − k)E(Ck )
E(Cn ) = E(Pn ) + n
2
(1)
k=1
Need: exact analysis of partitioning cost E(Pn ).
Then use known techniques (Generating functions, [Wild’13]) to solve (1).
M. Aumüller
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
5/23
How is the partitioning cost determined?
Must classify n − 2 entries x into three parts:
x <p p
p<x <q
q
x >q
small , medium , or large
M. Aumüller
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
6/23
How is the partitioning cost determined?
Must classify n − 2 entries x into three parts:
x <p p
p<x <q
q
x >q
small , medium , or large
1 or 2 comparisons.
Unavoidable: 1 comparison for small/large x, 2 comparisons for medium x.
M. Aumüller
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
6/23
How is the partitioning cost determined?
Must classify n − 2 entries x into three parts:
x <p p
p<x <q
q
x >q
small , medium , or large
1 or 2 comparisons.
Unavoidable: 1 comparison for small/large x, 2 comparisons for medium x.
Extra: small x compared with q first and large x compared with p first.
M. Aumüller
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
6/23
How is the partitioning cost determined?
Must classify n − 2 entries x into three parts:
x <p p
p<x <q
q
x >q
small , medium , or large
1 or 2 comparisons.
Unavoidable: 1 comparison for small/large x, 2 comparisons for medium x.
Extra: small x compared with q first and large x compared with p first.
Partitioning strategy determines for next element x whether to compare
x with p first or with q first.
An implementation (Y./S./. . . ) implicitly defines such a strategy.
M. Aumüller
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
6/23
Assumptions of the Analysis
Input is random permutation of {1, . . . , n}
Choose two elements as pivots
Relabel small elements as σ, medium elements as µ, and large
elements as λ
M. Aumüller
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
7/23
Assumptions of the Analysis
Input is random permutation of {1, . . . , n}
Choose two elements as pivots
Relabel small elements as σ, medium elements as µ, and large
elements as λ
q
=
=
p
3
M. Aumüller
2
8
5
1
4
7
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
6
7/23
Assumptions of the Analysis
Input is random permutation of {1, . . . , n}
Choose two elements as pivots
Relabel small elements as σ, medium elements as µ, and large
elements as λ
q
=
=
p
3
M. Aumüller
σ
8
5
1
4
7
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
6
7/23
Assumptions of the Analysis
Input is random permutation of {1, . . . , n}
Choose two elements as pivots
Relabel small elements as σ, medium elements as µ, and large
elements as λ
q
=
=
p
3
M. Aumüller
σ
λ
5
1
4
7
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
6
7/23
Assumptions of the Analysis
Input is random permutation of {1, . . . , n}
Choose two elements as pivots
Relabel small elements as σ, medium elements as µ, and large
elements as λ
q
=
=
p
3
M. Aumüller
σ
λ
µ
1
4
7
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
6
7/23
Assumptions of the Analysis
Input is random permutation of {1, . . . , n}
Choose two elements as pivots
Relabel small elements as σ, medium elements as µ, and large
elements as λ
q
=
=
p
3
M. Aumüller
σ
λ
µ
σ
4
7
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
6
7/23
Assumptions of the Analysis
Input is random permutation of {1, . . . , n}
Choose two elements as pivots
Relabel small elements as σ, medium elements as µ, and large
elements as λ
q
=
=
p
3
M. Aumüller
σ
λ
µ
σ
µ
7
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
6
7/23
Assumptions of the Analysis
Input is random permutation of {1, . . . , n}
Choose two elements as pivots
Relabel small elements as σ, medium elements as µ, and large
elements as λ
q
=
=
p
3
M. Aumüller
σ
λ
µ
σ
µ
λ
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
6
7/23
Assumptions of the Analysis
Input is random permutation of {1, . . . , n}
Choose two elements as pivots
Relabel small elements as σ, medium elements as µ, and large
elements as λ
q
=
=
p
3
σ
λ
µ
σ
µ
λ
6
Read input left to right, classify each element by one or two comparisons.
M. Aumüller
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
7/23
Average Cost
For the average partitioning/classification cost E(Pn ) we get:
E(Pn ) =
4
3 (n
− 2) + 1 + “average number of extra comparisons”
For all classification algorithms:
small [large] elements have
to be compared to p [q]
“Extra” comparisons:
small x compared to q first
large x compared to p first
medium elements have to be
compared to both
M. Aumüller
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
8/23
Yaroslavskiy’s Dual-Pivot Quicksort Algorithm
Assume we are in round i.
Strategy “Yaroslavskiy”
σ
λ
σ
µ
?
?
?
?
−→
Next element
Seen elements in
M. Aumüller
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
9/23
Yaroslavskiy’s Dual-Pivot Quicksort Algorithm
Assume we are in round i.
Strategy “Yaroslavskiy”
σ
λ
σ
µ
?
?
?
?
−→
Next element
Seen elements in
Classification Strategy:
previous element large: compare with larger pivot first.
otherwise: compare with smaller pivot first.
M. Aumüller
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
9/23
Yaroslavskiy’s Dual-Pivot Quicksort Algorithm
Assume we are in round i.
Strategy “Yaroslavskiy”
σ
λ
σ
µ
?
?
?
?
−→
Next element
Seen elements in
Classification Strategy:
previous element large: compare with larger pivot first.
otherwise: compare with smaller pivot first.
[WN12]: Average sorting cost is 1.9n ln n + O(n).
M. Aumüller
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
9/23
Yaroslavskiy’s Dual-Pivot Quicksort Algorithm
Assume we are in round i.
Strategy “Yaroslavskiy”
σ
λ
σ
µ
?
?
?
?
−→
Next element
Seen elements in
Classification Strategy:
previous element large: compare with larger pivot first.
otherwise: compare with smaller pivot first.
[WN12]: Average sorting cost is 1.9n ln n + O(n).
[WNN15]: Analysis of the distribution of Cn (and other cost measures).
M. Aumüller
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
9/23
The Comparison-Optimal Dual-Pivot Quicksort Algorithm
Assume we are in round i.
Strategy “Count”
σ
λ
σ
µ
?
?
?
?
−→
Next element
Seen si−1 small and `i−1 large elements in
M. Aumüller
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
10/23
The Comparison-Optimal Dual-Pivot Quicksort Algorithm
Assume we are in round i.
Strategy “Count”
σ
λ
σ
µ
?
?
?
?
−→
Next element
Seen si−1 small and `i−1 large elements in
Classification Strategy:
`i−1 > si−1 : compare with larger pivot first.
`i−1 ≤ si−1 : compare with smaller pivot first.
M. Aumüller
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
10/23
The Comparison-Optimal Dual-Pivot Quicksort Algorithm
Assume we are in round i.
Strategy “Count”
σ
λ
σ
µ
?
?
?
?
−→
Next element
Seen si−1 small and `i−1 large elements in
Classification Strategy:
`i−1 > si−1 : compare with larger pivot first.
`i−1 ≤ si−1 : compare with smaller pivot first.
[AD15]: Average sorting cost is 1.8n ln n + O(n).
M. Aumüller
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
10/23
The Comparison-Optimal Dual-Pivot Quicksort Algorithm
Assume we are in round i.
Strategy “Count”
σ
λ
σ
µ
?
?
?
?
−→
Next element
Seen si−1 small and `i−1 large elements in
Classification Strategy:
`i−1 > si−1 : compare with larger pivot first.
`i−1 ≤ si−1 : compare with smaller pivot first.
[AD15]: Average sorting cost is 1.8n ln n + O(n).
Here: “Count” is optimal + exact average comparison count.
M. Aumüller
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
10/23
Proof idea: “Count” is optimal (among all algorithms)
Pólya urn with three colors to model input sequence of {σ, µ, λ}.
M. Aumüller
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
11/23
Proof idea: “Count” is optimal (among all algorithms)
Pólya urn with three colors to model input sequence of {σ, µ, λ}.
1
Put one light green ball, one green ball, one dark green ball in urn.
M. Aumüller
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
11/23
Proof idea: “Count” is optimal (among all algorithms)
Pólya urn with three colors to model input sequence of {σ, µ, λ}.
1
2
Put one light green ball, one green ball, one dark green ball in urn.
Round i = 1, . . . , n − 2:
M. Aumüller
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
11/23
Proof idea: “Count” is optimal (among all algorithms)
Pólya urn with three colors to model input sequence of {σ, µ, λ}.
1
2
Put one light green ball, one green ball, one dark green ball in urn.
Round i = 1, . . . , n − 2:
Choose ball from urn at random.
M. Aumüller
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
11/23
Proof idea: “Count” is optimal (among all algorithms)
Pólya urn with three colors to model input sequence of {σ, µ, λ}.
1
2
Put one light green ball, one green ball, one dark green ball in urn.
Round i = 1, . . . , n − 2:
Choose ball from urn at random.
M. Aumüller
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
11/23
Proof idea: “Count” is optimal (among all algorithms)
Pólya urn with three colors to model input sequence of {σ, µ, λ}.
1
2
Put one light green ball, one green ball, one dark green ball in urn.
Round i = 1, . . . , n − 2:
Choose ball from urn at random. Take down its color c, put it back,
λ
M. Aumüller
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
11/23
Proof idea: “Count” is optimal (among all algorithms)
Pólya urn with three colors to model input sequence of {σ, µ, λ}.
1
2
Put one light green ball, one green ball, one dark green ball in urn.
Round i = 1, . . . , n − 2:
Choose ball from urn at random. Take down its color c, put it back,
λ
M. Aumüller
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
11/23
Proof idea: “Count” is optimal (among all algorithms)
Pólya urn with three colors to model input sequence of {σ, µ, λ}.
1
2
Put one light green ball, one green ball, one dark green ball in urn.
Round i = 1, . . . , n − 2:
Choose ball from urn at random. Take down its color c, put it back,
and put another ball of the same color in the urn.
λ
M. Aumüller
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
11/23
Proof idea: “Count” is optimal (among all algorithms)
Pólya urn with three colors to model input sequence of {σ, µ, λ}.
1
2
Put one light green ball, one green ball, one dark green ball in urn.
Round i = 1, . . . , n − 2:
Choose ball from urn at random. Take down its color c, put it back,
and put another ball of the same color in the urn.
λ
M. Aumüller
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
11/23
Proof idea: “Count” is optimal (among all algorithms)
Pólya urn with three colors to model input sequence of {σ, µ, λ}.
1
2
Put one light green ball, one green ball, one dark green ball in urn.
Round i = 1, . . . , n − 2:
Choose ball from urn at random. Take down its color c, put it back,
and put another ball of the same color in the urn.
λ
M. Aumüller
σ
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
11/23
Proof idea: “Count” is optimal (among all algorithms)
Pólya urn with three colors to model input sequence of {σ, µ, λ}.
1
2
Put one light green ball, one green ball, one dark green ball in urn.
Round i = 1, . . . , n − 2:
Choose ball from urn at random. Take down its color c, put it back,
and put another ball of the same color in the urn.
λ
M. Aumüller
σ
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
11/23
Proof idea: “Count” is optimal (among all algorithms)
Pólya urn with three colors to model input sequence of {σ, µ, λ}.
1
2
Put one light green ball, one green ball, one dark green ball in urn.
Round i = 1, . . . , n − 2:
Choose ball from urn at random. Take down its color c, put it back,
and put another ball of the same color in the urn.
λ
M. Aumüller
σ
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
11/23
Proof idea: “Count” is optimal (among all algorithms)
Pólya urn with three colors to model input sequence of {σ, µ, λ}.
1
2
Put one light green ball, one green ball, one dark green ball in urn.
Round i = 1, . . . , n − 2:
Choose ball from urn at random. Take down its color c, put it back,
and put another ball of the same color in the urn.
λ
M. Aumüller
σ
µ
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
11/23
Proof idea: “Count” is optimal (among all algorithms)
Pólya urn with three colors to model input sequence of {σ, µ, λ}.
1
2
Put one light green ball, one green ball, one dark green ball in urn.
Round i = 1, . . . , n − 2:
Choose ball from urn at random. Take down its color c, put it back,
and put another ball of the same color in the urn.
λ
M. Aumüller
σ
µ
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
11/23
Proof idea: “Count” is optimal (among all algorithms)
Pólya urn with three colors to model input sequence of {σ, µ, λ}.
1
2
Put one light green ball, one green ball, one dark green ball in urn.
Round i = 1, . . . , n − 2:
Choose ball from urn at random. Take down its color c, put it back,
and put another ball of the same color in the urn.
λ
M. Aumüller
σ
µ
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
11/23
Proof idea: “Count” is optimal (among all algorithms)
Pólya urn with three colors to model input sequence of {σ, µ, λ}.
1
2
Put one light green ball, one green ball, one dark green ball in urn.
Round i = 1, . . . , n − 2:
Choose ball from urn at random. Take down its color c, put it back,
and put another ball of the same color in the urn.
λ
M. Aumüller
σ
µ
σ
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
11/23
Proof idea: “Count” is optimal (among all algorithms)
Pólya urn with three colors to model input sequence of {σ, µ, λ}.
1
2
Put one light green ball, one green ball, one dark green ball in urn.
Round i = 1, . . . , n − 2:
Choose ball from urn at random. Take down its color c, put it back,
and put another ball of the same color in the urn.
λ
M. Aumüller
σ
µ
σ
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
11/23
Proof idea: “Count” is optimal (among all algorithms)
Pólya urn with three colors to model input sequence of {σ, µ, λ}.
1
2
Put one light green ball, one green ball, one dark green ball in urn.
Round i = 1, . . . , n − 2:
Choose ball from urn at random. Take down its color c, put it back,
and put another ball of the same color in the urn.
λ
M. Aumüller
σ
µ
σ
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
11/23
Proof idea: “Count” is optimal (among all algorithms)
Pólya urn with three colors to model input sequence of {σ, µ, λ}.
1
2
Put one light green ball, one green ball, one dark green ball in urn.
Round i = 1, . . . , n − 2:
Choose ball from urn at random. Take down its color c, put it back,
and put another ball of the same color in the urn.
λ
σ
µ
σ
Can show: This leads to the same distribution on {σ, µ, λ} sequences as the
one generated by random permutation/pivots/relabel.
M. Aumüller
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
11/23
Proof idea: “Count” is optimal (among all algorithms)
Pólya urn with three colors to model input sequence of {σ, µ, λ}.
1
2
Put one light green ball, one green ball, one dark green ball in urn.
Round i = 1, . . . , n − 2:
Choose ball from urn at random. Take down its color c, put it back,
and put another ball of the same color in the urn.
M. Aumüller
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
11/23
Proof idea: “Count” is optimal (among all algorithms)
Pólya urn with three colors to model input sequence of {σ, µ, λ}.
Put one light green ball, one green ball, one dark green ball in urn.
2 Round i = 1, . . . , n − 2:
Choose ball from urn at random. Take down its color c, put it back,
and put another ball of the same color in the urn.
In Round i:
si−1 + 1
Pr(element is small) =
.
i +2
1
(analogous formulæ for medium/large elements.)
M. Aumüller
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
11/23
Proof idea: “Count” is optimal (among all algorithms)
Pólya urn with three colors to model input sequence of {σ, µ, λ}.
1
2
Put one light green ball, one green ball, one dark green ball in urn.
Round i = 1, . . . , n − 2:
Choose ball from urn at random. Take down its color c, put it back,
and put another ball of the same color in the urn.
M. Aumüller
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
11/23
Proof idea: “Count” is optimal (among all algorithms)
Pólya urn with three colors to model input sequence of {σ, µ, λ}.
Put one light green ball, one green ball, one dark green ball in urn.
2 Round i = 1, . . . , n − 2:
Choose ball from urn at random. Take down its color c, put it back,
and put another ball of the same color in the urn.
Next step: Show that every partitioning strategy S with decision in round i
based on the elements seen in rounds up to i − 1 makes at least as many
comparisons in round i as “Count” (on average).
1
M. Aumüller
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
11/23
“Count” is optimal (among all algorithms)
If si−1 ≥ `i−1 , “Count” compares with small pivot first. Probability to
generate additional cost 1 in this step:
`i−1 + 1
.
Pr(“Count” gets extra comparison in step i) =
i +2
M. Aumüller
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
12/23
“Count” is optimal (among all algorithms)
If si−1 ≥ `i−1 , “Count” compares with small pivot first. Probability to
generate additional cost 1 in this step:
`i−1 + 1
.
Pr(“Count” gets extra comparison in step i) =
i +2
If S uses small pivot first, no difference.
M. Aumüller
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
12/23
“Count” is optimal (among all algorithms)
If si−1 ≥ `i−1 , “Count” compares with small pivot first. Probability to
generate additional cost 1 in this step:
`i−1 + 1
.
Pr(“Count” gets extra comparison in step i) =
i +2
If S uses small pivot first, no difference.
If S takes large pivot first:
si−1 + 1
Pr(S gets extra comparison in step i) =
,
i +2
at least as big as the probability for “Count”.
M. Aumüller
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
12/23
“Count” is optimal (among all algorithms)
If si−1 ≥ `i−1 , “Count” compares with small pivot first. Probability to
generate additional cost 1 in this step:
`i−1 + 1
.
Pr(“Count” gets extra comparison in step i) =
i +2
If S uses small pivot first, no difference.
If S takes large pivot first:
si−1 + 1
Pr(S gets extra comparison in step i) =
,
i +2
at least as big as the probability for “Count”.
Conclusion: “Count” is the comparison-optimal strategy.
Next: Analysis of average comparison count.
M. Aumüller
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
12/23
Define “random walk” :
Xi = si−1 − `i−1 ,
M. Aumüller
for 1 ≤ i ≤ n − 1.
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
13/23
Define “random walk” :
Xi = si−1 − `i−1 ,
for 1 ≤ i ≤ n − 1.
Classification Strategy:
Xi ≥ 0: compare with smaller pivot first.
Xi < 0: compare with larger pivot first.
M. Aumüller
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
13/23
Define “random walk” :
Xi = si−1 − `i−1 ,
for 1 ≤ i ≤ n − 1.
Classification Strategy:
Xi ≥ 0: compare with smaller pivot first.
Xi < 0: compare with larger pivot first.
Study simplified situation, ignore medium elements.
M. Aumüller
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
13/23
Define “random walk” :
Xi = si−1 − `i−1 ,
for 1 ≤ i ≤ n − 1.
Classification Strategy:
Xi ≥ 0: compare with smaller pivot first.
Xi < 0: compare with larger pivot first.
Study simplified situation, ignore medium elements.
We classify n0 = s + ` elements.
M. Aumüller
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
13/23
Random Walks & Analysis of Count
Xi
3
2
1
0
1
i
n0
−1
−2
−3
−→
λ
M. Aumüller
λ
σ
σ
σ
λ
σ
σ
λ
λ
λ
σ
λ
λ
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
σ
σ
σ
14/23
Random Walks & Analysis of Count
Xi
3
2
1
0
1
i
n0
λ
−1
−2
−3
−→
λ
M. Aumüller
λ
σ
σ
σ
λ
σ
σ
λ
λ
λ
σ
λ
λ
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
σ
σ
σ
14/23
Random Walks & Analysis of Count
Xi
3
2
1
0
1
i
n0
λ
−1
λ
−2
−3
−→
λ
M. Aumüller
λ
σ
σ
σ
λ
σ
σ
λ
λ
λ
σ
λ
λ
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
σ
σ
σ
14/23
Random Walks & Analysis of Count
Xi
3
2
1
0
1
i
n0
λ
−1
σ
λ
−2
−3
−→
λ
M. Aumüller
λ
σ
σ
σ
λ
σ
σ
λ
λ
λ
σ
λ
λ
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
σ
σ
σ
14/23
Random Walks & Analysis of Count
Xi
3
2
1
0
1
σ
λ
i
n0
−1
σ
λ
−2
−3
−→
λ
M. Aumüller
λ
σ
σ
σ
λ
σ
σ
λ
λ
λ
σ
λ
λ
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
σ
σ
σ
14/23
Random Walks & Analysis of Count
Xi
3
2
1
0
σ
1
σ
λ
i
n0
−1
σ
λ
−2
−3
−→
λ
M. Aumüller
λ
σ
σ
σ
λ
σ
σ
λ
λ
λ
σ
λ
λ
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
σ
σ
σ
14/23
Random Walks & Analysis of Count
Xi
3
2
1
0
σ
1
λ
σ
λ
i
n0
−1
σ
λ
−2
−3
−→
λ
M. Aumüller
λ
σ
σ
σ
λ
σ
σ
λ
λ
λ
σ
λ
λ
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
σ
σ
σ
14/23
Random Walks & Analysis of Count
Xi
3
2
1
0
σ
1
σ
λ
σ
λ
i
n0
−1
σ
λ
−2
−3
−→
λ
M. Aumüller
λ
σ
σ
σ
λ
σ
σ
λ
λ
λ
σ
λ
λ
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
σ
σ
σ
14/23
Random Walks & Analysis of Count
Xi
3
2
σ
1
0
σ
1
σ
λ
σ
λ
i
n0
−1
σ
λ
−2
−3
−→
λ
M. Aumüller
λ
σ
σ
σ
λ
σ
σ
λ
λ
λ
σ
λ
λ
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
σ
σ
σ
14/23
Random Walks & Analysis of Count
Xi
3
2
σ
λ
1
0
σ
1
σ
λ
σ
λ
i
n0
−1
σ
λ
−2
−3
−→
λ
M. Aumüller
λ
σ
σ
σ
λ
σ
σ
λ
λ
λ
σ
λ
λ
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
σ
σ
σ
14/23
Random Walks & Analysis of Count
Xi
3
2
σ
λ
1
0
σ
1
σ
λ
λ
σ
λ
i
n0
−1
σ
λ
−2
−3
−→
λ
M. Aumüller
λ
σ
σ
σ
λ
σ
σ
λ
λ
λ
σ
λ
λ
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
σ
σ
σ
14/23
Random Walks & Analysis of Count
Xi
3
2
σ
λ
1
0
σ
1
σ
λ
λ
σ
λ
i
n0
λ
−1
σ
λ
−2
−3
−→
λ
M. Aumüller
λ
σ
σ
σ
λ
σ
σ
λ
λ
λ
σ
λ
λ
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
σ
σ
σ
14/23
Random Walks & Analysis of Count
Xi
3
2
σ
λ
1
0
σ
1
σ
λ
λ
σ
λ
σ
λ
i
n0
−1
σ
λ
−2
−3
−→
λ
M. Aumüller
λ
σ
σ
σ
λ
σ
σ
λ
λ
λ
σ
λ
λ
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
σ
σ
σ
14/23
Random Walks & Analysis of Count
Xi
3
2
σ
λ
1
0
σ
1
σ
λ
λ
σ
λ
σ
λ
i
n0
λ
−1
σ
λ
−2
−3
−→
λ
M. Aumüller
λ
σ
σ
σ
λ
σ
σ
λ
λ
λ
σ
λ
λ
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
σ
σ
σ
14/23
Random Walks & Analysis of Count
Xi
3
2
σ
λ
1
0
σ
1
σ
λ
λ
σ
λ
σ
λ
i
n0
λ
−1
σ
λ
λ
−2
−3
−→
λ
M. Aumüller
λ
σ
σ
σ
λ
σ
σ
λ
λ
λ
σ
λ
λ
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
σ
σ
σ
14/23
Random Walks & Analysis of Count
Xi
3
2
σ
λ
1
0
σ
1
σ
λ
λ
σ
λ
σ
λ
i
n0
λ
−1
σ
λ
σ
λ
−2
−3
−→
λ
M. Aumüller
λ
σ
σ
σ
λ
σ
σ
λ
λ
λ
σ
λ
λ
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
σ
σ
σ
14/23
Random Walks & Analysis of Count
Xi
3
2
σ
λ
1
0
σ
1
σ
λ
λ
σ
λ
σ
λ
σ
λ
i
n0
−1
σ
λ
σ
λ
−2
−3
−→
λ
M. Aumüller
λ
σ
σ
σ
λ
σ
σ
λ
λ
λ
σ
λ
λ
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
σ
σ
σ
14/23
Random Walks & Analysis of Count
Xi
3
2
σ
λ
1
0
σ
1
σ
λ
σ
λ
σ
λ
σ
λ
σ
λ
i
n0
−1
σ
λ
σ
λ
−2
−3
−→
λ
M. Aumüller
λ
σ
σ
σ
λ
σ
σ
λ
λ
λ
σ
λ
λ
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
σ
σ
σ
14/23
Random Walks & Analysis of Count
Xi
3
2
σ
λ
1
0
σ
1
σ
λ
σ
λ
σ
λ
σ
λ
σ
λ
i
n0
−1
σ
λ
σ
λ
−2
−3
−→
λ
λ
σ
σ
σ
λ
σ
σ
λ
λ
λ
σ
λ
λ
σ
σ
σ
Observation: Extra comparison in round i ⇔
M. Aumüller
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
14/23
Random Walks & Analysis of Count
Xi
3
2
σ
λ
1
0
σ
1
σ
λ
σ
λ
σ
λ
σ
λ
σ
λ
i
n0
−1
σ
λ
σ
λ
−2
−3
−→
λ
λ
σ
σ
σ
λ
σ
σ
λ
λ
λ
σ
λ
λ
σ
σ
σ
Observation: Extra comparison in round i ⇔
move towards zero in round i,
M. Aumüller
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
14/23
Random Walks & Analysis of Count
Xi
3
2
σ
λ
1
0
σ
1
σ
λ
σ
λ
σ
λ
σ
λ
σ
λ
i
n0
−1
σ
λ
σ
λ
−2
−3
−→
λ
λ
σ
σ
σ
λ
σ
σ
λ
λ
λ
σ
λ
λ
σ
σ
σ
Observation: Extra comparison in round i ⇔
move towards zero in round i, plus move down from a zero.
M. Aumüller
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
14/23
Random Walks & Analysis of Count
Xi
3
2
σ
λ
1
0
σ
1
σ
λ
σ
λ
σ
λ
σ
λ
σ
λ
i
n0
−1
σ
λ
σ
λ
−2
−3
−→
λ
λ
σ
σ
σ
λ
σ
σ
λ
λ
λ
σ
λ
λ
σ
σ
σ
Observation: Extra comparison in round i ⇔
move towards zero in round i, plus move down from a zero.
Exactly min(s, `) many “move towards zero ”situations.
M. Aumüller
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
14/23
Counting Zeros in Random Walks
Define the “number of zeros”: Zn0 := #{i | Xi = 0}.
M. Aumüller
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
15/23
Counting Zeros in Random Walks
Define the “number of zeros”: Zn0 := #{i | Xi = 0}.
Can only have zero at odd positions i.
M. Aumüller
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
15/23
Counting Zeros in Random Walks
Define the “number of zeros”: Zn0 := #{i | Xi = 0}.
Can only have zero at odd positions i.
Central argument
For each i ∈ {1, 3, 5, . . .}, i ≤ n0 + 1,
1
P(Xi = 0) = .
i
M. Aumüller
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
15/23
Counting Zeros in Random Walks
Define the “number of zeros”: Zn0 := #{i | Xi = 0}.
Can only have zero at odd positions i.
Central argument
For each i ∈ {1, 3, 5, . . .}, i ≤ n0 + 1,
1
P(Xi = 0) = .
i
Linearity of expectation yields:
E(Zn0 ) =
X
1≤i≤n0 +1
i odd
M. Aumüller
1
.
i
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
15/23
Counting Zeros in Random Walks
Define the “number of zeros”: Zn0 := #{i | Xi = 0}.
Can only have zero at odd positions i.
Central argument
For each i ∈ {1, 3, 5, . . .}, i ≤ n0 + 1,
1
P(Xi = 0) = .
i
Linearity of expectation yields:
E(Zn0 ) =
X
1≤i≤n0 +1
i odd
1
.
i
0
1
[n even]
0
E(#(extra comps) | s + ` = n ) = min(s, `) +
E(Zn0 ) − 0
.
2
n +1
M. Aumüller
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
15/23
Exact Average Partitioning Cost
Back to general situation including medium elements.
Averaging over all pivot choices yields:
3n 19
2 + (−1)n
1
E(Pn ) =
−
+
+
2
8
8(n − [n even]) 2
X
1≤i≤n−2
i odd
1
.
i
Can also write down generating functions.
M. Aumüller
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
16/23
Exact Average Sorting Cost
Solving dual-pivot quicksort recurrence with generating functions [Wild’13]
gives expected comparison count, explicitly, and then asymptotically:
1
0
0
0
E(Cn ) = 1.8n ln n + A n + B ln n + C + O
,
n
where A0 ≈ −2.38, B’ = 1.675, C 0 ≈ 1.82.
M. Aumüller
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
17/23
Part II
Memory Accesses in Multi-Pivot Quicksort
http://arxiv.org/abs/1510.04676
M. Aumüller
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
18/23
Quicksort with k Pivots
Input: Random permutation of {1, . . . , n}.
First k entries sorted are pivots p1 , . . . , pk .
Partitioning: Split n − k remaining entries x into k + 1 classes
x < p1
A0
p1
p1 < x < p2
p2
···
A1
pk
x > pk
Ak
Then sort classes recursively.
M. Aumüller
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
19/23
Other Cost Measures
Number of branch misses: Thorough analysis of classical QS and
Yaroslavskiy’s algorithm by Martı́nez/Nebel/Wild (ANALCO’15).
Conclusion: No improvement over classical QS.
Number of cache misses: (Kushagra et al., ALENEX’14):
Dual-Pivot and three-pivot algorithm cause
far fewer cache misses than classical QS.
M. Aumüller
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
20/23
A Generalized Partitioning Algorithm
A0
A1
g1
M. Aumüller
A2
g2
A3
g3
?
?
i
j
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
A4
g4
A5
A6
g5
21/23
A Generalized Partitioning Algorithm
A0
A1
g1
M. Aumüller
A2
g2
A3
g3
A5
A1
i
j
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
A4
g4
A5
A6
g5
21/23
A Generalized Partitioning Algorithm
A0
A1
g1
M. Aumüller
A2
g2
A3
g3
A5
A1
i
j
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
A4
g4
A5
A6
g5
21/23
A Generalized Partitioning Algorithm
A0
A1
g1
M. Aumüller
A2
g2
A3
g3
A5
A1
i
j
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
A4
g4
A5
A6
g5
21/23
A Generalized Partitioning Algorithm
A0
A1
g1
M. Aumüller
A2
g2
A3
g3
A5
A1
i
j
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
A4
g4
A5
A6
g5
21/23
A Generalized Partitioning Algorithm
A0
A1
g1
M. Aumüller
A2
g2
A3
g3
A5
A1
i
j
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
A4
g4
A5
A6
g5
21/23
A Generalized Partitioning Algorithm
A0
A1
g1
M. Aumüller
A2
g2
A3
g3
A5
A1
i
j
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
A4
g4
A5
A6
g5
21/23
A Generalized Partitioning Algorithm
A0
A1
g1
A2
g2
A3
g3
A5
A1
i
j
A4
g4
A5
A6
g5
5 pointer visits.
M. Aumüller
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
21/23
A Generalized Partitioning Algorithm
A0
A1
g1
A2
g2
A3
g3
A5
A1
i
j
A4
g4
A5
A6
g5
5 pointer visits.
Analysis for partitioning yields:
M. Aumüller
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
21/23
A Generalized Partitioning Algorithm
A0
A1
g1
A2
g2
A3
g3
A5
A1
i
j
A4
A5
g4
A6
g5
5 pointer visits.
Analysis for partitioning yields:

k +3


 4 · (n − k),
E(Pptr-visit
)= n

k +3
3


+
· (n − k),
4
k +1
M. Aumüller
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
for odd k,
for even k.
21/23
A Generalized Partitioning Algorithm
avg. # pointer visits /n ln n
“Pointer visits” for sorting with k pivots: Minimum at k = 5.
M. Aumüller
2
1.8
1.6
1.4
1
2
3
4
5
6
# pivots
7
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
8
9
22/23
A Generalized Partitioning Algorithm
avg. # pointer visits /n ln n
“Pointer visits” for sorting with k pivots: Minimum at k = 5.
2
1.8
1.6
1.4
1
2
3
4
5
6
# pivots
7
8
9
Observations:
“pointer visits” predict L1 cache misses accurately
closest w.r.t. predicting empirical running time
M. Aumüller
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
22/23
Conclusion & Open Problems
Theory:
Intuitive algorithm “Count” is comparison-optimal
Exact average comparison count for “Count”
“Count” variant for multi-pivot quicksort also optimal
But: Average comparison count unknown for k ≥ 5!
Theory & Practice:
Multi-Pivot quicksort has advantages in cache behavior
Does “pointer visits”/“scanned elements” cost measure produce
reliable running time predictions for other algorithms?
M. Aumüller
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
23/23
Conclusion & Open Problems
Theory:
Intuitive algorithm “Count” is comparison-optimal
Exact average comparison count for “Count”
“Count” variant for multi-pivot quicksort also optimal
But: Average comparison count unknown for k ≥ 5!
Theory & Practice:
Multi-Pivot quicksort has advantages in cache behavior
Does “pointer visits”/“scanned elements” cost measure produce
reliable running time predictions for other algorithms?
Thank you!
M. Aumüller
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
23/23
Running Time Experiments
QS15
QS1
QS9
21
22
23
QS7
QS2
QS3
QS5
24
25
Items [log2 (n)]
26
27
Time /n ln n [ns]
4.4
4.2
4
M. Aumüller
Multi-Pivot Quicksort: Comparison-Optimal Algorithms and Beyond
1/1