Divide and Conquer
The most well known algorithm design strategy:
1. Divide instance of problem into two or more
smaller instances
2. Solve smaller instances recursively
3. Obtain solution to original (larger) instance
by combining these solutions
Design and Analysis of Algorithms - Chapter 4
1
Divide-and-conquer technique
a problem of size n
subproblem 1
of size n/2
subproblem 2
of size n/2
a solution to
subproblem 1
a solution to
subproblem 2
a solution to
the original problem
Design and Analysis of Algorithms - Chapter 4
2
Divide and Conquer Examples





Sorting: mergesort and quicksort
Tree traversals
Binary search
Matrix multiplication - Strassen’s algorithm
Convex hull - QuickHull algorithm
Design and Analysis of Algorithms - Chapter 4
3
General Divide and Conquer recurrence:
T(n) = aT(n/b) + f (n)
1.
2.
3.
a < bk
a = bk
a > bk
where f (n) ∈ Θ(nk)
T(n) ∈ Θ(nk)
T(n) ∈ Θ(nk lg n )
T(n) ∈ Θ(nlog b a)
Note: the same results hold with O instead of Θ.
Design and Analysis of Algorithms - Chapter 4
4
Mergesort
Algorithm:
 Split array A[1..n] in two and make copies of each half in
arrays B[1.. n/2 ] and C[1.. n/2 ]
 Sort arrays B and C
 Merge sorted arrays B and C into array A as follows:
• Repeat until no elements remain in one of the arrays:
– compare the first elements in the remaining unprocessed
portions of the arrays
– copy the smaller of the two into A, while incrementing the index
indicating the unprocessed portion of that array
• Once all elements in one of the arrays are processed,
copy the remaining unprocessed elements from the other
array into A.
Design and Analysis of Algorithms - Chapter 4
5
Mergesort Example
7 2 1 6 4
Design and Analysis of Algorithms - Chapter 4
6
Efficiency of mergesort
 C(n)
= 2 C(n/2) + Cmerge(n) for n>1, C(1)=0
 All cases have same efficiency: Θ(n log n)
 Number of comparisons is close to
theoretical minimum for comparison-based
sorting:
• log n ! ≈ n lg n - 1.44 n
 Space requirement: Θ(n) (NOT in-place)
 Can be implemented without recursion
(bottom-up)
Design and Analysis of Algorithms - Chapter 4
7
Quicksort
Select a pivot (partitioning element)
 Rearrange the list so that all the elements in the
positions before the pivot are smaller than or equal
to the pivot and those after the pivot are larger than
the pivot (See algorithm Partition in section 4.2)
 Exchange the pivot with the last element in the first
(i.e., ≤ sublist) – the pivot is now in its final position
 Sort the two sublists

p
A[i]≤p
A[i]>p
Design and Analysis of Algorithms - Chapter 4
8
The partition algorithm
Design and Analysis of Algorithms - Chapter 4
9
Quicksort Example
15 22 13 27 12 10 20 25
Design and Analysis of Algorithms - Chapter 4
10
Efficiency of quicksort
Best case: split in the middle — Θ( n log n)
 Worst case: sorted array! — Θ( n2)
 Average case: random arrays — Θ( n log n)
 Improvements:

• better pivot selection: median of three partitioning
avoids worst case in sorted files
• switch to insertion sort on small subfiles
• elimination of recursion
these combine to 20-25% improvement

Considered the method of choice for internal
sorting for large files (n ≥ 10000)
Design and Analysis of Algorithms - Chapter 4
11
Multiplication of large integers
To multiply two integers of n1 and n2 digits by a
pen-and-pencil algorithm, we perform n1n2
multiplications
 However, remark the example of 23*14, where
23=2.101+3.100 and 14=1.101+4.100
 In general: c=a*b=c2.10n+c1.10n/2+c0.100 where
c2=a1*b1, c0=a0*b0, c1=(a1+a0)*(b1+b0)-(c2+c0)
a0,a1,b0,b1 be the left and right parts of a and b.
 Recurrence relation
 Efficiency

Design and Analysis of Algorithms - Chapter 4
12
Closest Pair Problem
Problem: find closest points among n ones in kdimensional space
 Assume points are sorted by x-coordinate values
 Divide the points in two subsets S1 and S2 of n/2
points by drawing a vertical line at x=c
 Select the closest pair among the closest pair of the
left subset (distance d1), of the closest pair of the
right subset (distance d2), and the closest pair with
points from both sides
 We need to examine a stripe of width 2d, where
d=min[d1,d2]

Design and Analysis of Algorithms - Chapter 4
13
Closest Pair Problem (2)

Geometric explanation
Recurrence relation T(n) = 2T(n/2) + M(n)
 Efficiency in O(n logn)
 Optimality

Design and Analysis of Algorithms - Chapter 4
14
QuickHull Algorithm
Inspired by Quicksort compute Convex Hull:
 Assume points are sorted by x-coordinate values
 Identify extreme points P1 and P2 (part of hull)
 The set S of points is divided in two subsets S1 and S2
 Compute Convex Hull for S1
 Compute Convex Hull for S2 in a similar manner
Pmax
P2
P1
Design and Analysis of Algorithms - Chapter 4
15
QuickHull Algorithm (2)

How to compute the Convex (upper) Hull for S1
• find point Pmax that is farthest away from line P1P2
• compute the hull of the points to the left of line P1Pmax
• compute the hull of the points to the left of line PmaxP2

How to find the Pmax point
• Pmax maximizes the area of the triangle PmaxP1P2
• if tie, select the Pmax that maximizes the angle PmaxP1P2
The points inside triangle PmaxP1P2 can be excluded
from further consideration
 How to find geometrically the point Pmax and the
points to the left of line P1Pmax

Design and Analysis of Algorithms - Chapter 4
16
QuickHull Algorithm (3)

How to find geometrically the point Pmax and the points to
the left of line P1Pmax
• Given points P1(x1,y1), P2(x2,y2), Pmax(xmax,ymax)
• The area of the triangle is half of the magnitude of the
determinant
x1
y1
1
x2
y2
1
=x1y2+xmaxy1+x2ymax–xmaxy2–x2y1-x1ymax
xmax ymax 1
• The sign of the expression is positive if and only if the
point Pmax is to the left of the line P1P2
Design and Analysis of Algorithms - Chapter 4
17
Efficiency of QuickHull algorithm
Finding point farthest away from line P1P2 can be
done in linear time
 This gives same efficiency as quicksort:

• Worst case: Θ(n2)
• Average case: Θ(n log n)
If points are not initially sorted by x-coordinate
value, this can be accomplished in Θ(n log n) —
no increase in asymptotic efficiency class
 Other algorithms for convex hull:

• Graham’s scan
• DCHull
also in Θ(n log n)
Design and Analysis of Algorithms - Chapter 4
18
Strassen’s matrix multiplication

Strassen observed [1969] that the product of two matrices
A and B (of size 2nx2n) can be computed as follows:
C00 C01
A00 A01
=
C10 C11
B00 B01
*
A10 A11
B10 B11
M1 + M4 - M5 + M7
M3 + M5
=
M2 + M4
Design and Analysis of Algorithms - Chapter 4
M1 + M 3 - M2 + M6
19
Submatrices:
M1 = (A00 + A11) * (B00 + B11)
 M2 = (A10 + A11) * B00
 M3 = A00 * (B01 - B11)
 M4 = A11 * (B10 - B00)
 M5 = (A00 + A01) * B11
 M6 = (A10 - A00) * (B00 + B01)
 M7 = (A01 - A11) * (B10 + B11)

Design and Analysis of Algorithms - Chapter 4
20
Efficiency of Strassen’s algorithm





If n is not a power of 2, matrices can be
padded with zeros
Number of multiplications:
Number of additions:
Recurrence relations:
Other algorithms have improved this result,
but are even more complex
Design and Analysis of Algorithms - Chapter 4
21