UNIT_II
Divide and Conquer
By
Dr. A.S.Alvi
Divide and Conquer


Divide and Conquer algorithms consist of two parts:
 Divide: Smaller problems are solved recursively (except, of
course, the base cases).
 Conquer: The solution to the original problem is then
formed from the solutions to the subproblems.
Divide-and-conquer technique for algorithm design. Example
problems:
 Integer Multiplication
 Subset Sum Recursive Problem
 Closest Points Problem
 Skyline Problem
 Strassen’s Algorithm
Divide and Conquer
The most-well known algorithm design strategy:
1.
Divide instance of problem into two or more smaller
instances
2.
Solve smaller instances recursively
3.
Obtain solution to original (larger) instance by combining
these solutions
Divide & conquer

Divide & conquer is a general algorithm design strategy with a
general plan as follows:
1. DIVIDE: A problem’s instance is divided into several
smaller
instances of the same problem, ideally of about the same size.
2. RECUR: Solve the sub-problem recursively.
3. CONQUER: If necessary, the solutions obtained for the smaller
instances are combined to get a solution to the original instance.
Divide-and-Conquer Technique
a problem of size n
subproblem 2
of size n/2
subproblem 1
of size n/2
a solution to
subproblem 1
a solution to
subproblem 2
a solution to
the original problem
It general leads to a
recursive algorithm!
A simple example

4 -6
finding the maximum of a set S of n numbers
Multiplication
981
X 1234
------------
Multiplication

Take elementary arithmetic as an example. Suppose you have
to multiply two positive integers using only pencil and paper.
981
981
1234
1234
3924
981
2943
1962
1962
2943
981
3924
1210554
1210554
(a)
(b)
 Figure 1.1. Multiplication (a) American (b) English
A third, different algorithm for doing the same thing is illustrated
in Figure 1.2.
981
490
245
122
61
30
15
7
3
1

1234
2468
4936
9872
19744
39488
78976
157952
315904
631808
1234
4936
19744
78976
157952
315904
631808
1210554
Figure 1.2. Multiplication i la russe
Multiplication by Divide-and-Conquer
Multiply
09 12
09
34
81
12
81
34
Shift
4
2
2
0
Result
108 . . . .
306 . .
972 . .
2754
1210554
A Multiplication Algorithm
xy = P1 * 10N + (P3-P1-P2) * 10N/2 + P2
Where
P1= ac
P2= bd
P3= (a+b)(c+d)
N = no of digits
Ex. x= 2345
y = 6789
Strassen Matrix Multiplication

A fundamental numerical operation is the multiplication of 2
matrices.
 The standard method of matrix multiplication of two n x n
matrices takes T(n) = O(n3).
X

=
The following algorithm multiples n x n matrices A and B:
// Initialize C.
for i = 1 to n
for j = 1 to n
for k = 1 to n
C [i, j] = A[i, k] * B[k, j];
Strassen’s Algorithm

We can use a Divide and Conquer solution to solve matrix
multiplication by separating a matrix into 4 quadrants:
X

=
Then we know have:
a 
a
A   11 12 
 a21 a22 
b 
b
B   11 12 
 b21 b22 
c 
c
C   11 12 
 c21 c22 
if C  AB , then we have the following: c11  a11b11  a12b21
c12  a11b12  a12b22
c21  a21b11  a22b21
c22  a21b12  a22b22
Strassen’s Algorithm

Strassen showed how two matrices can be multiplied using
only 7 multiplications and 18 additions:Consider calculating
the following 7 products:








q1 = (a11 + a22) * (b11 + b22)
q2 = (a21 + a22) * b11
q3 = a11*( b12 – b22)
q4 = a22 * (b21 – b11)
q5 = (a11 + a12) * b22
q6 = (a21 – a11) * (b11 + b12)
q7 = (a12 – a22) * (b21 + b22)
It turns out that
 c11 = q1 + q4 – q5 + q7
 c12 = q3 + q5
 c21 = q2 + q4
 c22 = q1 + q3 – q2 + q6
Divide and Conquer Matrix Multiply

A
A
B
C
D
B
=
E
F
G
H

=
C
S1+S2-S4+S6
S4+S5
S6+S7
S2+S3-S4+S5
•S1=(B-D)*(G+H)
•S6= D*(G-E)
• S2=(A+D)*(E+H)
•S7=(C+D)*E
•S3=(A-C)*(E+F)
•S4=(A+B)*H
•S5= A*(F-H)
Closest Pair Of Points

Given n points in 2D, find the pair that are closest.
Applications



We plan to drill holes in a metal sheet.
If the holes are too close, the sheet will tear during drilling.
Verify that no two holes are closer than a threshold distance (e.g.,
holes are at least 1 inch apart).
Divide-And-Conquer Solution

When n is small, use simple solution.

When n is large

Divide the point set into two roughly equal parts A and B.

Determine the closest pair of points in A.

Determine the closest pair of points in B.

Determine the closest pair of points such that one point is in A
and the other in B.

From the three closest pairs computed, select the one with
least distance.
Example
A
•
B
Divide so that points in A have x-coordinate <= that
of points in B.
Example
d1
A
•
B
Find closest pair in A.
• Let d1 be the distance between the points in this pair.
Example
d1
d2
A
•
B
Find closest pair in B.
• Let d2 be the distance between the points in this pair.
Example
d1
d2
A
•
B
Let d = min{d1, d2}.
• Is there a pair with one point in A, the other in B and distance < d?
Example
A
•
RA RB
Candidates lie within d of the dividing line.
• Call these regions RA and RB, respectively.
B
Example
q
A
•
RA RB
B
Let q be a point in RA.
• q need be paired only with those points in RB that are within d
of q.y.
Example
2d
q
d
A
•
RA RB
B
Points that are to be paired with q are in a d x 2d rectangle of RB
(comparing region of q).
• Points in this rectangle are at least d apart.
Time Complexity

Create a sorted by x-coordinate list of points.


Create a sorted by y-coordinate list of points.


O(n log n) time.
O(n log n) time.
Using these two lists, the required pairs of points from RA and RB
can be constructed in O(n) time.
Closest Pair

Given Input: A set of n points in a plane. P={p(1),p(2)…..p(n)}
Where p(i)=(x(i),y(i))

Desired Output: The distance between the two points that are
closest.

The distance DELTA(i,j) between p(i) and p(j) is defined by
[(x(i)-x(j))2 +(y(i)-y(j))2]1/2
Closest Pair
There are three possibilities:

The closest pair lie in P-LEFT.

The closest pair lie in P-RIGHT.

The closest pair contains:

One Point from P-LEFT, and

One Point from P-RIGHT
COMBINE stage: Determine whether there are points p(l) in
P-LEFT and p(r) in P-RIGHT with distance
(p(l), p(r) ) < DELTA. If there are such points, set DELTA to
be the smallest distance.
Closest Pair
















Algorithm 8: closest_pair (P: point set; n: integer)
float DELTA-LEFT, DELTA-RIGHT;
float DELTA ;
if n = 2 then
return distance from p(1) to p(2);
else
P-LEFT ( p(1), p(2) ,..., p(n/2));
P-RIGHT
( p(n/2+1), p(n/2+2) ,..., p(n));
DELTA-LEFT
closest_pair (P-LEFT, n/2);
DELTA-RIGHT
closest_pair (P-RIGHT,n/2);
DELTA minimum (DELTA-LEFT, DELTA-RIGHT);
------------------------[COMBINE stage]
------------------------return DELTA;
end
Lines, Segments, and Polygons

We can represent a line l as a triple (a,b,c), such that these values
are the coefficients a, b, and c of the linear equation
ax+by+c = 0
associated with l. Alternatively, we may specify instead two
different points, q1 and q2, and associate them with the line that
goes through both. Given the Cartesian coordinates (x1;y1) of q1
and (x2;y2) of q2, the equation of the line l through q1 and q2 is
given by
[(x−x1) / (x2 −x1)] = [(y−y1)/(y2−y1)]
from which we derive
a = (y2−y1) ; b = −(x2−x1) ; c = y1(x2−x1)−x1(y2 −y1):
Triangulation
Triangulation is a process that takes a region of
space and divides it into subregions.
The Line-side Test: Given a point C and a line AB. The line-side
test answer the question :” On Which side of AB is C?”
B
C
A
The Line-side Test
The line-side test is easily implemented through a matrix
determinant operation. Let the line be defined by points
A=(xa, ya) and B=(xb, yb) and the test point be C=(xc, yc)
 Let
xa ya 1
D= xb, yb 1
xc yc 1
The point C is above the line AB iff D >0
The area of the triangle ABC is given by D/2
Ex: A=(0,0) B=(2,0) C=(1,1)

The In-circle Test

Given the points A,B,C and D , the in-circle test answer the
question: “Is D is in the circle defined by A,B and C?”
B
D
C
A
The In-circle Test

Let A=(xa, ya) , B=(xb, yb) , C=(xc, yc) and D=(xd, yd) . The the
matrix determinant used for the in-circle test is:
xa ya xa2+ ya2 1
xb yb xb2+ yb2 1
xc yc xc2+ yc2 1
xd yd xd2+ yd2 1
Then the point D is in the circle defined by ABC iff ABC>0.
Convex Hull
 Given a set of pins on a pinboard
 And a rubber band around them
 How does the rubber band look
when it snaps tight?
4
5
3
6
2
 We represent the convex hull as
the sequence of points on the convex
hull polygon, in counter-clockwise
order.
0
1
Example: Convex Hull
Input: p1,p2,p3,p4,p5,p6,p7,p8,p9,p10,p11
Output: p2,p9,p11,p4,p5,p6,p8,p2
p9
p11
p4
p1
p2
p3
p7
CH is a convex polygon
p10
p8
p6
p5
Convex Hull: Divide & Conquer
 Preprocessing: sort the points by xcoordinate
 Divide the set of points into two
sets A and B:
 A contains the left n/2 points,
 B contains the right n/2 points
Recursively compute the convex
hull of A
Recursively compute the convex
hull of B
 Merge the two convex hulls
A
B
Convex Hull: Runtime
 Preprocessing: sort the points by x-
O(n log n) just once
coordinate
 Divide the set of points into two
O(1)
sets A and B:
 A contains the left n/2 points,
 B contains the right n/2 points
Recursively compute the convex
T(n/2)
hull of A
Recursively compute the convex
hull of B
 Merge the two convex hulls
T(n/2)
O(n)
Merging in O(n) time
 Find upper and lower tangents in O(n) time
 Compute the convex hull of AB:
 walk counterclockwise around the convex
4
5
3
hull of A, starting with left endpoint of
lower tangent
 when hitting the left endpoint of the upper 2
6
tangent, cross over to the convex hull of B
 walk counterclockwise around the convex
hull of B
 when hitting right endpoint of the lower
tangent we’re done
This takes O(n) time
7
1
A
B
Gift-wrapping algorithm
Idea: Think of
wrapping a package.
Put the paper in
contact with the
package and continue
to wrap around from
one surface to the
next until you get all
the way around.
Gift-wrapping algorithm
1.Start
at lowest point (this is
necessarily on the convex hull)
2.Rotate the line until we hit another
point (ditto)
 All other points will lie on one
side of this line
 Look for the point that gives you
the largest angle with the current
line
3.Repeat
4.You’re done when you get back to
the starting point
41
Figure credit: Craig Gotsman
Graham Scan Algorithm

Form a simple polygon (connect the
dots as before)

Remove points at concave angles
(one at a time, backtracking one step
when any point is removed).

Continue until you get all the way
around.
Graham Scan How Does it Work?

Start with the lowest point (anchor point)
Graham Scan: Phase 1

Now, form a closed simple path traversing the points by increasing
angle with respect to the anchor point
Graham Scan: Phase 2

The anchor point and the next point on the path must be on the
hull (why?)
Graham Scan: Phase 2


keep the path and the hull points in two sequences
 elements are removed from the beginning of the path sequence and
are inserted and deleted from the end of the hull sequence
orientation is used to decide whether to accept or reject the next point
next
cur
prev
n
(p,c,n) is a
right turn!
c
p
Discard c
n
c
p
n
p
(p,c,n) is a
right turn!
c
n
c
p
(p,c,n) is a
right turn!
n
c
p
Merge Sort

It is a sorting technique, which divides the array into two parts
whose size are same, sorting these parts by recursive calls, and
then merging the solutions for each part.

To do this , we need an efficient algorithm for merging two
sorted array whose length is the sum of the lengths of two
sorted arrays.

Ex. 33 66 22 40 55 88 11 60 20 80 44 50 30 77
Merge Sortcont…….
8 3 2 9 7 1 5 4
8 3 2 9
8 3
8
7 1 5 4
2 9
3
2
3 8
7 1
9
2 9
7
5 4
1
5
1 7
2 3 8 9
4 5
1 4 5 7
1 2 3 4 5 7 8
9
4
Merge Sortcont…….

Timing analysis
The number of comparisons ƒ(n) needed to sort an nelement array T using the merge sort algorithm requires at
most (log n) passes and also each passes merges a total of n
elements, so each pass will require at most n comparisons.
Let t(n) be the time taken by the algorithm to sort an array
of n elements. The running time for both the worst case and
average case is given by
T(n) Є O(n lon n)
Merge Sortcont…….
Complexity Analysis:
Assuming that time taken to merge is proportional to n , the
time taken by merge sort is given by the recursive formula:
a
n=1 , a is a constant
T(n) =
2T(n/2+cn) n>1 , c is a constant
If n is a power of 2, we can do successive substitution and
T(n) = an +cn log2n
Hence the time complexity is O(n log n)
Merge Sortcont…….

Algorithm : Mergesort(integer low, high)
global array a[low:high];[having high-low+1 values]
if low < high then
mid
Floor((low+high)/2);
mergesort(low,mid);
mergesort(mid+1,high);
mergsort(low,mid.high);
end
Binary Search


To do binary search, first we need to sort the array elements.
The logic behind this technique is:
 First find the middle element of the array.
 Compare the middle element with an item.
 There are three cases possible:
o If it is a desired element then search is successful.
o If the desired item is less then middle then search only
the first half of the array.
o If the desired item is greater the middle then search only
the second half of the array.
Repeat the same step until an element is found or exhaust
in the search area.
Binary Searchcont….. (Iterative method)
high n-1;
while low ≤ high do
mid
((low+high)/2);
if x = a[mid] then
j
mid;
return;
if x < a[mid] then
high mid-1;
else
low
mid+1;
end
j=n;
The End
Binary Searchcont….. (Recursive method)
local int mid;
if low > high then
return high +1;
mid = Floor((low+high)/2);
if a[mid] = x then
return mid;
if x < a[mid] then
return BSR( a , low , mid-1 , x);
else
return BSR( a , mid-1,high , x);
end
end
The End
Large Positive Integer Power
Find , where
.
naive solution: p =
perform (n-1) floating point multiplications.
Divide as two sub-problems
, then combine the
results by multiplying, require far fewer mult overall.
Done recursively: =
, =
, and p =
, then for large n, number of mult needed
roughly
.
This algorithm will require, for example, 9
multiplications to get
power of any number, but
14 multiplications to get
power.
Large Power by D&C
Algorithm 1: Algorithm power(x,n)
power(float x, integer n);
local float a;
if n = 0 then
return 1; //x0 =1
a
power(x,n/2);
if n % 2 then
[ odd n ]
return x*a*a;
else
[ even n ]
return a*a;
end
Limitations of D&C strategy
Find the maximum value in an array a[1...n].
Algorithm 2: Maximum : Naive Approach
max
a[1];
for i
2 to n do
if a[i] max then
max
a[i];
end
end
In this naive approach, the number of iterations of the
for loop is (n-1). Time to execute will be
proportional to n.
Algorithm 3: Maximum : D&C Approach
Inputs : x,y;
if y-x 1 then
return max(a[x],a[y]);
else
);
max1 = maximum(x,
+1,y);
max2 = maximum(
return max(max1,max2)
end
The D&C algorithm executes max() operation (n-1)
times. Thus the overall time is of order of .
Seemingly D&C is not effective.
Reason: size of the problem space does not get
reduced at each iteration. It remains of order of n.
Problem space size should reduce for each iteration.
That gives the D&C strategy its real power and appeal.
Advantages of Divide & Conquer technique
• For
solving conceptually difficult problems like Tower Of
Hanoi, divide & conquer is a powerful tool
• Results in efficient algorithms
• Divide & Conquer algorithms are adapted for execution in
multi-processor machines
• Results in algorithms that use memory cache efficiently.
Limitations of divide & conquer technique:
• Recursion is slow
• Very simple problem may be more complicated than an
iterative approach. Example: adding n numbers etc
Thank you
for your attention
Nearest neighbour heuristic algorithm:
1. Arbitrarily select a starting city.
2. To select a next city, look at all cities not yet visited. Select
the one closest to the current city.
3. Repeat step 2 until all cities have been visited.
Inference Rules
Modus Ponens
From P and P → Q , infer Q.
P
P→Q
-----------Q
For ex:
Given :
(John is a student)
And :
(John is a student) → (John is a Engg student)
Conclude : (John is a Engg student)
Inference Rules
 Modus
tollens:
From P and P → Q , infer Q.
Q
P→Q
-----------P
For ex:
Given : (John is a Engg student)
And : (John is a student) → (John is a Engg student)
Conclude: (John is a student)
Inference Rules
 Chain
rule :
From P → Q , Q → R infer P → R.
P→Q
Q→R
-----------P→R
Depth-first searching

A
B
C

D
E
F
H
L
M
I
N
O
G
J
P
K

Q

82
A depth-first search (DFS)
explores a path all the way to
a leaf before backtracking and
exploring another path
For example, after searching
A, then B, then D, the search
backtracks and tries another
path from B
Node are explored in the
order A B D E H L M N I O P
CFGJKQ
N will be found before J
Breadth-first searching

A
B
C

D
E
F
G

H
L
M
I
N
O
J
P
K
Q

A breadth-first search (BFS)
explores nodes nearest the
root before exploring nodes
further away
For example, after searching
A, then B, then C, the search
proceeds with D, E, F, G
Node are explored in the
order A B C D E F G H I J K
LMNOPQ
J will be found before N
Criterion
Time
Breadth-First Depth-First
bd
bm
Space
bd
bm
Optimal?
Yes
No
Complete?
Yes
No
b: branching factor
d: solution depth
m: maximum depth
Tower of Hanoi
There are three towers
The disks, with decreasing sizes, placed on the first tower
You need to move all of the disks from the first tower to the
second tower
Larger disks can not be placed on top of smaller disks
The third tower can be used to temporarily hold disks
a
b
c
Tower of Hanoi
a
b
c
Problem Characteristics

Is the problem decomposable?

Can solution steps be ignored or undone?

Is the universe predictable?

Is a good solution absolute or relative?

Is the solution a state or a path?

What is the role of knowledge?

Does the task require human-interaction?
Is the problem decomposable?


Can the problem be broken down to smaller problems to be
solved independently?
Decomposable problem can be solved easily.
2
(x2 + 3x + sin2x.cos x) dx
2
x dx
x3/3
3x dx
3x dx
2
2
 sin x.cos x dx
2
(1− cos x)cos2xdx
3x2/2

 cos2xdx
−  cos4xdx
Can solution steps be ignored or undone?

Ignorable (e.g. Theorem Proving), in which solution steps
can be ignored.

Recoverable (e.g. 8-puzzle), in which solution steps can be
undone.

Irrecoverable (e.g. Chess), in which solution steps can be
undone.
Is the universe predictable?

The 8-Puzzle: Every time we make a move, we know exactly
what will happen.
Certain outcome!

Playing Bridge: We cannot know exactly where all the cards
are or what the other players will do on their turns.
Uncertain outcome!
Is a good solution absolute or relative?

1. Marcus was a man
2. Marcus was a Pompeian
3. Marcus was born in 40 A.D
4. All men are mortal.
5. All Pompeians died when the volcano erupted in
79 A.D
6. No mortal lives longer than 150 years.
7. It is now 2008 A.D

Question: Is Marcus alive?







What is the role of knowledge?

Playing Chess
Knowledge is important only to constrain the search for a
solution.

Reading Newspaper
Knowledge is required even to be able to recognize a
solution.
Does the task require human-interaction?
Is the solution a state or a path?

The Water Jug Problem
• The path that leads to the goal must be reported.

A path-solution problem can be reformulated as a state-
solution problem by describing a state as a partial path to a
solution.

The question is whether that is natural or not.