Design and Analysis of Algorithm
Algorithm: The method of solving a problem is known as an algorithm. It can also be
defined as a sequence of instruction that act on some input data to produce some output in
finite number of steps.
Properties of an algorithm
a) Input: An algorithm must receive some input data supplied externally.
b) Output: An algorithm must produce at least one output as the result.
c) Finiteness: No matter what is the input, the algorithm must terminate after a finite
number of steps.
d) Definiteness: The Steps to be performed in the algorithm must be clear and
unambiguous.
e) Effectiveness: One must be able to perform the steps in the algorithm without
applying any intelligence.
Algorithm fall under two broad categories:
1. Iterative algorithms: Typically uses loops and conditional statements.
2. Recursive algorithms: Uses a ‘divide and conquer’ strategy. This strategy
divides a large problem into small pieces and then applies the algorithm to each of
these small pieces.
Analysis: In designing algorithms we need methods to separate bad algorithms from
good ones, Analysis of algorithms helps us to determine which algorithm should be
chosen to solve the problem.
Consider the following two algorithms to find the biggest of four values:
Algorithm One:
big = a
if ( b > big )
big = b
endif
if ( c > big )
big = c
endif
if ( d > big )
big = d
endif
return big
Jayesh Patel
Page 1
8/1/2017
Design and Analysis of Algorithm
Algorithm Two:
if ( a > b )
if ( a > c)
if ( a > d )
return a
else
return d
endif
else
if ( c > d )
return c
else
return d
endif
endif
else
if ( b > c )
if ( b > d )
return b
else
return d
endif
else
if ( c > d)
return c
else
return d
endif
endif
endif
Both the above algorithms do exactly three comparisons to find the biggest number. First
is easy to read and understand but both are of same level of complexity for a computer to
execute. In terms of time, these two algorithms are same but in terms of space, the first
needs more because of temporary variable called big. This extra space is not significant
if we are comparing numbers or characters, but it may be with other types of data, like
say record of an employee. In such cases number of comparisons is used to determine the
algorithm that solves the problem more efficiently.
Jayesh Patel
Page 2
8/1/2017
Design and Analysis of Algorithm
Case to consider during Analysis:
Choosing the input to consider when analyzing an algorithm can have a significant
impact on how an algorithm will perform.
1. Best Case Input – This represents the input set that allows an algorithm to perform
most quickly. With this input algorithm takes shortest time to execute.
2. Worst Case Input – This represents the input set that allows an algorithm to
perform most slowly.
3. Average Case Input – This represents the input set that allows an algorithm to
deliver an average performance.
________________________________________________________________________
Stack: Is a data structure in which addition of new element or deletion of an existing
element always takes place at the same end. This end is known as top of stack.
Two operations that can be performed on stack are:
1. Push: When an item is added to a stack, operation is called push.
2. Pop: When an item is removed from the stack, operation is called pop.
Stack is also called as last-in-first-out (LIFO) list. If the elements are added continuously
to the stack it grows at one end. On deletion of elements stack shrinks at the same end.
Stack does not have any fixed size. It keeps on changing, as the elements in stack are
popped or pushed.
Diagram:
Representation of Stack after Inserting
Item
Representation of Stack after Deleting Item
top
top
10
10
8
8
8
8
-4
-4
-4
-4
-4
-4
2
2
2
2
2
2
top
top
top
top
top = NULL
Jayesh Patel
Page 3
top = NULL
8/1/2017
Design and Analysis of Algorithm
Algorithm:
Algorithm: Add(item)/Push Operation
Add(item)
{
if( top >= n -1) then
{
write(“Stack is Full”);return false;
}
else
{
top := top + 1;
stack[top] := item;
return true;
}
}
Jayesh Patel
Page 4
Algorithm: Delete(item)/Pop Operation
Delete(item)
{
if( top < 0) then
{
write(“Stack is Empty”);return false;
}
else
{
item := stack[top];
top := top - 1;
return true;
}
}
8/1/2017
Design and Analysis of Algorithm
Array Implementation of Stack
#include<stdio.h>
#include<conio.h>
#define MAX 4
struct stack
{
int item[MAX];
int top;
};
void initstack(struct stack*);
void push(struct stack*,int value);
void pop(struct stack*);
void main()
{
struct stack s;
int i;
clrscr();
initstack(&s);
push(&s,11);
push(&s,23);
push(&s,-12);
push(&s,3);
push(&s,4);
printf("\n\n");
pop(&s);
pop(&s);
pop(&s);
pop(&s);
pop(&s);
getch();
}
void initstack(struct stack *s)
{
printf("\n\tImplementation of stack using Arrays\n\n");
s->top = -1;
}
Jayesh Patel
Page 5
8/1/2017
Design and Analysis of Algorithm
void push(struct stack *s, int value)
{
if(s->top == MAX - 1)
{
printf("\n\nSTACK FULL");
}
else
{
printf("\nItem Pushed %d",value);
s->top++;
s->item[s->top] = value;
}
}
void pop(struct stack *s)
{
int data;
if(s->top == -1)
{
printf("\n\nSTACK EMPTY.");
}
else
{
data = s->item[s->top];
s->top--;
printf("\nItem Popped %d",data);
}
}
Jayesh Patel
Page 6
8/1/2017
Design and Analysis of Algorithm
Queue: Is a linear data structure that permits addition of new element at one end and
deletion of an element at the other end. The end at which the deletion of the element takes
place is called front and the end at which insertion of elements can take place is called
rear.
The first element that gets added into the queue is also referred to as first one to get
removed from the list. Hence, queue is also referred as first-in-first-out (FIFO) list.
Diagram:
Linear Queue :
arr[0] arr[1] arr[2] arr[3] arr[4] arr[5]
5
9
2
4
7
3
rear
front
Circular Queue :
front
5
3
rear
9
4
2
7
Jayesh Patel
Page 7
8/1/2017
Design and Analysis of Algorithm
Algorithm:
Algorithm: AddQ(item)
Algorithm: DeleteQ(item)
AddQ(item)
DeleteQ(item)
{
{
rear := (rear + 1) mod n;
if( front = rear) then
if( front = rear) then
{
{
write(“Queue is Empty”);
write(“Queue is Full”);
return false;
if( front = 0) then
}
rear := n -1;
else
else
{
rear := rear -1;
front := (front + 1) mod n;
return false;
item := q[front];
}
return true;
else
}
{
}
q[rear] := item;
return true;
}
}
Note: Algorithm for circular Queue and program is on linear Queue
Jayesh Patel
Page 8
8/1/2017
Design and Analysis of Algorithm
Array Implementation of Queue
#include<stdio.h>
#include<conio.h>
#define MAX 4
struct queue
{
int item[MAX];
int front;
int rear;
};
void initqueue(struct queue*);
void qadd(struct queue*,int value);
void qdelete(struct queue*);
void main()
{
struct queue q;
int i;
clrscr();
initqueue(&q);
qadd(&q,11);
qadd(&q,23);
qadd(&q,-12);
qadd(&q,3);
qadd(&q,4);
printf("\n\n");
qdelete(&q);
qdelete(&q);
qdelete(&q);
qdelete(&q);
qdelete(&q);
getch();
}
void initqueue(struct queue *q)
{
printf("\n\tImplementation of Queue using Arrays\n\n");
q->rear = -1;
q->front = -1;
}
Jayesh Patel
Page 9
8/1/2017
Design and Analysis of Algorithm
void qadd(struct queue *q,int value)
{
if(q->rear == MAX - 1)
{
printf("\n\nQUEUE FULL");
}
else
{
printf("\nItem Pushed %d",value);
q->rear++;
q->item[q->rear] = value;
if(q->front == -1)
{
q->front = 0;
}
}
}
void qdelete(struct queue *q)
{
int data;
if(q->front == -1)
{
printf("\n\nQUEUE EMPTY.");
}
else
{
data = q->item[q->front];
q->item[q->front] = 0;
if(q->front == q->rear)
{
q->front = q->rear = -1;
}
else
{
q->front++;
}
printf("\nItem Popped %d",data);
}
}
Jayesh Patel
Page 10
8/1/2017
Design and Analysis of Algorithm
Tree: A tree is a finite set of one or more nodes such that there is a specially designated
node called as root and the remaining nodes are partitioned into n >= 0 disjoint sets T1,
T2,….Tn are called the subtrees of the root.
Diagram:
Tree:
Level 1
A
B
E
K
F
L
G
Level 2
D
C
H
I
M
J
Level 3
Level 4
Tree Terminology:
Degree: Number of subtrees of a node is called its degree.
Degree of node A is 3, C is 1 and of F is 0.
Leaf: Nodes that have degree zero are called leaf or terminal nodes.
Nodes {K, L, F, G, M, I, J} is the set of leaf nodes.
Non Terminal Node: Node with non zero degree are called as non terminal node.
Parent-Children: Consider node X, Let n be number of subtree originating from this
node then roots of this subtrees will be called children of node X and node X will be
called parent of this children. Thus the children of D are H, J, I and the parent of D is A.
Siblings: Children of the same parent are said to be siblings. For example H, I, J are
siblings.
Ancestors: The ancestors of a node are all the nodes along the path from the root to that
node. The ancestors of M are A, D and H.
Jayesh Patel
Page 11
8/1/2017
Design and Analysis of Algorithm
Degree of Tree: Degree of a tree is maximum degree of the nodes in the tree. Degree of
above tree is 3.
Level: Level of a node is defined by initially letting the root be at level one. If the node is
at level p, then its children are at level p +1.
Height or Depth: The height or depth of a tree is defined to be the maximum level of
any node in the tree.
Forest: Forest is a set of n >= 0 disjoint trees. Forest can be obtained by deleting the root
node of the tree.
________________________________________________________________________
Binary Tree: A binary tree is a finite set of nodes that is either empty or consists of a
root and two disjoint binary trees called the left and right subtrees.
The maximum number of nodes on level i of a binary tree is 2i-1. Also, the maximum
number of nodes in a binary tree of depth k is 2k -1, k > 0.
Full Binary Tree: Binary tree of depth k that has exactly 2k -1 nodes is called a
Full binary tree.
Diagram:
Binary Tree
Full Binary Tree
1
A
B
D
H
2
C
E
F
G
4
3
5
6
7
I
Jayesh Patel
Page 12
8/1/2017
Design and Analysis of Algorithm
Binary Search Trees: A binary search tree is a binary tree. It may be empty. If it is not
empty, then it satisfies the following properties:
1. Every element has a key no two elements have the same key (i.e., the keys are
distinct).
2. The keys (if any) in the left subtree are smaller than the key in the root
3. The keys (if any) in the right subtree are larger than the key in the root
4. The left and right subtrees are also binary search trees.
Diagram:
Not Binary Searh Tree
Binary Search Tree
30
20
15
12
5
25
10
Jayesh Patel
22
2
Binary Search Tree
60
40
70
65
Page 13
8/1/2017
80
Design and Analysis of Algorithm
Algorithm:
Recursive search of binary search tree
Iterative search of binary search tree
Search(t , x)
{
if ( t = 0 ) then
return 0;
else
if ( x = t -> data ) then
return t;
else
if ( x < t-> data ) then
return Search( t-> lchild , x );
else
return Search( t-> rchild ,x );
}
Search ( x )
{
found := false;
t := tree;
while( ( t <> 0 ) and not found ) do
{
if ( x = ( t -> data)) then
found := true;
else
if ( x < ( t->data)) then
t := (t -> lchild);
else
t := (t -> rchild);
}
if ( not found) then
return 0;
else
return t;
}
Jayesh Patel
Page 14
8/1/2017
Design and Analysis of Algorithm
Insertion into binary search tree
Insert( x )
{
found := false;
t := tree;
while( ( t <> 0 ) and not found ) do
{
q := t;
if ( x = ( t -> data)) then
found := true;
else
if ( x < ( t->data)) then
t := (t -> lchild);
else
t := (t -> rchild);
}
if ( not found) then
{
t := new TreeNode;
(t -> lchild) := 0;
(t -> rchild) := 0;
t-> data := x;
if ( tree <> 0 ) then
{
if ( x < (q->data)) then
(q->lchild) := p;
else
(q->rchild) := p;
}
else
tree := p;
}
}
Jayesh Patel
Page 15
8/1/2017
Design and Analysis of Algorithm
Graph: A graph G consist of two sets V and E. The set V is finite, nonempty set of
vertices. The set E is a set of pairs of vertices, these pairs are called edges. The notation
V (G) and E (G) represent the sets of vertices and edges respectively, of graph G.
G = (V, E) represents a graph.
Undirected Graph: In undirected graph the pair of vertices representing any edge is
unordered. Thus, the pairs (u, v) and (v, u) represents the same edge.
The maximum number of edges in any n-vertex, undirected graph is n(n-1)/2.
Undirected graph with exactly n (n-1)/2 edges is called complete.
Directed Graph: In directed graph each edge is represented by a directed pair (u, v). u is
the tail and v the head of the edge.
The maximum number of edges in any n-vertex, directed graph is n(n-1).
Directed graph with exactly n (n-1) edges is called complete
Graph G1
Graph G2
Graph G3
1
1
2
1
3
3
2
2
4
4
5
6
7
3
Graph Terminology:
Adjacent and Incident: If (u, v) is an edge in E(G), then we say vertices u and v are
adjacent and edge (u, v) is incident on vertices u and v.
Example:
The vertices adjacent to vertex 2 in G2 are 4, 5 and 1.
The edges incident on vertex 3 in G2 are (1, 3), (3, 6) and (3, 7).
Jayesh Patel
Page 16
8/1/2017
Design and Analysis of Algorithm
Degree: Degree of a vertex is the number of edges incident to that vertex. The degree of
vertex 1 in G1 is 3.
If G is directed graph, we define in-degree of a vertex v to be the number of edges for
which v is the head.
The out-degree is defined to be number of edges for which v is the tail.
Example: Vertex 2 of G3 has in-degree 1, out-degree 2, and degree 3.
Sub Graph: A sub graph of G is a graph G’ such that V(G’) is subset of V(G) and E(G’)
is subset of E(G). Some of the sub graphs of G1 and G3 are shown in the figure below.
Length of Path: Is number of edges on it.
Simple Path: Is a path in which all vertices except possibly the first and last are distinct.
Example: A path such as (1, 2), (2, 4), (4, 3), is also written as 1,2,4,3. Paths 1, 2, 4, 3 and
1, 2, 4, 2 of G1 are both of length 3. The first is simple path and the second is not.
Cycle: A cycle is a simple path in which the first and last vertices are the same. The path
1, 2, 3, 1 is a cycle in G1 and 1, 2, 1 is a cycle in G3.
Strongly Connected: A directed graph G is said to be strongly connected if and only if
for every pair of distinct vertices u and v in V(G), there is a directed path from u to v and
also from v to u.
Example: Graph G3 is not strongly connected since there is no path from u to v.
Weighted Edges: Represents the distance from one vertex to another or the cost of going
from one vertex to an adjacent vertex. A graph with weighted edges is called a network.
Graph Representation:
Adjacency Matrix: Let G = (V, E) be a graph with n vertices, n >= The adjacency
matrix of G is a two-dimensional n x n array, say a, with the property that a [i, j] = 1if and
only if the edge (i, j) is in E(G). The element a[i, j] = 0 if there is no such edge in G. The
adjacency matrices for the Graph G1, G3 are shown below.
G1
Jayesh Patel
G2
Page 17
8/1/2017
Design and Analysis of Algorithm
Divide and Conquer:
Binary Search:
Let ai, 1<=i<=n, be list of elements that are sorted in non decreasing order. Consider the
problem of determining whether a given element x is present in the list.
Let P = (n, ai, .. al,x) denote the arbitrary instance of the search problem where n is the
number of elements in the list, ai, .. al is the list of elements, and x is the element to be
searched.
Divide and Conquer can be used to solve this problem. In the above problem P gets
divided into new sub problem. This division takes only O(1) time. After a comparison
with aq , the instance remaining to be solved (if any) can be solved by using this divide
and conquer scheme again. If q is always chosen such that aq is the middle element (that
is, q = [(n+1)/2], then the resulting search algorithm is known as binary search.
The answer to the new sub problem is also the answer to the original problem P.
Recursive Binary Search
BinSrch ( a, i, l, x )
{
if ( l = i ) then
{
if ( x = a[i] ) then return i;
else return 0;
}
else
{
mid := [(i+l)/2];
if ( x = a[mid] ) then return mid;
else if ( x < a[mid] ) then
return BinSrch ( a, i, mid - 1, x );
else
return BinSrch ( a, mid + 1, l, x );
}
}
Jayesh Patel
Page 18
8/1/2017
Design and Analysis of Algorithm
Iterative Binary Search
BinSrch ( a, n, x )
{
low := 1; high := n;
while ( low <= high ) do
{
mid := [(low + high)/2];
if ( x < a[mid]) then high := mid – 1;
else if ( x > a[mid]) then low := mid + 1;
else return mid;
}
return 0;
}
Jayesh Patel
Page 19
8/1/2017
Design and Analysis of Algorithm
FINDING MAXIMUM AND MINIMUM:
Consider the problem to find the maximum and minimum items from the given set of n
elements. This problem can be solved by using divide and conquer algorithm as follows:
Let P = (n , a[i],….,a[j]) denote an arbitrary instance of the problem. Here n is the number
of elements in the list a[i],…., a[j]. and we are interested in finding the maximum and
minimum of this list. For example we can divide P into the two instances P1 =
([n/2],a[1],…,a[(n/2)] ) and P2 = (n – [n/2], a[(n/2) +1],….,a[n]). After having divided P
into two smaller sub problems, this can be solved by recursively invoking the same
divide and conquer algorithm.
If MAX(P) and MIN(P) are the maximum and minimum of the elements in P, then
MAX(P) is larger of MAX(P1) and MAX(P2). Also, MIN(P) is the smaller of MIN(P1)
and MIN(P2).
The Algorithm MaxMin is recursive that finds the maximum and minimum of the set of
elements {a(i),a(i+1),….,a(j) }. Which is divide into two sets that are then handled
separately. For set containing more than two elements, the midpoint is determined and
two new sub problems are generated. When the maxima and minima of these sub
problems are determined, the two maxima and minima are compared to achieve the
solution for the entire set.
MaxMin Algorithm
MaxMin (i, j, max, min)
{
if ( i = j ) then max := min := a[i];
else if ( i = j – 1 ) then
{
if ( a[i] < a[j] )
{
max := a[j]; min := a[i];
}
else
{
max := a[i]; min := a[j];
}
}
else
{
mid := [(i+j)/2];
MaxMin(i, mid, max, min);
MaxMin(mid + 1, j, max1, min1);
If ( max < max1 ) then max := max1;
If ( min > min1 ) then min := min1;
Jayesh Patel
Page 20
8/1/2017
Design and Analysis of Algorithm
}
}
MERGE SORT:
Is a sorting algorithm that has the nice property that in the worst case its complexity is
O(nlogn).Given a sequence of n elements (also called as keys) a[1],…..,a[n], the general
idea is to imagine them split into two sets a[1], ….,a[(n/2)] and a[(n/2)+1],…,a[n]. Each
set is individually sorted, and the resulting sorted sequences are merged to produce a
single sorted sequence of n elements. Merge sort is an ideal example of divide and
conquer strategy in which the splitting is into two equal-sized sets and the combining
operation is the merging of two sorted sets into one.
Example: Consider the array of ten elements
a[1:10] = (310,285,179,652,351,423,861,254,450,520). Algorithm merge sort begins by
splitting a[] into sub arrays each of size five (a[1:5] and a[6:10]). The elements in a[1:5]
are then split into two sub arrays of size three (a[1:3]) and two (a[4:5]). Then the items in
a[1:3] are split into sub arrays of size two (a[1:2]) and one (a[3:3]). The two values in
a[1:2] are split a final time into one – element sub arrays, and now the merging begins.
Following steps in which the file can be viewed pictorially.
(310 | 285 | 179 | 652, 351 | 423, 861, 254, 450, 520)
(285,310 | 179 | 652, 351 | 423, 861, 254, 450, 520)
(179, 285, 310 | 652, 351 | 423, 861, 254, 450, 520)
(179, 285, 310 | 351, 652 | 423, 861, 254, 450, 520)
(179, 285, 310, 351, 652 | 423 | 861 | 254 | 450, 520)
(179, 285, 310, 351, 652 | 254, 423, 861 | 450, 520)
(179, 285, 310, 351, 652 | 254, 423, 450, 520, 861)
At this point there are sorted sub arrays and the final merge produces the fully sorted
result.
(179, 254, 285, 310, 351, 423, 450, 520, 625, 861)
Jayesh Patel
Page 21
8/1/2017