Course Information
Search Algorithms
Other Variations
573 Intelligent Systems
Sri Lanka Institute of Information Technology
Dr. E.C. Kulasekere
3ed July, 2005
Dr. E.C. Kulasekere
573 Intelligent Systems
Course Information
Search Algorithms
Other Variations
Course Outline
Reading Material
Course Setup
Course Outline
Intelligent agents.
Agent types, agent environments and characteristics.
Problem solving through searching.
Performance measures in problem solving, search
techniques, heuristic search techniques.
Knowledge representation and inference.
Predicate logic, frames and semantic nets.
Planning.
Planners, search in planning.
Learning algorithms.
Concept learning, decision tree learning.
Dr. E.C. Kulasekere
573 Intelligent Systems
Course Information
Search Algorithms
Other Variations
Course Outline
Reading Material
Course Setup
Text Book and Other Reading
Artificial Intelligence: A modern approach/2e,
Stuart Russell and Peter Norvig
Pearson Education, Inc. 1995 ISBN 81-7808-554-2
Machine Learning,
Tom M. Mitchell
McGraw-Hill 1997 ISBN 0-07-115467-1
Essentials of Artificial Intelligence,
Matt Ginsberg
Morgan Kaufman Publishers, Inc. 1993 ISBN
1-55860-221-6
Dr. E.C. Kulasekere
573 Intelligent Systems
Course Information
Search Algorithms
Other Variations
Course Outline
Reading Material
Course Setup
Course Expectations
Some amount of independent study.
Occasional Class room discussion on certain topics.
Continuous assessment in the form of following
assignments
Assignments (20%)
Final examination (80%)
Writing an approximately 10 page report on a selected topic
in IS for extra credit.
Some amount of home work to supplement the course
material in the form of reading and tutorials.
Dr. E.C. Kulasekere
573 Intelligent Systems
Course Information
Search Algorithms
Other Variations
Introduction to Searching
Depth First Search
Breadth First Search
Problem Solving and Search
Search is a problem solving technique t attempts to find a
solution by looking at possible actions and systematically
exploring the space of problem states until a solution is
found.
The Promises:
Many problems can be cast as search problems.
Search potentially offers generality.
Only need to know the “rules of the game”, not how to play
well.
The Problems:
Search is a weak method (knowledge is poor)
Combinatoric explosion (The chess game its 10120 ).
The Solution:
Use domain specific knowledge.
Dr. E.C. Kulasekere
573 Intelligent Systems
Course Information
Search Algorithms
Other Variations
Introduction to Searching
Depth First Search
Breadth First Search
Problem Solving Agents
A problem solving agent decides what to do by finding
sequences of actions that lead to desirable states.
The first step is goal formulation. A goal is a set of world
states–the states in which the goal is satisfied. Goals help
organize behavior by limiting the objectives.
Actions result in transitions between world states.
Problem formulation is the process of deciding what
actions and states to consider. This often involves
abstracting the problem at the right level of granularity.
A search algorithm takes a problem and returns a
solution (an action sequence)
When a solution is found, the actions recommended can
be carried out, which is called the execution phase.
Dr. E.C. Kulasekere
573 Intelligent Systems
Course Information
Search Algorithms
Other Variations
Introduction to Searching
Depth First Search
Breadth First Search
Defining the Problem
1. Often we are not given an algorithm to solve a problem, but
only a specification of what is a solution – we have to
search for a solution.
2. Search is a way to implement don’t know nondeterminism.
3. We need to know how to convert a semantic problem of
finding logical consequence to a search problem of finding
derivations.
Dr. E.C. Kulasekere
573 Intelligent Systems
Course Information
Search Algorithms
Other Variations
Introduction to Searching
Depth First Search
Breadth First Search
Graph Searching
1. A graph consists of a set N of nodes and a set A of
ordered pairs of nodes, called arcs.
2. Node n2 is a neighbor of n1 if there is an arc from n1 to n2 .
That is, if < n1 , n2 > in A.
3. A path is a sequence of nodes < n0 , n1 , . . . , nk > such that
< ni−1 , ni > in A.
4. Given a set of start nodes and goal nodes, a solution is a
path from a start node to a goal node.
Dr. E.C. Kulasekere
573 Intelligent Systems
Course Information
Search Algorithms
Other Variations
Introduction to Searching
Depth First Search
Breadth First Search
Graph Searching Cont ...
Generic search algorithm: given a graph, start nodes, and
goal nodes, incrementally explore paths from the start
nodes.
Maintain a frontier of paths from the start node that have
been explored.
As search proceeds, the frontier expands into the
unexplored nodes until a goal node is encountered.
The way in which the frontier is expanded defines the
search strategy.
Dr. E.C. Kulasekere
573 Intelligent Systems
Course Information
Search Algorithms
Other Variations
Introduction to Searching
Depth First Search
Breadth First Search
Graph Searching Example
Dr. E.C. Kulasekere
573 Intelligent Systems
Course Information
Search Algorithms
Other Variations
Introduction to Searching
Depth First Search
Breadth First Search
Problem Spaces
Another method of looking at the search problem involves the
setting up a problem space as follows.
1. the states of the world that form the search space
2. the operators that can transform one state of the world into
another
3. the initial state (Root of the tree), and
4. the goal state (Leaf of the tree).
Dr. E.C. Kulasekere
573 Intelligent Systems
Course Information
Search Algorithms
Other Variations
Introduction to Searching
Depth First Search
Breadth First Search
Illustration of Problem Spaces
A Water Jug Problem
You are given two jugs, a 4-gallon one and a 3-gallon one.
Neither has any measuring markers on it. There is a pump that
can be used to fill the jugs with water. How can you get exactly
2 gallons of water into the 4-gallon jug?
State Space Definition
Space consists of a set of ordered pairs (x, y) such that
x = 0, 4 and y = 0, 3.
x-Number of gallons of water in 4-gallon jug; y-Number
gallons of water in 3-gallon jug.
Start state is (0, 0).
Goal state is (2, n) for any value of n.
Dr. E.C. Kulasekere
573 Intelligent Systems
Course Information
Search Algorithms
Other Variations
Introduction to Searching
Depth First Search
Breadth First Search
Setting up of Problem
Unstated assumptions:
One can fill a jug from the pump.
One can pour water out of a jug onto the ground.
One can pour water from one jug to another.
No other measuring devices are available.
Select rules or actions that will move one state to another.
The rules can also have redundant information. In which
case the search space will be unnecessarily large and
cycles may exist.
Dr. E.C. Kulasekere
573 Intelligent Systems
Course Information
Search Algorithms
Other Variations
Introduction to Searching
Depth First Search
Breadth First Search
Example Rules
Initial
(x, y) if x < 4
(x, y) if y < 3
(x, y) if x > 0
(x, y) if x > 0
(x, y) if (x + y) ≤ 3 and x > 0
Final
(4, y)
(x, 3)
(0, y)
(x − d, y)
(0, x + y)
Description
Fill the 4 gallon jug
Fill the 3 gallon jug
Empty the 4 gallon jug on the ground
Pour some out of the 4 gallon jug
pour from 4gal to 3gal
We can use the rules based on the following transitions.
Partial fillings are not considered since there are no
gradings on the jugs and hence only complete filling and
emptying makes sense.
The state of only one jug is changed at a time. That is
multiple jugs cannot be filled at a time.
We only consider the three states, filling one jug at a time
from the tap, filling one jug from the other, emptying one jug
at a time.
Dr. E.C. Kulasekere
573 Intelligent Systems
Course Information
Search Algorithms
Other Variations
Introduction to Searching
Depth First Search
Breadth First Search
Questions to be answered
Can redundant information be removed from the search
tree?
How do we determine if the search is optimal? Is the
number of steps an appropriate choice?
What is the difference between the following two solutions?
(0, 0) → (0, 3) → (3, 0) → (3, 3) → (4, 2) → (0, 2) → (2, 0).
(0, 0) → (4, 0) → (1, 3) → (1, 0) → (0, 1) → (4, 1) → (2, 3).
Draw the search space eliminating redundant information.
Dr. E.C. Kulasekere
573 Intelligent Systems
Course Information
Search Algorithms
Other Variations
Introduction to Searching
Depth First Search
Breadth First Search
Missionaries and Cannibals
Problem: On one bank of a river there are three
missionaries and three cannibals. There is one row-boat
available that can hold up to two people and that they
would like to use to cross the river. If the cannibals ever
outnumber the missionaries on either of the river’s banks
(including the occupants of the boat), the missionaries will
get eaten, How can the row-boat be used to safely carry all
of the missionaries and cannibals across the river?
Draw the search space for this problem.
Use the breath first search to solve it.
Use the depth first to solve this.
What is preferable and why?
Dr. E.C. Kulasekere
573 Intelligent Systems
Course Information
Search Algorithms
Other Variations
Introduction to Searching
Depth First Search
Breadth First Search
General Procedure to Solve a Search Problem
1. Set L to be a list of the initial nodes in the problem. At any
given point in time, L s a list of nodes that have not yet
been examined by the program.
2. If L is empty, fail. Otherwise pick a node n from L.
3. If n is a goal node, stop and return in and the path from the
initial node to n.
4. Otherwise, remove n from L and add to L all of n’s
children, labeling each with its path from e initial node.
Return to Step 2.
Dr. E.C. Kulasekere
573 Intelligent Systems
Course Information
Search Algorithms
Other Variations
Introduction to Searching
Depth First Search
Breadth First Search
Search Techniques Categories
Blind Search (uninformed search, Brute force search): e.g.
Depth-first search, breadth-first search algorithms etc. This
type uses no information other than the initial state, the
search operators, and a test for a solution. Time
consuming.
Heuristic (informed search): e.g. hill climbing, best-first
search etc. Uses domain specific knowledge to restrict the
search space. Suboptimal solution.
Dr. E.C. Kulasekere
573 Intelligent Systems
Course Information
Search Algorithms
Other Variations
Introduction to Searching
Depth First Search
Breadth First Search
Depth First Search
Depth-first search treats the frontier as a stack
It always selects one of the last elements added to the
frontier.
If the frontier is [p1 , p2 , . . .],
p1 is selected. Paths that extend p1 are added to the front of
the stack (in front of p2 ).
p2 is only selected when all paths from p1 have been
explored.
Dr. E.C. Kulasekere
573 Intelligent Systems
Course Information
Search Algorithms
Other Variations
Introduction to Searching
Depth First Search
Breadth First Search
Depth First Search Illustration
Dr. E.C. Kulasekere
573 Intelligent Systems
Course Information
Search Algorithms
Other Variations
Introduction to Searching
Depth First Search
Breadth First Search
Depth First Search Algorithm
1. Set L to be a list of the initial nodes in the problem.
2. Let n be the first node on L. If L is empty, fail.
3. If n is a goal node, stop and return it and the path from the
initial node to n.
4. Otherwise, remove n from L and add to front of L all of n’s
children, labeling each with its path from the initial node.
Return to step 2.
Dr. E.C. Kulasekere
573 Intelligent Systems
Course Information
Search Algorithms
Other Variations
Introduction to Searching
Depth First Search
Breadth First Search
Complexity of Depth First Search Algorithm
Depth-first search isn’t guaranteed to halt on infinite graphs
or on graphs with cycles.
The space complexity is linear in the size of the path being
explored.
Search is unconstrained by the goal until it happens to
stumble on the goal.
Dr. E.C. Kulasekere
573 Intelligent Systems
Course Information
Search Algorithms
Other Variations
Introduction to Searching
Depth First Search
Breadth First Search
Analysis of Depth-First Search
L is considered a stack.
Space requirement: Store b − 1 nodes at each depth (the
siblings of the nodes that have already been expanded)
together with the additional node at depth d = d(b-1)+1
Time requirement: If the goal is at the far left of the tree the
time requirement is d + 1; if its on the far right one has to
examine the entire tree with a total of
1 + . . . + bd =
bd+1 − 1
nodes in all.
b−1
Take the average to find the average number of nodes that
have to be visited.
Dr. E.C. Kulasekere
573 Intelligent Systems
Course Information
Search Algorithms
Other Variations
Introduction to Searching
Depth First Search
Breadth First Search
Advantages/Disadvantages of Depth-First Search
Less memory is required because nodes on the current
path are stored.
For a search with many goal locations this method is
efficient. Because one can find a solution without
searching the entire set of nodes.
However can get trapped in a blind alley. Searching a
single unfaithful path deeper.
Minimality of the solution is not guaranteed.
Dr. E.C. Kulasekere
573 Intelligent Systems
Course Information
Search Algorithms
Other Variations
Introduction to Searching
Depth First Search
Breadth First Search
Breadth-First Search
Breadth-first search treats the frontier as a queue.
It always selects one of the earliest elements added to the
frontier.
If the frontier is [p1 , p2 , . . . , pr ]:
p1 is selected. Its neighbors are added to the end of the
queue, after pr .
p2 is selected next.
Dr. E.C. Kulasekere
573 Intelligent Systems
Course Information
Search Algorithms
Other Variations
Introduction to Searching
Depth First Search
Breadth First Search
Breadth First Search Illustration
Dr. E.C. Kulasekere
573 Intelligent Systems
Course Information
Search Algorithms
Other Variations
Introduction to Searching
Depth First Search
Breadth First Search
Breadth First Search Algorithm
1. Set L to be a list of the initial nodes in the problem.
2. Let n be the first node on L. If L is empty, fail.
3. If n is a goal node, stop and return it and the path from the
initial node to n.
4. Otherwise, remove n from L and add to th end of L all of
n’s children, labeling each with its path from the initial
node. Return to step 2.
Dr. E.C. Kulasekere
573 Intelligent Systems
Course Information
Search Algorithms
Other Variations
Introduction to Searching
Depth First Search
Breadth First Search
Complexity of Breadth First Search Algorithm
The branching factor of a node is the number of its
neighbors.
If the branching factor for all nodes is finite, breadth-first
search is guaranteed to find a solution if one exists.
It is guaranteed to find the path with fewest arcs.
Time complexity is exponential in the path length: bn,
where b is branching factor (number of child branches), n is
path length.
The space complexity is exponential in path length: bn.
Search is unconstrained by the goal.
Dr. E.C. Kulasekere
573 Intelligent Systems
Course Information
Search Algorithms
Other Variations
Introduction to Searching
Depth First Search
Breadth First Search
Analysis of Breadth-First Search
L is considered a queue.
Space requirements: Since the total number of nodes at a
depth k is bk , all of these nodes need to be stored before
depth k + 1 is examined. Then the requirement is b d−1 .
Time requirement: A node is checked to see if it is a goal
node just before the node is expanded to generate
successors, and not when the node itself is added to the
list L, in oder to reach a goal at d the internal nodes that
need to be examined
1 + b + . . . + bd−1 =
Dr. E.C. Kulasekere
bd − 1
b−1
573 Intelligent Systems
Course Information
Search Algorithms
Other Variations
Introduction to Searching
Depth First Search
Breadth First Search
Advantages/Disadvantages of Breadth-First Search
Minimal solution is guaranteed.
Storage is high since all nodes at a level needs to be
stored.
All parts of a level have to be examined to goto the next
level. Hence it takes a long time to reach a goal.
Will not get trapped in a blind alley.
Dr. E.C. Kulasekere
573 Intelligent Systems
Course Information
Search Algorithms
Other Variations
Introduction to Searching
Depth First Search
Breadth First Search
Measuring Performance in Problem Solving
Algorithms
Criterion
Time
Space
Optimal?
Complete?
BreadthFirst
UniformCost
DepthFirst
bd
bd
Yes
Yes
bd
bd
Yes
Yes
bm
bm
No
No
DepthLimited
bl
bl
No
Yes, if l
Iterative
Deepening
Bidirectional
(if applicable)
bd
bd
Yes
Yes
bd/2
bd/2
Yes
Yes
d
Completeness : guaranteed to find a solution?
Time Complexity : How long it takes to find the solution?
Space Complexity : How much memory is required to perform
the search?
Optimality : does it find the highest-quality solution?
∗ We will now look at some variations to the basic search
methods we have looked into.
Dr. E.C. Kulasekere
573 Intelligent Systems
Course Information
Search Algorithms
Other Variations
Algorithm Explanations
Analysis of Algorithms
Uniform Cost Search
BFS finds the shallowest goal but may not be the least
cost. (depends on how cost is defined of course)
UCS modifies the BFS by always expanding the
lowest-cost node on the fringe rather than the lowest depth.
BFS is USC with g(n) = DEPTH(n).
The path with the cost value is stored in a queue before the
final decision is made.
We assume that the costs cannot be negative. If they are
positive then a optimal goal is found always.
Note the reason why C is not expanded because if it is
expanded the cost would be greater than the other two and
will be useless exercise. (shown in the next page)
Dr. E.C. Kulasekere
573 Intelligent Systems