CS414-Artificial Intelligence
Lecture 5: Basic Search Algorithms
Waheed Noor
Computer Science and Information Technology,
University of Balochistan,
Quetta, Pakistan
Waheed Noor (CS&IT, UoB, Quetta)
CS414-Artificial Intelligence
September-October 2014
1 / 48
Outline
1
Basic search algorithms
2
Infrastructure for Search Algorithms
3
Search Strategy
4
Uninformed Search Algorithms
Breadth-First Search
Uniform-cost search
Depth-first search
Depth-limited Search
Iterative Deepening Search
5
Informed Search
Waheed Noor (CS&IT, UoB, Quetta)
CS414-Artificial Intelligence
September-October 2014
2 / 48
Outline
1
Basic search algorithms
2
Infrastructure for Search Algorithms
3
Search Strategy
4
Uninformed Search Algorithms
Breadth-First Search
Uniform-cost search
Depth-first search
Depth-limited Search
Iterative Deepening Search
5
Informed Search
Waheed Noor (CS&IT, UoB, Quetta)
CS414-Artificial Intelligence
September-October 2014
3 / 48
Search Algorithms
Search plays important role in solving AI problems.
Normally, the result of a search algorithm is a solution in the form
of a set of actions that lead to the goal. (But not always)
For any problem, once the problem is properly formulated and
initial state is identified, search algorithm is applied to identify best
possible sequences of state-action to achieve the goal.
Waheed Noor (CS&IT, UoB, Quetta)
CS414-Artificial Intelligence
September-October 2014
4 / 48
Searching & Optimization
In almost every problem:
Initial situation and goal is given.
At any state in the state space, we are faced with the set of actions
Executing a particular sequence of such actions may or may not
achieve the goal.
Search is the process of identifying several such sequences and
choosing one that achieves the goal.
For some applications, each sequence of actions may be
associated with a certain cost.
A search problem that is not only to reach to a goal but also
achieving the minimal cost is an optimization problem.
Waheed Noor (CS&IT, UoB, Quetta)
CS414-Artificial Intelligence
September-October 2014
5 / 48
An Example
(a)
(b)
Figure : Source:http://www.ics.forth.gr/cvrl/
Waheed Noor (CS&IT, UoB, Quetta)
CS414-Artificial Intelligence
September-October 2014
6 / 48
Searching the State Space
The set of all possible sequences of legal moves form a tree:
The nodes of the tree are labeled with states (the same state
could label many different nodes).
The initial state is the root of the tree.
For each of the legal follow-up moves of a given state, any node
labeled with that state will have a child labeled with the follow-up
state.
Each branch corresponds to a sequence of states (and thereby
also a sequence of moves).
Waheed Noor (CS&IT, UoB, Quetta)
CS414-Artificial Intelligence
September-October 2014
7 / 48
State Vs. Node
A state is a (representation of) a physical configuration
A node is a data structure constituting part of a search tree
includes parent, children, depth, path cost g(x)
States do not have parents, children, depth, or path cost!
The E XPAND function creates new nodes, filling in the various fields
and using the S UCCESSOR F N of the problem to create the
corresponding states.
Waheed Noor (CS&IT, UoB, Quetta)
CS414-Artificial Intelligence
September-October 2014
8 / 48
Basic Search Algorithm: In General
Basic idea:
offline, simulated exploration of state space by generating successors
of already-explored states (i.e.,expanding states)
Algorithm 1: Basic Tree Search Algorithm
Input: problem, strategy
Output: solution or failure
initialize the search tree using the initial state of problem
while Expansion is possible do
choose a leaf node for expansion according to strategy
if the node contains a goal state then
Return: the corresponding solution
else
expand the node and add the resulting nodes to the search tree
end
end
Return failure
Waheed Noor (CS&IT, UoB, Quetta)
CS414-Artificial Intelligence
September-October 2014
9 / 48
Search Tree Example
Waheed Noor (CS&IT, UoB, Quetta)
CS414-Artificial Intelligence
September-October 2014
10 / 48
Outline
1
Basic search algorithms
2
Infrastructure for Search Algorithms
3
Search Strategy
4
Uninformed Search Algorithms
Breadth-First Search
Uniform-cost search
Depth-first search
Depth-limited Search
Iterative Deepening Search
5
Informed Search
Waheed Noor (CS&IT, UoB, Quetta)
CS414-Artificial Intelligence
September-October 2014
11 / 48
Infrastructure for Search Algorithm
For each node n of the tree, we have a structure that contains five
components:
n.STATE: the state in the state space to which the node
corresponds.
n.PARENT: the node in the search tree that generated this node.
n.ACTION: the action that was applied to the parent to generate the
node.
n.DEPTH: the depth of tho node n, i.e., number of nodes in the path
from the rood to this node n.
n.PATH-COST: the cost, traditionally denoted by g(n), of the path from
the initial state to the node, as indicated by the parent
pointers.
Waheed Noor (CS&IT, UoB, Quetta)
CS414-Artificial Intelligence
September-October 2014
12 / 48
Infrastructure for Search Algorithm
General search tree algorithm, shown above, also needs to know the
list of unexpanded nodes and among them which node to choose for
expansion.
Node selection for expansion depends on the search strategy that
we will see shortly.
The list of available nodes for expansion (unexpanded yet) is
maintained in the form a collection called fringe or frontier.
Generally, this collection of nodes is implemented as a queue.
Since, sometime it may be necessary for search strategy to check
each node in the list for possible expansion.
Waheed Noor (CS&IT, UoB, Quetta)
CS414-Artificial Intelligence
September-October 2014
13 / 48
Outline
1
Basic search algorithms
2
Infrastructure for Search Algorithms
3
Search Strategy
4
Uninformed Search Algorithms
Breadth-First Search
Uniform-cost search
Depth-first search
Depth-limited Search
Iterative Deepening Search
5
Informed Search
Waheed Noor (CS&IT, UoB, Quetta)
CS414-Artificial Intelligence
September-October 2014
14 / 48
Search Strategy
Definition
A strategy is defined by picking the order of node expansion
Strategies are evaluated based on:
completeness—does it always find a solution if one exists?
time complexity—number of nodes generated/expanded
space complexity—maximum number of nodes in memory
optimality—does it always find a least-cost solution?
Time and space complexity are measured in terms of
b—maximum branching factor of the search tree
d—depth of the least-cost solution
m—maximum depth of the state space (may be ∞)
Waheed Noor (CS&IT, UoB, Quetta)
CS414-Artificial Intelligence
September-October 2014
15 / 48
Types of Search Strategy
Definition (Uninformed Search Algorithms)
Algorithms that are given no information about the problem other than
its definition. Although some of these algorithms can solve any
solvable problem, none of them can do so efficiently.
Definition (Informed Search Algorithms)
Informed search algorithms, on the other hand, can do quite well given
some guidance on where to look for solutions. That is, it may have
access to some function that estimate the cost of the solution.
Waheed Noor (CS&IT, UoB, Quetta)
CS414-Artificial Intelligence
September-October 2014
16 / 48
Outline
1
Basic search algorithms
2
Infrastructure for Search Algorithms
3
Search Strategy
4
Uninformed Search Algorithms
Breadth-First Search
Uniform-cost search
Depth-first search
Depth-limited Search
Iterative Deepening Search
5
Informed Search
Waheed Noor (CS&IT, UoB, Quetta)
CS414-Artificial Intelligence
September-October 2014
17 / 48
Outline
1
Basic search algorithms
2
Infrastructure for Search Algorithms
3
Search Strategy
4
Uninformed Search Algorithms
Breadth-First Search
Uniform-cost search
Depth-first search
Depth-limited Search
Iterative Deepening Search
5
Informed Search
Waheed Noor (CS&IT, UoB, Quetta)
CS414-Artificial Intelligence
September-October 2014
18 / 48
Breadth-First Search
Definition
The shallowest unexpanded node is chosen for expansion, i.e., root
node is expanded first, then all the successors of the root node are
expanded next, then their successors, and so on.
They are implemented using FIFO data structures, the algorithm will
maintain a list of the currently active paths and in each round of the
algorithm repeat following three steps until solution is not found or no
further expansion is possible.
1
Remove the first path from the list of paths.
2
Generate a new path for every possible follow-up nodes.
3
Append the list of newly generated paths to the end of the list of
paths (to ensure paths are really being visited in breadth-first
order).
Waheed Noor (CS&IT, UoB, Quetta)
CS414-Artificial Intelligence
September-October 2014
19 / 48
Breadth-First Search: Example
Waheed Noor (CS&IT, UoB, Quetta)
(a)
(b)
(c)
(d)
CS414-Artificial Intelligence
September-October 2014
20 / 48
Breadth-First Search: Rating
Complete Yes (if b is finite)
Time 1 + b + b2 + b3 + . . . + bd + b(bd − 1) = O(bd+1 ), i.e.,
exponential in d
Space O(bd+1 ) (keeps every node in memory)
Optimal Yes (if cost = 1 per step); not optimal in general
Space is the big problem; can easily generate nodes at 100MB/sec
so 24hrs = 8640GB.
Waheed Noor (CS&IT, UoB, Quetta)
CS414-Artificial Intelligence
September-October 2014
21 / 48
Outline
1
Basic search algorithms
2
Infrastructure for Search Algorithms
3
Search Strategy
4
Uninformed Search Algorithms
Breadth-First Search
Uniform-cost search
Depth-first search
Depth-limited Search
Iterative Deepening Search
5
Informed Search
Waheed Noor (CS&IT, UoB, Quetta)
CS414-Artificial Intelligence
September-October 2014
22 / 48
Uniform-cost search
Definition
Unlike BFS, instead of expanding shallowest node, uniform-cost
Search (UCS) expands the node with lowest path cost.
Explanation
UCS finds the least-cost path through a graph by maintaining an
ordered list of nodes in order of least-greatest cost(Priority Queue).
This allows us to evaluate the least cost path first.
Waheed Noor (CS&IT, UoB, Quetta)
CS414-Artificial Intelligence
September-October 2014
23 / 48
How UCS Works
The algorithm uses the accumulated path cost and a priority
queue to determine the path to evaluate.
The priority queue (least cost) contains the nodes to be evaluated.
As node children are evaluated, we add their cost to the node with
the aggregate sum of the current path.
This node is then added to the queue, and when all children have
been evaluated, the queue is sorted in order of ascending cost.
When the first element in the priority queue is the goal node, then
the best solution has been found.
Waheed Noor (CS&IT, UoB, Quetta)
CS414-Artificial Intelligence
September-October 2014
24 / 48
Example
Figure : Graph of an example problem with respective tree.
Waheed Noor (CS&IT, UoB, Quetta)
CS414-Artificial Intelligence
September-October 2014
25 / 48
Solution
Step0:
Step1:
Step2:
Step3:
Step4:
Step5:
Step6
Step-by-Step Execution of UCS
{[S,0]}
{ [ S->A , 1 ] , [ S->G , 12 ] }
{ [ S->A->C , 2 ] , [ S->A->B , 4 ] , [ S->G , 12] }
{ [ S->A->C->D , 3 ] , [ S->A->B , 4 ] ,
[ S->A->C->G , 4 ] , [ S->G , 12 ] }
{ [ S->A->B , 4 ] , [ S->A->C->G , 4 ] ,
[ S->A->C->D->G , 6 ] , [ S->G , 12 ] }
{ [ S->A->C->G , 4 ] , [ S->A->C->D->G , 6 ] ,
[ S->A->B->D , 7 ] , [ S->G , 12 ] }
Output as S->A->C->G.
Waheed Noor (CS&IT, UoB, Quetta)
CS414-Artificial Intelligence
September-October 2014
26 / 48
UCS: Rating
Complete:
Time:
Space:
Optimal:
Yes (if b is finite)
O(bd )
O(bd )
Yes, if cost on the edges are non-negative.
Good things about UCS
Can set to prune (remove) duplicate entries in the queue.
Can set to search exhaustively for optimal solution, if possible,
among different lest-cost candidate paths to the goal.
Waheed Noor (CS&IT, UoB, Quetta)
CS414-Artificial Intelligence
September-October 2014
27 / 48
Class Activity
Class Activity
Draw expected graph based on BFS and UCS algorithms and output
possible resulting optimal path.
Waheed Noor (CS&IT, UoB, Quetta)
CS414-Artificial Intelligence
September-October 2014
28 / 48
Quiz
Waheed Noor (CS&IT, UoB, Quetta)
CS414-Artificial Intelligence
September-October 2014
29 / 48
Outline
1
Basic search algorithms
2
Infrastructure for Search Algorithms
3
Search Strategy
4
Uninformed Search Algorithms
Breadth-First Search
Uniform-cost search
Depth-first search
Depth-limited Search
Iterative Deepening Search
5
Informed Search
Waheed Noor (CS&IT, UoB, Quetta)
CS414-Artificial Intelligence
September-October 2014
30 / 48
Depth-First Search
Definition (DFS)
A graph searching algorithm that begins at the root node, and
exhaustively searches each branch to its greatest depth before
backtracking to previously unexplored branches.
Explanation
– Nodes found but yet to be reviewed are stored in a LIFO queue
(stack).
– Exhaustive means DFS can search every node in the graph to find
the goal. If the goal is not present in the graph, the algorithm will
terminate, but will search each and every node in a systematic way.
– Therefore, the DFS need to store only single path from node to leaf
along with unexpanded sibling at each level of depth. – Each node
explored is put on the front of frontier.
Waheed Noor (CS&IT, UoB, Quetta)
CS414-Artificial Intelligence
September-October 2014
31 / 48
Progress of DFS
A
A
B
C
D
H
E
J
I
F
L
K
C
D
G
N
M
O
H
E
J
I
H
J
G
N
M
L
H
D
O
E
I
H
J
G
N
M
L
J
H
O
H
I
J
J
H
G
N
M
L
I
J
L
Waheed Noor (CS&IT, UoB, Quetta)
G
M
N
O
H
J
O
H
E
I
J
F
N
M
L
K
G
B
F
K
O
O
A
E
I
N
C
D
C
D
G
M
L
K
A
B
F
K
O
F
B
F
K
O
A
E
C
E
I
N
E
C
D
G
N
M
C
D
G
M
L
K
A
B
D
F
L
K
B
F
B
F
K
E
J
I
A
C
E
I
H
C
A
B
D
D
O
A
B
F
K
G
N
M
C
A
C
E
I
B
F
L
K
A
B
D
A
B
L
G
M
N
C
D
O
CS414-Artificial Intelligence
H
E
I
J
F
K
L
G
M
N
O
September-October 2014
32 / 48
Variants of DFS
1
2
3
4
Graph Version Vs. Tree Version
Graph version avoids redun- Can not avoid loops, however
dant paths and it is complete modification can be made to
in finite state spaces, since it avoid loops. General, both
will eventually expand every are incomplete when depth
node.
spaces are infinite.
Non-Optimal
Non-Optimal
Time complexity is bounded O(bm ), where m is the maxiby the size of the state space: mum depth of any node: TerNot good if state space is infi- rible.
nite
Space complexity is not better Linear O(bm), since nodes
than BFS
from root to leaf along the,
and unexplored siblings are
stored at a time
Waheed Noor (CS&IT, UoB, Quetta)
CS414-Artificial Intelligence
September-October 2014
33 / 48
Outline
1
Basic search algorithms
2
Infrastructure for Search Algorithms
3
Search Strategy
4
Uninformed Search Algorithms
Breadth-First Search
Uniform-cost search
Depth-first search
Depth-limited Search
Iterative Deepening Search
5
Informed Search
Waheed Noor (CS&IT, UoB, Quetta)
CS414-Artificial Intelligence
September-October 2014
34 / 48
Depth-limited Search
Definition
Depth-limited search (DLS) is the variant of DFS with pre-specified
depth limit. That is if depth limit is denoted by ` then no successor are
added further to the node at the depth `. The DLS overcomes the
failure of DFS in infinite space and solve the infinite path problem.
If ` is not properly chosen in the case when d is unknown, ` < d,
then DLS is incomplete that is if the goal is beyond the depth limit.
The depth limit also reduces the scope of the search.
It is also suboptimal, since the algorithm may find first path to the
goal instead of shortest path.
Waheed Noor (CS&IT, UoB, Quetta)
CS414-Artificial Intelligence
September-October 2014
35 / 48
DLS Continue
The depth limit is often based on the domain knowledge of the
problem.
Example
In the map of Romania, we can see that there are 20 cities, therefore
possibly ` = 19 since if solution exists that would reach at the depth of
19. Or if we look at the map carefully every city from any city can be
reach in 9 steps that would be more precise depth limit. In such case it
is called diameter of the state space.
DLS Rating
Complete:
Time Complexity:
Space Complexity:
Optimal:
Waheed Noor (CS&IT, UoB, Quetta)
No (Generally, if ` < d)
O(b` )
O(b`)
No.
CS414-Artificial Intelligence
September-October 2014
36 / 48
An Example
Figure : DLS example with different depth limit
Home Work
– Given the map of Romania, what will be the minimum depth limit `
and generate graphs using DLS based on your reported ` for:
– From Arad to Busharest.
– From Iasi to Zerind.
Waheed Noor (CS&IT, UoB, Quetta)
CS414-Artificial Intelligence
September-October 2014
37 / 48
Outline
1
Basic search algorithms
2
Infrastructure for Search Algorithms
3
Search Strategy
4
Uninformed Search Algorithms
Breadth-First Search
Uniform-cost search
Depth-first search
Depth-limited Search
Iterative Deepening Search
5
Informed Search
Waheed Noor (CS&IT, UoB, Quetta)
CS414-Artificial Intelligence
September-October 2014
38 / 48
Iterative Deepening Search
Reading
Read it by your self and solve the above problem for IDS as well.
You are also supposed to read Bidirectional Search.
Waheed Noor (CS&IT, UoB, Quetta)
CS414-Artificial Intelligence
September-October 2014
39 / 48
Outline
1
Basic search algorithms
2
Infrastructure for Search Algorithms
3
Search Strategy
4
Uninformed Search Algorithms
Breadth-First Search
Uniform-cost search
Depth-first search
Depth-limited Search
Iterative Deepening Search
5
Informed Search
Waheed Noor (CS&IT, UoB, Quetta)
CS414-Artificial Intelligence
September-October 2014
40 / 48
Informed Search
What we have seen so far are all those search algorithms that
operate with out any domain knowledge, and try to solve the
problem in brute-force fashion.
Such methods are insufficient for most problems specially when
the problem is large and complex.
Now we will look into another type of search techniques called
informed search.
Definition
Informed Search Incorporates a heuristic in the search that determines
the quality of any state in the search space. In a graph search, this
results in a strategy for node expansion. The heuristic may consider
the problem knowledge to guide the search strategy.
Waheed Noor (CS&IT, UoB, Quetta)
CS414-Artificial Intelligence
September-October 2014
41 / 48
Best First Search I
In Best-FS state space is evaluated on the basis of an evaluation
function f (n) that incorporates a heuristic function h(n) and
estimated cost g(n) from start node to n, i.e., f (n) = h(n).
This search strategy is also called A* Search.
The h(n) can be an educated guess such that the cheapest path
from the state at node n to the goal.
In Romania example, this may be a straight line between state at
node n to goal.
The g(n) is the cost from the start node to node n.
The algorithm is often implemented by maintaining two list, an
open one and a closed one.
Waheed Noor (CS&IT, UoB, Quetta)
CS414-Artificial Intelligence
September-October 2014
42 / 48
Best First Search II
Open list is a priority queue consist nodes yet to be visited sorted
by their evaluation function.
Closed list contains nodes that have already been evaluated.
Best-FS specializes to BFS when f (n) = h(n) and to UFS when
f (n) = g(n).
Waheed Noor (CS&IT, UoB, Quetta)
CS414-Artificial Intelligence
September-October 2014
43 / 48
An Example
Consider again the Romania example
71
75
Oradea
Neamt
Zerind
87
151
Iasi
Arad
140
92
Sibiu
99
Fagaras
118
Vaslui
80
Rimnicu Vilcea
Timisoara
111
Pitesti
97
Lugoj
142
211
70
98
Mehadia
75
Dobreta
146
85
101
138
120
Hirsova
Urziceni
86
Bucharest
90
Craiova
Giurgiu
Eforie
Straight−line distance
to Bucharest
Arad
366
Bucharest
0
Craiova
160
Dobreta
242
Eforie
161
Fagaras
178
Giurgiu
77
Hirsova
151
Iasi
226
Lugoj
244
Mehadia
241
Neamt
234
Oradea
380
Pitesti
98
Rimnicu Vilcea 193
Sibiu
253
Timisoara
329
Urziceni
80
Vaslui
199
Zerind
374
Figure : Romania map with SLD
Waheed Noor (CS&IT, UoB, Quetta)
CS414-Artificial Intelligence
September-October 2014
44 / 48
Greedy Fashion
(a) The initial state
Arad
(b) After expanding Arad
Arad
366
Sibiu
Timisoara
Zerind
253
329
374
Timisoara
Zerind
329
374
Timisoara
Zerind
329
374
(c) After expanding Sibiu
Arad
Sibiu
Arad
366
Fagaras
176
Oradea
380
Rimnicu Vilcea
193
(d) After expanding Fagaras
Arad
Sibiu
Arad
366
Fagaras
Oradea
380
Sibiu
Bucharest
253
0
Rimnicu Vilcea
193
Figure : In the greedy fashion
Waheed Noor (CS&IT, UoB, Quetta)
CS414-Artificial Intelligence
September-October 2014
45 / 48
A* Search Example
(a) The initial state
Arad
366=0+366
(b) After expanding Arad
Arad
Sibiu
Timisoara
Zerind
393=140+253
447=118+329
449=75+374
(c) After expanding Sibiu
Arad
Sibiu
Arad
Fagaras
Oradea
Timisoara
Zerind
447=118+329
449=75+374
Rimnicu Vilcea
646=280+366 415=239+176 671=291+380 413=220+193
(d) After expanding Rimnicu Vilcea
Arad
Sibiu
Arad
Fagaras
Oradea
Timisoara
Zerind
447=118+329
449=75+374
Rimnicu Vilcea
646=280+366 415=239+176 671=291+380
Craiova
Pitesti
Sibiu
526=366+160 417=317+100 553=300+253
(e) After expanding Fagaras
Arad
Sibiu
Arad
Fagaras
Oradea
Sibiu
Bucharest
591=338+253
450=450+0
Craiova
Pitesti
Fagaras
646=280+366
Sibiu
Arad
Sibiu
Timisoara
Zerind
447=118+329
449=75+374
Rimnicu Vilcea
Oradea
671=291+380
Sibiu
Bucharest
Craiova
591=338+253
450=450+0
526=366+160
Bucharest
418=418+0
Waheed Noor (CS&IT, UoB, Quetta)
Zerind
449=75+374
526=366+160 417=317+100 553=300+253
(f) After expanding Pitesti
Arad
Timisoara
447=118+329
Rimnicu Vilcea
671=291+380
646=280+366
Pitesti
Sibiu
553=300+253
Craiova
Rimnicu Vilcea
615=455+160 607=414+193
Intelligence
Figure :CS414-Artificial
The A* implementation
September-October 2014
46 / 48
A* Search
A* Search Rating
Complete:
Time Complexity:
Space Complexity:
Optimal:
Yes
O(bd )
O(bd )
Yes.
A* search algorithm have two flavors, the graph based and tree based.
Conditions for Optimality: Admissibility and Consistency
Admissibility: That the heuristic is admissible i.e., that h(n) will never
overestimate the cost to reach the goal. For A*, h(n) is admissible,
since straight line is the shortest path between any two points.
Consistent: Consistency (also called monotonicity) describes that the
cost along the path is increasing or decreasing. For A* graph version,
the g(n) is the increasing function of n along the path..
Waheed Noor (CS&IT, UoB, Quetta)
CS414-Artificial Intelligence
September-October 2014
47 / 48
References I
Waheed Noor (CS&IT, UoB, Quetta)
CS414-Artificial Intelligence
September-October 2014
48 / 48