Notes for CS3310
Artificial Intelligence
Part 8: Heuristic search
Prof. Neil C. Rowe
Naval Postgraduate School
Version of January 2006
Search
• Search = reasoning about "states".
• A state is a set of facts true at some instant of time.
That is, a state is a snapshot of facts. (Or to save
space, just the relevant facts in some problem.)
• Search and states permit reasoning about time.
• In artificial intelligence, states are discrete, and
transitions between states ("branches") are discrete
(all at once) too.
• Search among states permits hypothetical reasoning
about effects of actions. Each possible action can
result in a different state.
• A state can be represented by a linked list. Possible
state transitions can be represented by a directed
graph, a "search graph".
• Search is the oldest area of artificial intelligence.
More search terms
• State: a set of facts (whose predicate names are usually past
participles, e.g. "opened", and abstract nouns, e.g. "location").
• Successor state: an immediate next state.
• Operator: a label on a state transition (usually a verb, e.g.
"open")
• Starting state: the first state of a search.
• Goal states: states in which objectives have been achieved, so
search can stop.
• Level: the number of state transitions from the starting state to
a given state.
• Preconditions of an operator: facts that must occur in a state
before you can use the operator to it.
• Postconditions of an operator: facts that become true after you
use the operator from a state.
Search examples
• Intercity route planning: given a road map, find the best route
between two cities for a car.
• City route planning: Given intersections A and B, find a route
between them for a car.
• Car repair: Given part to repair, access it, fix it, and put the car
back together.
• Smarter forward chaining: Forward chaining but making an
intelligent decision at each step about which fact to pursue
next. (The goal is to infer interesting facts sooner and save
time.)
• Scheduling classes: At a university, assign classes to times so
every student can take the classes they want.
• Mission planning: Given a model of targets and costs of
attacking them, and a model of enemy responses, figure the
best sequence of attacking actions.
Search example
I-80 (20)
[location(Oakland)] {15}
[location(San Francisco)] {0}
I-280 (50)
Cal-17 (50)
Cal-17
Cal-1
[location(San Jose)] {45}
(30)
US-101 (35)
(85)
[location(Santa Cruz)] {65}
[location(Gilroy)] {74}
Cal-1 (45)
[location(Monterey)] {100}
US-101 (35)
The state is in brackets; air-mileage to San Francisco in curly
brackets; mileages in parentheses; road names are the operators.
Prefer westmost branch. Compute order of visit and solution
path: Use abbreviations: M=Monterey, SC=Santa Cruz,
G=Gilroy, SJ=San Jose, SF=San Francisco, Oak=Oakland.
Work page for comparison of search algorithms
Search States Agenda
method visited
Depthfirst
Breadthfirst
Best-first
A*
Solut
ion
A search graph for car repair
[dead(battery), in(battery,car), bolted(battery), cabled(battery)]
remove bolts
remove cables
[dead(battery), in(battery,car), cabled(battery)]
remove
cables
[dead(battery), in(battery,car), bolted(battery)]
[dead(battery), in(battery,car)]
remove bolts
Complete the search graph to the goal state [ok(battery),
in(battery,car), cabled(battery), bolted(battery)]. Possible
operators: "attach cables", "attach bolts", and "replace battery".
Note: not(cabled(battery)) is a precondition of the action "remove
battery"; ok(battery) is a postcondition of the action "replace
battery"; facts not mentioned are assumed false in a state.
Work page for car battery problem
Classic search methods
• These differ in the order in which states are "visited". "Visit”
means "find the successors of."
• Depth-first: visit an unvisited successor of current state, else
back up and try a new successor of previous state.
• Breadth-first: visit next-found unvisited state at same level,
else the first-found state at next level.
• Hill climbing: visit the unvisited successor of current state
which has lowest evaluation-function value, else back up.
• Best-first: visit the known unvisited state with lowest
evaluation-function value, which can be anywhere in search
graph.
• Branch-and-bound: like best-first but use cost from start to
state.
• A*: like best-first but use sum of evaluation-function value
and cost from start to state.
Notes on the search methods
• Search stops when you visit a goal state (that is, when you try to
find its successors).
• Heuristics can give criteria for choosing the next state with all the
search methods (and break ties in best-first and A*).
• Never visit the same state twice (or add a duplicate state to the
agenda) -- but for A*, consider new routes to the same state.
• Depth-first can be implemented with a stack; breadth-first with a
queue; and the other methods with a sorted "agenda" of states
known but not yet visited.
• When you find successors of a state, set the "backpointer" for each
successor to point to the state. When done, follow backpointers
from the goal state to the start state to get the solution path.
• Agendas should store for each state its description (list of facts
true for it), its backpointer, and any numbers associated with the
state.
• With A* search, if the evaluation function is a lower bound on the
subsequent cost, the first path found to a state is guaranteed to be
the best path to that state.
Heuristics
= Any piece of nonnumeric advice for choosing among
possible branches in a search.
Heuristics are a form of meta-rules.
Examples:
For city route planning, prefer not to turn twice in
succession.
For car repair, do work from top of the engine before
work from below the engine.
For buying decisions, don't buy anything advertised on
television.
In mission planning, withdraw from an engagement if
you do not have at least 2-to-1 superiority in assets.
Search terms that are easy to confuse
• evaluation function: a "distance-to-goal" measure, a
number. A function of the state. Makes search more
intelligent, if you pick direction to search with lowest
evaluation function.
• heuristic: a non-numeric way for choosing a direction
for search at a state.
• cost function: a "cost-accumulated" measure, a
number. A function of the states in a path.
Note: Evaluation function concerns future; cost
function concerns past.
An example route-planning program, not so good
Microsoft Automap Road Atlas Route Planning Demo
The quickest route from Monterey, California, to San Francisco, California (going the speed limit), is as follows:
Total Time: 2 hours 31 minutes
Total Distance: 121 miles
Time
Road
For
Dir
Towards
00:00
DEPART Monterey (California)
2 miles
E
on the Fremont St
00:04
Go onto S1
17 miles
E
Santa Cruz
00:23
At Castroville stay on the S1
14 miles
N
Santa Cruz
00:39
At Freedom stay on the S1
15 miles
NW
Santa Cruz
00:55
At Santa Cruz turn S9
23 miles
N
Saratoga right onto
01:23
Turn left onto S35
14 miles
N
Woodside
01:40
Turn right onto S84
5 miles
E
Redwood City
01:46
At Woodside stay on the S84
1/2 mile
NE
Redwood City
01:47
Turn left onto I280
16 miles
W
San Mateo
02:07
At Burlingame stay on the I280
1 mile
N
Brisbane
02:09
Turn off onto I380
1 mile
E
San Bruno
02:10
At San Bruno stay on the I380
1 mile
E
02:12
Turn off onto U101
3 miles
N
South San Francisco
02:17
At South San Francisco U101
3 miles
NE
Brisbane stay on the
02:22
At Brisbane stay on the U101
3 miles
N
02:26
Turn off onto I80
3 miles
E
San Francisco
02:31
ARRIVE San Francisco(California)
If you are impressed with this demo, you'll be wowed by our full product, Microsoft Automap Road Atlas 4.0.
We invite you to join our mailing list, read more information about Automap Road Atlas, or give us
feedback on this demo.
More about evaluation functions
What if states aren’t locations? Then add "amount-of-worknecessary" numbers for certain facts to get the evaluation-function
value. For instance for the car repair, start at 0 and then:
Add 10 if a dead(battery) fact is in the state;
Add 5 if no bolted(battery) fact is in the state;
Add 3 if no cabled(battery) fact is in the state.
So for instance:
[dead(battery), in(battery,car), bolted(battery), cabled(battery)] has
evaluation 10
[dead(battery), in(battery,car), cabled(battery)] has evaluation 15
[dead(battery), in(battery,car)] has evaluation 18
[ok(battery), in(battery,car)] has evaluation 8
[ok(battery), in(battery(car), cabled(battery)] has evaluation 5
[ok(battery), in(battery(car), bolted(battery), cabled(battery)] has 0
Problems with evaluation functions
Evaluation functions don't always work well. Some
classic problems:
– "pond" problem: you may find a local (instead of
global) minimum
– "valley" problem: most directions may be worse,
and it may be hard to find the few directions that
are better
– "plateau" problem: you may have lots of ties in the
evaluation function value
– "pit" problem: evaluation function may not be
useful unless you're extremely close to the goal
In these case a set of heuristics may be better.
Another kind of cost function: The probability
of an interpretation of a situation
Overall probability can be product of near-independent factors.
So to find the best interpretation, minimize its logarithm:
n
 log( p1 p2 p3 ...)   log( pi )
i 1
Do this by an A* search where branches cost:  log( pi )
This is important in language understanding. For instance,
interpret "navy fighter wing" with probabilities:
of each word sense of "navy” ; of each word sense of "fighter”;
of each word sense of "wing”; that "navy" is a kind of "fighter”;
that "navy" is part of "fighter”; that "fighter" is part of "navy”;
that "navy" is owner of "fighter”;
that "fighter" is a kind of "wing”; that "wing" is a part of "fighter”
that "wing" is a kind of "fighter”.
In a goal state each word has a sense and all senses are linked by
relationships.
Stochastic state graph for rootkit-installation example
167
ping
180
0.8
22
research
vulnerabilities
scan
0
2
ping
33
3
65
11
0.2
100
10
75
close ftp
26
62 25
decompress
secureport
0.17
9
100
0.6
download rootkit
test
rootkit
14
36
0.1
test
rootkit
15
download
secureport
25
0.1
test
rootkit
16
close ftp
24
0.83
24
20
ftp
8
download
secureport
85
86
Numbers in boxes are
state numbers.
Integers outside boxes are
duration from goal state.
Decimals are transition
probabilities in 10 runs.
18
install
secureport
7
test
rootkit
test
rootkit
17
decompress
secureport
login
74
0.4
download rootkit
become
admin
35
0.8 decompress
51
0.8
74
55
rootkit
decompress rootkit
122 48
0.2 29
32
50
0.8
27
30
decompress
secureport
0.2
55
0.25 0.75
install rootkit
install rootkit
test rootkit
0.9
0.67
0.9
0.17
34
13
22
23
28
download 21
close ftp
install
decompress
secureport
11
secureport
39
36
secureport 30
40
0.33 47
ftp
96
0.1
close
port
0.83
download secureport
61
70
install
rootkit
download
secureport
97
6
99
101
decompress
rootkit
31
close
port
5
4
107
close ftp
ftp
become
admin
0.9
1
research 0.2
vulnerabilities
overflow
buffer
open port
1
19
logout
0
Definitions needed to solve a search
problem
(1) successor: Defines all possible state transitions
(with successor(State,Newstate) in Prolog; in Java,
successor(State) which returns the array Newstates).
(2) goalreached: Defines all possible goal states.
(3) eval (for best-first and A*): Returns the evaluation
function value for any state.
(4) piece_cost (for A*): Returns the cost between any
pair of states.
(5) To start search, call depthsearch, breadthsearch,
bestsearch, or astarsearch with argument the
starting state; it should return the solution path.
Example Prolog definition of a search
problem: finding a route to San Francisco
successor(R1,R2) :- successor2(R1,R2).
successor(R1,R2) :- successor2(R2,R1).
successor2(monterey,gilroy).
successor2(monterey,santa_cruz).
successor2(gilroy,san_jose).
successor2(santa_cruz,san_jose).
successor2(santa_cruz,san_francisco).
successor2(san_jose,oakland).
successor2(san_jose,san_francisco).
successor2(oakland,san_francisco).
goalreached(san_francisco).
eval(monterey,100).
eval(gilroy,74).
eval(santa_cruz,65).
eval(san_jose,45).
eval(oakland,15).
eval(san_francisco,0).
Example Prolog search definition, cont.
/* This totals up the pair_cost2 numbers along a path */
piece_cost(X,Y,C) :- piece_cost2(X,Y,C).
piece_cost(X,Y,C) :- piece_cost2(Y,X,C).
piece_cost2(monterey,gilroy,35).
piece_cost2(monterey,santa_cruz,45).
piece_cost2(gilroy,san_jose,35).
piece_cost2(santa_cruz,san_jose,30).
piece_cost2(santa_cruz,san_francisco,85).
piece_cost2(san_jose,oakland,50).
piece_cost2(san_jose,san_francisco,50).
piece_cost2(oakland,san_francisco,20).
Then give the starting state, and search returns to sequence of
states to the goal, like: ?- depthsearch(monterey,A).
A=[san_francisco,san_jose,gilroy,monterey]
Another example: the robot housekeeper problem
Define behavior of a housekeeping robot.
1. In the starting state:
– The robot is at the trash chute (use predicate at);
– Offices 1 and 2 need dusting (use dusty);
– Office 1 has trash in its trash basket, but Office 2 doesn't
(use trashy);
– The carpet in Office 1 does not need vacuuming, while the
carpet in Office 2 does (use vacuumable).
Write the starting state using predicate expressions.
2. A goal state is one in which all these are true: Every office is
dusted; every carpet is vacuumed; every trash basket is
emptied; the robot is not holding any basket (use
holdingbasketfrom); and the robot is at office 1. Give the
conditions defining a goal state in logical terms.
The cost function for the robot housekeeper
3. Assume these energy costs:
– 10 units to vacuum a single room (the robot has a built-in
vacuum);
– 6 units to dust a room (the robot has a built-in duster);
– 3 units to pick up a trash basket (the robot can hold several
at once);
– 1 unit to put down a trash basket;
– 3 units to travel between offices;
– 8 units to travel between an office and the trash chute;
– 5 units to dispose of the contents of a trash basket down the
trash chute.
Assume the last action taken by the robot is the argument to a
lastact fact in each state. Define a piece_cost function.
The eval function for the robot housekeeper
4. For A* search, we prefer an evaluation function that is a lower
bound on the subsequent cost. So we'll add up numbers
("piece_eval"s) for each expression in the state.
The default number will be zero, but "dusty(X)" for instance will
have a piece_eval of 6. Compute the other piece_evals.
Successor definition for the robot housekeeper
5. The successor definition must follow these requirements:
a. The robot can vacuum the floors, dust, and empty the trash
baskets.
b. Vacuuming the floor generates dust that goes into the trash basket
of the room it is in.
c. Emptying the trash baskets in each room requires a trip to the
trash chute.
d. It doesn't make sense to vacuum or dust if the room isn't
vacuumable or dusty respectively.
e. Dusting puts dust on the floor, requiring vacuuming.
f. It doesn't make sense to vacuum if a room is dusty.
g. It doesn't make sense to pick up a trash basket if a room is
vacuumable or dusty.
h. A lastact fact in each state should hold what the last action was.
Write "successor" definition for the
"putdown", "dust", and "go" actions
Example: For "dispose(Basket)” (meaning to empty the trash
basket whose name is Basket). Preconditions: you are at the
chute, have a basket, and the basket contains trash.
Postconditions: Delete the fact the basket has trash, delete the
old “lastact” fact, and add a new “lastact” fact with argument
“dispose(Basket)”.
Successor definitions in Prolog
successor(S,NS):-member(at(chute),S),
member(fullbasket(X),S),
member(holdingbasketfrom(X),S),
delete(S, fullbasket(X),S2),
delete(S2, lastact(A),S3),
NS=[lastact(dispose(X)) | S3].
successor(S,NS) :- member(at(X),S),
\+ member(dusty(X),S),
member(vacuumable(X),S),
delete(S, vacuumable(X), S2),
delete(S2, fullbasket(X), S3),
delete(S3, lastact(A), S4),
NS=[lastact(vacuum(X)), fullbasket(X) | S4].
successor(S,NS) :- member(at(X),S),
\+ member(dusty(X),S),
\+ member(holdingbasketfrom(X),S),
\+ member(vacuumable(X),S),
member(fullbasket(X),S),
delete(S, lastact(A), S2),
NS=[lastact(pickup(X)), holdingbasketfrom(X) |S2].
successor(S,NS) :- member(at(X),S),
member(dusty(X),S),
delete(S, dusty(X), S2), delete(S2,
vacuumable(X), S3),
delete(S3, lastact(A), S4),
NS=[lastact(dust(X)), vacuumable(X) | S4].
successor(S,NS) :- member(at(X),S),
member(holdingbasketfrom(X),S),
delete(S, holdingbasketfrom(X), S2),
delete(S2, lastact(A), S3),
NS=[lastact(putdown(X)) | S3].
successor(S,NS) :- member(at(X),S),
delete(S, at(X), S2), places(PL),
member(Y,PL),\+ X=Y,
delete(S2, lastact(A), S3),
NS=[lastact(go(X,Y)), at(Y) | S3].
Eval function in Prolog
eval([],0).
eval([X|L],N) :- piece_eval(X,N2), eval(L,N3),
N is N2+N3.
piece_eval(dusty(X),6).
piece_eval(vacuumable(X),10).
piece_eval(fullbasket(X),8).
piece_eval(holdingbasketfrom(X),1).
piece_eval(P,0)
Cost function in Prolog
cost([_],0).
cost([S1,S2|L],C) :- S1=[FS1|_], S2=[FS2|_],
piece_cost(FS1,FS2,C2), cost([S2|L],C3), C is C2+C3, !.
piece_cost(lastact(vacuum(X)),_,10).
piece_cost(lastact(dust(X)),_,6).
piece_cost(lastact(pickup(X)),_,3).
piece_cost(lastact(dispose(X)),_,5).
piece_cost(lastact(putdown(X)),_,1).
piece_cost(lastact(go(chute,Y)),_,8).
piece_cost(lastact(go(X,chute)),_,8).
piece_cost(lastact(go(X,Y)),_,3).
piece_cost(lastact(none),_,0).
Statistics on the housekeeping problem
These programs all eliminate states whose fact lists are
permutations of another. "Cells" means the number of cons
cells created. Breadth-first search ran extremely slowly. All
runs were on SunOS machine ai9 in 1990.
depth-first
search
Allegro
Lisp,
interpreted
433 msec.
Allegro
Lisp,
compiled
33 msec.
Quintus
Prolog,
semicompiled
267 msec.
breadthfirst search
33144 msec. 3224 msec. > 1 hour
best-first
search
1450 msec.
83 msec.
1150 msec.
A* search
2000 msec.
117 msec.
9250 msec.
The depth-first search algorithm
• Initialize S to current state; initialize stack to hold
starting state.
• Until S is a goal state:
– Put (“push”) S onto the stack;
– Find the successors of S;
– Set S to its best successor (as per heuristics) that is
not in the stack;
– If no such successor, remove (“pop”) the last state
P from stack and go to a different successor of P.
• The solution path is the stack items when goal is
found.
The breadth-first search algorithm
• Erase the oldstate array; initialize queue to the starting
state with a null backpointer.
• Until the first state on the queue is a goal state:
– Remove the first item on the queue and call it S.
– If S is a goal state, stop.
– Otherwise, add successors of S not already on the
agenda or in the oldstate array to the end of the queue,
using heuristics to determine the order you add them;
give each a backpointer to S.
– Add S to the oldstate array.
• The solution path is found by creating a list of the states
encountered in following the backpointers from the goal
state (using the oldstate array), then reversing the list.
The A* search algorithm
• Initialize agenda to just the starting state with null
backpointer and starting-state evaluation-function value.
• Loop until you pick a goal state:
– Remove the first (best) item on the agenda and call it S.
– If S is a goal state, stop.
– Otherwise, insert successors of S into the agenda if
they are not already in the agenda or in the oldstate
array, maintaining sorted order of total evaluation.
Give each successor state T a backpointer to S and a
total evaluation U(T) = U(S) + C(S,T) + E(T) - E(S)
where C is the cost from S to T and E is the evaluation
function. If T is already on the agenda and this U is
better, replace the state on the agenda, else ignore.
– Add S to the oldstate array.
• Create the solution path by following backpointers from
the goal state, then reversing the list.