CS440/ECE448: Intro to Artificial Intelligence!
Tuesdayʼs key concepts"
Lecture 4:
Heuristic search
and local search"
Problem solving as search:"
Prof. Julia Hockenmaier!
[email protected]!
!
http://cs.illinois.edu/fa11/cs440!
!
!
Systematic (blind) search algorithms"
Solution = a finite sequence of actions
"
State graphs and search trees"
Which one is bigger/better to search?
"
Breadth-first vs. depth-first; properties?
"
"
CS440/ECE448:
Intro AI!
"
Blind search: deterministic queuing functions"
Depth-first search "(LIFO)!!
Expand deepest node first!
QF(old, new): !Append(new, old)
Graph search"
Initial !
state!
A
b3!
a1!
B
2!
C
D
c5!
!
"!
!
!
Goal!
state!
!
b4!
"!
"!
!
"!
"!
a2!
!
E
F
G
H
I!
J
a3!
"!
b1!
!
!
Breadth-first (FIFO)"
A
"!
B
Explored set!
Frontier
(Queue)!
Unexplored set!
!
b2!
"!
Expand nodes level by level!
C
!
D
"!
!
"!
QF(old, new): !Append(old, new);!
!
….
E
F
G
H
I!
J
!
!
"!
….
"!
….
!
"!
Goal!
state!
!
"!
Goal!
state!
Todayʼs key questions"
How can we find the optimal solution?!
We need to assign values to solutions
Informed (heuristic) search"
How can we find a better solution if we
can only foresee the effect (=value) of
the next action? !
This is local search.!
!
CS440/ECE448: Intro AI!
5!
Considering the cost of solutions"
We may not just want to find any solution,
but the cheapest solution, if:!
Heuristic search algorithms sort the nodes
on the queue according to a cost function: !
–  Each action has a (positive, finite) cost !
QF(a,b): !sort(append(a,b), CostFunction)
!
The cost function is an estimate of the true cost.!
Nodes with the lowest estimated cost have the highest priority.!
–  Some solutions may be cheaper than others
!
!
CS440/ECE448: Intro AI!
Heuristic search: priority queue"
7!
8!
Heuristic graph search"
Cost from root to node: g(n)"
SEARCH(Problem P, Queuing Function QF):
local: n, q, e; q ! new List(Initial_State(P));
Loop:
!if q == () return failure;
!n ! Pop(q);
!if n solves P return n;
add n.STATE to e for m in Expand(n):
if m is not in e or q:
q ! QF(q,{m});!
/*NEW: we want to find the cheapest goal!*/ ! else if m.STATE in q with higher cost:
!
!
! q ! replace(q, m.STATE, m);
9!
end!
Uniform-cost search"
g*(n): Minimum cost from root to n!
g*(n) is the sum of the costs for each action from the root to node n.!
This requires a cost function for actions!
g(n): Computable approximation to g*(n)
For trees: g(n) == g*(n)!
CS440/ECE448: Intro AI!
10!
Uniform-cost search illustrated"
S" 1
g = 0!
Sort the queue by path cost g(n):"
First expand the node with lowest g(n)!
!
QF(a,b): sort(Append(a,b), g)!
80!
99!
R" 2
g=80!
97!
F" 3
g = 99!
4
P"
g = 177!
211!
101!
5
B"
g = 278!
S:0 ![R:80, F:99]!
R:80 ![F:99, P:177]!
F:99 ![P:177, B:310]!
P:177 ![B:278, B:310]!
B:278 [B:310] !
CS440/ECE448: Intro AI!
11!
CS440/ECE448: Intro AI!
12!
Properties of uniform-cost search"
How close are we to the goal?"
I! Initial state!
Complete if b is finite, and each action has
positive (non-zero) cost (gets stuck in loops of zero-cost actions)
Optimal."
!
Time and space complexity similar to
breadth-first search if costs are uniform!
(possibly much worse otherwise)!
}
g*(n)!
n! Node n!
}
???!
G! Goal state!
CS440/ECE448: Intro AI!
13!
Cost from node to goal: h(n)"
14!
h*(n): the minimum cost from n to any goal
Greedy best-first search"
Sort the queue by heuristic function h(n):"
First expand the node with lowest h(n)!
!
QF(a,b): sort(Append(a,b), h)!
h*(n) is generally unknown !
h(n): computable approximation to h*
h(n) is called the heuristic function!
!
n!
}
!
Problem: This ignores g(n)!
h*(n)!
G
CS440/ECE448: Intro AI!
15!
CS440/ECE448: Intro AI!
16!
Properties of greedy best-first search"
Total cost: f*(n)"
Similar to DFS: !
Tree-search version is incomplete. "
Graph-search version is complete in finite state spaces. Both are not optimal."
!
Worst-case time and space complexity
similar to DFS (actual complexity depends on h)!
I! Initial state!
}
f*(n) = g*(n) + h*(n)"
f*(n) is the lowest cost
solution from the
initial state to a goal
constrained through n !
g*(n)!
n! Node n!
}
h*(n)!
G! Goal state!
CS440/ECE448: Intro AI!
17!
18!
A* search"
Total estimated cost: f(n)"
f(n) = g(n) + h(n)"
f(n) approximates
the lowest cost
solution from the
initial state to a
goal constrained
through n !
I! Initial state!
Sort the queue by total estimated cost f(n):"
First expand the node with lowest f(n)!
!
QF(a,b): sort(Append(a,b), f)!
!
!
}
g(n)!
n! Node n!
}
h(n)!
G! Goal state!
19!
CS440/ECE448: Intro AI!
20!
Properties of A* search"
Tree-search A* is optimal if…"
A* is complete if b is finite, and each action
has positive (non-zero) cost !
Its complexity depends on the heuristic
function h(n)!
!
Is A* optimal?!
…h(n) is an admissible heuristic"
CS440/ECE448: Intro AI!
21!
Admissible heuristics never overestimate
the future cost.
!
Definition
h(n) is admissible: !n " nodes: h(n) # h*(n)
22!
!
!
Tree-search A* is optimal if h(n) is admissible "
In tree-search:
g(n) = g*(n)"
If n is a goal:
f(n) = f*(n) "
Graph-search A* is optimal if…"
Confused?!
This is
optional
material!
䚎The goal that is returned is the cheapest on the queue! !
If n is not a goal:
f(n) ≤ f*(n) "
g(n) + h(n) ≤ g*(n) + h*(n)!
f*(n): true cost of cheapest goal that can be reached from n. !
If gno is a non-optimal goal, and nopt is an ancestor
of the optimal goal gopt: f(nopt) ≤ f(gopt) < f(gno)"
䚎nopt will be explored first!"
Monotonic heuristics obey the triangle inequality:
Going from n to the goal via nʼ is at least as
expensive as going from n to the goal.!
!
n! cost(n, a, nʼ)!
!n!n’ " nodes,
nʼ!
h(n)!
!a " actions with a(n)$n’
h(nʼ)!
h(n) # cost(n, a, n’) + h(n’)
G!
!
NB.: every monotonic heuristic is also admissible!
!
!
!
…h(n) is monotonic (=consistent)!
Confused?!
This is optional
material!
23!
24!
Graph-search A* is optimal if…"
…h(n) is monotonic (=consistent)!
h(n) # cost(n, a, n’) + h(n’)
!
Proof sketch:"
1.  If h is monotonic, the values of f(n) along any
path are non-decreasing. (true by definition)!
2. When n is expanded, g(n) = g*(n) !
[we have found the cheapest path to n.]!
Thus, sorting by f finds the cheapest goal first !
Confused?!
This is optional
material!
Possible Heuristic Functions"
Often simplify by relaxing some constraint!
!
!h1: 3!
!h2 (8-puzzle): count # tiles out of place!
!h3 (route-finding): sum Manhattan metric distances!
!
Uniform-cost: huniform-cost = 0!
25!
26!
Informed heuristics"
Given: !A1* with heuristic function h1
and !A2* with heuristic function h2
! ! !Both A1* and A2* are admissible!
Local search"
!
A1* is more informed than A2* iff for all non-goal nodes n h1(n) > h2(n)!
!
“More informed”: “guaranteed not to search more”!
!
27!
Motivation for local search"
How can we find the goal when:
!
-  we canʼt keep a queue? - 
Local search algorithms:"
Consider only the current node and the next action!
!
Also useful for optimization:"
finding the best state according to
an objective function!
(because we donʼt have the memory) !
we donʼt want to keep a queue?
(because we just need to find the goal state, ) !
!
-  we canʼt enumerate the next actions?
(because thereʼs an infinite number)
!
!
!
CS440/ECE448: Intro AI!
8-queens"
30!
8-queens by local search"
Initial state:"
A board with 8 queens, one in each column.!
(we use a complete-state formulation).!
"
Action: Move one queen to another square
in its column.!
"
Heuristic function (score): How many queens attack each other? "
!
CS440/ECE448: Intro AI!
31!
CS440/ECE448: Intro AI!
32!
Possible successor states"
If queen in
first column
moves here,
score = 15!
18 12 14 13 13 12 14 14
14 16 13 15 12 14 12 16
14 12 18 13 15 12 14 14
14 17 15
current
state!
14 16 16
16 18 15
18 14
The best
next
states!
13 16 13 16
15 14 14
17
cost!
The state space landscape:
minimizing cost…"
15 15 14
15
goal state!
16
14 14 13 17 12 14 12 18
CS440/ECE448: Intro AI!
state space!
33!
…or maximizing an objective function"
objective!
function!
= −cost!
CS440/ECE448: Intro AI!
34!
The state space landscape"
objective!
function!
goal state!
global
maximum!
current
state!
current
state!
plateau!
local
maximum!
state space!
CS440/ECE448: Intro AI!
state space!
35!
CS440/ECE448: Intro AI!
36!
Goal: reaching the global
maximum"
objective!
function!
global
maximum!
current
state!
Generic Search search"
Function
Hill-climbing
HillClimb(Problem P):!
You win!!
!local: !n !/*current node*/
n ! InitialState(P);!
plateau!
Loop:!
local
maximum!
! n’ ! highestValueNeighborOf(n);!
!/* Are we at the peak yet? */!
if n’.VALUE ≤ n.VALUE return n; ! !
else n = n’ /* Keep climbing up to n’ */!
state space!
CS440/ECE448: Intro AI!
end!
37!
Problem: local maxima are
difficult to avoid"
objective!
function!
current
local
state!
maximum!#!@*!!
A local maximum for 8 queens"
global
maximum!
plateau!
state space!
CS440/ECE448: Intro AI!
39!
CS440/ECE448: Intro AI!
40!
Problem: plateaus are difficult
to navigate"
Ideal case:
Convex optimization"
objective!
objective!
function!
function!
local
maximum!
global
maximum!
current
state!
#!@*!!
global
maximum!
plateau!
state space!
When the objective function is convex, search/optimization becomes vastly simplified.!
state space!
CS440/ECE448: Intro AI!
41!
CS440/ECE448: Intro AI!
Random
restart
Generic
Search
Function
Dealing with local maxima"
hill-climbing search"
Random restart hill-climbing:"
k trials starting from random positions!
!
k-best (beam) hill climbing:"
Pursue k trials in parallel; only keep the k best
successors at each time step!
!
Simulated annealing:"
Accept downhill moves with non-zero prob.!
CS440/ECE448: Intro AI!
43!
RandomRestartHillClimb(Problem P):!
!local: !n, nMax; /* current best node*/
For r = 1 … R: /* try R times*/!
n ! RandomInitialState(P); !
!
nMax ! n;!
!Loop (climb):!
! !
n’ ! highestValueNeighborOf(n);!
! !
if n’.VALUE > nMax.VALUE nMax ! n’;!
! !
if n’.VALUE > n.VALUE n ! n’;!
!
else exit Loop;!
return nMax;!
42!
Generic
Search
k-best
(beam)
hill Function
climbing"
Generic Search
Function
Simulated
annealing"
K-bestHillClimb(Problem P, int k):!
local: N = Array[k] /*vector of K node*/!
for i = 0…k-1: !
N[i] ! RandomInitialState(P);!
N ! sort(N)!
Loop: !
! nMax = N[0]; !
! !local: N’ = Array[k], M = Array[2k] !
! !for i = 0…k-1:!
! !
N’[i] !! highestValueNeighborOf(n[i]);
M ! sort(append(N, N’)); !
N ! M[0…k-1];!
if N[0].VALUE ≤ nMax.VALUE return nMAx;!
end!
A version of stochastic hill-climbing:!
go downhill with non-zero probability pdown!
!
pdown depends on step size: "
pdown is greater for smaller steps.!
!
pdown depends on current temperature:"
pdown is greater at high temperature.!
Temperature is a function of time:"
Start with high temperature, lower gradually.!
!
Computing pdown "
Temperature = 10 "
1
∆Value: neighbor.VALUE – this.VALUE!
In a downhill move, ∆Value is negative.!
!
Temperature: "
We define temperature to be a decreasing
function of time. !
!
pdown = e∆Value/Temperature!
CS440/ECE448: Intro AI!
T=10
0.9
0.8
0.7
pdown
0.6
0.5
0.4
0.3
0.2
0.1
0
-50
-40
-30
-20
-10
0
6 Value
47!
CS440/ECE448: Intro AI!
48!
Temperature = 5 "
1
1
T=10
T=5
0.9
0.8
0.8
0.7
0.7
0.6
0.6
0.5
0.5
0.4
0.4
0.3
0.3
0.2
0.2
0.1
0.1
0
-50
-40
-30
-20
-10
T=10
T=5
T=2
0.9
pdown
pdown
Temperature = 2 "
0
-50
0
-40
6 Value
-30
-20
-10
0
6 Value
CS440/ECE448: Intro AI!
49!
CS440/ECE448: Intro AI!
Generic Search
Function
Simulated
annealing"
Temperature = 0.0001 "
1
SimAnnealing(Problem P):!
T=10
T=5
T=2
T=0.0001
0.8
50!
!local: !n !
n ! InitialState(P);!
For time = 1 to ∞ do:!
0.6
pdown
!Temperature = temp(time); !
!if Temperature == 0 return n; !
0.4
!n’ ! randomNeighborOf(n);!
0.2
0
-50
∆Value = n’.VALUE - n.VALUE;
!if ∆Value > 0
-40
-30
-20
-10
n ! n’!
!else n ! n’ with prob. e∆Value/Temperature!
0
6 Value
end!
CS440/ECE448: Intro AI!
! !
51!
Todayʼs key concepts"
Heuristic search:"
Actions and solutions have costs!
Heuristic function: estimate of future cost!
Uniform cost, best-first, A*!
!
Local search:"
Agent only sees the next steps.!
Features of the state space landscape!
Hill-climbing, random restart, beam,
simulated annealing!
To conclude…"
Your tasks"
Reading:!
Chapter 3.5, Chapter 4.1!
!
Compass quiz:!
Up at 2pm!
!
Assignments:
MP 1 will go out this afternoon.!
CS440/ECE448: Intro AI!
55!