State Space Search
OPTIMIZATION PROBLEM
Problem
Problem
Instance 1
•Description of
valid input
• Description of
desired output
Instance 2
…
Instance 3
Instance N
Algorithm
Instance X
Algorithm
Solution for
X
Optimization Problem
• Most problems can be described as an
Optimization Problem (OP)
is described by a
function
– A domain of must also be describe
• Called , the set of possible solutions
– The solution of the instance is
such that
is maximized (or minimized)
Optimization Problem Instance
Instance X
Input = Ix
D: Set of
possible
solution
Fx(S)
Evaluation
value of S
How to solve OP?
•
•
•
•
Simple
Iterate every possible
Calculate
for each
Remember that maximize
THE METHOD for OP
• Try everything!!!
• Pick the best one!!!!
Is this efficient?
• Depends on the size of possible solution
• In many problems, we have a better
approach
Example
• Find Max in the previous lab
• Shortest Path
– Find min = find max of the negative
– i.e., Minimize F(S)  Maximize -F(S)
More Subtle Example
• Sorting
• Define inversion
– Inversion = number of pairs that is out of order
– Ex
• (3,1,2) has two inversions (a pair of 3,1 and 3,2)
• Find permutation that minimize inversion
Wait
• “Most problems can be described as an
Optimization Problem (OP)”
– Is this true?
• What is a “problem”?
•We know how to “ check”
whether the output is
desired.
•Hence, we can use that as
F(S)
•Description of
valid input
• Description of
output
Optimization Example: Finding Max Value
in an Array
25 2 34 43 4
9
0
-5 87 0
• There are N possible answers
– The first element
– The second element
– 3rd, 4th …
• Try all of them
– Remember the best one
5
6
1
STATE SPACE SEARCH
State Space Search Framework
VERY
IMPORTANT!!
• Define the set of admissible solution
• Generate all of them
(
• For each generated solution
– Test whether it is the one we want
– By the “evaluation” function
(
)
)
– for optimization: remember the best one so far
– For decision: report when we found the
“correct” one
Solving Problem by State Space Search
1. Define the set of admissible solution (the
search space)
– What is it?
– How large?
– How to represent the solution
2. Determine how to generate all solutions
3. Determine how to check each solution
Generating in Steps
• In most problem, admissible solutions can
be generated iteratively
– Step-by-step
• Example
– Maximum Sum of Subsequence
– Minimal Spanning Tree
– Pachinko Problem
Search: Maximum sum of
Subsequence
• Search Space
– Every possible sequence
– Described by a pair of starting,ending element
• Generating all solutions
– Step 1: select the position of the first element
– Step 2: select the position of the last element
• Checking each solution
– Sum and test whether it is maximum
Search: Minimal Spanning Tree
• Search Space
– Every subset of edges
– Described by a set of edges
• Generating all solutions
– For every edge
• Either include or exclude it from the set
• Checking each solution
– Sum the cost in the set
• and test whether it is maximum
– Test whether it is a tree
– Test whether it is connected
Search: Minimal Spanning Tree 2nd
attemp
• Search Space
– Every subset of edges of size |V|-1
– Described by a set of edges
• Generating all solutions
– For every edge
• Either include or exclude it from the set
• Do not select more than |V|-1 edges
• Checking each solution
– Sum the cost in the set
• and test whether it is maximum
– Test whether it is a tree
Search: Minimal Spanning Tree 3rd
attemp
• Search Space
– Every subgraph of size |V|-1 edge that is a tree
– Described by a set of edges
• Generating all solutions
– For every edge in X in “cut property”
• Either include or exclude it from the set
– Update the tree
• Do not select more than |V|-1 edges
• Checking each solution
– Sum the cost in the set
• and test whether it is maximum
Search: Pachinko
• Search Space
– Every path from the top pin to the bottom pin
– Described by a sequence of direction
• (left,left,right,left)
• Generating all solutions
– For every level
• Choose either the left side or right side
• Checking each solution
– Sum the cost in the path
• and test whether it is maximum
GENERATING ALL POSSIBLE
ANSWERS
Combination and Permutation
• In many case, the set of the admissible
solutions is a set of “combination” or
“permutation” of something
• We need to knows how to generate all
permutations and combinations
Combination
• Given N things
• Generate all possible selections of K things
from N things
• Ex. N = 3, k = 2
Combination with replacement
• Given N things
• Generate all possible selections of K things
from N things
– When something is selected, we are permit to
select that things again (we replace the selected
thing in the pool)
• Ex. N = 3, k = 2
Breaking the Padlock
Breaking the Padlock
• Assuming we have four rings
• Assuming each ring has following mark
–
• We try
–
–
– ….
–
–
Undone the second
step, switch to
another value
Key Idea
• A problem consists of several similar steps
– Choosing a things from the pool
• We need to remember the things we’ve done
so far
General Framework
Initial
Step
1. Get a step that is not complete
Storage
Engine
Gemerated (partial)
solution
2. Try all possible
choice in next step
3. Store each newly
generated next step
Solution Generation
Storage s
s  ‘’
While s is not empty
Curr  s.get
If Curr is the last step
evaluate
Else
Generate all next step from Curr
put them into S
If length(curr) == 4
For all symbol i
New = curr+I
Storage.push(new)
Search Space
• Set of all admissible solutions
• E.g., Combination Padlock
– Search space = 0000  3333
Search Space Enumeration
• Key Idea: generate step-by-step,
– Undo previous step when necessary
• Usually using some data structure to
implement the storage
– E.g., using stack, either explicitly or implicitly
from the processor stack
• i.e., using the “recursive” paradigm
– Queue is a possible choice
Example: Padlock
• Generate all combinations of lock key
• Represent key by int
–
–
–
–
=0
=1
=2
=3
• Step = sequence of selected symbols
– E.g.,
• ’01’  
• ‘0003’  
Example: Padlock
void search(int step,int *sol)
{
if (step < num_step) {
for (int i = 0; i < num_symbol; i++) {
sol[step] = i;
search(step + 1,sol);
}
} else {
check(sol);
}
}
• The process automatically remember the
step (by sol array) and by stack (int i)
Search Tree
• A tree representing every step in the seach
• Similar to Recursion Tree
– Actually, it is related
• Node:
– Each step in solution generation
• Edge:
– Connects two nodes such that one node is
generated from applying one step to the other
node
Search Tree
0
00
…
…
…
01
…
…
…
1
2
02
03
…
…
…
…
…
…
3
…
Search Tree
• Sols in each step
–
–
–
–
–
–
–
–
–
–
–
–
–
–
0
00
000
0000  check
000
0001  check
000
0002  check
000
0003  check
000
00
001
0010  check
8-QUEEN PROBLEM
8-queen problem
• Given a chess board
with 8 queens
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
8-queen problem
• Try to place the
queens so that they
don’t get in the others’
ways
Q
Q
Q
Q
Q
Q
Q
Q
8-queen problem
• Input:
– None!
• Output:
– Every possible placement of 8-queens that does not
jeopardize each other
Solving the Problem
• Define the search space
– What is the search space of this problem?
– How large it is?
– Choose an appropriate representation
st
1
attemp
• Every possible placement of queens
– Size: 648
– Representation: a set of queens position
• E.g., (1,1) (1,2) (2,5) (4,1) (1,2) (3,4) (8,8) (7,6)
• This includes overlapping placement!!!
nd
2
attemp
• Another representation
– Try to exclude overlapping
• Use combination without replacement
– This is a combination
• Selecting 8 positions out of 64 positions
• Size: (64)! / (64 – 8)! * 8!
– Implementation: in the “generating next step”,
check for overlapping
nd
2
attemp (first implementation)
• We go over all positions
– For each position, we either “choose” or “skip”
that position for the queen
1
2
3
4
1
2
3
4
5
6
7
8
5
6
7
8
9
10
11
12
9
10
11
12
13
14
15
16
13
14
15
16
Combination without replacement
Mark_on_board is a binary array
to indicate whether the position is
selected
void e_queen(int step,int *mark_on_board)
{
if (step < 64) {
mark_on_board[step] = 0;
e_queen(step + 1,mark_on_board);
mark_on_board[step] = 1;
e_queen(step + 1,mark_on_board);
} else {
check(mark_on_board);
}
}
* Check if OK
* Also has to check whether we mark
exactly 8 queens
nd
2
attemp Issue
• The generated mark_on_board includes
– 000000000000  select no position (obviously
not the answer)
– 111111000000  select 6 positions (obviously
not the answer)
• We must limit our selection to be exactly 8
2nd attemp (second implementation)
void e_queen(int step,int *mark_on_board,int chosen)
{
Number of
if (step < 64) {
possible 0
if ((64 – 8) – (step – chosen) > 0) {
mark_on_board[step] = 0;
e_queen(step + 1,mark_on_board,chosen);
}
if (8 - chosen > 0) {
mark_on_board[step] = 1;
e_queen(step + 1,mark_on_board,chosen+1);
}
} else {
Number of
check(mark_on_board);
possible 1
}
}
e_queen(0,mark,0);
2nd attemp (second implementation)
rev 2.0
void e_queen(int step,int *mark_on_board,int one,int zero)
{
if (step < 64) {
Number of
if (zero > 0) {
possible 1
mark_on_board[step] = 0;
e_queen(step + 1,mark_on_board,one,zero - 1);
}
if (one > 0) {
Number of
mark_on_board[step] = 1;
possible 0
e_queen(step + 1,mark_on_board,one – 1,zero);
}
} else {
check(mark_on_board);
}
}
e_queen(0,mark,8,64 - 8);
rd
3
attemp
• Any better way?
• For each row, there should be only one
queen
– The problem consists of 8 step
• Placing each queen
– Size: 88
– Representation: sequence of columns
• E.g., (1,2,3,4,5,6,7,8)
rd
3
attemp Implementation
• There are eight possible ways in each step
• There are eight steps
• Very similar to the combination problem
void e_queen(int step,int *queen_pos)
{
if (step < 8) {
for (int i = 0; i < 8; i++) {
queen_pos[step] = i;
e_queen(step + 1, queen_pos);
}
} else {
check(queen_pos);
}
}
th
4
attemp
• Queen should not be in the same column
– The solution should never have any column
repeated
• E.g.,
– (1,2,3,4,5,6,7,1) is bad (column collision
– (1,1,3,4,5,6,7,5) is bad as well….
– (1,2,3,4,5,6,7,8) is good
» There should be no duplicate column index!!!
Permutation
• Given N symbols
• A permutation is the element arrange in any
order
– E.g., 1 2 3 4
– Shows
•
•
•
•
•
•
1234
1243
1324
1342
…
4321
• For each step, we have to known which one is
used
Permutation
• The problem consists of several similar
steps
• Special condition
– Symbols never repeat
• How to do?
– Easy way:
• Generate all combination (as done before)
– Check for ones that symbols do not repeat
– Better way:
• Remember what symbols are used
Permutation
void search(int step,int *sol)
{
if (step <
) {
for (int i = 0; i <
if not_used(sol,i,step) {
sol[step] = i;
search(step,sol);
}
}
} else {
check(sol);
}
}
; i++) {
Bool not_used(int *sol,int value,int step)
{
for (int i = 0;i < step; i++) {
if (sol[i] == value) return false;
}
return true;
}
Permutation
• More proper ways
void search(int step,int *sol,bool *used)
{
if (step < num_step) {
for (int i = 0; i < num_symbol; i++) {
if (!used[i]) {
used[i] = true;
sol[step] = i;
search(step,sol,used);
used[i] = false;
}
}
} else {
check(sol);
}
}
Incresing Sequence
• Given N
– Find any sequence of (a1,a2,a3,…) such that
•
•
•
•
a1 +a2 + a3 +… + ak = N
ai > 0
ai <= aj for all i < j
ai is an integer
Example
• N=4
–1+1+1+1
–1+1+2
–1+3
–2+2
–4
DFS AND BFS ALGORITHM
Breadth First Search and Depth First
Search
Storage s
s  ‘’
While s is not empty
Curr  s.get
If Curr is the last step
evaluate
Else
Generate all next step from Curr
put them into S
• Can we draw a search tree from this pseudocode?
– Assume that we know how to generate next step?
• Do we know the growth (in term of structure,
not size) of the search tree?
Breadth First Search and Depth First
Search
Storage s
s  ‘’
While s is not empty
Curr  s.get
If Curr is the last step
evaluate
Else
Generate all next step from Curr
put them into S
• Search Tree grows according to S
– and how to generate all next step as well,
• If S is “stack”, the algorithm is called DFS
• If S is “queue”, the algorithm is called BFS
Triple and Half Problem
• Input:
– A number
• Output:
– A sequence of either * 3 or / 2 operation, start from 1 that
is evaluated as the given number
• Special version: number of operations should be minimal
• Example:
–
–
–
–
Input: 10
Output: 1 * 3 * 3 * 3 * 3 / 2 / 2 / 2
Input: 31
Output: 1 * 3 * 3 * 3 * 3 * 3 / 2 / 2 / 2 / 2 / 2 * 3 * 3 / 2
Triple and Half Problem
• Search Space
– Every possible sequence of triple or half
– Described by a sequence of operation
• (T,H,T,H,T)
– Issue: There is no clear length-limit of sequence!!
• Generating all solutions
– Step: operator
• Choose either the triple or half
• Checking each solution
– Evaluate the expression
• and test whether it equals to target
Solution
st
(1
attemp)
void DFS(int goal) {
Stack S;
// store string
S.put ("1");
while (!S.isEmpty()) {
curr = S.pop();
if ( test(curr) == goal) {
printf("found! %s",curr);
return ;
}
char a[10000],b[10000];
strcpy(a,curr); strcat(a,"*3");
S.push(a);
strcpy(b,curr); strcat(b,"/2");
S.push(b);
}
}
• Issue
– Is [10000]
enough?
– Repeat
solution?
• 1*3*3/
2*3/2
6
• 1*3*3*
3/2/2
6
– When will
the program
stop?
– Can you
draw the
search tree?
Solution
nd
(2
attemp)
void DFS(int goal) {
Stack S;
// store int
Hash h;
S.put (1); H.put(1);
while (S.isEmpty() = false) {
curr = S.pop();
if (curr == goal) {
printf("found!\n”); return ;
}
if (h.contain(curr * 3) == false) {
s.push(curr * 3);
h.add(curr * 3);
}
if (h.contain(curr / 2) == false) {
s.push(curr / 2);
h.add(curr / 2);
}
}
}
• Issue
– Path is
lost
• Never
mind
that for
now
Solution
nd
(2
attemp) by recursive
void DFS(int goal,int curr,Hash h) {
if (curr == goal) {
printf(“found\n”);
} else {
if ( !h.contain(curr * 3)) {
h.add(curr * 3);
DFS(goal,curr * 3,h);
} if ( !h.contain(curr / 2)) {
h.add(curr / 2);
DFS(goal,curr / 2,h);
}
}
}
• Issue
– Path is
lost
• Never
mind
that for
now
Solution
rd
(3
attemp) BFS
void DFS(int goal) {
Queue q;
// store int
Hash h;
q.enq(1); H.put(1);
while (S.isEmpty() = false) {
curr = q.deq();
if (curr == goal) {
printf("found!\n”); return ;
}
if (h.contain(curr * 3) == false) {
s.enq(curr * 3);
h.add(curr * 3);
}
if (h.contain(curr / 2) == false) {
s.enq(curr / 2);
h.add(curr / 2);
}
}
}
• What’s the
difference
?
Conclusion
• DFS
– Stack-based
– Use less space
– Might loops forever if number of steps is not
fixed
• BFS
– Queue-based
– Use large space
– Found solution nearest to the root node in the
search tree
BACKTRACKING AND BRANCH &
BOUND
Technique to reduce enumeration
Main Idea
• We should not enumerate solution that will
never produce a solution
• We have done that!!!
– 8-queens
– By naïve combination, we will have to do all 648
• But, by each improvement, we further reduce what
we have to do
Another Example: Permutation by
Combination
0
1
2
00
01
02
10
11
12
20
21
22
000
010
020
100
110
120
200
210
220
001
011
021
101
111
121
201
211
221
002
012
022
102
112
122
202
212
222
Another Example: Permutation by
Combination
0
1
2
00
01
02
10
11
12
20
21
22
000
010
020
100
110
120
200
210
220
001
011
021
101
111
121
201
211
221
002
012
022
102
112
122
202
212
222
Partial Solution
Backtracking and B&B
• Partial Solution
– Solution that we are generating
• Not complete
– Should be apprehensible in some sense
• We can make something out of the partial solution
• Backtracking and B&B work with problem
with partial solution
• Pushing the concept of “do not generating
something that won’t lead to answer”
Backtracking
• If we know, at any step, that the solution is
not feasible
– Then, it is futile to further search along that
path
• Try drawing search tree of 4-queen problem
4-queen
Q
Q
Q
Q
Q
Q
Q
…
Q
Q
Q
Q
Q
Q
Q
Q
Should we proceed on these state?
If not, how much state do we save?
Q
…
Sum of Subset Problem
• Input:
– Array D of positive integer
– A number K
• Output
– A subset of D whose summation is K
• Example
– D = {2,5,7,1,3,8}
–K=9
– Solution is {2,7} or {8,1} or {5,3,1}
Sum of Subset
• Search Space
– Every possible subset of elements
– Described by a sequence of selection bit
• (1,0,1,0,0,0)
• Generating all solutions
– Step: choose either we select each item
• Checking each solution
– Sum of the selected item
• and test whether it equals to target
Sum of Subset by DFS
void ss(int step,int* sol) {
if (step < n) {
sol[step] = 0;
ss(step + 1,sol);
sol[step] = 1;
ss(step + 1,sol);
} else {
int sum = 0;
for (int i = 0;i < n;i++)
if (sol[i] == 1) sum += D[i];
if (sum == K) { printf("YES!\n"); }
}
}
k = target
D = given array
n = size of array
Backtracking for Sum of Subset
• If summation of selected element is more
void
ss(int
step,int* sol,int total) {
than
K,
stop
if (step < n) {
if (total > K) break;
sol[step] = 0;
ss(step + 1,sol,total);
sol[step] = 1;
ss(step + 1,sol,total+D[i]);
} else {
int sum = 0;
for (int i = 0;i < n;i++)
if (sol[i] == 1) sum += D[i];
if (sum == K) { printf("YES!\n"); }
}
}
Backtracking for Sum of Subset
• If summation of selected element is more
void
ss(int
step,int* sol,int total) {
than
K,
stop
if (step < n) {
if (total
sol[step]
ss(step +
sol[step]
ss(step +
} else {
if (total
}
> K) break;
= 0;
1,sol,total);
= 1;
1,sol,total+D[i]);
== K) { printf("YES!\n"); }
}
We don’t really need to compure
sum at the final step
Because total already do that for us
Branch and Bound
• B&B is for optimization problem
• Consider “Maximizing Problem” for example
• Need bounding heuristic
– Something that can tell us “what is the guaranteed
minimal of our solution in the remaining steps”
• If the value of the current partial solution +
value of the heuristic is less than that of any
candidate solution
– Stop
• A special version of Backtracking
Branch & Bound in Optimization Problem
• For many problems, it is possible to assert
its goodness even the solution is not
complete
• If we can predict the best value for the
remaining steps, then we can use that value
to “bound” our search
Example
• Assuming that we have 10 steps
• At step 7, the goodness of the partial
solution is X
• Assuming that we know that the remaining
step could not gain more than Y
– If we have found a solution having value better
than X+Y
• We can simply “bound” the search
Keys
• We must know the so-called “upper bound”
of the remaining step
– It should be computed easily
Example
If we know that this path
never bet higher than 13
(which make 10 + 13 < 35)
We can neglect it
23
35
2
Let value at this
point be 10
Knapsack Problem
• Given a sack, able to hold W kg
• Given a list of objects
– Each has a weight and a value
• Try to pack the object in the sack so that the
total value is maximized
The Problem
• Input
– A number W, the capacity of the sack
– n pairs of weight and price
((w1,p1),(w2,p2),…,(wn,pn))
• wi = weight of the ith items
• pi = price of the ith item
• Output
– A subset S of {1,2,3,…,n} such that
•
p
iS
i
is maximum
wi  W
•
iS
Knapsack by Search
void knapsack(int step,int* sol) {
if (step < n) {
sol[step] = 0; knapsack(step + 1,sol);
sol[step] = 1; knapsack(step + 1,sol);
} else {
int sumP = 0;
int sumW = 0;
for (int i = 0;i < n;i++)
if (sol[i] == 1) {
sumP += p[i];
sumW += w[i];
}
if (sumP > max && sumW <= W) { max = sum; }
}
}
Knapsack by Search rev 2.0
void knapsack(int step,int* sol,int sumP,int sumW) {
if (step < n) {
sol[step] = 0;
knapsack(step + 1,sol,sumP,sumW);
sol[step] = 1;
knapsack(step + 1,sol,sumP + v[step],sumW + w[step]);
} else {
if (sumP > max && sumW <= W) { max = sum; }
}
}
Knapsack with Backtracking
• Knapsack is similar to sum of subset
• In SS, we need summation of element to be
equal to K
• In KS, we need summation of weight to be
less than W
• So, obviously, we should backtrack when
weight sum of selected item is more than W
Knapsack with Backtracking
void knapsack(int step,int* sol,int sumP,int sumW) {
if (sumW > W) return ;
if (step < n) {
sol[step] = 0;
knapsack(step + 1,sol,sumP,sumW);
sol[step] = 1;
knapsack(step + 1,sol,sumP + v[step],sumW + w[step]);
} else {
if (sumP > max && sumW <= W) { max = sum; }
}
}
Stop when sumW is more than W
Branch and Bound in Knapsack
• Assume that
– we have 10 items (N = 10)
– Right now, our max is 100
– Right now, we are searching at step = 5
– Right now, our sumP = 20
• If v[5] + v[6] + … + v[9] is 75, should we
proceed?
Knapsack with Backtracking + B&B
tail [n-1] =v[n-1];
for (int i = n – 2;i > =0;i--) tail[i] = tail [i+1] + v[i];
void knapsack(int step,int* sol,int sumP,int sumW) {
if (sumW > W) return ;
if (tail[step] + sumP < max) return ;
if (step < n) {
sol[step] = 0;
knapsack(step + 1,sol,sumP,sumW);
sol[step] = 1;
knapsack(step + 1,sol,sumP + v[step],sumW + w[step]);
} else {
if (sumP > max && sumW <= W) { max = sum; }
}
}
Stop when sumW is more than W
Stop when remaining + sumP is less than current MAX
Knapsack with Backtracking + B&B rev
2.0
tail [n-1] = v[n-1];
for (int i = n – 2;i > =0;i--) tail[i] = tail [i+1] + v[i];
void knapsack(int step,int* sol,int sumP,int sumW) {
if (sumW > W) return ;
if (tail[step] + sumP < max) return ;
if (step < n) {
sol[step] = 1;
knapsack(step + 1,sol,sumP + v[step],sumW + w[step]);
Wh
sol[step] = 0;
y?
knapsack(step + 1,sol,sumP,sumW);
} else {
if (sumP > max && sumW <= W) { max = sum; }
}
}
Does the search tree differ?
Variation
• Rational Knapsack
– Object is like a gold bar, we can cut it in to piece
with the same value/weight
• Can be solved by greedy
• Sort object according to value/weight ratio
• Pick objects by that ratio
– If object is larger than the remaining capacity,
just divide it
0-1 Knapsack with B&B
• 0-1 knapsack is very suitable for B&B
– We can calculate the goodness of the partial
solution
• Just sum the value of the selected objects
– We have fast, good upper bounds (several one)
• The sum of remaining unselected objects
– The sum of remaining unselected object that don’t exceed
the capacity
• The solution of the “rational knapsack” of the
remaining objects with the remaining capacity
Bounding Heuristic
• Maximization problem (finding highest value solution)
– Bound must be higher than (or equal to) the real value
(upper bound)
• Good bound  lowest value that is >= the real value
• Minimization problem (finding lowest value solution)
– Bound must be lower than (or equal to) the real value (lower
bound)
• Good bound  highest value that is <= the real value
• Key
– Never under estimate!!!
– We stop when current cost + future cost is sure loser.
– Future cost can be over estimate
• optimistic
LEAST COST SEARCH
a.k.a Best First Search
Least Cost Search
• For optimization problems, good solution
helps backtracking and B&B
– The better the value, the higher chance that
backtracking and B&B could benefit
• Why do DFS or BFS?
• We have “incentive” to find “good” solution
Guided search
earlier
• Search toward “promising” solution
Least Cost Search
• Use priority queue in search framework
• Key value in PQ is calculated from the
partial solution
• It’s called “Best First Search”
– Guided by the value of partial solution
– Suffers similar problem as BFS (since we need
to maintain the queue) In practice, LC-Search always employ
B&B and Backtracking
LC-Search with Branch and Bound
• Use bounding heuristic for value in PQ
• value stored in the PQ is X + Y
– X = current value of the partial solution
– Y = value from the heuristic of the partial solution
• I.e., use the bound of total value for guiding the
search
– For maximization problem use upper bound of total
value
– For minimization problem use lower bound of total
value
• Can guarantee minimality of the first solution
15 Puzzle Problem
• Given a puzzle board
• Objective is to move
pieces around so that the
board be like the figure
on the right
15 Puzzle Problem
• Input
– A board (solvable)
• Output
– Movement of pieces to solve
the puzzle
• Using minimal moves
– i.e., trying to minimize move
• It’s Minimization problem
15 Puzzle Problem
• Search Space
– Every possible movement
– Described by a sequence of move of the “empty”
piece
• (U,D,D,L,U,R,…)
– Don’t know the limit on the length of sequence
• Generating all solutions
– Step: choose directions of empty piece
• Checking each solution
– Simulate the move, see whether it leads to solution
Bounding Heuristic for 15-Puzzle
• Number of misplaced piece
– Obviously is a lower bound
1
3
6 2 11
5 8 7
14 12 15
left
1
6 2
5 8
14 12
4
10
9
13
Misplace = 11
down Moved = 1
right
3
11
7
15
4
10
9
13
Misplace = 13
Moved = 1
1
6
5
14
Misplace = 12
3
4
2 11 10
8 7 9
12 15 13
1 2 3 4
6
11 10
5 8 7 9
14 12 15 13
Misplace = 13
Moved = 1
11 + 2
1
10 + 2
2
6
5 8
14 12
3
11
7
15
4
10
9
13
1
6
11 + 2
5
14
2 3 4
11
10
8 7 9
12 15 13
1 2 3 4
6 8 11 10
5
7 9
14 12 15 13
Another Bounding Heuristic
• L-1 Distance of misplace piece
• For example, consider piece #12
– It is at row 4 col 2
– It should be at row 3 col 4
– Distance = 1 row 2 col = 1 + 2 = 3
 Obviously
•
current
1
3
6 2 11
5 8 7
14 12 15
4
10
9
13
goal
1
5
9
13
2
6
10
14
is a lower bound as well
Better lower bound (closer to the actual # of moves)
3 4
7 8
11 12
15
Conclusion
• Search Space
– Solution space
– State space tree, search tree
• Solution
Generating procedure gives
“structure” of the search
tree
Algorithm gives “order” of
the search tree exploration
(including prunning)
– Partial solution
– Candidate solution, solution state
• Algorithm
–
–
–
–
–
DFS (Depth first search)
BFS (Breadth first search)
LCS (Best first search)
Backtracking
Branch and Bound