CS541 Review
Jim Blythe
Planning for the Grid
USC INFORMATION SCIENCES INSTITUTE
2
Interferom
eter
LIGO’s Pulsar Search
(Laser Interferometer Gravitational-wave Observatory)
archive
Extract
channel
transpose
Long time frames
raw channels
Single Frame
Extract
frequency
range
Short
Fourier
Transform
30 minutes
Short time frames
Time-frequency
Image
Construct
image
Hz
USC INFORMATION SCIENCES INSTITUTE
Find Candidate
Store
Time
event
DB
3
Operators include data dependencies,
host and resource constraints.
(operator pulsar-search
(preconds
((<host> (or Condor-pool Mpi))
(effects
(<start-time> Number)
()
(<channel> Channel)
(
(<fcenter> Number)
(add (created <file>))
(<right-ascension> Number)
(add (at <file> <host>))
(<sample-rate> Number)
(add (pulsar <start-time> <end-time> <channel>
(<file> File-Handle)
<instrument> <format>
;; These two are parameters for the frequency-extract.
<fcenter> <fband>
(<f0> (and Number (get-low-freq-from-center-and-band
<fderv1> <fderv2> <fderv3> <fderv4> <fderv5>
<fcenter> <fband>)))
<right-ascension> <declination> <sample-rate>
(<fN> (and Number (get-high-freq-from-center-and-band
<file>))
<fcenter> <fband>)))
)
(<run-time> (and Number
))
(estimate-pulsar-search-run-time
<start-time> <end-time> <sample-rate>
<f0> <fN> <host> <run-time>)))
…)
(and (available pulsar-search <host>)
(forall ((<sub-sft-file-group>
(and File-Group-Handle
(gen-sub-sft-range-for-pulsar-search
<f0> <fN> <start-time> <end-time>
<sub-sft-file-group>))))
(and (sub-sft-group <start-time> <end-time>
<channel> <instrument> <format>
<f0> <fN> <sample-rate> <sub-sft-file-group>)
(at <sub-sft-file-group> <host>)))))
USC INFORMATION SCIENCES INSTITUTE
4
High-level specs of
desired results and
intermediate data
products
Metadata Catalog
Service
Request Manager
Workflow
Planning
AI-based
Planner
Current
State
Generator
Globus Replica
Location Service
Models and
current state
information
Concrete
Workflow
Dynamic
information
Submission and
Monitoring System
Resource Models
ng
ori
t
i
n
Mo
workflow executor
(DAGman)
Execution
Globus Monitoring
and Discovery
Service
a
rm
o
f
in
n
tio
Information and
Models
s
ta
ks
Grid
Raw data
detector
USC INFORMATION SCIENCES INSTITUTE
5
Temporal logics for planning
USC INFORMATION SCIENCES INSTITUTE
6
Fahiem Bacchus
USC INFORMATION SCIENCES INSTITUTE
7
Fahiem Bacchus
USC INFORMATION SCIENCES INSTITUTE
8
Heuristic search planning
USC INFORMATION SCIENCES INSTITUTE
9
Derive cost estimate from a relaxed
planning problem


Ignore the deletes on actions
BUT – still NP-hard, so approximate:

For individual propositions p:
d(s, p) = 0 if p is true in s
= 1 + min(d(s, pre(a))) otherwise
[min over actions a that add p]
USC INFORMATION SCIENCES INSTITUTE
10
HSP2 overview

Best-first search, using h+

Based on WA* - weighted A*:
f(n) = g(n) + W * h(n).
If W = 1, it’s A* (with admissible h).
If W > 1, it’s a little greedy – generally finds solutions
faster, but not optimal.

In HSP2, W = 5
USC INFORMATION SCIENCES INSTITUTE
11
HSPr problem space

States are sets of atoms (correspond to sets of states
in original space)

initial state is the goal G

Goal states are those that are true in s0 (initial state
in planning problem)

Still use h+. h+(s) = sum g(s0, p)
USC INFORMATION SCIENCES INSTITUTE
12
Mutexes in HSPr, take 2

Better definition:
A set M of pairs R = {p, q} is a mutex set if
(1) R is not true in s0
(2) for every op o that adds p,
either o deletes q
or o does not add q, and for some precond r of o,
{r, q} is in M.
Recursive definition allows for some interaction of the
operators
USC INFORMATION SCIENCES INSTITUTE
13
Temporal reasoning and scheduling
USC INFORMATION SCIENCES INSTITUTE
14
Temporal planning with mutual exclusion
relation

Propositions and actions are monotonically
increasing, no-goods monotonically decreasing:
USC INFORMATION SCIENCES INSTITUTE
15
ASPEN

Combine planning and scheduling steps as
alternative ‘conflict repair’ operations

Activities have start time, end time, duration

Maintain ‘most-commitment’ approach – easier to
reason about temporal dependencies with full
information

C.f. TLPlan
USC INFORMATION SCIENCES INSTITUTE
16
Contributors for a non-depletable
resource violation
USC INFORMATION SCIENCES INSTITUTE
17
Contributors for a depletable resource
violation
USC INFORMATION SCIENCES INSTITUTE
18
Learning search control knowledge
and case-based planning
USC INFORMATION SCIENCES INSTITUTE
19
Using EBL to improve plan quality


Given: planning domain, evaluation function
planner’s plan, a better plan
Learn: control knowledge to produce the better plan

Explanation used: explain why the alternative plan is
better

Target concept: control rules that make choices
based on the planner state and meta-state
USC INFORMATION SCIENCES INSTITUTE
20
Architecture of Quality system
USC INFORMATION SCIENCES INSTITUTE
21
Explaining better plans recursively:
target concept: shared subgoal
USC INFORMATION SCIENCES INSTITUTE
22
Hamlet: blame assignment
USC INFORMATION SCIENCES INSTITUTE
23
Probabilistic planning
USC INFORMATION SCIENCES INSTITUTE
24
Sources of uncertainty

Incomplete knowledge of the world (uncertain initial
state)

Non-deterministic effects of actions

Effects of external agents or state dynamics.
USC INFORMATION SCIENCES INSTITUTE
25
Dealing with uncertainty:
re-planning and conditional planning

Re-planning:





Deal with contingencies (plans for bad outcomes) at planning
time, before they occur.



Make a plan assuming nothing bad will happen
Build a new plan if a problem is found
(either re-plan to the goal state or try to repair the plan)
In some cases, this is too late.
Can’t plan for every contingency, so need to prioritize
Implies sensing
Build a plan that reduces the number of contingencies requires
(conformant planning)

May not be possible
USC INFORMATION SCIENCES INSTITUTE
26
A Buridan plan based on SNLP
USC INFORMATION SCIENCES INSTITUTE
27
Computing the probability of success
2: Bayes nets
Time-stamped literal node
Action outcome node
What is the
worst-case
time
complexity
of this
algorithm?
USC INFORMATION SCIENCES INSTITUTE
28
MAXPLAN



Inspired by SATPLAN. Compile planning problem to
an instance of E-MAJSAT
E-MAJSAT: given a boolean formula with variables
that are either choice variables or chance variables,
find an assignment to the choice variables that
maximizes the probability that the formula is true.
Choice variables: we can control them


Chance variables: we cannot control them


e.g. which action to use
e.g. the weather, the outcome of each action, ..
Then use standard algorithm to compute and
maximize probability of success
USC INFORMATION SCIENCES INSTITUTE
29
Probabilistic planning:
exogenous events
USC INFORMATION SCIENCES INSTITUTE
30
Representing external sources of change
Model actions that external agents can take in the same
way as actions that the planner can take.
(event oil-spills
(probability 0.1)
(preconds
(and (oil-in-tanker <sea-sector>)
(poor-weather <sea-sector>)))
(effects
(del (oil-in-tanker <sea-sector>))
(add (oil-in-sea <sea-sector>))))
USC INFORMATION SCIENCES INSTITUTE
31
Computing the probability of success
using a Bayes net
USC INFORMATION SCIENCES INSTITUTE
32
Example: the weather events and the
corresponding markov chain



The markov chain shows possible states
independent of time.
As long as transition probabilities are independent of
time, the probability of the state at some future time t
can be computed in logarithmic time complexity in t.
The computation time is polynomial in the number of
states in the markov chain.
USC INFORMATION SCIENCES INSTITUTE
33
The event graph


Captures the dependencies between events needed
to build small but correct markov chains.
Any event whose literals should be included will be
an ancestor of the events governing objective literals.
USC INFORMATION SCIENCES INSTITUTE
34
Probabilistic planning:
structured policy iteration
USC INFORMATION SCIENCES INSTITUTE
35
Craig Boutilier
Structured representation

States decomposable into state variables

Structured representations the norm in AI




STRIPS, Sit-Calc., Bayesian networks, etc.
Describe how actions affect/depend on features
Natural, concise, can be exploited computationally
Same ideas can be used for MDPs



actions, rewards, policies, value functions, etc.
dynamic Bayes nets [DeanKanazawa89,BouDeaGol95]
decision trees and diagrams [BouDeaGol95,Hoeyetal99]
USC INFORMATION SCIENCES INSTITUTE
36
Craig Boutilier
Action Representation – DBN/ADD
USC INFORMATION SCIENCES INSTITUTE
37
Craig Boutilier
Structured Policy and Value Function
USC INFORMATION SCIENCES INSTITUTE
38