Policy 2004
June 7-9
Yorktown Heights, NY
An AI Perspective
on Autonomic
Computing Policies
Jeffrey O. Kephart and William E. Walsh
IBM T.J. Watson Research Center
{kephart, wwalsh1}@us.ibm.com
© 2004 IBM Corporation
Policy 2004
Autonomic Computing, Policy, and AI
Autonomic Computing
Self-managing: configuration,
optimization, healing, protection
•Don’t want all behavior
hard-coded
•High-level description
of how to self-manage
Policy
formal behavioral guide
2
Unified Framework
•Rationality as guide
in designing policies
• Action
• Goal
• Utility Function
•Automated decision making
•Rational self-management
Artificial Intelligence
design of rational agents
•Perceives and acts upon environment
•Makes the “right” (best/optimal)
decisions
•with respect to objective
•based on knowledge
© 2004 IBM Corporation
Policy 2004
Some Types of Policy

Action Policy
Specifies action a that should be
taken in current state S
IF(Condition) THEN (Action)
Condition specifies state or set of
states
Objective:
•Just apply policy
•Resulting state not explicit
Knowledge:
•Current state S
•Action to take a
Possible
State
s1
a1
a2
Current
State
S
a3
Possible
State
s2
Possible
State
s3
Policy author (human or computer)
knows exactly what should be done
Rationality is compiled into the policy
3
© 2004 IBM Corporation
Policy 2004
Policy Sets

Set of action policies:
 Conditions fully cover the state space
 Each state is mapped to a unique action
No policy conflicts
Necessary for
 Single coherent, consistent mapping
rational behavior
from state to action
 Policy sets can be hard for humans
to understand
 Need automated tools to assist in creation and analysis
of policy sets

4
© 2004 IBM Corporation
Policy 2004
Some Types of Policy

Action Policy

Goal Policy
Specifies desired resulting state  or
criteria for set of states
•Any member of desired states
acceptable
System must compute action
a: S →
Objective: Desired state 
Knowledge
•Current state S
•System model: (S, a)
Rational behavior is generated by
optimizer/planner
5
Possible
State
s1
a1
Possible
State
s2
a2
Current
State
S
a3
Possible
State
s3
(S, a)
Compare to action policies:
•What we want, rather than what to do
•Higher-level
•More flexible
•Requires sophisticated models,
optimization/planning algorithms
© 2004 IBM Corporation
Policy 2004
Some Types of Policy

Action Policy
Goal Policy

Utility Function Policy

Function assigns a single real value
to each resulting state
U() → 
Tradeoffs directly encoded, thus no
conflicts
System must compute optimal action
Objective: Maximize U()
Knowledge
•Current state S
•System model: (S, a)
Rational behavior is generated by
optimizer/planner
6
Possible
State
U(s1)
a1
Possible
State
U(s2 )
a2
Current
State
S
a3
(S, a)
Possible
State
U(s3)
Compare to other policy types:
•High-level & flexible (like Goal)
•Range of state values (rather than
binary Goal classification)
•Strict generalization of Goal
•No conflicts (like Action and Goal)
•Utility elicitation can be hard!
© 2004 IBM Corporation
Policy 2004
Unified Framework
Coexistence of different policy types requires us to
understand relationships between policy types
Modeling,
Optimization
Actions
Modeling,
Generative
Planning
Goals
Feasible States,
Optimization
Utility Functions
Higher-level behavioral specifications
7
© 2004 IBM Corporation
Policy 2004
Policy A Data Center Model
Resource
Arbiter
Application
Manager
Router
Servers
Servers
Servers
Application Environment 1
8
Application
Manager
Router
Servers
Servers
Servers
Application Environment 2
© 2004 IBM Corporation
Policy 2004
Decision Making in an Application
Environment
Policy
Application
Manager
Gold
Transactions
Router
Silver
Transactions
9
Servers
Servers
Servers
Objective: “Good” Gold & Silver
performance
Decision:
Set control parameters affecting
proportion of CPU allocated to
Gold and Silver
Application Environment i
© 2004 IBM Corporation
Policy 2004
Action Policy Set for Decision Making in
Application Manager
Gold-Silver Action Policies
G: IF (RTG > 100 msec)
THEN (Increase CPUG by 5%)
S: IF (RTS > 200 msec)
THEN (Increase CPUS by 5%)
Overlapping Action Policies
Conflict if CPU (almost) fully
utilized!
10
© 2004 IBM Corporation
Policy 2004
Conflict Resolution: Extending the State Space


Policy precondition also
includes CPU utilization
Two techniques:
Revise original policies:
Silver policy only applicable at
≤ 90% utilization OR if RTG ≤
100 msec
Meta-policy:
IGNORE Silver action if > 90%
utilization and RTG > 100 msec


11
Potential explosion of # of
policies in number of classes
and description of state
Finer-grain control leads to a
bigger explosion
CPUG + 5%
Only
CPUG + 5%
Only
CPUG + 5%
Only
Apparent
simplicity of
Action Policies
is deceptive
Policy sets may
need to be very
complex!
© 2004 IBM Corporation
Policy 2004
Conflict Resolution: Priorities
G: IF (RTG > 100 msec)
THEN (Increase CPUG by 5%)
: Priority = 10
S: IF (RTS > 200 msec)
THEN (Increase CPUS by 5%)
: Priority = 5
 No explosion of state space

3%
2%
Do we always want to strictly
favor Gold over Silver?
Should we compromise?
What if we can make Silver a
lot better by making Gold a little
worse?
12
© 2004 IBM Corporation
Policy 2004
Goal Policies
It’s all bad!
What to do?
Gold-Silver Tradeoff
G: RTG < 100 msec
S: RTS < 200 msec
1
Performance model:
(e.g., M/M/1 Queue)
  
T ( , C )
Gives feasible response time curves
It’s all good! Conflict:
What is best? Gold/Silver Tradeoff
What to do?
13
© 2004 IBM Corporation
Policy 2004
Resolving Conflicts in Goal Policies
Priorities
G : RTG > 100 msec
: Priority 10
S: RTS > 200 msec
: Priority 5
B: RTB > 250 msec
: Priority 3
Typical priority semantics:
1. Satisfy top priority goal (if feasible)
2. Satisfy second priority goal (if feasible)
....
N. Satisfy Nth priority goal (if feasible)
Do we always want to satisfy
Gold at all expense?
•Better to partially satisfy all classes?
•Better to satisfy both Silver and Bronze
at expense of Gold?
Simple goals and priorities provide a limited language


14
Could enumerate compound goals with associated priorities
Inefficient way to compose a utility function!
© 2004 IBM Corporation
Policy 2004
Utility Function Policies
U(RTG, RTS) = UG(RTG) + US(RTS)
Gold & Silver Utility Functions
Utility
UG(RTG)
US(RTS)
Response Time (ms)




15
Utility
Silver
Response
Time (ms)
Gold
Response
Time (ms)
States have real value, rather than binary good/bad classification
Map all states of interest in to single unique value
Tradeoffs directly encoded
No conflicts!
© 2004 IBM Corporation
Policy 2004
Using Utility Policies
  
T ( , C )
Compute (CG, CS) that maximizes
U(RTG, RTS) in feasible region
Silver RT
Performance model:
Feasibility Curves Overlayed
on Utility Topography
10
5
Higher Utility
50 Servers
Lower Utility
16
Gold RT
© 2004 IBM Corporation
Policy 2004
Mixing Policy Within a Component

Goals & Utility Functions
 Goals add constraints
 Constrained optimization problem
 Goal issues apply here:
• Failure to satisfy goals
• Goal conflicts

Action Policies and Goal/Utility
 Action conflicts hard to avoid: actions
explicit in Action Policy but arise from
optimization in Goal/Utility
Silver RT
 Naturally compatible: both deal with
desired state
1
5
10
50 Servers
RTG ≤ 200 msec Gold RT
 Reduce conflicts: apply to different
variables/domains
• But indirect interactions could occur
17
© 2004 IBM Corporation
Policy 2004
Resource Allocation
Objective:
Obtain best tradeoff between
needs of Application
Environments
Resource
Arbiter
Decision problem:
Choose allocation to Application
Environments
Policy
Resource requests:
Specify resources
needed to achieve
application objectives
Policy
Router
Application
Manager
Servers
Servers
Servers
Application Environment 1
18
Policy
Router
Application
Manager
Servers
Servers
Servers
Application Environment 2
© 2004 IBM Corporation
Policy 2004
Mixing Policies Across Components
Request Resource (AM)

Compatible
Protocols
Action:
Allocate Resources (Arbiter)

IF (CPU Overutilized) THEN
(Request 1 more server)

Goal:
C*
Compute
needed to meet all
goals.
IF C* ≥ C0 THEN (Request C*−
C0 more servers).

Utility Function:
Send U(Cr) to Arbiter.
For each Cr, U(Cr) is max
utility possible with Cr servers.
19
Action:
Arb1: IF (AMA requests n servers)
THEN (Grant) : Priority = 10
Arb2: IF (AMB requests n servers)
THEN (Grant) : Priority = 8

Goal:
Ensure that at least k fraction of AMs
satisfied.

Utility Function:
Maximize sum of utility over all
Application Managers.
© 2004 IBM Corporation
Policy 2004
Conclusions

Introduced and critiqued 3 types of policy

Policy Sets required to ensure rational behavior


Unified framework provides principled way to
combine different policy types in the same system
All 3 policy types likely to co-exist for a long time
 Goals, utility functions will become more prevalent
• Greater need: increasingly complex systems
• Better technology: planning, modeling, optimization
20
© 2004 IBM Corporation