Input and Output
Thanks: I. Witten and E. Frank
1
The weather problem

Conditions for playing an outdoor game
Outlook
Temperature
Humidity
Windy
Play
Sunny
Hot
High
False
No
Sunny
Hot
High
True
No
Overcast
Hot
High
False
Yes
Rainy
Mild
Normal
False
Yes
…
…
…
…
…
If outlook = sunny and humidity = high then play = no
If outlook = rainy and windy = true then play = no
If outlook = overcast then play = yes
If humidity = normal then play = yes
If none of the above then play = yes
2
Classification vs. Association
rules

Classification rule: predicts value of prespecified attribute (the classification of an
example)
If outlook = sunny and humidity = high then play = no

Associations rule: predicts value of arbitrary
attribute or combination of attributes
If temperature = cool then humidity = normal
If humidity = normal and windy = false then play = yes
If outlook = sunny and play = no then humidity = high
If windy = false and play = no
then outlook = sunny and humidity = high
3
Weather data with mixed
attributes

Two attributes with numeric values
Outlook
Temperature
Humidity
Windy
Play
Sunny
85
85
False
No
Sunny
80
90
True
No
Overcast
83
86
False
Yes
Rainy
75
80
False
Yes
…
…
…
…
…
If
If
If
If
If
outlook = sunny and humidity > 83 then play = no
outlook = rainy and windy = true then play = no
outlook = overcast then play = yes
humidity < 85 then play = yes
none of the above then play = yes
4
The contact lenses data
Age
Spectacle prescription
Astigmatism
Tear production rate
Young
Young
Young
Young
Young
Young
Young
Young
Pre-presbyopic
Pre-presbyopic
Pre-presbyopic
Pre-presbyopic
Pre-presbyopic
Pre-presbyopic
Pre-presbyopic
Pre-presbyopic
Presbyopic
Presbyopic
Presbyopic
Presbyopic
Presbyopic
Presbyopic
Presbyopic
Presbyopic
Myope
Myope
Myope
Myope
Hypermetrope
Hypermetrope
Hypermetrope
Hypermetrope
Myope
Myope
Myope
Myope
Hypermetrope
Hypermetrope
Hypermetrope
Hypermetrope
Myope
Myope
Myope
Myope
Hypermetrope
Hypermetrope
Hypermetrope
Hypermetrope
No
No
Yes
Yes
No
No
Yes
Yes
No
No
Yes
Yes
No
No
Yes
Yes
No
No
Yes
Yes
No
No
Yes
Yes
Reduced
Normal
Reduced
Normal
Reduced
Normal
Reduced
Normal
Reduced
Normal
Reduced
Normal
Reduced
Normal
Reduced
Normal
Reduced
Normal
Reduced
Normal
Reduced
Normal
Reduced
Normal
Recommended
lenses
None
Soft
None
Hard
None
Soft
None
hard
None
Soft
None
Hard
None
Soft
None
None
None
None
None
Hard
None
Soft
None
None
5
A complete and correct rule
set
If tear production rate = reduced then recommendation = none
If age = young and astigmatic = no and tear production rate = normal
then recommendation = soft
If age = pre-presbyopic and astigmatic = no and
tear production rate = normal then recommendation = soft
If age = presbyopic and spectacle prescription = myope and
astigmatic = no then recommendation = none
If spectacle prescription = hypermetrope and astigmatic = no and
tear production rate = normal then recommendation = soft
If spectacle prescription = myope and astigmatic = yes and
tear production rate = normal then recommendation = hard
If age young and astigmatic = yes and tear production rate = normal
then recommendation = hard
If age = pre-presbyopic and spectacle prescription = hypermetrope
and astigmatic = yes then recommendation = none
If age = presbyopic and spectacle prescription = hypermetrope and
astigmatic = yes then recommendation = none
6
A decision tree for this
problem
7
Predicting CPU performance
Cycle time
(ns)
Main memory
(Kb)
Cache
(Kb)
Channels
Performance
MYCT
MMIN
MMAX
CACH
CHMIN
CHMAX
PRP
1
125
256
6000
256
16
128
198
2
29
8000
32000
32
8
32
269
208
480
512
8000
32
0
0
67
209
480
1000
4000
0
0
0
45
…
PRP = -55.9 + 0.0489 MYCT + 0.0153 MMIN + 0.0056 MMAX
+ 0.6410 CACH - 0.2700 CHMIN + 1.480 CHMAX
8
Data from labor negotiations
Attribute
Duration
Wage increase first year
Wage increase second year
Wage increase third year
Cost of living adjustment
Working hours per week
Pension
Standby pay
Shift-work supplement
Education allowance
Statutory holidays
Vacation
Long-term disability assistance
Dental plan contribution
Bereavement assistance
Health plan contribution
Acceptability of contract
Type
(Number of years)
Percentage
Percentage
Percentage
{none,tcf,tc}
(Number of hours)
{none,ret-allw, empl-cntr}
Percentage
Percentage
{yes,no}
(Number of days)
{below-avg,avg,gen}
{yes,no}
{none,half,full}
{yes,no}
{none,half,full}
{good,bad}
1
1
2%
?
?
none
28
none
?
?
yes
11
avg
no
none
no
none
bad
2
2
4%
5%
?
tcf
35
?
13%
5%
?
15
gen
?
?
?
?
good
3
3
4.3%
4.4%
?
?
38
?
?
4%
?
12
gen
?
full
?
full
good
…
40
2
4.5
4.0
?
none
40
?
?
4
?
12
avg
yes
full
yes
half
good
9
Decision trees for the labor
data
10
Instance-based representation

Simplest form of learning: rote learning






Training instances are searched for instance that
most closely resembles new instance
The instances themselves represent the
knowledge
Also called instance-based learning
Similarity function defines what’s “learned”
Instance-based learning is lazy learning
Methods: nearest-neighbor, k-nearest-
neighbor, …
11
Learning prototypes/Case
Based Reasoning

Only those instances involved in a
decision need to be stored
12
Representing clusters I
Simple 2-D representation
d
Venn diagram
d
e
a
h
k
f
g
a
c
j
i
e
b
c
j
h
k
f
g
b
i
Overlapping clusters
13
Representing clusters II
Probabilistic assignment
a
b
c
d
e
f
g
h
…
1
2
3
0.4
0.1
0.3
0.1
0.4
0.1
0.7
0.5
0.1
0.8
0.3
0.1
0.2
0.4
0.2
0.4
0.5
0.1
0.4
0.8
0.4
0.5
0.1
0.1
Dendrogram
g a c i e d k b j f h
NB: dendron is the Greek
word for tree
14