CSE 4705
Artificial Intelligence
Jinbo Bi
Department of Computer Science & Engineering
http://www.engr.uconn.edu/~jinbo
1
Machine learning (2)
Cluster analysis
– hierarchical clustering
2
Types of hierarchical clustering
Two main types of hierarchical clustering
Agglomerative:
Start with the points as individual clusters
At each step, merge the closest pair of clusters until only one
cluster (or k clusters) left
Divisive:
Start with one, all-inclusive cluster
At each step, split a cluster until each cluster contains a point
(or there are k clusters)
Traditional hierarchical algorithms use a similarity or
distance matrix
Merge or split one cluster at a time
3
Divisive hierarchical clustering
Start with all points in a single cluster
Create subsequent clusters by splitting some
clusters
Find a cluster and bi-section that cluster
Build a spanning tree which gives a tree structure that
links all data points (a connected component that has
all points)
Then find an edge in the tree to break so a connected
component is broken into two connected components
4
Divisive hierarchical clustering
Build MST (Minimum Spanning Tree)
Start with a tree that consists of any point
In successive steps, look for the closest pair of points
(p, q) such that one point (p) is in the current tree
but the other (q) is not
Add q to the tree and put an edge between p and q
5
Divisive hierarchical clustering
Use MST for constructing hierarchy of clusters
6
Divisive hierarchical clustering
Example
0.8
X4
0.7
X3
0.6
0.5
0.4
X2
0.3
X1
0.2
0.1
0
0.1
X5
0.2
0.3
0.4
0.5
0.6
0.7
0.8
7
Divisive hierarchical clustering
Example
0.8
X4
0.7
X3
0.6
0.5
0.4
X2
0.3
X1
0.2
0.1
0
0.1
X5
0.2
0.3
0.4
0.5
0.6
0.7
0.8
8
Divisive hierarchical clustering
Example
0.8
X4
0.7
X3
0.6
0.5
0.4
X2
0.3
X1
0.2
0.1
0
0.1
X5
0.2
0.3
0.4
0.5
0.6
0.7
0.8
9
Divisive hierarchical clustering
Example
0.8
X4
0.7
X3
0.6
0.5
0.4
X2
0.3
X1
0.2
0.1
0
0.1
X5
0.2
0.3
0.4
0.5
0.6
0.7
0.8
10
Divisive hierarchical clustering
Example
0.8
X4
0.7
X3
0.6
0.5
0.4
X2
0.3
X1
0.2
0.1
0
0.1
X5
0.2
0.3
0.4
0.5
0.6
0.7
0.8
11
Density-based clustering (DBSCAN)
DBSCAN is a density-based algorithm.
Density = number of points within a specified
radius (Eps)
A point is a core point if it has more than a
specified number of points (MinPts) within Eps
These are points that are at the interior of a
cluster
A border point has equal or fewer than MinPts
within Eps, but is in the neighborhood of a core
point
A noise point is any point that is not a core point
or a border point.
12
Density-based clustering (DBSCAN)
DBSCAN is a density-based algorithm.
Density = number of points within a specified
radius (Eps)
A point is a core point if it has more than a
specified number of points (MinPts) within Eps
These are points that are at the interior of a
cluster
A border point has equal or fewer than MinPts
within Eps, but is in the neighborhood of a core
point
A noise point is any point that is not a core point
or a border point.
13
DBSCAN: core, border, and noise points
14
DBSCAN algorithm
Eliminate noise points
Perform clustering on the remaining points
0
15
DBSCAN algorithm
Eliminate noise points
Perform clustering on the remaining points
0
16
DBSCAN (more examples)
Original Points
Point types: core,
border and noise
Eps = 10, MinPts = 4
17
When DBSCAN works well
Original Points
Clusters
• Resistant to Noise
• Can handle clusters of different shapes and sizes
When DBSCAN does not work well
(MinPts=4, Eps=9.75).
Original Points
• Varying densities
• High-dimensional data
(MinPts=4, Eps=9.92)
When DBSCAN does not work well
(MinPts=4, Eps=9.75).
Original Points
• Varying densities
• High-dimensional data
(MinPts=4, Eps=9.92)
DBSCAN: determining EPS and MinPts
Idea is that for points in a cluster, their kth nearest
neighbors are at roughly the same distance
Noise points have the kth nearest neighbor at farther
distance
So, plot sorted distance of every point to its kth
nearest neighbor
DBSCAN: in-class practice
EPS
MinPts = 3
Questions?
23