Sparse Additive Subspace Clustering
-- A Nonparametric Approach for Subspace Clustering
Xiao-Tong Yuan
Nanjing University of Information Science and Technology
NLPR, CASIA
2014. 08.26
Less Structured Clustering …
Some popular clustering algorithms
•
•
•
•
•
•
Kmeans
Mean-Shift (mode-seeking)
Mixture models (e.g., GMM)
Hierarchical methods
…
Spectral clustering
Subspace Clustering
Data generation: Sampling
Segmentation/Clustering
Spectral Clustering
Affinity matrix W
Laplacian matrix L
Ideal W shall be block-diagonal
Eigen-analysis
Learning the Affinity Matrix
?
Self-Representation
Model:
?
?
.
.
.
?
?
?
?
.
.
.
?
…
…
.
.
.
…
?
?
?
.
.
.
?
Additive Self-Expressive Model
f
f
?
?
?
?
?
.
.
.
?
?
?
.
.
.
?
?
?
.
.
.
?
Basic idea
There exists an unknown function for each data point such that
the transformed data point lies near a subspace
Identifiability
Identifiability: As both qij and f are unknown , the model is
i
generally unidentifiable.
q
Restrict the solution space: fi (y i ) =
n
е
a il j
il (y i )
l= 1
where {j il (y i ) }i = 1 are a set of basis functions, e.g., Fourier basis
functions or polynomial basis functions.
Re-parametrization
The additive self-expressive model can be re-parametrized in
terms of the basis functions:
It is expected that the parameters b jtil exhibit group-level sparsity in
terms of the groups defined over the q bases.
Parameter Estimation
A convex formulation: group Lasso
Strongly convex with a unique global solution
Optimization: proximal gradient descent
Spectral Clustering
Similarity matrix W
= (wij )
Construct clusters by applying spectral clustering
algorithms to the affinity matrix
Algorithm
Analysis
Additive Subspace Detection Property
Let W be the constructed similarity matrix. We say the additive
subspace detection property holds if
(1) for all wij № 0, x i and x j belong to the same subspace
(2) For all i, the entries {wij }j №i are not all zero
Main result
If the K-subspaces respectively spanned by the basis functions
are weakly correlated to each other, and the regularization
parameter is well bounded from both sides, then the additive
subspace detection property holds.
Monte-Carlo Simulation
5 overlapping subspaces of dimensionality 50
20 data vectors are sampled from each subspace by
The observed samples are generated by
Monte-Carlo Simulation
Motion Segmentation
Grouping of motion trajectories according to motion patterns.
Motion Segmentation
Results on Hopkins 155 data
Motion Capture
CMU motion capture data
Summary
1. Model: A non-parametric extension for sparse subspace
clustering
2. Merit: Capture complex perturbations beyond additive
random noises in the observed data
3. Guarantee: Under mild conditions, SASC is able to
successfully recover the underlying subspace structure.
4. Applications in motion data clustering: outperforms the
state-of-the-art methods
References
• X.-T. Yuan, P. Li. Sparse Additive Subspace Clustering. ECCV, 2014.
• E. Elhamifar and R. Vidal. Sparse Subspace Clustering. CVPR, 2009.
• R. Vidal. A Tutorial on Subspace Clustering. IEEE Signal Processing
Magazine, 2010.
Thank you!
Questions?