Deep Symmetry Networks
Robert Gens
Pedro Domingos
University of Washington
NIPS, 2014
Motivation
Get rid of unimportant variations
Make important ones easy to detect
SYMMETRY: Transformation of an object that
maps the object to itself.
Groups
A group is a set of elements together with an
operation that satisfies four axioms, namely
closure, associativity, identity and invertibility.
E.g.: Reals with addition
Symmetry groups
Elements: functions
Operator: function composition
For e.g.: affine transformation functions such as
scale, rotation, translation, shear
Convolutional Neural Nets
Feature Map = Feature Function + Translations
Deep Symmetry Nets
Feature Map = Feature Function + Symmetry
Groups
CNNs - DSNs relationship
Deep Affine Networks
Symmetry Group is Affine.
Curse of Dimensionality
6 dimensions in affine space
10 points along each axis
106 points in affine space
Making DSNs tractable
Compute feature maps at N control points.
How? Random is okay. Better to select local
optima where neurons are expected to strongly
fire
Finally, Gaussian interpolation for pooling
Experiments - MNIST rot, 1 layer
Experiments - NORB, 2 layers
Food for Thought - Semantic Parsing
A symmetry of a sentence is a syntactic
transformation that preserves its meaning.
E.g.: synonyms, active/passive
Application - Question Answering
Bill wore shades.
William donned glasses.
Sunglasses were donned by William.
Were shades donned by Bill?