S. Mandayam/ ANN/ECE Dept./Rowan University
Artificial Neural Networks
ECE.09.454/ECE.09.560
Fall 2008
Lecture 11
November 17, 2008
Shreekanth Mandayam
ECE Department
Rowan University
http://engineering.rowan.edu/~shreek/fall08/ann/
S. Mandayam/ ANN/ECE Dept./Rowan University
Plan
• ANN Pre-processing
• Feature Extraction
• Approximation Theory
• Universal approximation
• Final Project Discussion
S. Mandayam/ ANN/ECE Dept./Rowan University
Feature Extraction
Objective:
• Increase information content
• Decrease vector length
• Parametric invariance
• Invariance by structure
• Invariance by training
• Invariance by transformation
S. Mandayam/ ANN/ECE Dept./Rowan University
Approximation Theory:
Distance Measures
• Supremum Norm
• Infimum Norm
• Mean Squared Norm
S. Mandayam/ ANN/ECE Dept./Rowan University
Recall: Metric Space
• Reflexivity
• Positivity
• Symmetry
• Triangle Inequality
S. Mandayam/ ANN/ECE Dept./Rowan University
Approximation Theory:
Terminology
• Compactness
K
• Closure
F
S. Mandayam/ ANN/ECE Dept./Rowan University
Approximation Theory:
Terminology
• Best Approximation
E
M
min
f
u0
• Existence Set
M
u0
E
min ALL
f
S. Mandayam/ ANN/ECE Dept./Rowan University
Approximation Theory:
Terminology
• Denseness
F
f
e
g
S. Mandayam/ ANN/ECE Dept./Rowan University
Fundamental Problem
E
M
min
?
g
• Theorem 1: Every compact set is an
existence set (Cheney)
• Theorem 2: Every existence set is a
closed set (Braess)
S. Mandayam/ ANN/ECE Dept./Rowan University
Stone-Weierstrass Theorem
F
f
• Identity
x
• Separability
e
g
1
x1
f(x1)
x2
f(x2)
• Algebraic Closure
F
af+bg
S. Mandayam/ ANN/ECE Dept./Rowan University
Final Project Discussion