Ghent University
An overview of Reservoir Computing:
theory, applications and implementations
Benjamin Schrauwen
David Verstraeten and Jan Van Campenhout
Electronics and Information Systems Department
Ghent University – Belgium
April 27 2007 – ESANN
Intro
•
In ML and pattern recognition: mostly FF structures
•NN,
Bayesian models, kernel methods …
•Well
•
understood, non-temporal
Many applications: temporal domain
•Time
series prediction
•Financial
data
•Dynamic
systems and control
•Robotics
•Vision,
speech,…
•
Takens’ theorem: explicit embedding
•
Or introducing recurrence: loopy belief propagation, RNN
An overview of Reservoir Computing: theory, applications and implementations
ESANN - April 27 2007
2/19
Intro
•
Recurrent Neural Networks:
•Hopfield
(1982):
•Specific
topologies with symmetric weights
•Information
•Werbos
(1974):
•BackProp
•Problem
•Few
Through Time (and all its improvements)
of fading gradient, mathematically difficult
applications, difficult to master
•Special
•
stored in attractors
topologies: LSTM (Schmidhuber)
RNN are universal approximators (ESANN special session 2005)
An overview of Reservoir Computing: theory, applications and implementations
ESANN - April 27 2007
3/19
Intro
•
Recurrent structures without the training: Reservoir Computing
•Early
related work by Buonomano (1995) and Laurenco (1994)
•Independently
•Jaeger
(2001): Echo State Networks (engineering)
•Maass
(2002): Liquid State Machines (neuroscience)
•Shortly
afterwards:
•Steil
•
discovered:
(2003): weight dynamics of Atiya-Parlos equivalent
RC:
•Fixed
(random) topology operated in correct dynamic regime
•Linear
“readout” function which is trained
An overview of Reservoir Computing: theory, applications and implementations
ESANN - April 27 2007
4/19
Reservoir Computing
An overview of Reservoir Computing: theory, applications and implementations
ESANN - April 27 2007
5/19
Reservoir Computing
•
•
•
Properties of reservoir:
•
Exact topology, connectivity, weights: not important
•
Has to have fading memory: when not chaotic
•
Longest memory if at the edge of stability: memory = nr. nodes
•
Reservoir size can be large: no over-fitting
Training with linear regression (pseudo-inv, ridge regression):
•
No local minima, no problems with recurrent structure, one shot
learning
•
Can do regression, classification, prediction
On-line learning also possible with LMS and RLS
An overview of Reservoir Computing: theory, applications and implementations
ESANN - April 27 2007
6/19
Reservoir Computing
• RC
does on-line computing: prediction at every time-step
• Theoretically:
•
Any time-invariant filter with fading memory can be learned
•
But: unable to implement generic FSMs
• Recently
Maass (2006): when adding output feedback
•
Also non-fading memory filters: generic FSMs
•
Ability to simulate any n-th order dynamical system
•
Turing universal
An overview of Reservoir Computing: theory, applications and implementations
ESANN - April 27 2007
7/19
RC: tone generation example
Taken from H. Jaeger
An overview of Reservoir Computing: theory, applications and implementations
ESANN - April 27 2007
8/19
Usual setup and training
•
Create random weight matrices
•
Rescale reservoir weights so that max absolute eigenvalue
close to one (edge of stability)
•
Excite reservoir with input and record all states
•
Train readouts by minimizing (Aw-b)2
time
space
A
w
B
An overview of Reservoir Computing: theory, applications and implementations
ESANN - April 27 2007
9/19
error
error
Influence of parameters
dynamic regime
reservoir size
error
Not very important:
•Connection fraction
•Exact topology
•Weight distribution
timescale
An overview of Reservoir Computing: theory, applications and implementations
ESANN - April 27 2007
10/19
A useful ESANN analogy
Compute output
Input
Reservoir state
Reservoir
2008
An overview of Reservoir Computing: theory, applications and implementations
ESANN - April 27 2007
11/19
State space view
An overview of Reservoir Computing: theory, applications and implementations
ESANN - April 27 2007
12/19
Link to kernel machines
y
*●
*
*
*
●
•
●
●* ●
*
* ●
x
Kernel
y'
Kernel: Projection of input space into highdimensional feature space.
●
●
•
Conventional methods rely on 'kernel trick' to
avoid explicitly going to feature space.
Reservoir computing works in feature space,
but reservoir state contains temporal
information.
Temporal to spatial transformation
● ●
●
●z' ●
●
*
* * *
*
*
*
x'
An overview of Reservoir Computing: theory, applications and implementations
ESANN - April 27 2007
13/19
Link to FSMs
FSM
RC
RC
with output feedback
An overview of Reservoir Computing: theory, applications and implementations
ESANN - April 27 2007
14/19
RC: Applications
•
Chaotic time series prediction: order
of magnitude better than SOA
•
Speech recognition on small
vocabulary: outperform HMM-based
recognizer (Sphinx)
•
Digits recognition: better than SOA
•
Robot control
•
System identification
•
Noise removal/modelling
•
…
An overview of Reservoir Computing: theory, applications and implementations
ESANN - April 27 2007
15/19
Larger example: speech
Speech
Pre-processing
t
Reservoir
Readout
Post-processing
Σ
Mean
t
An overview of Reservoir Computing: theory, applications and implementations
ESANN - April 27 2007
...
Σ
...
...
Downsampling
Reservoir state
WTA
'6'
Mean
t
16/19
RC: novel computing paradigm
•
RC presents a novel way of looking at computation
•
“Random” dynamic systems can be used by only training a linear
readout layer
•
RC already used to show general computing capabilities of:
•
•
Microcolumn structure in the cortex
•
Gene regulatory network
•
The visual cortex of a real cat
Implementation:
•
Toolbox (freely available at http://www.elis.ugent.be/rct)
•
“Bucket of water”, aVLSI, digital hardware
•
Photonics (in progress)
An overview of Reservoir Computing: theory, applications and implementations
ESANN - April 27 2007
17/19
Current research topics
•Theoretical:
•Proper
understanding of importance of dynamics
•Regularisation
•Reservoir
optimisation
•Intrinsic
plasticity: unsupervised reservoir adaptation based on
infomax to set dynamic regime
•Timescales
•Modular
•Generic
reservoirs
reservoir idea
•Applications
An overview of Reservoir Computing: theory, applications and implementations
ESANN - April 27 2007
18/19
This session
•Reservoir
optimisation
•Intrinsic
plasticity: Steil, Verstraeten et al., Wardermann et al.
•Reservoir
•Alternate
•Gao
pruning: Dutoit et al.
reservoir ideas
et al.
•Lourenco
An overview of Reservoir Computing: theory, applications and implementations
ESANN - April 27 2007
19/19