RC tutorial
David Verstraeten
RCT tutorial
1
Reservoir Computing
Random and fixed
Trained
RCT tutorial
2
Steps in training and testing
• Create random weight matrices
– Input to reservoir
– Reservoir to reservoir
– (Output to reservoir)
•
•
•
•
Scale weight matrices globally to the desired value
Feed all input samples to the reservoir
Collect reservoir states
Train readout weight using linear regression or
ridge regression
RCT tutorial
3
Scaling reservoir matrix:
Spectral radius
LTI system
x[k 1]  Ax[k]  Bu[k 1]
y[k 1]  Cx[k 1]  Du[k 1]
RC system
res
in
x[k 1]  f W res
x[k]  W res
u[k 1]
out
in
y[k 1]  W out
x[k 1]  W out
u[k 1]


res
Spectral radius   max eig W res
RCT tutorial

4
Training
Inputs
Reservoir
Outputs
M
N
P
T1
1
1
1
T2
2
2
2
A
B
…
…
…
S
S
S
TS
RCT tutorial
5
Linear/ridge regression
Linear regression
w opt  arg min Aw  b
w
T
1
2
A
wopt  (A A) A B
T
Ridge regression
w opt  arg min Aw  b   w
w
wopt  (AT A  I) 1 AT B
2
RCT tutorial
X
W
=
B
2
6
Pattern generation tasks: training
Fixed connections
Trained connections
RCT tutorial
7
Pattern generation:
generation/testing
Fixed connections
Warmup
Freerun
RCT tutorial
Trained connections
8
Leaky integrator neurons
RCT tutorial
9
Global optimization parameters
•
•
•
•
•
Input scaling
Reservoir scaling (spectral radius)
Leak rate
Feedback scaling
Regularization parameter (through crossvalidation)
• Not:
– connection fraction (only for spiking neurons)
– reservoir size
RCT tutorial
10
RC toolbox
RCT tutorial
11
Outline
•
•
•
•
•
Rationale, usage and structure
Topology
Datasets
Cross-validation
Parameter sweeps and custom scripts
RCT tutorial
12
Outline
•
•
•
•
•
Rationale, usage and structure
Topology
Datasets
Cross-validation
Parameter sweeps and custom scripts
RCT tutorial
13
Rationale
• Reservoir Computing toolbox:
• Box of tools for
– Quick-and-dirty experiments
– Large scale parameter sweeps and optimizations
• Modular setup: function hooks for
customization
– Reservoir types
– Learning rules
– Scoring and evaluation methods
RCT tutorial
14
Getting started
• Download from http://snn.elis.ugent.be/rct
• Unpack, go to directory $RCT_ROOT and
type:
>> install
• For ‘Hello world’ experiment, type
>> rc_simulate
RCT tutorial
15
Usage
• Default settings
– Are ok as starting point for many experiments
– Contained in
$RCT_ROOT/default_settings/default
_*.m
– Look at these scripts! They give an overview
of RCToolbox functionality
RCT tutorial
16
Custom settings
• Two ways of overriding defaults:
– Command line
>> dataset_generation_function=… @my_dataset_function();
>> rc_simulate;
– Custom configuration file
my_conf.m
•dataset_generation_function=@my_dataset_function();
•topology.layer(3).scaling = .1;
>> custom_configuration=‘my_conf’
>> rc_simulate;
– Example: analog_speech.m
• Other examples: $RCTROOT/examples/configurations
RCT tutorial
17
Flow diagram for basic
experiment
Dataset
Topology
Reservoir
Evaluation
• Generate dataset
• Generate topology
• Simulate reservoir
• Train readout layer and evaluate on test data
RCT tutorial
18
Structure of rc_simulate
rc_simulate
rc_simulate_job
generic_simulate
Generate
dataset
Setup
parameter
sweeps
Generate
topology
Simulate
reservoir
Construct state
and weight
matrices
Simulate
system
Training and
testing
RCT tutorial
19
Outline
•
•
•
•
•
Rationale, usage and structure
Topology
Datasets
Cross-validation
Parameter sweeps and custom scripts
RCT tutorial
20
Standard topology for
classification/regression
Input
Mandatory
Bias
Reservoir
Optional
Readout
RCT tutorial
21
Connectivity matrix between layers
Input
Bias
Reservoir
Readout
RCT tutorial
From/to
Input
Bias
Reservoir
Readout
Input
0
0
1
0/1
Bias
0
0
0/1
0/1
Reservoir
0
0
1
1
Readout
0
0
0
0
22
Standard topology for
classification/regression
Mandatory
Reservoir
Bias
Optional
Readout
RCT tutorial
23
Connectivity matrix between layers
Reservoir
Bias
Readout
RCT tutorial
From/to
Bias
Reservoir
Readout
Bias
0
0/1
0/1
Reservoir
0
1
1
Readout
0
1
0/1
24
Topology datastructure in the
toolbox
• topology.
– layer(1,n)
– conn(n,n)
– with n = number of layers
• Default settings:
–
–
–
–
–
$RCTROOT/default_settings/default_topology.m
Layer 1 : input
Layer 2 : bias
Layer 3 : reservoir
Layer 4 : readout
RCT tutorial
25
Topology datastructure in the
toolbox
• topology.layer(1:n).
– size: integer
– is_trained_offline: boolean
– is_trained_online: boolean
– node_type: string
– training_function:function pointer
– nonlin_functions: array of
function pointers
RCT tutorial
26
Topology datastructure in the
toolbox
• topology.layer(1:n).
– init_simulation: function pointer
– finish_simulation: function
pointer
– dt: float
– regul_param: float
– scoring: function pointer
RCT tutorial
27
Topology datastructure in the
toolbox
• topology.conn(1:n,1:n).
– is_active: boolean
– creation_pipeline: array of
function pointers
– scaling_factor: float
– conn_frac: float [0,1]
– discrete_set: float array
RCT tutorial
28
Topology creation pipeline
• Array of function pointers which pass
weight matrices
• Generation functions
– @gen_rand, @gen_1D, @gen_load
• Assignment functions
– @assign_rand, @assign_discrete
• Scaling functions
– @scale_specrad, @scale_constant
RCT tutorial
29
Topology creation pipeline
• Example (default for reservoir to reservoir
connection):
– topology.conn(3,3).creation_pipeline= {@gen_rand,
@assign_randn, @scale_specrad}
gen_rand
assign_randn
scale_specrad
• Can be customized! Function hooks
RCT tutorial
30
Outline
•
•
•
•
•
Rationale, usage and structure
Topology
Datasets
Cross-validation
Parameter sweeps and custom scripts
RCT tutorial
31
Dataset generation
• Function hook
• Default
– dataset_generation_function=@dataset_narma_10(input_dt,
input_maxtime, n_samples)
• Signature
– function [inputs outputs] = my_dataset_function(…)
– inputs = cell(1,nr_samples)
•
•
inputs{:} = matrix (M,:);
M = input dimensionality
– outputs = cell(1,nr_samples)
•
•
outputs{:} = matrix (P,:) or [] for signal generation tasks.
P = output dimensionality
• Example: 10th order NARMA
RCT tutorial
32
Data struct
• data.layer(1:n).
– r : required/teacher forced states
– s : simulated states
RCT tutorial
33
Outline
•
•
•
•
•
Rationale, usage and structure
Topology
Datasets
Cross-validation
Parameter sweeps and custom scripts
RCT tutorial
34
Training and testing
• cross-validate
– Training and testing using cross-validation
• cross_validate_grid
- Training and testing using cross-validation with
optimization of regularization parameter
RCT tutorial
35
Cross-validate
• Example: three-fold cross-validation with
three samples.
Training
Results
1
1
2
2
3
Testing
error fold
1
3
Fold 1
RCT tutorial
36
Cross-validate
• Example: three-fold cross-validation with
three samples.
Training
Results
1
1
2
3
3
Testing
error fold
1
error fold
2
2
Fold 2
RCT tutorial
37
Cross-validate
• Example: three-fold cross-validation with
three samples.
Training
Results
1
2
2
3
3
Testing
error fold
1
error fold
2
error fold
3
1
Fold 3
RCT tutorial
38
Cross-validation
• Function hook:
– train_params.cross_val_set_function
– Default: @random_cross_val
– Other possibilities:
•
•
•
cross_val_only_training: no test set
no_cross_val: simple train and test set
random_freerun_cross_val : cross-validation for freerun tasks
• Freerun cross-validation
Training
1
1
3
2
2
3 3 3
Testing
3
RCT tutorial
3
…
Fold 1
39
Cross-validate with gridsearch
• For optimization of regularization parameter 
1
Training
n21
1
2
2
Val.
results
12
Subfold 2
1
RCT tutorial
Results
Testing
error 1
error 2
Validation
3
…
error n
opt
3
error fold
1
error fold
2
error fold
3
Fold 3
1
2
40
Outline
•
•
•
•
•
Rationale, usage and structure
Topology
Datasets
Cross-validation
Parameter sweeps and custom scripts
RCT tutorial
41
Parameter sweeps
• parameter_ranges = struct( 'name', {}, 'range', {});
• Example
– 1D sweep:
parameter_ranges = struct(
'name', {‘topology.layer(3).size’},
'range’, {10:10:100});
– 2D sweep (all combinations!):
parameter_ranges = struct(
'name', {‘topology.layer(3).size’, ‘topology.layer(1).scale_factor’},
'range', {10:10:100, .1:.1:1});
RCT tutorial
42
Parameter sweeps
• Results are saved in output_directory
• Set save_results=true!
• Plot results of sweep
– plot_error(output_directory)
• Get variables from simulation
– get_saved_data(output_directory,
variables)
RCT tutorial
43
Custom scripts
• Toolbox is: box of tools you can use to
build your own experiments
• Good starting points:
– $RCTROOT/examples/tutorial/tut1.m
– $RCTROOT/examples/framework/rc_sim
ulate_job.m
RCT tutorial
44
Questions
RCT tutorial
45