Stochastic Systems in programs
Contents
1 Basic programs
2
1.1
Simulation of the path of a launched missile . . . . . . . . . . . . . . . . . . . . .
2
1.2
Simulation of the temperature of heated water
3
1.3
Estimation of consumption in a driven car: simple
1.4
Estimation of consumption in a driven car: basic
. . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . .
4
. . . . . . . . . . . . . . . . . .
4
1.5
Prediction of daily trac intensity: one-step prediction . . . . . . . . . . . . . . .
5
1.6
Simulation with the third-order dierence equation
. . . . . . . . . . . . . . . .
9
1.7
Simulation with the rst order regression model . . . . . . . . . . . . . . . . . . .
10
1.8
Simulation with the categorical model
. . . . . . . . . . . . . . . . . . . . . . . .
10
1.9
Estimation of regression model: o-line
. . . . . . . . . . . . . . . . . . . . . . .
11
1.10 Estimation of regression model: on-line . . . . . . . . . . . . . . . . . . . . . . . .
13
1.11 Estimation of regression model: least squares
. . . . . . . . . . . . . . . . . . . .
14
1.12 Estimation of categorical model . . . . . . . . . . . . . . . . . . . . . . . . . . . .
16
1.13 Prediction with regression model: one-step . . . . . . . . . . . . . . . . . . . . . .
17
1.14 Prediction with regression model: multi-step
. . . . . . . . . . . . . . . . . . . .
18
. . . . . . . . . . . . . . . . . . . . . .
20
1.15 Prediction with regression model: on-line
1.16 Prediction with categorical model: known parameters
. . . . . . . . . . . . . . .
21
1.17 Prediction with categorical model: o-line . . . . . . . . . . . . . . . . . . . . . .
22
1.18 Prediction with categorical model: on-line . . . . . . . . . . . . . . . . . . . . . .
23
1.19 State estimation
25
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.20 Control with 1st-order regression model
. . . . . . . . . . . . . . . . . . . . . . .
1.21 Adaptive control with regression model: 1st-order
1.22 Adaptive control with regression model:
1.23 Adaptive control with categorical model
1
26
. . . . . . . . . . . . . . . . .
27
. . . . . . . . . . . . . . . . .
29
. . . . . . . . . . . . . . . . . . . . . . .
31
nth-order
2 Supporting subroutines
33
2.1
Introductory le for any session . . . . . . . . . . . . . . . . . . . . . . . . . . . .
33
2.2
Normalization of data: zero mean and variance one . . . . . . . . . . . . . . . . .
34
2.3
Dierent values of a discrete variable . . . . . . . . . . . . . . . . . . . . . . . . .
34
2.4
Generation of regression vector
[yt−1 , ut−1 , · · · yt−n , ut−n ]
2.5
Generation of regression vector
[ut , yt−1 , ut−1 , · · · yt−n , ut−n ]
. . . . . . . . . . .
35
2.6
Generation of set-point . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
35
2.7
Transformation of regression model to state-space one
. . . . . . . . . . . . . . .
36
2.8
Coding of discrete variable . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
36
2.9
Uncodiong of discrete variable . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
37
. . . . . . . . . . . . .
2.10 Computation of point estimates of regression model from statistics
2.11 Kalman lter
. . . . . . . .
38
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
38
3 Supplement
1
35
39
3.1
Connection between dierence and dierential equations . . . . . . . . . . . . . .
39
3.2
Codes of discrete variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
39
3.3
Output estimation and prediction . . . . . . . . . . . . . . . . . . . . . . . . . . .
40
Basic programs
1.1 Simulation of the path of a launched missile
The missile moves in a plane, orthogonal to the surface with coordinates
(vertical). The missile is shot by a force
and horizontal
F
under the angle
α.
x
y
F sin (α)
(longitudinal) and
So, the vertical force is
F cos (α) . Against the vertical force acts gravitational force - at each time period
it is subtracted.
Against horizontal force goes the friction and causes its decrease which is
proportional to the actual speed. So the its increment
∆t
is
ϕ∆t−1 ,
where
// Simulation of the missile path
// A missile is fired in the angle al with the force F.
// Simulate its path.
// ---------------------------------------------------exec SCIHOME/ScIntro.sce, mode(0)
nd=100;
g=1;
al=15;
F=50;
fi=.99;
Fx=F*cos(al*%pi/180);
//
//
//
//
//
//
number of steps of simulation
gravitational acceleration
angle of the shot
force of the shot
air resistance
force decompositio - horizontal
2
ϕ ∈ (0, 1).
Fy=F*sin(al*%pi/180);
// force decompositio - vertical
x=0; xt=x;
y=0; yt=y;
// initial position
// initial position
// Loop of simulation
for t=1:nd
Fx=fi*Fx;
x=x+Fx;
Fy=Fy-g;
y=y+Fy;
xt=[xt x];
yt=[yt y];
end
//
//
//
//
//
//
- horizontal
- vertical
air resistance action on x-force
horizontal coordinate of the missile
gravitational action on y-force
vertical coordinate of the missile
storing
the missile path cordinates
// Resulta
j=find(yt<0); yt(j)=0;
plot(xt,yt,'linewidth',3)
set(gca(),'data_bounds',[0 2000 0 1200])
1.2 Simulation of the temperature of heated water
A boiler with water is exposed to heating. The dependence of the water temperature is monitored
in dependence on the amount of heat applied.
The increase of the temperature should be
governed by the 1st-order dierential equation.
// Heating water in the boiler
// Simulate the teperature of heated water in dependence
//
on the heating u=0 no heating, u=1 maximum heating
// ---------------------------------------------------------exec SCIHOME/ScIntro.sce, mode(0)
nd=200;
// length of simulation
a=.95;
// time constant of heating
b=.8;
// effect of heating
sd=.05;
// noise std
u=[zeros(1,20) ones(1:130) zeros(1,50)];// indicator of heating
y(1)=2;
// initial temperature
// SIMULATION LOOP
for t=2:nd
y(t)=a*y(t-1)+b*u(t)+sd*rand(1,1,'n');
end
// Results
plot(1:nd,y,1:nd,u)
title('Time course of heating water')
legend('temperature','heating');
3
1.3 Estimation of consumption in a driven car: simple
// Model of consumption in dependence on gas, revs and altitude
// - first model order
// ---------------------------------------------------------exec SCIHOME/ScIntro.sce, mode(0)
load _data/dataKF.dat
dt=DataKF.dataOrig([1 4 8 15],1:10:10*500);
y=dt(1,:);
x=dt(2:4,:);
Y=y';
X=[x' ones(y)'];
th=inv(X'*X)*(X'*Y)
yp=X*th;
set(scf(),'position',[500 100 600 500])
plot([y' yp])
e=y'-yp;
SE=e'*e/length(y)
1.4 Estimation of consumption in a driven car: basic
The subject of monitoring is a driven car and its consumption with regard to other variables
measured on the car. They are mainly: speed, used gas, engine moment and engine revs. For
modelling, the regression model is used. The estimation is performed by least squares.
// Model of consumption of a car in dependence
// on gas, revs and altitude
// - general model order
// ---------------------------------------------------------exec SCIHOME/ScIntro.sce, mode(0)
load _data/dataKF.dat
dt=DataKF.dataOrig([1 4 8 15],1:10:10*500);
ord=2;
y=dt(1,:);
x=dt(2:4,:);
nd=length(y);
for i=1:size(x,1)
x(i,:)=(x(i,:)-mean(x(i,:)))/stdev(x(i,:));
end
4
for t=(ord+1):nd
Y(t,1)=y(t);
xx=x(:,t)';
for i=1:ord
xx=[xx x(:,t-i)'];
end
X(t,:)=[xx 1];
end
th=inv(X'*X)*(X'*Y)
yp=X*th;
set(scf(),'position',[500 100 600 500])
plot([y' yp])
e=y'-yp;
SE=e'*e/nd
1.5 Prediction of daily trac intensity: one-step prediction
The task is to predict the trac intensity during the day which has the well known shape: in
the morning it rapidly grows to its maximum. This level with some variations stays the whole
morning and early afternoon.
Then, alter the working hours nish, it slowly decreases and
in the late afternoon it approaches zero.
This value stays the whole night.
Such a function
cannot be modelled by a simple regression model - without a control it can model only decaying
exponentials. It requires zero (or at least constant) expectation. Here the expectation (locally)
is time varying,
A way, how to cope with this, is either to model deviations from some average daily course of
the intensity or to take this daily course as an external variable into the model.
of modelling
(i)
without the daily course,
(ii)
The results
taking the daily course as external variable and
(iii) modelling the deviations of the intensity and the daily course are presented in the following
examples.
(i) without daily course
···
here, and estimation experiment is performed with the daily intensity without using the
known daily course. It can be seen, that the prediction, verifying correctness of the estimation,
totally fails.
// Prediction of daily traffic intnsity
// - attempt without daily course
// - even if the order is high, the results are no good
// ---------------------------------------------------------exec SCIHOME/ScIntro.sce, mode(0)
load _data/dataTunel.dat
5
// DATA CONSTRUCTION
y=dt(1,((3*288+1):35*288));
yy=[];
for k=([1 2 3 4 5 8 9 10 11 12 15 16 17 18 19 22 23 24 25 26]+2)
yy=[yy; y(((k-1)*288+1):(k*288))];
end
// averaging
yc=mean(yy,1);
// filtration (means from 5 values)
yf=([yc(1:286) 0 0]+[yc(1:287) 0]+yc+[0 yc(1:287)]+[0 0 yc(1:286)])/5;
// ESTIMATION
ord=5;
V=[];
for t=(ord+1):288
Ps=y(t);
for i=1:ord
Ps=[Ps y(t-i)];
end
Ps=[Ps 1];
V=V+Ps'*Ps;
end
Vyp=V(2:$,1);
Vp=V(2:$,2:$);
th=inv(Vp)*Vyp
// PREDICTION
np=15;
for t=(ord+1):288-np
yi=y(1:t-1);
// past y is just measured
yi(t)=[yi((t-1):-1:t-ord) 1]*th;
// first partial pradiction
for i=1:np
yi(t+i)=[yi((t+i-1):-1:t+i-ord) 1]*th; // successive predictions
end
yp(t+np)=yi(t+np);
// final value of prediction
end
// RESULTS
// prediction error variance
s=5:288;
e=y(s)-yp(s)';
SE=e*e'/288
// plot of y and its prediction
set(scf(),'position',[500 100 600 500])
plot(1:288,y(1:288),1:288,yp)
(ii) with daily course as external variable
···
here, the model of daily intensity used the knowledge of the daily course. Even if the daily
6
course for the used data is not precise, the estimation is able to t it to the data, so that the
prediction is satisfactory.
// Prediction of daily traffic intnsity based on daily course
// - even if the order is always 1, the results are good
// ---------------------------------------------------------exec SCIHOME/ScIntro.sce, mode(0)
load _data/dataTunel.dat
// DATA CONSTRUCTION
y=dt(1,((3*288+1):35*288));
yy=[];
for k=([1 2 3 4 5 8 9 10 11 12 15 16 17 18 19 22 23 24 25 26]+2)
yy=[yy; y(((k-1)*288+1):(k*288))];
end
yc=mean(yy,1);
yf=([yc(1:286) 0 0]+[yc(1:287) 0]+yc+[0 yc(1:287)]+[0 0 yc(1:286)])/5;
// choice of one specific daily intensity
nI=2;
y=y(nI*288:(nI+1)*288);
// ESTIMATION
V=[];
for t=2:288
Ps=[y(t) y(t-1) yf(t) 1]';
V=V+Ps*Ps';
end
Vyp=V(2:$,1);
Vp=V(2:$,2:$);
th=inv(Vp)*Vyp
// PREDICTION
np=15;
for t=2:288-np
yi(t)=[y(t-1) yf(t) 1]*th;
for i=1:np
yi(t+i)=[yi(t+i-1) yf(t+i) 1]*th;
end
yp(t+np)=yi(t+np);
end
// yf is filtered daily course
// first partial prediction
// successive predictions
// final value of prediction
// RESULTS
// prediction error variance
s=5:288;
e=y(s)-yp(s)';
SE=e*e'/288
// plot of y and its prediction
set(scf(),'position',[500 100 600 500])
7
plot(1:288,y(1:288),1:288,yp,1:288,yf,':')
legend('y','yp','yf');
(iii) with modelling a deviation of intensity and its daily course
···
here, instead of adding the daily course as an external variable (as in the previous case), the
dierence between the modelled intensity and the daily course is directly modelled. The result
is very similar to the previous case.
// Prediction of daily traffic intnsity based on daily course
// - even if the order is always 1, the results are good
// - model is based directly on deviations y from yf
// ---------------------------------------------------------exec SCIHOME/ScIntro.sce, mode(0)
load _data/dataTunel.dat
// DATA CONSTRUCTION
y=dt(1,((3*288+1):35*288));
yy=[];
for k=([1 2 3 4 5 8 9 10 11 12 15 16 17 18 19 22 23 24 25 26]+2)
yy=[yy; y(((k-1)*288+1):(k*288))];
end
yc=mean(yy,1);
yf=([yc(1:286) 0 0]+[yc(1:287) 0]+yc+[0 yc(1:287)]+[0 0 yc(1:286)])/5;
// choice of one specific daily intensity
nI=2;
y=y(nI*288:(nI+1)*288);
// ESTIMATION
V=[];
d(1)=0;
for t=2:288
d(t)=y(t)-yf(t);
Ps=[d(t) d(t-1) 1]';
V=V+Ps*Ps';
end
Vyp=V(2:$,1);
Vp=V(2:$,2:$);
th=inv(Vp)*Vyp
// PREDICTION
np=15;
for t=2:288-np
di(t)=[d(t-1) 1]*th;
for i=1:np
di(t+i)=[di(t+i-1) 1]*th;
end
yp(t+np)=yf(t+np)+di(t+np);
end
// yf is filtered daily course
// first partial prediction
// successive predictions
// final value of prediction
8
// RESULTS
// prediction error variance
s=5:288;
e=y(s)-yp(s)';
SE=e*e'/288
// plot of y and its prediction
set(scf(),'position',[500 100 600 500])
plot(1:288,y(1:288),1:288,yp,1:288,yf,':')
legend('y','yp','yf');
1.6 Simulation with the third-order dierence equation
In this program, we can investigate the properties of the response of rst order dierence equation. Let us recall that the dierence equation
y 0 (t) = αy (t)
where
a = exp {αT },
where
T
yt = ayt−1
comes from a dierential equation
is the period of sampling. The dierential equation
mentioned produces an exponential which in practically reasonable cases is decaying. The input
signal used (ut and
ut−1 )
is a driving force each step pushing the output somewhere. During
the period it falls towards zero following the exponential. The indication of the relation between
dierence and dierential equations is given in Supplement. The equation is
yt = b0 ut + a1 yt−1 + b1 ut−1 + a2 yt−2 + b2 ut−2 + a3 yt−3 + b3 ut−3 + k
// Response to difference equation of the third order
// with an external input u (sinusoidal signal)
exec("ScIntro.sce",-1),mode(0)
N=100;
a=[.6 .2 -.4];
b=[1 -.5 .1 0];
k=1;
u=sin(10*(1:N)/N);
y=zeros(1,N);
th=[b(1) a(1) b(2) a(2) b(3) a(3) b(4) k];
y(1)=0;
n1=length(a)+1;
for t=n1:N
ps=[u(t) y(t-1) u(t-1) y(t-2) u(t-2) y(t-3) u(t-3) 1];
y(t)=th*ps';
end
// RESULTS
s=n1:N;
plot(s,y(s),s,u(s))
set(gcf(),"position",[800 100 500 400])
9
1.7 Simulation with the rst order regression model
Regression model is nothing more than the dierence equation with the added noise term
et .
Its response is similar to that of the dierence equation but noisy. The model is
yt = b0 ut + a1 yt−1 + b1 ut−1 + k + et
// Simulation with regression model
//
y(t) = a1*y(t-1) + b0*u(t) + b1*u(t-1) + k + e(t)
// - try various inputs (puls, ones, other)
// - try various model parametrs (also noise std. s)
// --// - test rules for 1st order
// - extend to 2nd order and test it
exec("ScIntro.sce",-1),mode(0)
nd=200;
th=[.96 1 -.3 .1];
s=.1;
u=0*ones(1,nd);
y(1)=10;
//
//
//
//
//
// Simulation loop
for t=2:nd
ps=[y(t-1) u(t) u(t-1) 1];
y(t)=ps*th'+s*rand(1,1,'norm');
end
length of simulation
reg. coef. a1,b0,b1,k
noise std.dev.
input
initial condition
// regression vector
// model
// Results
plot(1:nd,u,'b:.',1:nd,y,'r:.')
set(gcf(),"position",[800 100 500 400])
legend('u','y');
1.8 Simulation with the categorical model
Categorical model diers fundamentally from the regression one. It van be used if all variables
entering the model are discrete. It is based on the fact, that all combinations of the variables
have nite number (even if it can be very large). The model can be represented by a table where
in columns are values of the modelled variable and at rows are codes of variables in regression
vector. The codes are explained in Supplement. The model is
f (yt |ut , yt−1 )
ut , yt−1
1, 1
1, 2
2, 1
2, 2
yt = 1
p1|11
p1|12
p1|21
p1|22
10
yt = 2
p2|11
p2|12
p2|21
p2|22
// Simulation with dynamic discrete model
// (controlled coin with memory)
// - test some pre-defined cases (e.g. y=1,0,1,0 etc.)
// - test different uncertainty in the model
exec("ScIntro.sce",-1),mode(0)
nd=20;
// y= 1
2
th=[.48 .52
//y1=1 u=1
.49 .51
//y1=1 u=2
.53 .47
//y1=2 u=1
.51 .49];
//y1=2 u=2
y(1)=1;
u=(rand(1,nd,'unif')>.2)+1;
parameters of discrete model
// initial condition
// Simultion loop
for t=2:nd
j=2*(y(t-1)-1)+u(t);
py=th(j,:);
// choice of the row of parameters
// [xxx y(t)]=max(py);
// index with maximal probability
y(t)=(rand(1,1,'unif')>py(1))+1; // output generation
end
// Result
plot(1:nd,y,'r:o')
set(gcf(),"position",[800 100 500 400])
set(gca(),"data_bounds",[0 nd+1 .9 2.1])
1.9 Estimation of regression model: o-line
Construction of a model consists of two parts: determination of the model structure and estimation of its parameters. Structure determination is a dicult task where we choose the model
variables, type of the model, model order etc. Once the model structure is determined, we can
estimate its parameters and thus to t it to the modelled reality. Good model is such one, that
provides good output estimates (in other words, good model gives similar outputs as the real
system when the same inputs are applied).
Here, the data are produced by a simulation - their character is determine by the structure and
parameters of the simulation model. The most signicant characteristics of the model are its
parameters and magnitude of the noise.
For estimation we prepare another model with its structure and estimate its parameters. If the
model structures are identical, we should obtain the same parameters values as those used for
simulation. If the structures are dierent (e.g. the model order is not the same), the estimation
must approximate and the parameters are dierent. The model is
yt = b0 ut + a1 yt−1 + b1 ut−1 + a2 yt−2 + b2 ut−2 + et
Update of statistics
0
Vt = Vt−1 + Ψt Ψt
11
Point estimates
V =
Vy
Vyp
0
Vyp
Vp
, θ̂ = Vp−1 Vyp
// Approximated estimation with regression model
// - simulade data with 2nd order reg. model
// Task one
// - estimate the data but with 1st order reg. model
//
you obtain (i) estimated parameters, (ii) prediction
// Task two
// - estimate the same data sample, however, with 2n order reg. model
// - again, you obtain (i) estimated parameters, (ii) prediction
// Notice: the parametrs differ (for 2n order they resemble simulated),
//
however, the predictions are simular (the 1st order
//
approximates the 2nd oreder one).
exec("ScIntro.sce",-1),mode(0)
nd=100;
a1=.6; a2=.1; r=.1;
b0=1; b1=-1.5; b2=.2;
// SIMULATION (second order reg. model)
y=zeros(1,nd);
u=1+rand(1,nd,'norm');
// input
th=[b0 a1 b1 a2 b2];
for t=3:nd
ps=[u(t) y(t-1) u(t-1) y(t-2) u(t-2)]; // regression vector
y(t)=ps*th'+sqrt(r)*rand(1,1,'norm'); // output
end
// ESTIMATION (first order model)
V=zeros(4,4);
for t=2:nd
Ps=[y(t) u(t) y(t-1) u(t-1)];
V=V+Ps'*Ps;
end
Vy=V(1,1);
Vyp=V(2:$,1);
Vp=V(2:$,2:$);
Eth1=inv(Vp)*Vyp
Er1=(Vy-Vyp'*inv(Vp)*Vyp)/nd
// extended regression vector
// update of statistics
// partitioning of
//
information
//
matrix
// point estimates of reg. koef
// -"of norise variance
// PREDICTION (simulation with estimated parameters)
yp1=zeros(1,nd);
for t=2:nd
ps=[u(t) yp1(t-1) u(t-1)];
// regression vector
yp1(t)=ps*Eth1+sqrt(Er1)*rand(1,1,'norm'); // output from estimated pars
end
// RESULTS
12
e1=y-yp1;
PE1=sqrt(e1*e1')/nd
// prediction error
// sum of squares of pred. err.
subplot(121)
plot(1:nd,y,'.:',1:nd,yp1,'r.:')
//set(gcf(),'position',[70 100 600 500])
title('Simulation with 1st order model')
legend('data','simulation');
// ESTIMATION (second order model)
// - for the same simulated data we use 2nd order
//
model for estimation
V=zeros(6,6);
for t=3:nd
Ps=[y(t) u(t) y(t-1) u(t-1) y(t-2) u(t-2)]; // ext. regression vector
V=V+Ps'*Ps;
// update of statistics
end
Vy=V(1,1);
// partitioning of V
Vyp=V(2:$,1);
Vp=V(2:$,2:$);
Eth2=inv(Vp)*Vyp
// point estimates
Er2=(Vy-Vyp'*inv(Vp)*Vyp)/nd
// PREDICTION (simulation with estimated parameters)
yp2=zeros(1,nd);
for t=3:nd
ps=[u(t) yp2(t-1) u(t-1) yp2(t-2) u(t-2)]; // regression vector
yp2(t)=ps*Eth2+sqrt(Er2)*rand(1,1,'norm'); // prediction
end
// RESULTS
e2=y-yp2;
PE2=sqrt(e2*e2')/nd
// sum of squares of pred. err.
subplot(122)
plot(1:nd,y,'.:',1:nd,yp2,'r.:')
set(gcf(),'position',[100 100 600 500])
title('Simulation with 2nd order model')
legend('data','simulation');
1.10 Estimation of regression model: on-line
The previous example shows o-line estimation when rst the data sample is simulated (as if
it is preliminarily measured) and only then the estimation is performed.
Here, we show an
on-line estimation. I.e. the data are measure and immediately used for estimation. Thus, we
can monitor the evolution of the parameter point estimates. Model and statistics see previous
example.
// On-line estimation of regression model
13
// initial statistics are set from ni measurements
// - try to change ni and compare the beginning of est. parameters
// - try to vary model parameters and model noise variance
exec("ScIntro.sce",-1), mode(0)
ni=5; nd=200;
// # of initial and measured data
b0=1; a1=.6; b1=-.2; k=-1.5; r=2; // coefficients of simulation
y=zeros(1,ni+nd);
u=1*rand(1,ni+nd,'n');
// u=0 est. fails
// SIMULATION
for t=2:ni+nd
e=sqrt(r)*rand(1,1,'norm');
y(t)=b0*u(t)+a1*y(t-1)+b1*u(t-1)+k+e;
end
// simulated output
subplot(121)
plot(1:(ni+nd),y,'.:',1:(ni+nd),u,'.:')
set(gcf(),'position',[100 50 900 500]);
// INITIAL ESTIMATION from measured data
V=1e-8*eye(5,5); Ett=[];
for t=2:ni
Ps=[y(t) u(t) y(t-1) u(t-1) 1];
V=V+Ps'*Ps;
end
// REGULAR ESTIMATION from measured data
for t=(2:nd)+ni
Ps=[y(t) u(t) y(t-1) u(t-1) 1];
V=V+Ps'*Ps;
// update of statistics
Vy=V(1,1);
// decomposition of inf. matrix
Vyp=V(2:$,1);
Vp=V(2:$,2:$);
Eth=inv(Vp+1e-8*eye(Vp))*Vyp;
// point estimates
Ett=[Ett Eth];
// remembering (for graph)
end
subplot(122)
plot(Ett')
plot([nd-50 nd],[b0 b0],'--',[nd-50 nd],[a1 a1],'--',...
[nd-50 nd],[b1 b1],'--',[nd-50 nd],[k k],'--')
legend('b0','a1','b1','k');
1.11 Estimation of regression model: least squares
Here, a very simple way of estimation is shown. It can be used only for o-line estimation. It
consists in writing the model for the measured data thus creating the vector
outputs and the matrix
X
with corresponding regression vectors in rows.
14
Y
of the measured
Model
yt = b0 ut + a1 yt−1 + b1 ut−1 + a2 yt−2 + b2 ut−2 + et
Matrices
Y
and
X


y3
 y4 

Y =
 y5 
···

u3
 u4
X=
 u5
···
Point estimates are
−1
θ̂ = (X 0 X)
y2
y3
y4
u2
u3
u4
···
y1
y2
y3

u1
u2 

u2 
···
X 0Y
// Approximated estimation with regression model
// - simulade data with 2nd order reg. model
// Task one
// - estimate the data but with 1st order reg. model
//
you obtain (i) estimated parameters, (ii) prediction
// Task two
// - estimate the same data sample, however, with 2n order reg. model
// - again, you obtain (i) estimated parameters, (ii) prediction
// Notice: the parametrs differ (for 2n order they resemble simulated),
//
however, the predictions are simular (the 1st order
//
approximates the 2nd oreder one).
exec("ScIntro.sce",-1),mode(0)
nd=100;
a1=.6; a2=.1; r=.1;
b0=1; b1=-1.5; b2=.2;
// SIMULATION (second order reg. model)
y=zeros(1,nd);
u=1+rand(1,nd,'norm');
// input
th=[b0 a1 b1 a2 b2];
for t=3:nd
ps=[u(t) y(t-1) u(t-1) y(t-2) u(t-2)]; // regression vector
y(t)=ps*th'+sqrt(r)*rand(1,1,'norm'); // output
end
// ESTIMATION
for t=3:nd
Y(t,1)=y(t);
X(t,:)=[u(t) y(t-1) u(t-1) y(t-2) u(t-2) 1];
end
Eth=inv(X'*X)*X'*Y;
yp=X*Eth;
ep=y'-yp;
Er=ep'*ep/nd;
// PREDICTION (simulation with estimated parameters)
yp1=zeros(1,nd);
15
for t=3:nd
ps=[u(t) y(t-1) u(t-1) y(t-2) u(t-2) 1]; // regression vector
yp1(t)=ps*Eth+sqrt(Er)*rand(1,1,'norm'); // output from estimated pars
end
// RESULTS
disp(Eth,'estimated parameters')
disp(Er,'estimated noise variance')
set(scf(),'position',[500 100 800 600]);
plot(1:nd,y,'.:',1:nd,yp1,'r.:')
title('Simulation with 1st order model')
legend('data','simulation');
1.12 Estimation of categorical model
Categorical estimation creates statistics in the form of matrix where columns are indexed by the
values of the output and rows correspond to the codes of regression model (for coding see the
Supplement). Each entry of the table contains the frequency of occurrence of the corresponding
combination of output and regression vector.
The on-line update consists of adding 1 to the statistics entry which corresponds with the
measured output and regression vector.
Model
f (yt |ut , yt−1 )
ut , yt−1
yt = 1
p1|11
p1|12
p1|21
p1|22
1, 1
1, 2
2, 1
2, 2
yt = 2
p2|11
p2|12
p2|21
p2|22
Update of statistics
Vi|jk = Vi|jk + δi|jk
where
δi|jk = 1
for
yt = i, ut = j
and
yt−1 = k
and is zero otherwise.
// Estimation of discrete model
// - change the uncertainty of the simulated model (compare results)
exec("ScIntro.sce",-1),mode(0)
nd=100;
th=[.4 .6
.7 .3];
//th=[.01 .99; .99 .01];
u=(rand(nd,1,'unif')>.2)+1;
// model parameter
// alternative model parameter
// input
16
// SIMULATION
for t=1:nd
y(t)=(rand(1,1,'unif')>th(u(t)))+1; // output
end
// ESTIMATION
V=zeros(2,2);
for t=1:nd
V(u(t),y(t))=V(u(t),y(t))+1; // update of statistics
end
// POINT ESTIMATES
disp('Simulated'),th
disp('Estimated')
Eth=V./(sum(V,2)*ones(1,2))
// estimate of parameter
// RESULTS
plot(1:nd,y,'.:',1:nd,u,'r.:')
set(gcf(),'position',[600 80 600 500])
set(gca(),"data_bounds",[1, nd, .9, 2.1])
title('Data for estimation')
1.13 Prediction with regression model: one-step
One step prediction (better output estimation) can be computed directly from the regression
model by substitution of regression model, current parameter point estimates and zeroing the
noise part of the model.
Model
yt = b0 ut + a1 yt−1 + et
Point prediction
ŷ1 = b0 u1 + a1 y0
ŷ2 = b0 u2 + a1 ŷ1
ŷ3 = b0 u3 + a1 ŷ2
where
y0
is known initial condition.
// One step prediction with known normal 1st order regression model
// - very very simple case - propagation trhough the model
// - for normal model nosise its covariance estimate can be evoluted
// - vary covariance r and compare the prediction results
exec("ScIntro.sce",-1),mode(0)
nd=160;
a1=.6; b0=1; r=.1;
u=1-.5*rand(1,nd,'norm');
yt=zeros(1,nd);
// number of data
// model parameters
// indut variable
17
yp=zeros(1,nd);
rp=zeros(1,nd);
// SIMULATION AND PREDICTION LOOP
for t=2:nd
// time loope
yt(t)=a1*yp(t-1)+b0*u(t)+sqrt(r)*rand(1,1,'norm'); // output
yp(t)=a1*yp(t-1)+b0*u(t);
// expectation evolution
rp(t)=r+a1^2*rp(t-1);
// noise vaiance evolution
end
// Results
printf('\nVariances: (simulated, computed, estimated, realired)\n\n')
printf('Var_e_sim =
%g\n',r)
printf('Var_e_comp =
%g\n',r*(1+a1^2))
printf('Var_e_est =
%g\n',rp($))
printf('Var_e_realized = %g\n',sum((yt-yp).^2)/nd)
plot(1:nd,yt,'b',1:nd,yp,'r.:')
// outp. and predic.
set(gcf(),'position',[500 100 600 500])
set(gca(),"data_bounds",[1 nd -1 5])
legend('output','prediction');
title('Data and prediction')
1.14 Prediction with regression model: multi-step
In this example we use the model
yt = b0 ut + a1 yt−1 + b1 ut−1 + a2 yt−2 + b2 ut−2 + k + et
Multi-step prediction with point estimates can be computed as follows:
ψt = [ut , yt−1 , ut−1 , yt−2 , ut−2 , 1]
0
is regression vector and
ψ̂t
is the same regression model but
with some/all outputs replaced by their point estimates.
At step 2 (y1 and
y0
are known, parameters are estimated form
d (1)
0
ŷ2 = ψ2 θˆ1
0
ŷ3 = ψ̂3 θˆ1
0
ŷ4 = ψ̂4 θˆ1
···
At step 3 (y2 ,
y1
and
y0
are known, parameters are estimated form
0
ŷ3 = ψ̂3 θˆ2
0
ŷ4 = ψ̂4 θˆ2
0
ŷ5 = ψ5 θˆ2
18
d (2)
···
And so on.
(For o-line estimation the parameters are all the same and come form the estimation part)
// np-step prediction with continuous model (known parameters)
// Change: - np = number of steps of prediction
//
- r = noise variance
//
- th = model parametrs
//
- u = input signal
exec("ScIntro.sce",-1),mode(0)
nd=100;
// number of data
np=5;
// length of prediction
// b0 a1 b1 a2 b2 k
th=[1 .4 -.3 -.5 .1 1]';
// regression coefficients
r=.02;
// noise variance
u=sin(4*%pi*(1:nd)/nd)+rand(1,nd,'norm'); // input
y=ones(1,nd);
nu=zeros(4,2);
yp=ones(1,nd);
// TIME LOOP
for t=3:(nd-np)
// time loop (on-line tasks)
// simulation
ps=[u(t) y(t-1) u(t-1) y(t-2) u(t-2) 1]'; // regression vector for sim.
y(t)=ps'*th+sqrt(r)*rand(1,1,'norm');
// output generation
// prediction
ps=[u(t+1) y(t) u(t) y(t-1) u(t-1) 1]'; // first reg. vec for prediction
yy=ps'*th;
// first prediction at t+1
for j=2:np
// loop of predictions for t+2,...,t+np
tj=t+j;
// future times for prediction
ps=[u(tj); yy; ps(1:$-3); 1]; // reg.vecs with predicted outputs
yy=ps'*th;
// new prediction (partial)
end
yp(tj)=yy;
// final prediction for time t+np
end
// Results
s=(np+3):nd;
plot(s,y(s),'.:',s,yp(s),'rx')
set(gca(),"data_bounds",[1 nd -3 5])
legend('output','prediction');
title([string(np),'-steps ahead prediction'])
19
1.15 Prediction with regression model: on-line
Here we show on-line prediction. It means that model parameters are unknown and currently
estimated. So, the prediction at the very beginning can be less precise due to the insuciently
estimated model parameters.
Here the situation is similar to the previous example. Only the parameters entering the prediction evolve in time.
// n-step prediction with continuous model (wit estimation)
// Change: - np = number of steps of prediction
//
- r = noise variance (effect on estimation)
//
- th = model parametrs
//
- u = input signal (effect on estimation)
exec("ScIntro.sce",-1),mode(0)
nd=100;
// number of data
np=5;
// length of prediction
//
b0 a1 b1 a2 b2 k
th=[1 .4 -.3 -.5 .1 1]';
// regression coefficients
r=.02;
// noise variance
u=sin(4*%pi*(1:nd)/nd)+rand(1,nd,'norm'); // input
y=ones(1,nd);
nu=zeros(4,2);
yp=ones(1,nd);
nz=3;
for t=nz:(nd-np)
// time loop (on-line tasks)
// simulation
ps=[u(t) y(t-1) u(t-1) y(t-2) u(t-2) 1]'; // regression vector for sim.
y(t)=ps'*th+sqrt(r)*rand(1,1,'norm');
// output generation
// estimation
Ps=[y(t) u(t) y(t-1) u(t-1) y(t-2) u(t-2) 1]'; // reg.vect. for estim.
if t==nz, V=1e-8*eye(length(Ps)); end // initial information matrix
V=V+Ps*Ps';
// update of statistics
Vp=V(2:$,2:$);
Vyp=V(2:$,1);
Eth=inv(Vp+1e-8*eye(Vp))*Vyp;
// point estimates
Et(:,t)=Eth(:,1);
// stor est. parameters
// prediction
ps=[u(t+1); Ps(1:$-3); 1];
// first reg. vec for prediction
yy=ps'*Eth;
// first prediction at t+1
for j=2:np
// loop of predictions for t+2,...,t+np
tj=t+j;
// future times for prediction
ps=[u(tj); yy; ps(1:$-3); 1]; // reg.vecs with predicted outputs
yy=ps'*Eth;
// new prediction (partial)
end
yp(tj)=yy;
// final prediction for time t+np
20
end
// Results
disp(' Simulated parameters')
disp(th)
disp(' Estimated parameters')
disp(Eth)
subplot(121),plot(Et')
set(gcf(),'position',[100 100 1200 400])
set(gca(),"data_bounds",[0 nd+1 -1 2])
title('Evolution of estimated parameters')
subplot(122)
s=(np+3):nd;
plot(s,y(s),'.:',s,yp(s),'rx')
set(gca(),"data_bounds",[1 nd -3 5])
legend('output','prediction');
title([string(np),'-steps ahead prediction'])
1.16 Prediction with categorical model: known parameters
This prediction is a bit unusual.
Having an estimated model as a probabilistic table, we de-
termine the code of the regression vector. According it we choose the corresponding row of the
table. The maximal entry indicates the predicted value.
Model
f (yt |ut , yt−1 )
ut , yt−1
1, 1
1, 2
2, 1
2, 2
yt = 1
p1|11
p1|12
p1|21
p1|22
yt = 2
p2|11
p2|12
p2|21
p2|22
// Prediction with discrete model (off-line)
// Vary: - length of prediction
//
- uncertainty of the simulated model
exec("ScIntro.sce",-1),mode(0)
nd=100;
np=5;
th1=[0.98 0.01 0.04 0.97]';
th=[th1 1-th1];
u=(rand(1,nd)>.3)+1;
y=ones(1,nd);
// simulation
for t=2:nd
i=2*(u(t)-1)+y(t-1);
// parameters for simulation (for y=1)
// all parameters
// input
// row of the table
21
y(t)=(rand(1,1,'unif')>th(i,1))+1; // output generation
end
// prediction
yj=ones(1,nd);
// fictitious predicted output
for t=2:(nd-np)
yj(t)=y(t);
// at time t start with real output
for j=1:np
tj=t+j;
// future times
i=2*(u(tj)-1)+yj(tj-1);
// row of the table
yj(tj)=(rand(1,1,'unif')>th(i,1))+1; // prediction generation
end
yp(tj)=yj(tj);
// np-step predction
end
// Results
disp(' Model parameters')
th
s=(np+3):nd;
plot(s,y(s),'.:',s,yp(s),'rx')
set(gcf(),'position',[400 100 800 400])
set(gca(),"data_bounds",[0 nd+1 .9 2.1])
legend('output','prediction');
title(string(np)+'-steps ahead prediction')
1.17 Prediction with categorical model: o-line
This prediction is, surprisingly, very similar to that with regression model. What diers is only
the output estimation.
For details see the previous example.
// Prediction with discrete model (off-line)
// Vary: - length of prediction
//
- uncertainty of the simulated model
//
- imput signal (effect on estimation)
exec("ScIntro.sce",-1),mode(0)
nd=100;
np=5;
th1=[0.8 0.1 0.4 0.7]';
th=[th1 1-th1];
u=(rand(1,nd)>.3)+1;
y=ones(1,nd);
// simulation
for t=2:nd
i=2*(u(t)-1)+y(t-1);
// parameters for simulation (for y=1)
// all parameters
// input
// row of the table
22
y(t)=(rand(1,1,'unif')>th(i,1))+1; // output generation
end
// estimation
nu=zeros(4,2);
for t=2:nd
i=2*(u(t)-1)+y(t-1);
nu(i,y(t))=nu(i,y(t))+1;
end
Eth=nu./(sum(nu,2)*ones(1,2));
// row of the table
// statistics update
// estimate of parameters
// prediction
yj=ones(1,nd);
// fictitious predicted output
for t=2:(nd-np)
yj(t)=y(t);
// at time t start with real output
for j=1:np
tj=t+j;
// future times
i=2*(u(tj)-1)+yj(tj-1);
// row of the table
yj(tj)=(rand(1,1,'unif')>Eth(i,1))+1; // prediction generation
end
yp(tj)=yj(tj);
// np-step predction
end
// Results
disp(' Simulated parameters')
disp(th)
disp(' Estimated parameters')
disp(Eth)
s=(np+3):nd;
plot(s,y(s),'.:',s,yp(s),'rx')
set(gcf(),'position',[300 100 500 400])
set(gca(),"data_bounds",[0 nd+1 .9 2.1])
legend('output','prediction');
title(string(np)+'-steps ahead prediction')
1.18 Prediction with categorical model: on-line
This is again similar to the continuous case. The estimation is performed concurrently - each
prediction is based on the current parameters estimate.
For details see the last but one example.
// Prediction with discrete model (on-line)
// Vary: - length of prediction
//
- uncertainty of the simulated model
//
- imput signal
//
- study the beginning when estimation is not finished
//
how can we secure quicker transient phase of estimation?
23
exec("ScIntro.sce",-1),mode(0)
nd=100;
np=5;
th1=[0.8 0.1 0.4 0.7]';
th=[th1 1-th1];
u=(rand(1,nd,'unif')>.3)+1;
y=ones(1,nd);
//
//
//
//
//
number of data
length of prediction
parameters (for y=1)
all parameters
input
nu=1e-8*ones(4,2);
yp=ones(1,nd);
Et=zeros(4,nd-np);
for t=2:nd-np
// time loop
// simulation
i=2*(u(t)-1)+y(t-1);
y(t)=(rand(1,1,'unif')>th(i,1))+1;
// estimation
i=2*(u(t)-1)+y(t-1);
nu(i,y(t))=nu(i,y(t))+1;
Eth=nu./(sum(nu,2)*ones(1,2));
Et(:,t)=Eth(:,1);
// prediction
yj(t)=y(t);
for j=1:np
tj=t+j;
i=2*(u(tj)-1)+yj(tj-1);
yj(tj)=(rand(1,1,'unif')>Eth(i,1))+1;
end
yp(tj)=yj(tj);
end
// Results
disp(' Simulated parameters')
disp(th)
disp(' Estimated parameters')
disp(Eth)
subplot(121),plot(Et')
set(gcf(),'position',[500 100 1000 400])
set(gca(),"data_bounds",[0 nd+1 -.1 1.1])
subplot(122)
s=(np+3):nd;
plot(s,y(s),'.:',s,yp(s),'rx')
set(gca(),"data_bounds",[0 nd+1 .9 2.1])
legend('output','prediction');
title([string(np),'-steps ahead prediction'])
24
1.19 State estimation
State is a (generally multivariate) system variable which is unknown and unmeasurable. We can
observe input and output of the system and on its basis estimate the state. The connections
between state, input and output is given by a state-space model. The parameters of the model
are supposed to be known (which in practice is rather sever restriction). Estimation is performed
by so-called Kalman lter.
The state-space model is
xt
=
M xt−1 + ut + wt
yt
=A
xt−1 + But + vt
// State estimation (vector input and output)
// - simulation: 4-states, 3-inputs and 2-outputs
// - uses function Kalman (recomputation of state statistics)
exec("ScIntro.sce",-1),mode(0)
nd=100;
M=[.3 .5 -.2 .1
.2 .6 .3 -.2
.4 -.2 .6 .1
.1 .2 -.4 .8];
N=[1 .2 .5
.6 1 .2
.7 .1 1
.1 .2 .2];
A=[1 0 0 0
0 1 0 0];
B=[.1 .3 .2
.5 .3 .8];
rv=.01; rw=.01;
nx=max(size(M));
[ny,nu]=size(B);
ut=rand(nu,nd,'n');
yt=zeros(ny,nd);
xt=zeros(nx,nd);
x=zeros(nx,1);
// length of simulation/estimation
// parameters for simulation model
// input generation
// initial state
// SIMULATION
for t=1:nd
u=ut(:,t);
y=A*x+B*u+sqrt(rv)*rand(ny,1,'n');
yt(:,t)=y; xt(:,t)=x;
x=M*x+N*u+sqrt(rw)*rand(nx,1,'n');
end
x=zeros(nx,1);
Rv=.01*eye(ny,ny); Rw=.1*eye(nx,nx);
// output generation
// remembering
// state generation
// initial state estimate
// model covariance matrices
25
Rx=1e3*eye(nx,nx);
xe=zeros(nx,nd); yh=zeros(ny,nd);
// state covariance matrix
// FILTRATION
for t=1:nd
[x,xf,Rx,yp]=Kalman(x,yt(:,t),ut(:,t),M,N,A,B,Rw,Rv,Rx);
// statistics recomputation
xe(:,t)=xf;
// remembering
yh(:,t)=yp;
ee(:,t)=yt(:,t)-yp;
rr(:,t)=diag(Rx);
end
// RESULTS
s=1:nd;
// plot of output and predicted output
set(scf(1),'position',[70 70 500 500])
for i=1:ny
subplot(ny,1,i)
plot(s,yt(i,s),'b')
plot(s,yh(i,s),'r')
title(['Output ',string(i),' and its prediction'])
end
// plot of state and state estimate
set(scf(2),'position',[600 70 500 500])
for i=1:nx
subplot(nx,1,i),
plot(s,xt(i,s),'b')
plot(s,xe(i,s),'r')
title(['State ',string(i),' and its estimate'])
end
1.20 Control with 1st-order regression model
Optimal control construct the control variable so that the specied quality criterion is minimized.
One-step ahead control (so called minimum variance control) has practically no sense as it
frequently leads to unstable control.
Practically signicant control is that which designs the
control on the whole control interval simultaneously. The control is designed using well known
Dynamic programming as a solution so called Bellman equations that are computed recursively
against the direction of time.
For details see the text for lectures.
//
//
//
//
//
//
Control with scalar 1st order regression model
- simulated data y(t)=a*y(t-1)+b*u(t)+k+e(t);
- state realization of the model for synthesis
- control on a single control interval with the length nd
- following a settpoint s(t)
---------------------------------------------------------26
exec SCIHOME/ScIntro.sce, mode(0)
nd=100;
a=.996; b=1; k=-3;
om=1; la=.01;
s=sign(100*sin(18*(1:nd)/nd));
// conversion to state-space model
N=[b 1 0]';
Om=diag([1 la 0]);
// matrix penalization
S=list();
R=list();
R(nd+1)=zeros(Om);
// initial condition for dyn. progr.
// CONTROL
// computation of control-law
for t=nd:-1:2
M=[a 0 k-s(t); 0 0 0; 0 0 1];
T=R(t+1)+Om;
A=N'*T*N;
B=N'*T*M;
C=M'*T*M;
S(t)=inv(A)*B;
R(t)=C-S(t)'*A*S(t);
end
// length of control interval
// regression model parameters
// penalization (output, input)
// setpoint generation
(M is computed on-line)
// state matrix with set-point
// control-law realization
y(1)=5;
u(1)=0;
for t=2:nd
u(t)=-S(t)*[y(t-1) u(t-1) 1]';
y(t)=a*y(t-1)+b*u(t)+k+.1*rand(1,1,'n');
end
// optimaal control
// simulation
// RESULTS
x=1:nd;
plot(x,y(x),'--',x,u(x),x,s(x),':')
legend('y','u','s');
1.21 Adaptive control with regression model: 1st-order
Adaptive control is performed with the model with unknown parameters. The task is not feasible
in optimal. So suboptimal solutions are mostly accepted.
The most common procedure is called receding horizon. Here, we start at the beginning of the
overall control interval using the existing point estimates of parameters and perform the control
synthesis for some short interval of control (say for 5-10 steps). From this control we perform
only one step, apply the control and measure new system output. Obtaining new data we can
recompute the estimates of parameters and newly compute the control on the short interval.
And this way we can continue for the whole overall control interval.
27
For details see the text for lectures.
// Control with scalar 1st order regression model
// - simulated data y(t)=a*y(t-1)+b*u(t)+k+e(t);
// - state realization of the model for synthesis
// - control on a receding control interval with the length nc
// - following a settpoint s(t)
// Experiments
// - change setpoint and system parameters (slow - quick system)
// - penalization of input variable
// - initial condition for estimation (better or worse initial param.)
// ---------------------------------------------------------exec SCIHOME/ScIntro.sce, mode(0)
nd=100;
nc=5;
a=.6; b=1; k=-3;
aE=.1; bE=.5; kE=1;
om=1; la=.01;
s=sign(100*sin(18*(1:nd)/nd));
Om=diag([1 la 0]);
y(1)=5;
V=1e-8*eye(4,4);
//
//
//
//
//
//
//
//
//
number of data to be controlled
length of control interval
model
parameters for simulation
penalizations (output, input)
set-point
matrix penalization
initial output
initial information matrix
// COMPUTATION OF CONTROL-LAW
S=list();
R=list();
u(1)=0;
// initial control
for t=2:nd
// loop for comtrol
N=[bE 1 0]';
// state-space model
R(nc+1)=zeros(Om);
// initial condition for dyn.prog.
for i=nc:-1:1
// loop for receding horizon
M=[aE 0 kE-s(t); 0 0 0; 0 0 1];// state-space model
T=R(i+1)+Om;
// dynamic
A=N'*T*N;
// programming
B=N'*T*M;
C=M'*T*M;
S(i)=inv(A)*B;
R(i)=C-S(i)'*A*S(i);
end
// CONTROL REALIZATION
u(t)=-S(1)*[y(t-1) u(t-1) 1]';
y(t)=a*y(t-1)+b*u(t)+k+.1*rand(1,1,'n');
// ESTIMATION
Ps=[y(t) y(t-1) u(t) 1]';
V=V+Ps*Ps';
Vyp=V(2:4,1);
Vp=V(2:4,2:4);
28
// optimal control value
// simulation
th=inv(Vp)*Vyp;
aE=th(1);
bE=th(2);
kE=th(3);
end
// point estimates
// regression
// model
// parameters
// RESULTS
z=1:nd;
plot(z,y(z),'--',z,u(z),z,s(z),':')
legend('y','u','s');
1.22 Adaptive control with regression model: nth-order
The procedure is the same, only multivariate.
// ADAPTIVE CONTROL WITH DYNAMIC REG. MODEL WITH CONSTANT
//
multivariate input and output
//
control with setpoint
//
penalization of input increments
//
receding horizon in control
// -----------------------------------------------exec SCIHOME/ScIntro.sce,
mode(0), getd _func
// SETTING THE TASK
ni=50;
// length of prior estimation
nd=200;
// length of control
ord=1;
// model order
pen=1;
// penalization of u 1=increments, 0=absolute u
//
increm. only for ord>1
nh=3;
// length of control interval
om=1; la=.005;
// penalty: y(t)^2 and (u(t)-u(t-1))^2
ada=0;
typU=3;
// adaptive control 1=yes, 0=no,
//
else fix prior est.
// type of input 1=noise, 2=sin, 3=jumps
a=list(); b=list();
// full structure assumed:
a(1)=[.3 -.2; -.05 .6];
// parameters at y
a(2)=[.1 -.1; .02 -.1];
b(1)=[1 -.4; 1.3 -.4];
// parameters at u
b(2)=[.1 0;0 .2];
b(3)=[.1 0;0 .1];
k=[1; -1];
// model constant
sd=.05;
// noise standard deviation
ny=size(a(1),1);
nu=size(b(1),2);
// No of values of y
// and of u
29
th=b(1);
for i=1:ord
// parameters -> vector th
th=[th a(i) b(i+1)];
end
th=[th k];
//conversion to state model
[XX,XX,XX,nx,ny,nu]=reg2st(th,ord);
// penalization matrices
Om=zeros(nx,nx);
Om(1:ny,1:ny)=om*eye(ny,ny); // output
nn=ny+nu;
for i=1:nu
// input increment
Om((ny+i),(ny+i))=la;
if (ord>1)& (pen==1)
Om((ny+i+nn),(ny+i+nn))=la;
Om((ny+i),(ny+i+nn))=-la;
Om((ny+i+nn),(ny+i))=-la;
end
end
// setpoint generation
g=genstp(ny,nd,[3;5],[.95;.92],[-5;10]);
// PRIOR ESTIMATION
yi=zeros(ny,ni);
select typU
case 1, ui=rand(nu,ni,'norm');
case 2, ui=[sin(20*(1:ni)/ni);cos(20*(1:ni)/ni)];
case 3, ui=sign([sin(20*(1:ni)/ni);cos(20*(1:ni)/ni)]);
end
V=zeros(nx+ny+nu,nx+ny+nu);
for i=(ord+1):ni
ps=genps(ord,i,yi,ui);
yi(:,i)=th*ps + sd*rand(ny,1,'norm');
Ps=[yi(:,i); ps];
V=V+Ps*Ps';
end
theta=v2thN(V/ni,ny);
// TIME LOOP for adaptive control ==
S=list();
y=zeros(ny,nd); y(:,1:2)=[10 10;-10 -10];
u=zeros(nu,nd);
for t=(ord+2):(nd-nh)
// estimation
if ada==1
ps=genps(ord,t-1,y,u);
30
Ps=[y(:,t-1); ps];
V=V+Ps*Ps';
theta=v2thN(V/(t+ni),ny); // current estimates
elseif ada==0
theta=th';
// known parameters
else
// prior pt.estimates are used
end
[M,N,A,nx,ny,nu]=reg2st(theta',ord);
// generation of control law
R=.00001*eye(nx,nx);
// regularization
for i=nh:-1:1
M(1:ny,$)=-g(:,t+i)+k;
T=R+Om;
R=inv(inv(T)+N*inv(om)*N');
end
// generation of optimal control
S=inv(N'*T*N)*N'*T*M;
x=genph(ord,t,y,u);
u(:,t)=-S*x;
y(:,t)=th*[u(:,t); x]+sd*rand(ny,1,'norm');
end
// RESULTS
s=1:(nd-nh);
subplot(211)
plot(s,y(1,s),'b',s,u(1,s),'g--',s,g(1,s),'m.','markersize',2)
legend('output','input','setpoint',4);
subplot(212)
plot(s,y(2,s),'b',s,u(2,s),'g--',s,g(2,s),'m.','markersize',2)
set(gcf(),'position',[500 50 800 600])
legend('output','input','setpoint',1);
disp 'Comparison of simulated and estimated parametrs'
printf '\n - first two rows are simulated parameters'
disp ' - second two rows are estimated parameters'
disp([th;theta'])
1.23 Adaptive control with categorical model
For categorical model the control synthesis is the same as for regression model: we use dynamic
programming and receding horizon.
However, the computation of the expectation is a bit
uncommon, as it is based on probabilistic table.
For details see the text for lectures.
// Adaptive control with discrete model
// - first order controled discrete system
31
// - adaptive control with receding horizon
// Experiments
// - for specific control aim determine the penalization function
// - check performance of the control in ependence on the amount
//
of uncertainty in the system
//
// Dynamic programming
// fp = omega + fs (penalty function, old criterion)
// f = E[fp|d(t)] - expectation over y(t+1)
// fs = min f => u(t+1) - optimal u
// --------------------------------------------------------------exec("ScIntro.sce",-1),mode(0)
// VARIABLES TO BE SET
nh=5;
nd=30;
y(1)=1; y(2)=1;
u(2)=1;
nu=ones(4,2);
k=10;
// y 1 2
om=[3 4+k
4 5+k
4 5+k
5 6+k];
u
//
//
//
//
y1 =
1 1
1 2
2 1
2 2
// y 1 2
th=[.3 .7
.1 .9
.6 .4
.8 .2];
//
//
//
//
u
1
1
2
2
y1
1
2
1
2
//
//
//
//
//
length of control interval
length of control
initial condition for output
initial condition for input
initial statistics for estimation
criterion
= system model
// time loop of adaptive control
for t=1:nd
// ESTIMATION
i=2*(u(t+1)-1)+y(t);
nu(i,y(t+1))=nu(i,y(t+1))+1;
Et=nu./(sum(nu,2)*ones(1,2));
fs=zeros(1,2);
// CONTROL-LAW COMPUTATION
for i=nh:-1:1
fp=om+ones(4,1)*fs;
// expectation
f=sum((fp.*th),'c');
// minimization
// penalty + reminder from last step
// expectation over y
32
if f(1)<f(3),
us(i,1)=1; fs(1)=f(1);
else
us(i,1)=2; fs(1)=f(3);
end
if f(2)<f(4),
us(i,2)=1; fs(2)=f(2);
else
us(i,2)=2; fs(2)=f(4);
end
end
// for y(i-1)=1
// optimal control, minimum of criterion
// optimal control, minimum of criterion
// for y(i-1)=2
// optimal control, minimum of criterion
// optimal control, minimum of criterion
// CONTROL APPLICATION
u(t+2)=us(1,y(t+1));
// optimal control
i=2*(u(t+2)-1)+y(t+1);
// row in the model table
y(t+2)=double(rand(1,1,'unif')>th(i,1))+1; // simulation
end
// RESULTS
s=1:nd;
plot(s,y(s),'ro',s,u(s),'g+')
set(gcf(),'position',[200 40 1000 600])
set(gca(),"data_bounds",[min(s)-1 max(s)+1, .8 2.2])
legend('output','input');
title('Optimal control with discrete model')
2
Supporting subroutines
2.1 Introductory le for any session
The memory and the screen are cleared, all open gures are closed, the working directory is set
as that from which the program is loaded and all functions needed ale loaded.
// General introduction of a SciLab run
// - here you can write your own commands, necessary always
//
after start of your program; the program must start with
//
the command:
exec("ScIntro.sce",-1)
clc,
clear, clearglobal(),
xdel(winsid()),
funcprot(0), warning('off')
//rand('seed',139737)
//
//
//
//
//
// Creating librery from functions
// genlib("funlib","_func/");
// generation of library
33
clear console
claear variables
clear all figures
switch off warnings
seed for random generator
// Direct translation of functions
getd("_func/");
// call of functions to memmory
2.2 Normalization of data: zero mean and variance one
function fn=fnorm(f,i)
// fn=fnorm(f,i) normalization of probabilistic table
// fn normalized table
// f
table
// i
direction i=1 norm colunms, i=2 norm rows
if argn(2)==1,
sf=sum(f);
fn=f/sf;
else
[m n]=size(f);
if i==1
f1=sum(f,1);
fn=f./(ones(m,1)*f1);
else
f2=sum(f,2);
fn=f./(f2*ones(1,n));
end
end
endfunction
2.3 Dierent values of a discrete variable
function [h,f]=vals(a)
// [h f]=vals(a) find different nalues of a
//
and their frequencies
// h
values and frequencies [vals;abs_freq]
// f
relatine frequencies
a=a(:)';
b=gsort(a,'g','i');
[v,m]=unique(b);
dm=diff(m);
n1=length(b)+1;
n=[dm n1-m($)];
f=n/sum(n);
h=[v(:)';n];
if sum(n)~=max(size(a))
disp('Error: in vals.sci')
return
end
endfunction
34
2.4 Generation of regression vector [yt−1 , ut−1 , · · · yt−n , ut−n ]
function ps=genph(n,t,y,u)
// ps=genp(n,t,y,u)
generation of regression vector \phi
//
i.e. without y(t) and u(t)
// n
order
// t
time
// y
output
// u
input
ps=[];
for i=1:n
ps=[ps; y(:,t-i); u(:,t-i)];
end
ps=[ps; 1];
endfunction
2.5 Generation of regression vector [ut , yt−1 , ut−1 , · · · yt−n , ut−n ]
function ps=genps(n,t,y,u)
// ps=genps(n,t,y,u)
generation of regression vector \psi
//
i.e. without y(t)
// n
order
// t
time
// y
output
// u
input
ps=u(:,t);
for i=1:n
ps=[ps; y(:,t-i); u(:,t-i)];
end
ps=[ps; 1];
endfunction
2.6 Generation of set-point
The set-point is generated as a step function with dened jumps and their size.
function g=genstp(ny,nd,a,f,m)
// g=genstp(ny,nd,a,f)
generation of rectangular setpoint
// ny
numb. of rows
// nd
numb of columns
// a
vector of amplitudes
// f
frequency of switching
//
if rand>f: switch
35
// m
vertical shift
g=zeros(ny,nd);
for j=1:ny
z=a(j); s=f(j); r=m(j);
for i=1:nd
g(j,i)=r+z;
if rand(1,1,'unif')>s
z=-z;
end
end
end
endfunction
2.7 Transformation of regression model to state-space one
The theory is explained on lectures,
function [M,N,A,nx,ny,nu]=reg2st(theta,n)
// [M,N,A,nx,ny,nu,n]=reg2st(theta) conversion of reg.model to state-sp. one
// M,N,A
parameters of state model
x(t)=Mx(t-1)+Nu(t), y(t)=Ax(t)
// nx
length of the state
// ny,nu
number of outputs and inputs
// theta
reg.mod. parametrs - b0 a1 b1 .. an bn k (full str.)
// n
model order
[ny nt]=size(theta);
nu=(nt-n*ny-1)/(n+1);
nt=size(theta,2);
nx=n*(ny+nu)+1;
// length of parameters
if fix(nu)~=nu, disp(' Wrong theta'), end
// parameters of:
x(t)=Mx(t-1)+Nu(t), y(t)=Ax(t)
nn=(n-1)*(ny+nu);
M=[theta(:,(nu+1):$);
zeros(nu,nt-nu);
[eye(nn,nn) zeros((n-1)*(ny+nu),ny+nu+1)];
[zeros(1,nt-nu-1) 1]];
N=[theta(:,1:nu);eye(nu,nu);zeros(nt-ny-2*nu,nu)];
A=[eye(ny,ny) zeros(ny,nt-ny-nu)];
endfunction
2.8 Coding of discrete variable
The principle of coding is explained in Supplement.
function i=psi2row(x,b)
36
// i=psi2row(x,b) i is row number of a model table with
//
the regression vector x with the base b;
//
elements of x(i) are 1,2,...,nb(i)
// it is based on the relation
//
i=b(n-1)b(n-2)...b(1)(x(n)-1)+...+b(1)(x(2)-1)+x(1)
n=length(x);
if argn(2)<2, b=2*ones(1,n); end
bb=b(2:n);
bb=bb(:)';
b=[bb 1];
i=0;
for j=1:n
i=(i+x(j)-1)*b(j);
end
i=i+1;
endfunction
2.9 Uncodiong of discrete variable
The principle of coding is explained in Supplement.
function x=row2psi(i,b)
// x=row2psi(i,b) generates a discrete rgegression vector
//
with a base b that corresponds to the
//
i-th row of a table of the discrete model
// it is based on the relation
//
i=b(n-1)b(n-2)...b(1)(x(n)-1)+...+b(1)(x(2)-1)+x(1)
if isscalar(b),
n=fix(log(i+1)/(log(b)))+1;
b=b*ones(1,n);
else
n=length(b);
end
if i>prod(b)
disp('ERROR: The row number is too big')
return
end
i=i-1;
for j=1:n
i=i/b(n-j+1);
x(n-j+1)=round((i-fix(i))*b(n-j+1)+1);
i=fix(i);
end
x=x(:)';
endfunction
37
2.10 Computation of point estimates of regression model from statistics
After partitioning the information matrix
V
θ̂ = Vp−1 Vyp
into parts
and
r̂ =
Vy , Vyp
and
Vp
we have
Vy − θ̂Vyp
κ
function [th,s2]=v2thN(v,m)
// [th,s2]=v2thn(v,m) computation of par. point estimates
//
from normalized information matrix
// v
information matrix
// m
dimension of y
// th
regression coefficients
// s2
noise cobariance estimate
if argn(2)<2
m=1;
end
// check for number of input arguments
// partitioning of information matrix
vy=v(1:m,1:m);
vyf=v(m+1:$,1:m);
vf=v(m+1:$,m+1:$);
// computation of point estimates
th=inv(vf+1e-12*eye(vf))*vyf;
s2=(vy-vyf'*th);
endfunction
2.11 Kalman lter
Kalman lter recomputes characteristic of the normally distributed state variable. It updates
state expectations (point estimates of the state) and state variances (uncertainty of the state
estimates) on the basis of prediction error.
This is the dierence between the predicted and
measured output. With the amplication given by the Kalman gain it corrects the state estimates.
function [xt,xf,Rx,yp]=Kalman(xt,yt,ut,M,N,A,B,Rw,Rv,Rx)
// [xt,xf,Rx,yp]=Kalman(xt,yt,ut,M,N,A,B,Rw,Rv,Rx)
// Kalman filter for state estimtion with the model
//
xt = M*xt + N*ut + w
//
yp = A*xt + B*ut + e
// xt
state
// Rx
state estimate covariance matrix
// yp
output prediction
// yt
output
// ut
input
38
//
//
//
//
M,N
A,B
Rw
Rv
state model parameters
output model parameters
state model covariance
output model covariance
yp=A*xt+B*ut;
Ry=Rv+A*Rx*A';
Rx=Rx-Rx*A'*inv(Ry)*A*Rx;
ey=yt-yp;
KG=Rx*A'*inv(Rv);
xf=xt+KG*ey;
xt=M*xf+N*ut;
Rx=Rw+M*Rx*M';
endfunction
3
//
//
//
//
//
//
//
//
output prediction
noise covariance update
state est. coariance update
prediction error
Kalman gain
data update of the state
rime update of the state
time updt. of state covariance
Supplement
3.1 Connection between dierence and dierential equations
Consider an elementary homogeneous dierence equation
yt = ayt−1
This is an equation describing a straight line, saying that in a step of the length 1 we come form
yt−1
to
yt .
Now, let us have a period
T → 0.
The equation will become
yt = T αyt−T
where
.
α = y (t) =
0
yt −yt−T
.
T
We can write
yt − yt−T = (T α − 1) yt−T
and
and taking the limit for
yt − yt−T
=
T
T →0
α−
1
T
yt−T
we obtain
y 0 (t) = αy (t)
3.2 Codes of discrete variables
Let us have two discrete variables
x1
with values 1, 2, 3 and
x2
all combinations of values of these variables. We obtain a table
row number
x1
x2
1
1
1
2
1
2
3
2
1
4
2
2
5
3
1
6
3
2
39
with values 1, 2. Let us write
x
Now we can say that we have new (vector) variable
with values 1, 2, 3, 4, 5, 6. The values
correspond to the order of the rows of the table.
This can be easily extended to arbitrary number of variable with arbitrary number of dierent
values. However, the number of values of the coded variable grows very rapidly.
The process of assigning values to vector variable we will call coding.
3.3 Output estimation and prediction
Let us have model
f (yt |ψt , Θ)
set by regression equation
0
yt = ψt Θ + et
with the estimated parameters Θ = Θ̂t−1 (using data up to the time t − 1) and regression vector
0
ψt = [ut , yt−1 , ut−1 · · · yt−n , ut−n ] (i.e. without yt ).
The expectation of this model is
0
E [yt |ψt , Θ] = ψt Θ = yˆt
where
ŷt
is the
output estimate of yt
based on data
ut
and
d (t − 1).
Output prediction is estimation of some more remote ahead output yt+k , k ≥ 1 based again
on the data
ut
and
d (t − 1).
The pdf corresponding to this task is
f (yt+k |uk , d (t − 1)) .
The
task is mostly solved by sequential substitution of shifted models and using output predictions
instead of the future outputs.
The details can be read from the programs.
40