Intelligent Control
Module III- Neural Control
Lecture 4
Indirect Adaptive Control of a Robot Manipulator
Laxmidhar Behera
Department of Electrical Engineering
Indian Institute of Technology, Kanpur
Recurrent Networks – p.1/??
Topics to be Covered
System identification using a feed-forward
network
Neural network training and Data-insufficiency
Query based learning
Forward-inverse modeling
A network inversion approach
Simulation results
Recurrent Networks – p.2/??
A Robot Manipulator: The dynamic model
The vector equation of motion of N-link rigid
manipulator is of the form
M (q)q̈ + C(q, q̇)q̇ + G(q) = τ
where
τ = N x 1 vector of joint actuator torques
q = N x 1 vector of joint positions
M(q) = N x N inertial matrix
C(q, q̇)q̇ = Torque arising from Centrifugal and
Coriolis forces
G(q) = Gravitational torque vector
Recurrent Networks – p.3/??
The state model
State vector : x(k) = [q(k)T , q̇(k)T ]T order is 2N
x1
Control vector : u(k) = τ (k) order is N x 1
Desired trajectory :
xd (k + 1) = [qd (k + 1)T , q̇d (k + 1)T ]T
Recurrent Networks – p.4/??
Modeling using RBFN
Consider a generic nonlinear system:
x(k + 1) = f (x(k), u(k))
where x(k) ∈ <n and u(k) ∈ <p . The radial basis function
network that will model this system:
φ1
x1 (k)
P
x̂1 (k + 1)
P
x̂n (k + 1)
φ2
xn (k)
u1 (k)
up (k)
φ`
Figure 1: An n + p input and n output RBF network
Recurrent Networks – p.5/??
Modeling using RBF network
A Radial basis function network has following components:
Input nodes
Radial centers
Weight layer
Output layer
The ith output of such a network can be expressed as:
xi (k + 1) = fi (v) =
l
X
θij φj (k v − cj k)
j=1
where v = [x1 (k), .., xn (k), u1 (k), .., up (k)] and cj is the j th
radial center.
Recurrent Networks – p.6/??
Training an RBF network
Fixed centers + Weight update using gradient descent
Fixed centers + Weight update using Recursive least
square
Gradient descent based parameter tuning for centers
and weights
PRPE/EKF based parameter tuning for centers and
weights
Hybrid learning: Centers updated using clustering
techniques and weight update using RLS
Remark : Centre update reduces the number of radial
Recurrent Networks – p.7/??
centres
Dimensionally insufficient data
Robot manipulator: a case of open-loop
unstable system
Data are generated using a PD controller
Control input is state dependent
Addition of dither signal moves data to n + p
manifold.
State independent control input that can still
keep the system stable.
Recurrent Networks – p.8/??
Query based learning
The query based learning is proposed to generate
state independent control signals:
τ
q, q̇
Robot Manipulator
Neural Emulator
retraining
û
x̂ x
q, q̇
u
New Examples
Network Inversion
Learning Trajectory
Figure 2: Scheme for query based learning
Recurrent Networks – p.9/??
QBL - The algorithm
Step 1: Select a new training trajectory from a set of
finite number of learning trajectories spanning over
robot work space.
Step 2: For a desired target vector selected
sequentially as per the learning trajectory, compute
control input by inversion of RBFN model to get the
desired output taking present state into account.
Step 3: Actuate the control action to robot manipulator
and observe joint positions and velocities.
Recurrent Networks – p.10/??
QBL - The algorithm
Step 4: If actual states of robot manipulator is
following the desired trajectory, then go to step 2 and
repeat the process for next desired target vector, else
store the data set as a new training pair and go to
step 2.
Step 5: If the recall process for the complete trajectory
is over, select a new trajectory in Step 1 until all the
learning trajectories are exhausted.
The input-output pairs so generated can be used to retrain
the RBFN model so that it can adapt itself for newer inputs.
This training can be done both in online and offline since
the emulator is trained before implementing the controller.
Recurrent Networks – p.11/??
The Control Objective
Given a desired state trajectory vector i.e. output activation of the RBFN model, xd (k + 1), and actual
system state vector x(k); design a control law so that
the neural model g(·)is Lyapunov stable.
Recurrent Networks – p.12/??
Forward-inverse modeling
+Σ x̃
∂ x̂
∂u
Weight
Update rule
Ẇ
qd
Neural
q̂
Emulator
∂u
∂W
Neural Controller
τ ff
Σ
τ fb
Robot Manipulator
q
PD Controller
Figure 3:
Indirect adaptive controller using forwardinverse-modeling approach
Recurrent Networks – p.13/??
Control law for forward inverse modeling
Consider the Lyapunov function to be
1 T
V = (x̃ x̃)
2
where x̃ = xd − x̂.
The time derivative of the Lyapunov function can be
derived as follows
∂ x̂ ∂u
V̇ = x̃ x̃ = −x̃
Ẇ = −x̃T J Ẇ
∂u ∂W
∂ x̂ ∂u where J = ∂u ∂W .
The objective is to select a weight update law Ẇ such that
the derivative of Lyapunov function remains negative
semidefinite. We will now present two such weight update
laws that are converging in the sense of Lyapunov function.
Recurrent Networks – p.14/??
T
T
Control law for forward inverse modeling
Let’s select the weight update law to be
Ẇ =
k x̃ k2
k J T x̃ k
T
J
x̃
2
Combining the expressions for Ẇ and V̇ , we have
V̇ = −k x̃ k2 ≤ 0
Thus the weight update law for W is convergent, i.e. it
guarantees that the tracking error converges to zero.
However, the weight update law does not ensure the
boundedness of the weights.
Recurrent Networks – p.15/??
Control law for forward inverse modeling
Thus the update law is modified by adding the
gradient of a cost function H as follows:
∂H
k x̃ k2 T
Ẇ =
2 J x̃ − Ξ(W )
T
∂W
k J x̃ k
where
Ξ(W ) = I −
1
k J T x̃ k
T
T
J
J
x̃x̃
2
Using this update law, V̇ becomes:
V̇ = −k x̃ k2 ≤ 0
This update law ensures convergence of tracking error vector x̃ to
zero. Ξ(W ) is chosen such that x̃T J Ξ(W ) is zero which ensures the
time derivative of the Lyapunov function remains negative
T
1
be 2 W̃ W ,
we are
semi-definite. Simultaneously by selecting H to
intuitively providing a damping to the weight that is increasing. This
Recurrent Networks – p.16/??
ensures the boundedness of the weight vector W .
Forward inverse modeling: Weight update rules
Two weight update rules are derived which guarantees
stability of forward inverse modeling based adaptive
control:
Weight update rule 1:
Ẇ =
k x̃ k2
T
x̃
J
2
k J T x̃ k
Weight update rule 2:
∂H
k x̃ k2 T
J x̃ − Ξ(W )
Ẇ =
2
∂W
k J T x̃ k
where
Ξ(W ) = I −
1
T
T
x̃x̃
J
J
2
k J T x̃ k
Recurrent Networks – p.17/??
Network Inversion
τ
Robot
Manipulator
q, q̇
Neural
Emulator
û
x̃
Network
q̂, q̂˙
Control law
to Stabilize
neural emulator
û
J
û
predicted control input
Inversion
next desired states
present and past states
past inputs
Algorithm
Figure 4: Indirect Adaptive Controller Using Network Inversion
Recurrent Networks – p.18/??
Control law using network inversion
Such a function can be expressed as
1 T
T
x̃ x̃ + ũ ũ
V =
2
where x̃ = xd − x̂ and ũ = û − u. Here xd is the desired
output activation, x̂ is the actual control action. The time
derivative of the Lyapunov function is given by
∂ x̂
V̇ = −x̃
u̇ − ũT u̇ = −x̃T (J + D)u̇
∂u
T
where J =
∂ x̂
;
∂u
n × p Jacobian matrix and D =
T
1
2 x̃ũ .
kx̃k
Recurrent Networks – p.19/??
Control law using network inversion
Theorem 1: If an arbitrary initial input activation u(0) is
Z t0
updated by
u̇dt
u(t0 ) = u(0) +
0
where
2
k x̃ k
T
u̇ =
x̃
(J
+
D)
2
T
k (J + D) x̃ k
then x̃ converges to zero under the condition that u̇ exists
along the convergence trajectory.
Proof: Substituting u̇ in V̇ , we have V̇ = −k x̃ k2 ≤ 0
where V̇ < 0 for all x̃ 6= 0 and V̇ = 0 iff x̃ = 0 Q.E.D.
The iterative input activation update rule:
u(t) = u(t − 1) + µu̇(t − 1)
where µ is a small constant representing the updateRecurrent
rate.
Networks – p.20/??
Comparative Performances
neural model
before QBL
after QBL
rms error in input prediction
TD 1 TD 2 TD 3 TD 4
0.081 0.12 0.074 0.067
0.029 0.021 0.024 0.032
Table 1: Comparison of neural model before and
after query based learning over four different test
data (TD) sets
Note that query based learning improves network
training and the prediction capability.
Recurrent Networks – p.21/??
Forward-inverse modeling
+Σ x̃
∂ x̂
∂u
Weight
Update rule
Ẇ
qd
Neural
q̂
Emulator
∂u
∂W
Neural Controller
τ ff
Σ
τ fb
Robot Manipulator
q
PD Controller
Figure 5:
Indirect adaptive controller using forwardinverse-modeling approach
Recurrent Networks – p.22/??
Forward inverse modeling: Weight update rules
Two weight update rules are derived which guarantees
stability of forward inverse modeling based adaptive
control:
Weight update rule 1:
Ẇ =
k x̃ k2
T
x̃
J
2
k J T x̃ k
Weight update rule 2:
∂H
k x̃ k2 T
J x̃ − Ξ(W )
Ẇ =
2
∂W
k J T x̃ k
where
Ξ(W ) = I −
1
T
T
x̃x̃
J
J
2
k J T x̃ k
Recurrent Networks – p.23/??
Forward-inverse modeling: Simulation Results
rms tracking error over 50 repeated trials for a sinusoid trajectory: (1) - - using gradient
descent with weight update once per sampling interval, (2) - · - using gradient descent
with weight update thrice per sampling interval, (3)– – using adaptive tuning with weight
update once per sampling interval, (4)—- using adaptive tuning with weight update thrice
per sampling interval, and (5)· · · · · using adaptive tuning with weight update once per
sampling interval.
Recurrent Networks – p.24/??
Forward-inverse modeling: Simulation Results
(a) Sinusoid trajectory tracking of joint 1: — desired, - - using gradient descent, · · ··
using adaptive tuning. (b) Tracking error in joint 1: — using adaptive tuning , · · ·· using
gradient descent.
Recurrent Networks – p.25/??
Forward-inverse modeling: Simulation Results
(a) Sinusoid trajectory tracking of joint 2: — desired, - - using gradient descent, · · ··
using adaptive tuning (eqn 24). (b) Tracking error in joint 2: — using adaptive tuning, · · ··
using gradient descent.
Recurrent Networks – p.26/??
Network Inversion
τ
Robot
Manipulator
q, q̇
Neural
Emulator
û
x̃
Network
q̂, q̂˙
Control law
to Stabilize
neural emulator
û
J
û
predicted control input
Inversion
next desired states
present and past states
past inputs
Algorithm
Figure 6: Indirect Adaptive Controller Using Network Inversion
Recurrent Networks – p.27/??
The Control law
The control law for the network inversion based
adaptive scheme:
u(t) = u(t − 1) + µu̇(t − 1)
where
2
k x̃ k
T
u̇ =
x̃
(J
+
D)
2
k (J + D)T x̃ k
and µ is a small constant representing the update
rate.
Recurrent Networks – p.28/??
Network inversion based control: Simulation Results
(a)Tracking error in joint 1 and joint 2 using control law (eqn 35) for sinusoid trajectory:
—– after query based learning, · · ·· before query based learning. (b)Controller response
for joint 1 and joint 2 corresponding to figure 10(a): —- after query based learning, · · ··
before query based learning.
Recurrent Networks – p.29/??
Network inversion based control: Simulation Results
(a)Tracking error in joint 1 and joint 2 using control law (eqn 35) for exponential
trajectory: —– after query based learning, · · ·· before query based learning.
(b)Controller response for joint 1 and joint 2 corresponding to figure 11(a): — after query
based learning, · · ·· before query based learning.
Recurrent Networks – p.30/??
Summary
In this lecture following topics have been covered
Indirect adaptive control for a robot manipulator
Indirect adaptive control using forward-inverse
modeling approach
Indirect adaptive control using network inversion
Both the control schemes are shown to be
Lyapunov stable
Simulation results are provided to validate
efficacy of the proposed schemes.
Recurrent Networks – p.31/??
References
1. Laxmidhar Behera, Query based model learning and stable tracking of a robot
arm using radial basis function network, Computers and Electrical Engineering
29 (2003) 553-573
2. L. Behera, M. Gopal, and Santanu Chaudhury, On adaptive control of a robot
manipulator using inversion of its neural emulator, IEEE Trans. On Neural
Networks, vol. 7, no. 6, pp. 1401-1414, nov. 1996.
3. L. Behera, M. Gopal and Santanu Chaudhury, On inversion of RBF networks and
its application to adaptive control of nonlinear system, IEE Proceedings Control
Theory and Applications, vol. 142, no. 6, pp. 617-624, Non 1995
4. Bao-Liang Lu, Kita, H., Nishikawa, Y.; Inverting feedforward neural networks
using linear and nonlinear programming, IEEE Transactions on Neural Networks,
Volume 10, Issue 6, Nov. 1999 Page(s):1271 - 1290
5. Saad, E.W., Choi, J.J., Vian, J.L., Wunsch, D.C., II; Query-based learning for
aerospace applications, IEEE Transactions on Neural Networks, Volume 14,
Issue 6, Nov. 2003 Page(s):1437 - 1448
Recurrent Networks – p.32/??